Early iOS Export 28 Dec 2017
Taking a break from the posts outlining the inner workings of the compiler to report some exciting news: late last night, for the first time, Clojure code compiled with MAGIC successfully ran on an iPhone!
The TL;DR is that we’ve confirmed that MAGIC can achieve its original goal of bringing Clojure to more restrictive environments. In particular, the code in this prototype survived Unity’s IL2CPP translation into native code suited for iOS. We expected this to be possible, but seeing it running on a physical device puts to rest any lingering doubts.
The demo is an implementation of the classic boids flocking algorithm. It is available at this gist and for brevity I will not list it inline here.
boid.core is the first prototype I wrote using Clojure/MAGIC, and is most representative of what a namespace will look like when MAGIC is more complete and can stand on its own. For reasons that I get into below, a namespace like this does not “Just Work” yet.
boids.build is the namespace I used to produce the iOS-ready assemblies. You can see the bodies of functions and macros copied from
boid.core, but there’s a lot of compiler-specific machinery surrounding it. It should be reproducible on the latest MAGIC commit (
73531fc2 as of the time of this writing). I built the assemblies from within Unity using the Arcadia REPL.
BoidBehaviour.cs is the Unity component that invokes the Clojure code. It is attached to each of the 330 boids and passes the local
transform into the
Additional work had to be done to make the demo function given the current state of MAGIC. None of this reflects what a real-world workflow would be like given a finished compiler, but rather documents my process in getting this to work on MAGIC as it exists today.
This demo is restricted to the parts of Clojure that require no runtime support, namely:
- C# interop
- special forms (
MAGIC supports more than this, namely invoking Clojure vars, but looking up vars requires the support of the Clojure runtime, which requires
clojure.core to be loaded, which MAGIC cannot compile yet. In more usual uses of MAGIC this is not a problem, as MAGIC is usually run from within the stock compiler and leans on it to gain access to the Clojure runtime. In an exported context, this is not possible, because we have no way of getting
clojure.core to run on an iPhone without MAGIC being able to compile it.
Even given those limitations, MAGIC code is fairly expressive. Namespaces like
arcadia.linear are completely useable, and a lot of call sites that would normally be vars get turned into inlined bytecode by
inc and more. MAGIC’s macroexpander is working, so macros like
boids.build/transforms are also fair game.
Without runtime support, we cannot compile a proper Clojure namespace, so instead this demo compiles C# classes with static methods that contain the bodies of our functions. The default analysis passes are modified to generate static methods instead of instance methods, and the
boids.build/static-methods macro is designed to generate a CLR type with a named static method for each function.
There are some shenanigans involved in preparing the assemblies. First, note that we actually emit three assemblies in the comment at the end of
boids.build. This is to get around our inability to invoke vars. For a function to call another function, it must treat it as interop. The
flock function invokes the
alignment functions as static C# methods. This requires the
Boids.Rules assembly to be compiled first and loaded into memory. The same is true when
This is a bit tedious, but it works. I uncomment each of the forms in turn and emit each assembly to disk separately. The resulting assemblies (
boids.dll) can be merged into a single assembly using
ilrepack. Once dropped into a Unity project and referenced from your IDE, they are completely useable from C# and even participate in intellisense!
BoidBehaviour.cs can call our
update method, passing in its
BoidBehaviour component is attached to the boid GameObjects, which are positioned n a three dimensional grid in the scene. This works in the editor and survives IL2CPP export, because as far as Unity is concerned all it’s dealing with is C# code and one assembly full of IL it knows how to convert. I needed to add the
make-array intrinsic, but everything else converted without issue.
The goal is for something that looks like
boid.core to “Just Work” without additional intervention. To get there, MAGIC needs to be able to compile whole namespaces, closures, dynamic call sites, and the remaining special forms in a way that IL2CPP can convert. Compiling
clojure.core is the real hurdle. MAGIC’s approach to maximizing the flexibility of the bytecode emission means we can make our IL look how ever it needs to look to survive translation onto platforms like iOS. This early demo is an extremely promising milestone, and an indication that we’re on the right track.