Putting a Unity player inside a Flutter widget

#The requirement

A client wanted customers to rotate a product in three dimensions, change its finish, and see the price update as they went. Unity renders that comfortably. Flutter, in early 2019, had no way to host a real-time engine inside a widget tree.

The options were to build the whole app in Unity and lose every native control, or to build it in Flutter and give up the 3D. Neither was acceptable, so I looked at what a platform view could actually carry.

#What a platform view gives you

Flutter draws its own pixels. A platform view is the escape hatch: the framework asks the host operating system for a native view, composites it into the widget tree, and hands input to it. On Android that meant an AndroidView backed by a SurfaceView; on iOS it meant a UiKitView backed by the Unity player's own view.

Unity, in both cases, exposes a way to run as a library rather than as the whole application. Getting it to start, render into a surface owned by somebody else, and stop cleanly is the entire problem.

#The message channel

Rendering was only half of it. The configurator had to tell Unity which finish to apply, and Unity had to report back when the user tapped a part of the model. Flutter's platform channels carry that, but they carry strings and maps rather than typed values.

1// Flutter side: tell the scene which material to use.
2_unityWidgetController.postMessage(
3  'ProductRoot',      // GameObject name
4  'SetFinish',        // method
5  jsonEncode({'finish': 'brushed-steel'}),
6);
Copy

On the Unity side that arrives at a MonoBehaviour attached to a game object with a matching name, in a method with a matching signature. Both names are strings, resolved at runtime, matched by convention. Nothing checks them. Rename the game object and the message goes nowhere, quietly, with no error on either side.

That design decision followed the shape of the platform channel API, and I regretted it for six years. Almost every integration built its own encoding on top, and almost every one of them had a bug where a renamed object silently stopped receiving messages.

#Lifecycle is where the real work is

A rendering surface is easy compared to the question of what happens when the app goes to the background. The engine holds a rendering context, audio focus, and often a camera. Android may reclaim any of those. Flutter may dispose the widget while the engine is mid-frame. The user may rotate the device, which on Android recreates the activity underneath everything.

Getting this wrong produces the two worst bug classes in the tracker: a black rectangle where the scene should be, and a process that terminates with no useful stack. Both take a long time to reproduce, because both depend on timing.

The rule I settled on is that the widget owns the pause and resume calls and the engine never assumes it is visible. Anything that survives backgrounding lives in Unity's own state, and anything that does not is rebuilt on resume.

#Why publish it

I published because the alternative was that everyone with this requirement would spend the same weekend. Augmented and mixed reality inside conventional applications has been an active area for a long time [2], and the tooling gap in 2019 was not conceptual. It was the plumbing between one rendering framework and another.

The repository is public [1]. I did not expect to maintain it. I expected to solve one screen.