I was trying to get Claude to help me explain a seemingly simple maths problem to my daughter - how to work out 7/8 of 1000. I wanted to find a way to help her visualise the concept without jumping to solutions. Claude did help, but it had to spend a few minutes building an interactive app and uploading it as an artefact.
It worked, but it got me thinking: why did it need to generate an application in the first place?
What if we could skip generating an app and go directly from what we ask for to an interactive experience?
The idea
LLMs essentially generate a stream of text. So what happens if we treat that stream as instructions for drawing and animation instead of something for us to read?
The idea is to treat the entire output stream as a language intended for another system, rather than for us to read.
The AI and the receiving system share a common language, allowing instructions to be interpreted and acted upon as they're generated.
The receiving system can maintain its own state, which the AI can inspect and build upon without having to recreate everything from scratch.
In my experiment, that system is a browser, interpreting the stream into visual and interactive experiences. But the underlying approach isn't limited to browsers, or even graphical interfaces.
Where this could go
I've since been experimenting with interactive maths lessons, visualisations and prototyping, and the results are promising.
The interesting part is that it runs on my existing Claude Code subscription (tested it to work on Codex too), without additional API costs. And rather than being limited to a chat window, I now have a different interface for interacting with the model - one that can capture inputs like drawings, gestures and interactions that were previously difficult to communicate through text alone.
I believe this approach could unlock a much broader range of experiences. Rather than building every interaction in advance, the model could compose and adapt the experience around what someone is trying to do, using contextual input from text, drawings, voice, cameras and the current state of the interface.
It's still early, there are bugs, and AI-generated interfaces aren't new. But I think there's plenty of unexplored potential in generating experiences directly rather than building an application first.
Do note! There is a big difference in results if say using a sonnet model versus a fable model.
For all the video demonstrations I used Opus 5.5 on medium which seems to work reasonably well. Because it's essentially relying on the model to be the "brain" that means rendering things that require exact co-ordinates does require an intelligent model with thinking tuned appropriately.
For some more complex scenarios I've found Fable medium to be quite good. It's a very simple demonstration that goes to show how far the difference in models have come.
I think as models become more intelligent, cheaper and faster in the future, this method potentially opens up a much richer alternative suite of bespoke user experiences.
---
I've made the code used to generate the demo public under the MIT licence, so you can experiment with it yourself using Claude Code or Codex.
GitHub: https://github.com/deegeevince/conduit
Full write-up: https://lattris.com/by/b/memories/what-does-the-user-interface-of-the-future-look-like-8aff