How does AI build complex interactive applications?

Watching Gemini 2.5 Pro's ability to turn video directly into an interactive application, it dawned on me that the paradigm of AI building complex applications is changing qualitatively. We used to talk about AI as a tool, but now AI is more like a child that can put together its own Lego - not only can it read the instructions (multimodal understanding), but it can also build a finished product that can actually be played with according to the schematics (code generation). The evolution of the technology behind this is simply fascinating, especially when you realize that it can even handle the context of an hour-long video.

Multimodal Understanding: AI's "Sensing" Capability

Seriously, having AI understand video is nothing new, but what Gemini 2.5 Pro does is very human - it understands actions, speech, text, and even interface elements in a video as executable development requirements. The difference is that the AI "cooks" the video into runnable code, just as you can cook after watching a cooking video. There is a test case is particularly interesting: AI after watching the React teaching video, not only generated component code, but also automatically fill in the video did not demonstrate the error handling logic. This kind of ability to learn by example is still unimaginable in last year's model.

The Evolution of Interaction Logic: From Static to Dynamic

I remember in the early days when AI generated code, what came out were "dead" pages. Now the breakthrough is that AI is starting to really understand the word "interaction". User tests have shown that the memory matching games generated by Gemini 2.5 Pro already take into account the animation rhythm of card flipping and the consistency of scoring logic. More amazingly, a developer made a traffic light simulator in 20 minutes - this thing has to deal with vehicle generation, signal timing, collision detection and other dynamic logic, in the past at least a few hundred lines of code. Now the AI seems to have some kind of "program sense", can predict the chain reaction that may be triggered by the user's operation.

But then again, there are some interesting problems with this type of technology when it comes to implementation. For example, although the AI-generated code can be used, but the architecture style will carry a clear "AI flavor" - especially like to use Promise chain call, variable naming always has a strange poetry. A team has done statistics, 87% of AI-generated code needs to be manually adjusted in order to pass the strict Code Review, but from another point of view, this is not exactly the beauty of human-computer collaboration, AI is responsible for rapid prototyping, and human beings are responsible for polishing and optimization.

Future possibilities: AI as runtime

A crazy idea came up recently while chatting with a few developers: what if AI could parse user behavior in real time and dynamically adjust app logic? Imagine, you are using the weather app, AI found that you always check the UV index, automatically sunscreen advice to a more prominent position. With this level of interaction complexity, traditional development needs to bury points, analyze, and iteratively release the version... while AI may learn to adjust directly on the spot.The sketch-to-app demo shown by Google already vaguely has the beginnings of this - it generates a drawing board app that dynamically adjusts to the characteristics of the described stroke Stroke algorithms.

Of course, today's technology is still a long way from real "live" interactive applications. Multimodal understanding sometimes misses important contexts, and the complex state management code generated is prone to bugs, but look at the Gemini 2.5 Pro's 147 Elo scores compared to its predecessor, who can tell what breakthroughs AI will make by this time next year? Perhaps by then, we are no longer discussing "how AI builds applications", but "how AI and applications can coexist and coevolve".