What does the future hold for multimodal AI?

Looking at the launch of OpenAI's gpt-image-1 multimodal model, I can't help but think: This is just the beginning, isn't it? AI is evolving from a unimodal "biased student" into an all-around "generalist". Text, image, and voice are being integrated into an interactive chain, and the change is far more profound than we imagined. Take the design field, in the past, designers may need to write copy guide, then sketches, and finally do the rendering, and now as long as a natural language description, AI will be able to link these links to complete. This change is not simply "improving efficiency", but reshaping the entire creative production process.

How is multimodality changing human-computer interaction?

We have to admit that we are standing on the inflection point of the change of interaction mode. I remember a few years ago, the mainstream AI system is still basically in the stage of "question and answer machine", you have to search for keywords first, and then filter. Now this situation is being subverted - when I was testing gpt-image-1, I directly said to the phone "help me draw a cherry blossom pattern suitable for cell phone cases, pink gradient but not too girly", and even I was surprised that it could accurately get the aesthetic needs of this fuzzy. Even I am surprised that it can accurately get this kind of fuzzy aesthetic demand. This kind of "natural human expression + AI accurate understanding" interaction may be the most valuable part of multimodal.

Bottlenecks and breakthroughs behind the technology

But is it really smooth sailing for multimodal development? I don't think so. Recently, I found an interesting phenomenon in the test: the model's understanding of "style mix and match" is still not stable, so that when generating a "cyberpunk-style classical landscape painting", 80%'s works are more inclined to one of the styles. This in fact reveals the deeper problem of cross-modal representation understanding - the semantic alignment between different modalities requires amazing arithmetic power and data support.

But OpenAI has apparently found a way. Their latest tech blog mentions a "cross-modal contrast learning" strategy that allows models to be exposed to both textual descriptions and corresponding images during training. Interestingly, this is a bit like teaching a child how to read by pointing to a real object. The current model with 27B parameters can achieve this effect, what will happen when it exceeds 100B parameters? What will happen when the 100B parameter is exceeded?