Steering the Creative Machines

Now that image generation is mature, the techniques by which AI tools can be driven are diverging.

The quality of AI-generated images improved enough over the past eighteen months that the debate over whether machines can produce usable visual work has quietly resolved itself. Most commercial applications can now be served by generation tools at a fraction of what a photo shoot or an illustrator would cost. The public conversation about AI and visual creation, however, still orbits around output quality, as though the central question remains whether the images are good enough. Practitioners settled that question some time ago.

Over the past two weeks, three developments landed in the same territory. Stanford’s Institute for Human-Centered AI published research on tools that teach generative models to follow an artist’s spatial direction. Adobe launched a conversational AI assistant inside Photoshop that lets editors describe changes in plain language. Google expanded Pomelli, its automated marketing tool, from four countries to more than 170. All three are solving for the same constraint: how to tell the machine what to make. The answers they offer diverge sharply in whom they serve and what they assume about the user’s expertise.

A visual vocabulary

The Stanford project, funded by a Hoffman-Yee Research Grant through Stanford HAI, brings together computer scientists, cognitive psychologists, and education researchers to build what the team calls a shared conceptual grounding for human-AI visual collaboration. Maneesh Agrawala, professor of computer science and co-principal investigator, frames the central problem with a test case: ask a generative model for a suburban single-family home, and it produces a modern duplex. The model can render either structure. The gap between request and result is a failure of communication, rooted in the absence of a shared vocabulary through which the human can specify an intention and the model can register what it understood.

ControlNet, one of the tools that emerged from the project, addresses this gap by teaching diffusion models about spatial composition in two stages. The first, blocking, establishes the rough arrangement of elements in a scene, comparable to the way that an artist begins with a loose sketch to fix proportions and placement before committing to detail. The second, detailing, refines those compositional decisions into finished visual elements. Current models struggle with precisely the spatial dimensions of image-making that this two-stage process targets: pose, object placement, the relationship between foreground and background. A text prompt can describe mood, palette, and subject matter with reasonable fidelity, yet it offers no native mechanism for communicating that a figure should stand in a particular spot, facing a particular direction, with the horizon at a particular height.

The research extends beyond static images. FramePack, a second tool from the same group, applies a similar principle to 3D video, teaching models to prioritize scenes by narrative importance when composing multi-scene sequences. A third innovation uses neuro-symbolic AI to generate visual diagrams from text prompts through an intermediate coding language, making the generation process legible to the human directing it. The interdisciplinary composition of the team reinforces this framing. Judith Fan, an assistant professor of psychology who studies how humans establish common conceptual ground with one another, serves as a collaborator. Her involvement signals that the researchers locate the problem in the space between two agents, one human and one computational, rather than in the model’s capability alone.

Editorial assistance

Adobe’s response arrived on March 10 with the public beta of an AI Assistant inside Photoshop, available on web and mobile. The interaction model is conversational. A user describes the desired edit in natural language, and the assistant either applies it directly or walks the user through the steps that would achieve it manually. On mobile, the assistant accepts voice input, turning spoken description into applied changes. The choice between automatic execution and step-by-step guidance reveals whom Adobe designed this tool for. A tool that can apply an edit and a tool that can teach someone how to apply an edit occupy different positions in a creator’s development, and Adobe built both functions into the same interface.

A second feature, AI Markup, extends the conversational model with spatial specificity. Available in Photoshop on the web, it allows the user to draw directly on an image and attach a text prompt to the marked area, controlling both where a change happens and what that change should be. Text prompts alone force the user to describe spatial relationships in language, a medium poorly suited to the task. Drawing solves the spatial half of the communication problem while language handles the semantic half, and combining the two addresses the same gap that Stanford’s researchers identified from the academic side.

The Firefly Image Editor, updated alongside the AI Assistant, consolidates Adobe’s generative tools into a single workspace. Generative Fill, Remove, Expand, Upscale, and Background Removal all operate within one interface, and the editor now provides access to more than 25 third-party generation models, including Google’s Nano Banana 2, OpenAI’s image generation, Runway’s Gen-4.5, and Black Forest Labs’ Flux.2. Adobe has positioned Firefly as the place where generated imagery meets professional craft, and the consolidation reflects a bet that professionals will adopt AI tools faster when those tools live inside the workflows that they already inhabit.

Autonomous design

Google’s Pomelli occupies the opposite end of this spectrum. A user pastes a website URL, and Pomelli scans the site to extract what Google calls a Business DNA profile: the brand’s colors, fonts, tone of voice, and visual style. All subsequent content that the tool generates draws from this profile. The user does not need to articulate creative direction because the direction already exists, embedded in the brand’s published assets. Pomelli infers it and applies it.

The tool launched in October 2025 as a Google Labs experiment available in four English-speaking countries and expanded to more than 170 countries on March 9, 2026. Two features added since launch extend its scope beyond static social media posts. Photoshoot, introduced in February, transforms smartphone product photographs into studio-quality images using Google’s Nano Banana 2 model. Animate, launched in January, converts static marketing assets into branded video using Veo 3.1. The entire service remains free during the beta period.

Pomelli belongs to a category that includes Canva’s Magic Studio and a growing number of platforms where the user’s role has shifted from directing the creative process to selecting among its outputs. These tools handle layout, typography, color matching, and copy generation internally, presenting finished options for approval. The design philosophy that unites them assumes that their users want good results more than creative control, and builds accordingly.

Expert direction

The prevailing fear about AI image generation, through most of 2024 and 2025, held that these tools would make professional visual creators redundant. If anyone could produce a polished social media graphic or a product photograph by typing a sentence, the reasoning went, the market for people who do that work professionally would collapse. At the output level, some version of this prediction has materialized. A small business owner using Pomelli can produce branded social media content that would have required a designer or an agency two years ago, and the cost has fallen from hundreds of dollars per project to zero during the beta period.

The tools diverged, however, along lines that the displacement narrative failed to anticipate. Platforms serving people without design training absorbed creative decisions into automated pipelines, removing the need for visual judgment as a prerequisite for visual output. Platforms serving professionals moved in the opposite direction, building interfaces that reward specificity, spatial awareness, and iterative refinement. Pomelli’s user never needs to form a visual opinion. Adobe’s user, working with AI Markup or the conversational assistant, succeeds in proportion to the clarity and precision of the opinions that the user brings.

Agrawala’s observation that authoring original content requires having opinions and constantly making choices points toward the underlying dynamic. Execution, the ability to render a concept as a finished image, is the capability that generation tools have commoditized. Direction, the ability to know what should be rendered and to communicate that knowledge with precision, remains a human skill, and the new generation of professional-grade tools treats it as the primary input. The professional creator’s value has migrated from the capacity to produce finished work to the capacity to direct its production with specificity and taste.

A spectrum of service

The same two weeks of March produced tools aimed at researchers studying how humans and AI establish visual common ground, at professionals editing photographs through conversation and spatial markup, and at small business owners who have never opened an image editor. AI visual creation is developing a skill curve. The automated end of that curve serves anyone with a URL and a product to promote. The precise end rewards spatial thinking, aesthetic judgment, and the ability to articulate a visual intention clearly enough that a model can follow it. Both ends are expanding, and the people who spent years learning to see have found themselves at the end where the tools are getting sharper.