Press play to listen to this article
Voiced by artificial intelligence.
When Jason M. Allen entered a fine art competition at the Colorado State Fair, he wasn’t just trying to win — he wanted to make a point.
Allen is a video game designer. His submission, titled “Théâtre D’opéra Spatial,” was a baroque phantasmagoria depicting three gowned figures illuminated by the light of a large open window. It was only after the artwork took first place, in August 2022, that Allen revealed it had been created using Midjourney, an artificial intelligence program capable of converting textual descriptions into full-blown images.
His victory was met by outrage among other artists, who felt he had not played by the rules, and worried that works like his undermined their craft. Allen was unapologetic. More than that: He welcomed the backlash. “I wanted to make a statement,” he said. “Artists were saying that this would never happen, that ‘AI will never be able to do what I do.’ But it can: It won first place in a fine art competition.”
“We should be talking more about whether we are gonna let AI take over our lives,” he said.
In the months since Allen won his competition, AI-powered creativity has surged from the nerdiest fringes of the internet to the center of a conversation about what exactly constitutes art — and whether there’s still a future for traditional artists and the skills they’ve spent their lifetimes developing.
Spend any time on the internet, and you’re sure to come across AI art. These include images often created in seconds by programs like Midjourney, Stable Diffusion and Dall-E-2, and more recently, poems, lyrics and essays penned not by humans but by ChatGPT, a chat bot trained on billions of words scraped from the internet. In January, Google revealed it had created a program capable of turning text descriptions into high-fidelity music.
The AI illustrations span the full range of artistic styles — Picasso, Magritte, H. R. Giger — and can mimic brush strokes or, for photo-realistic images, the visual effects produced by different types of cameras. There are few limitations: Most models are programmed to refuse to generate pornographic or violent images, or images of living politicians or public figures. They also sometimes struggle with intricate details; most notably, they’re comically bad at drawing hands, often depicting them with mishappen digits or far too many fingers. But issues like these are expected to be resolved as the technology improves.
“Even hardened veterans of the AI industry couldn’t quite predict how quickly things would move,” said Abhishek Gupta, founder of the Montreal AI Ethics Institute, a nonprofit that monitors the industry. “Every time we were getting over our astonishment, at a pretty frequent interval something else would follow.”
Economic anxiety about galloping technological change dates back to at least the beginning of the industrial revolution when British textile workers rioted against mechanization, breaking machinery and fighting with the army. Taking their name from an apocryphal figure named Ned Ludd, or King Ludd, the rampaging weavers became known as the Luddites.
In the centuries since, the word “luddite” has taken on negative connotations, as waves of technological disruption led not to mass unemployment but to the creation of new industries, better jobs and higher standards of living. The question, as AI comes for the creative class, is not so much whether history will repeat itself — but how the process will be managed.
The automatization of functions that were once regarded as uniquely human is forcing artists, engineers, lawyers and regulators to wrestle with some of the biggest conundrums in the philosophy of art, including the nature of art, who should be credited with its creation, and whether intangible elements like style, feel or inspiration can be owned.
Creative destruction
Those seeking parallels with the Luddites don’t have to look very hard. Thousands of artists have posted “Do Not AI” signs on their Twitter accounts or online art showcases. One such platform, ArtStation, was so overwhelmed by anti-AI memes that it banned them in late December and announced it would soon require AI-made artwork to be labeled.
The people behind the AI programs are quick to point out that not all artists are grumbling. “I don’t think AI replaces artists,” said Emad Mostaque, founder of Stability AI, the company behind Stable Diffusion. “Photographers and digital artists didn’t replace conventional artists. AI creates brand new forms of expression.”
But there is little denying that some people — if not honest-to-god, groundbreaking artists, then workaday illustrators, commercial graphic designers and photo retouchers — are on track to be made redundant.
On the other side of the ledger, there are signs a new industry is already being created. In mid-October, Addition, an Illinois-based AI consultancy, put out a job ad for a new role: prompt engineer.
The job — annual salary $90,000-$100,000 — entails developing and testing “prompts,” the text that tells AI programs what to create. The ideal candidate would have an eclectic background, blending coding skills with creativity and, according to Addition’s CEO Paul Aaron, artistic sensibility.
Writing prompts has, in the view of many of its practitioners, become a bit of art in itself, and a small ecosystem of platforms — with names like PromptBase, KREA, PromptHero — has sprung up online offering high-performing prompts for a few dollars a pop. An 82-page guide to the image generator DALL-E-2 put together by the British artist Guy Parson features this example of a good prompt: “grainy abstract experimental expired film photo of a woman in a red dress, talking angrily on a mobile phone, gesticulating angrily, in 1960s New York City by Saul Leiter, 50mm lens, cinematic colors, oversaturated filter, blur, reflection, refraction, distortion, rain drops, smears, smudges, blur, cinestill 800t.”
The same prompt will return different results depending on the platform. But in general, detail, precision and the use of signpost words (a historical period, an art style, a type of camera) are essential. “It’s almost like [British logician] Bertrand Russell, when he was analyzing sentences,” said Aaron. “You don’t have to go that far — but you need to be, like, very precise.”
Prompts, Aaron said, can even be used to generate other prompts. “You can train a language model to generate an ‘art prompt,’” he said. The illustrations of the future might begin life as buzzword-laden emails, be metamorphosed by a chatbot into an art prompt and finally emerge fully grown as a digital image: a weathered gunslinger standing in the Arizona desert; a Shiba Inu puppy sporting pink shades; a sinister lunar landscape.
‘Secret recipe’
The explosion of AI-generated artwork has sparked a debate not just about the nature of art — but about the ingredients that go into the creative process, and who can legitimately claim ownership.
When Allen won the fine art competition, he infuriated illustrators, but he upset prompt writers too. By deciding not to reveal the words he used to create his artwork, he breached what had by then become an unwritten rule of the online AI art community. “I originally said I was going to publish the prompt when I was finished with my project,” Allen said. “I’ve since learned better: It’s like asking a chef to release his secret recipe.”
Even if a prompt is not guaranteed to return the same result every time it’s used, Allen said he had devised a general structure — a “seed” — able to consistently deliver images with a characteristic vibe. Like Vincent van Gogh, Allen’s seed won’t always create identical paintings; but like van Gogh, the image it generates will share a signature je-ne-sais-quoi. “It’s a high-value prompt,” he said. “That’s something people are searching for.”
Allen said he was talking to a lawyer to explore ways to protect his intellectual property, not only of the artwork, but of the prompt too. While he declined to disclose details, Allen said that copyrighting his prompt would “definitely be a possible direction we could take.” But he worried that the process might require sharing the prompt publicly. “It’d be in the public domain,” he said. “It’d be basically telling everyone not to think of an elephant.”
Traditional artists are also pressing their claims, seeking compensation when their work is being used to train AI models, or simply asking to be able to opt out altogether. Karla Ortiz, an award-winning concept artist who has worked with film studios like Marvel and Industrial Light & Magic, recalled stumbling upon a tutorial for an image generator that encouraged users to include an artist’s name in the prompt when gunning for a particular style.
“I found the works of over hundreds of artists, peers that I knew personally,” said Ortiz. “People are utilizing our full names to evoke those works and allow AIs to generate media that is supposed to be like our work.”
The story of fantasy illustrator Greg Rutkowski has become a shorthand for the problem. His name has been included in more than 93,000 prompts for Stable Diffusion, according to findings cited in the MIT Technology Review. That has created a glut of content à-la-Rutkowski — with Google searches for his name returning AI-made fake Rutkowskis among the top results.
In January, Ortiz and fellow artists Sarah Andersen and Kelly McKernan initiated a class-action lawsuit in California against Stability AI, Midjourney and the art website DeviantArt, alleging the unauthorized use of artists’ work to train the Stable Diffusion AI is a breach of copyright so massive to be comparable to historic “art heists.”
Ortiz is also an adviser at the Concept Art Association, a trade group that argues that the U.S. Federal Trade Commission should force companies to carry out what it describes as “algorithmic disgorgement” — the destruction of datasets trained on work by artists who did not give their consent for it to be used.
In early January, the photo agency Getty Images also announced it would bring Stability AI to court in London, accusing the company of scraping millions of its copyrighted images.
Human authorship
With the legal framework around AI art still to be developed, court cases like Ortiz’s and Getty’s are likely to play a central role in shaping the landscape on which future battles will be fought.
Regulators in the United States and Europe broadly agree that if a person is involved in the creation of an AI-generated artwork, that person holds the right to that work. In February 2022, the U.S. Copyright Office decided that an image randomly generated by an AI in 2018 could not be copyrighted by the AI’s owner because the work “lack[ed] the human authorship necessary.” But in September, it accepted the registration of a graphic novel whose visuals had been generated by an AI based off the author’s prompts.
European Union institutions — the European Court of Justice, the European Commission, the European Parliament — also distinguish between “AI-generated” outputs created fully by a computer, and “AI-assisted” outputs in which a person made an intellectual effort to extract something original out of the machines.
But that’s about as far as any agreement goes. Speaking at a town hall event on AI art in November, a U.S. Copyright Office official said that while simple prompts are unlikely to be copyrightable, intricate ones raise “more complicated questions.” Similarly, Renaud Dupont, a managing partner at Brussels legal firm CMS, said if a prompt meets the EU’s criteria of originality and distinctiveness, “rights can be claimed.” Tl;dr: You can probably lay claim to a super-convoluted prompt, but we don’t know for sure yet.
The EU is crafting legislation to regulate AI, but the draft proposal was put forward by the European Commission in 2021 when AI programs like ChatGPT or Midjourney seemed like distant possibilities, and so the challenges they pose have yet to be addressed.
Members of the European Parliament discussing the text recently agreed on an amendment that would require AI-generated text to be labeled — a principle that could easily be extended to images. Dragoș Tudorache, one of the MEPs steering the text through the Parliament, has hinted that AIs that generate content like illustrations or text will need some type of regulation.
Visual conversation
The hottest battle at the moment is over whether traditional artists can ask researchers and companies to remove their work from the databases used to train AI technology. Style is traditionally not copyrightable — but individual works are, so the question comes down to whether using them to train a model qualifies as fair use.
European artist associations are calling on lawmakers to require that artists give their consent before their work is included in training data. In the U.K. however, the government plans to make opting out from mined data impossible.
In theory, the EU’s Copyright Directive allows European artists to demand their images be removed from models. But the law includes an exemption for datasets initially created for research and scientific purposes, which could include those used by Stable Diffusion and Google’s model Imagen.
In the long run, AI companies will likely try to come to an understanding with artists and other visual entrepreneurs. Some of that is already happening. The new version of Stable Diffusion, released in late November, makes it harder to aim for a specific artist’s style — to the annoyance of some users.
One possibility that is being discussed as acceptable to both sides is a licensing system in which artists get rewarded to help train AIs. “If we transition to something ethical where the models are built using public domain images, and any expansion is done via licensing — then honestly, I’m not threatened by that,” Ortiz said, in an interview.
But while such an agreement might paper over the commercial disagreements, for artists like Ortiz, it won’t address the more fundamental issues. “Creating art is one of the most wonderful acts humanity can do,” Ortiz wrote in an email after our interview. “For me every brushstroke is a thought, an emotion, a decision.”
She described the creation of art as “a visual conversation between one human and another human,” she said. “For AI, the extent of the human in that conversation is typing ideas like ‘sunset, people, 4k, trending in art station, insert name of living artist here.’”
“I personally find that so empty,” she said. “As an artist, I would not find joy in that alone.”
