From storyboard artists to editors to assistants, will artificial intelligence put humans out of a job – or help them do what they do better?
This is Part Seven of a series about AI and its impact on Hollywood. Keep reading WrapPRO for upcoming stories on AI. Level up your entertainment career and subscribe!
The evolution of artificial intelligence in entertainment has sent shockwaves through Hollywood in the last several months — even becoming a core issue at the heart of the Writers Guild of America strike. But ChatGPT may be the least of film and television creators’ worries, as major studios like Paramount and Disney have already started adopting AI for various other uses.
You've reached your article limit.
Unlock premium content with a subscription.
Click Here Already a subscriber? LoginYves Berguist, a data scientist and the director of the USC Entertainment Technology Center’s AI in Neuroscience and Media project, predicts the technology will cause “a lot of disruption in production and post-production” in Hollywood in the coming years.
How much disruption? TheWrap spoke with industry and tech experts and asked about a variety of jobs on movie sets that are currently performed entirely by humans — and here’s who might want to start mentally updating their resumes.
Assistants
Assistants keep the wheels of Hollywood turning, from answering phone calls and making lists to arranging travel and appointments. Those are all tasks that can just as easily be done by an AI.
When it comes to concerns over job displacement, Bergquist warns that the first people in Hollywood production who will be impacted by artificial intelligence are those with “menial” jobs involving “small workflows that don’t involve super-high technical knowledge or super-high creative ability” — i.e. assistants.
“Assistants probably will need to upskill or think about their future. Throughout the production and post-production process, you’ll need less of them,” he said. “It’s going to be less of a manual process and so the shift will be on technical knowledge of how to build that stuff, how to understand and talk about it and creative work.”
For now, AI doesn’t pose a threat due to the collaborative nature of how production assistants work, a PA on a Ryan Murphy Netflix series told TheWrap.
“We have to build relationships amongst those around us to help get the job done,” the assistant said.
In fact, AI could actually enhance the role of production assistants, allowing them to focus on more creative and strategic tasks while leaving more mundane and administrative work to the machines. For example, the production assistant noted that “design departments can use AI for research.”
At the same time, the PA is fearful of the effects of AI in the future.
“I’m worried that content quantity over quality will dictate how we are to navigate this industry and move up within it,” the production assistant said.
Storyboard artists
Before on-set work can begin, a writer’s concept needs to be mapped out and visualized ahead of time, a task that typically falls on the shoulder of a storyboard artist working with a director.
However, OneDoor Studios, a development and distribution studio from film producer and finance legend John J. Lee Jr., is looking to streamline the time-consuming development process via a crowdfunded film project known as “Calculated” that utilizes ChatGPT, Jasper and Adobe Firefly for AI-generated storyboarding and previsualization.
The development process, which is carried out via a Discord channel, allows fans and investors to weigh in on everything from character design and sets to special effects that showcase the protagonist’s near-supernatural ability to predict the future through mathematical calculations. OneDoor, which has raised $3 million, plans to use the funds to develop two sequels, “Simulated” and “Activated.” The entire series’ release remains years away.
Leveraging AI for storyboarding is “going to make a better finished product,” OneDoor chief marketing officer Dan Cobb told TheWrap. “Instead of making a storyboard that looks like sketches and pen and ink, now we’re going to have storyboards that literally look like the scene from the movie. And they’re going to be so close that once you’re on set, you’re going to try to match it because the quality of that image is the vision of the artist.”
He emphasized that storyboard artists who “stick with the same old marker boards” could risk becoming “irrelevant.”
“Nobody’s going to be happy with that anymore,” he continued. “People are going to say, ‘I expect every movie to look like what I first felt when I saw “Star Wars.”‘ You’re going to have that feeling that ‘Wow, this is a new epic level of quality’ and if you don’t have it, it’s going to show that you don’t have it and you’re going to be behind because the quality that it’s delivering is beyond what human hands can do.”
At the same time, he doesn’t believe that AI will take storyboard artists’ jobs away outright and instead sees the technological advancement as an opportunity to help them do their jobs better. One example that Cobb noted is the “Calculated” character Red, who initially had a beard and looked similar to Confucius during early illustrations.
“The author came in and said, ‘No, there’s nobody in China who looks like Confucius anymore. It dates the structure of the film and the story and no one in China will buy this,'” he explained. “Within less than three minutes, we started seeing Red without the beard described exactly as the author mentioned inside of the chat environment in Discord. So you have this environment now where in real time it feels like you’re co-creating the idea with others whereas in the past, the artist would have to go back to their closed door, spend a few days drawing, come back and show you another iteration. It might be months to get to the idea you want. It’s minutes now… They just entered into [a] chat room and now they’re making movies.”
Makeup and VFX artists
Makeup artists have long been the unsung heroes that ensure actors are ready for their close-ups. But advancements in Hollywood visual effects over the last several years have allowed for everything from small digital touch-ups to de-aging of well-known actors like Harrison Ford, Peter Cushing and Carrie Fisher.
One VFX studio and AI technology startup, Monsters Alien Robots Zombies, is taking things a step further. The company, which has worked on over 120 TV series including “Stranger Things,” “Umbrella Academy,” “Moon Knight,” “Spiderman: No Way Home,” “Ms. Marvel,” “WandaVision” and “Ant-Man and the Wasp: Quantumania,” launched an automated, end-to-end solution in January known as Vanity AI.
Vanity AI enables the delivery of cosmetic VFX 300 times faster than traditional VFX pipelines and has been used on 27 major Hollywood productions, shaving 100 weeks off production schedules and saving nearly $8 million in costs. The technology makes creating a feature-film caliber shot as simple as adjusting dials on a single frame. For example, instead of decreasing an actor’s brow lines frame by frame, a VFX artist can now tell Vanity AI what changes to make on a single frame per shot and the software replicates those changes for the entire sequence, taking into account the actor’s movements, changes in lighting, etc.
“The reason that we started to work on AI so long ago as a business was because we felt the challenges [of VFX artists]. There’s never enough time, there’s never enough budget. What can we do about this? Could we start to develop technology that actually makes VFX faster and easier for the first time?” MARZ CEO Matt Panousis told TheWrap. “And we were in search of an application that was broadly applicable and scientifically feasible and landed on Vanity as the first use case.”
He noted that traditional VFX work can take anywhere from “a half a day per shot to all the way up to three artist days per shot.”
“So pretty much anywhere from four hours to 24 hours of artist labor applied and we can now accomplish those same shots in 20 to 40 minutes,” he said. “So from the artist’s perspective, it’s fantastic because you are increasing their output by orders of magnitude. Like the same artist that could do one shot to maybe only half a shot a day is now pumping out 20, 30 shots a day.”
MARZ plans to complete beta testing of Vanity AI midway through 2023, during which time the production-ready technology will be offered as a service.
While it’s easy to assume this technology could one day advance to the point where both VFX and makeup artists could potentially become obsolete, Panousis and the MARZ team don’t see it that way.
“It’s very unlikely and definitely not near-term. I think what’s super important to note about Hollywood is the requirements for what are needed on a particular show are always very specific and what we’ve found is you want a talented artist in the driver’s seat,” he said. “When somebody comes to us to do any kind of cosmetic work or de-aging, there is a very specific look that they would like to achieve and so in Vanity, the idea is the artist is able to be in full control of that look.”
He believes Vanity will allow VFX artists to work an eight-hour day as opposed to a 12- or 14-hour day.
“Not only is it going give Hollywood studios what they need, which is VFX faster and more affordable, but it’s also going to bring work life balance back to a group that historically has not had the luxury of any form of real work life balance,” he said.
When it comes to makeup artists, Panousis predicts that how much they’re displaced would depend on the type of makeup work being performed.
“There’s always a creative component to makeup and that’s not something our solution handles today. Our solution is there to make cosmetic fixes really, really fast and on a really affordable scale,” he added. “I don’t think it would make sense for any company to develop solutions for things that are really creatively unique because you’re going to put years and years and millions of dollars into development and what that particular director wants may only appear on one or two movies that year and that’s where you need really talented makeup artists. But, in the future, do you need a world where makeup artists are doing the very simple stuff like getting rid of eye bags, crow’s feet, laugh lines and acne? Maybe not.”
Directors
Directors are one of the most important members of a TV or film production, tasked with taking a writer’s concept and vision and translating it to the big or small screen.
“Directors already use image generators like Dall-E to more accurately explain their creative vision to production designers,” Andrew Palmer, an assistant director who worked on the television shows “Suits” and “The Boys” told TheWrap. “Professional art directors use the same tools to complete their jobs faster and cheaper, and amateurs use them to produce high quality work with minimal budgets.”
Owen Harris, the director of Peacock’s artificial intelligence-focused comedy-drama series “Mrs. Davis,” told TheWrap that Hollywood desperately wants AI to be a “force for good.”
“Certainly the attraction and the amount of investment we put into technology financially, but also in terms of our own time, reflects that we get something out of it,” he added. “But the question is how much of ourselves are we giving away to get that and is trade-off worth it? At the moment, it feels very much like we’re at the crossroads.”
Harris believes with “total certainty” that there will be a film or project made entirely with AI.
“I could say with almost equal certainty that it’s the sort of film that probably I wouldn’t enjoy,” he added. “I’d probably be not very interested in it.”
He also believes it’s will “definitely” displace jobs in Hollywood.
“It’s not just the writers, it’s going to be across the board how this affects everyone individually in terms of the impact it will have. It’s going down areas of the industry and then these creatives are going to need to find work elsewhere,” he said.
If used properly, Harris believes that the tool can be used to help break and communicate ideas visually. But he argued that storytelling is still a “very human interaction” that’s going to be “very difficult” for AI to replicate.
“I’d always hope that there will be a human intelligence that can then interact with that and be the bridge back into the creative discussion… but it’s all happening so quickly that it’s really difficult to sort of gauge,” he added. “I’d be amazed if there’s a piece of AI that could tell a story that is absolutely unique to you in a tone of voice that’s absolutely unique to you. Maybe you could mimic a bit of Quentin Tarantino or Damon Lindelof but that’s not the storytelling that I want to make.”
“It will be a long time before there is an AI-generated movie,” one Hollywood agent told TheWrap. “What will happen is that AI will significantly bring down the cost of CGI effects and digital makeup and de-aging.”
A franchise film producer agreed with the agent, arguing that AI isn’t even remotely capable yet of making a good movie. Filmmaking, after all, is a director’s medium.
“Film is art and art is a reflection of the individual,” the producer said. “A computer cannot make art.”
A spokesperson for the Directors Guild of America told TheWrap that it continues to “research and monitor the potential creative and economic impact of AI on directors and their teams” as the technology rapidly develops. The DGA, whose current contract with the Alliance of Motion Picture and Television Producers expires on June 30, is currently negotiating a new contract.
Editors
Researchers are currently working on what’s known as “content intelligence” to help editors and producers in the post-production process extract specific features of content.
“Right now, when you shoot a bunch of content, someone has to sit in front of the rushes and tag certain moments. So we’re making videos searchable by shot types, emotional arcs of the characters, scenes, objects, talent, colors, etcetera,” Bergquist said. “So that really helps the producers get faster in processing all that content.”
Brian Ahearn, the CEO of media asset management solution firm Evolphin Software, told TheWrap that editors can also use AI as an assistant editor to create rough cuts automatically for review and refining.
“For example, editors can use AI to collate all scenes with a particular celebrity into a rough cut for final “polishing” in seconds, not days,” he said.
Synthesia and Adobe’s Sensei are two AI-based technologies currently being used for microtasks in the editing process, like matching style and color and auto-tagging.
“AI tagging makes a huge difference in how raw movie footage and B-rolls are stored, organized, and retrieved when needed. By using AI to tag and apply metadata to video files, editors can quickly find faces, logos, scenes, objects and even sentiments for use in post-production,” Ahearn added. “This cuts down the time taken to scrub through clips from days to seconds, speeding up the movie production process and precision.”
Additionally, programs like SmartROTO, CopyCat and ElectricSheep can be used for automated or accelerated rotoscoping or shooting without green screen.
For sound editing, AI can help quickly scan through large sound effects libraries to create soundscapes, Robert Harari, a music professor at Stevens Institute of Technology, told TheWrap. He’s worked on multiple Grammy-nominated albums, Emmy Award-winning programming for children’s educational content on PBS and musical themes for major sporting events.
“You used to sit there and have to go through thousands and thousands of banks of sounds to try and identify something that’s going to have a certain sonic texture like a harsh sound for a storm versus a quiet rain,” he noted. “It’s helping speed up processes, but at the end of the day, it always comes back to the human curation of what’s right.”
While some who work in the editing space see AI as simply a tool to aid Hollywood’s artisans, the International Alliance of Theatrical Stage Employees, which has over 168,000 members from all sides of the entertainment industry, is more suspicious of its potential impact.
The union is launching a new commission which will engage in a comprehensive study of AI technologies, with a particular focus on how they might reshape the landscape of entertainment industry jobs under IATSE’s jurisdiction. It will also consider how contract provisions, legislation, and training programs can be adapted to ensure the fruits of increased productivity through AI are shared equitably among all stakeholders.
The group will be comprised of representatives from across IATSE’s jurisdiction, as well as subject matter experts from academia and the tech industry, and present a report of its findings and recommendations to be presented at the 2023 midsummer meeting of IATSE’s General Executive Board, held at the end of July.
“As AI continues to evolve and proliferate, it is critical that our union is at the forefront of understanding its impact on our members and industry,” union president Matthew Loeb said in a statement. “Just as when silent films became talkies and as the big screen went from black-and-white to full color, the IATSE Commission on Artificial Intelligence is part of our commitment to embracing new technologies. We will work to equip our members with the skills to navigate this technological advancement, and to ensure that the transition into this new era prioritizes the interests and well-being of our members and all entertainment workers.”
Keep reading WrapPRO for more from our AI series:
Read Part One: AI and the Rise of the Machines: Is Hollywood About to Be Overrun by Robots?
Read Part Two: How Hollywood’s Guilds Are Preparing for the Dangers – and Benefits – of AI
Read Part Three: This Story Was Not Written by a Robot: AI and the Future of News Media
Read Part Four: The Chatbot Stays in the Picture: How AI Could Transform Acting
Read Part Five: Chatbots Flip the Script: For Screenwriters, AI’s Evolution Brings New Tools and New Fears
Read Part Six: AI and the Law: It’s the ‘Wild, Wild West’ Out There
Lucas Manfredi
Lucas Manfredi is a TV Business reporter with TheWrap. He has a Bachelor of Science in Television-Radio from Ithaca College. He can be reached at lucas.manfredi@thewrap.com.
Umberto Gonzalez
Umberto has been covering the fanboy beat & breaking scoops for 20 years with numerous Hollywood trade, newspaper, & magazine mentions to his credit. Umberto has been profiled in such publications as The Washington Post, Variety and Grantland.