Video game artists: Can AI reproduce the screams of a man on fire? Video game artists want their work protected

For hours, motion capture Sensors placed on Noshir Dalal’s body tracked his movements as he threw overhead punches, one-handed attacks and one-handed attacks that would later appear in a video game. Eventually, he swung the sledgehammer he held in his hand so many times that he tore a tendon in his forearm. By the end of the day, he couldn’t open the door handle of his car.

The physical effort involved in this type of motion work and the hours dedicated to it are part of the reason he believes all video game artists should be equally protected from using unregulated software. artificial intelligence.

Video game artists Industry workers say they fear artificial intelligence could reduce or eliminate job opportunities because the technology could be used to replicate a performance in multiple movements without their consent. That’s a concern that led the Screen Actors Guild and the American Federation of Television and Radio Artists to go on strike in late July.

“If motion capture actors, video game actors in general, are only making the money they make that day… that can be a really slippery slope,” said Dalal, who played Bode Akuna in “Star Wars Jedi: Survivor.” “Instead of saying, ‘Hey, we’re going to bring you back’… they’re just not going to bring me back at all and they’re not going to tell me at all that they’re doing this. That’s why transparency and compensation is so important to us at AI Protections.”

Hollywood video game artists announced a work stoppage — the second in a decade — after more than 18 months of negotiations over a new interactive media deal with video game industry giants broke down over artificial intelligence protections. Union members have said they are not against AI. However, artists are concerned that the technology could provide studios with a means to displace them.

Dalal said he took it personally when he heard that video game companies were negotiating with SAG-AFTRA On a new contract, some work movement was to be considered “data” and not performance.

If players were to tally up the cutscenes they see in a game and compare them to the hours they spend controlling characters and interacting with non-player characters, they would find that they interact with the work of movers and stuntmen “a lot more than they interact with my work,” Dalal said. “They’re the ones that sell the world that these games live in, when you’re doing combos and doing crazy, cool moves using Force powers, or you’re playing Master Chief, or you’re Spider-Man swinging through the city,” he said.

Some actors argue that AI could deprive less experienced actors of the chance to land smaller supporting roles, such as non-lead characters, which they typically get their hands on before landing bigger jobs. Unchecked use of AI, actors say, could also lead to ethical issues if their voices or likenesses are used to create content they don’t agree with morally. That kind of ethical dilemma has recently arisen with game “mods,” in which fans alter and create new content. Last year, voice actors spoke out against such mods in the role-playing game “Skyrim,” which used AI to generate actors’ performances and cloned their voices for pornographic content.

In video game motion capture, actors wear special Lycra or neoprene suits with markers. In addition to more complex interactions, actors perform basic movements such as walking, running, or holding an object. Animators take images from those motion capture recordings and string them together to respond to what someone playing the game is doing.

“What AI allows game developers, or game studios, is to generate a lot of those animations automatically from previous footage,” said Brian Smith, an adjunct professor in the Department of Computer Science at Columbia University. “Studios no longer need to collect new footage for every game and every type of animation they want to create. They can also draw on their archive of previous animations.”

If a studio has motion capture from a previous game and wants to create a new character, he said, animators could use those stored recordings as training data.

“With generative AI, you can generate new data based on that pattern of previous data,” he said.

A spokeswoman for the video game makers, Audrey Cooling, said the studios offered “significant” artificial intelligence protections, but the SAG-AFTRA negotiating committee said the studios’ definition of who constitutes a “performer” is key to understanding the question of who would be protected.

“We’ve worked hard to put forward proposals with reasonable terms that protect the rights of artists while ensuring we can continue to use the most advanced technology to create a great gaming experience for fans,” Cooling said. “We’ve proposed terms that provide consent and fair compensation for anyone employed under the contract if an AI playback or digital replica of their performance is used in the games.”

The video game companies offered pay increases, he said, with an initial 7% increase in scale rates and a further 7.64% increase taking effect in November. That’s a 14.5% increase over the life of the contract. The studios had also agreed to increases in per diem, overnight travel pay and an increase in overtime rates and bonus payments, he added.

“Our goal is to reach an agreement with the union that will end this strike,” Cooling said.

A 2023 report on the global video game market by industry tracking firm Newzoo predicted that video games would begin to include more AI-generated voices, similar to the voices featured in Squanch Games’ “High on Life.” Game developers, the Amsterdam-based firm claimed, will use AI to produce unique voices, thereby avoiding the need to hire voice actors.

“Voice actors may see fewer opportunities in the future, especially as game developers use AI to reduce costs and development time,” the report notes, noting that “big AAA prestige games like ‘The Last of Us’ and ‘God of War’ use motion capture and voice acting in a similar way to Hollywood.”

Other games, like “Cyberpunk 2077,” feature celebrities in their cast.

Actor Ben Prendergast said the data points collected for motion capture don’t capture the “essence” of someone’s performance as an actor. The same is true, he said, of AI-generated voices that can’t offer the nuanced choices used in big scenes, or smaller, more strenuous endeavors like screaming for 20 seconds to depict a character’s death in a fire.

“The big problem is that someone, somewhere, has this massive amount of data and I no longer have control over it,” said Prendergast, who voices Fuse in the “Apex Legends” game. “Nefarious or not, someone can collect that data now and say, ‘We need a character who’s nine feet tall, who sounds like Ben Prendergast, and who can fight in this battle scene.’ And I have no idea that’s happening until the game comes out.”

Studios could “get away with it,” he said, unless SAG-AFTRA can secure the artificial intelligence protections they are fighting for.

“It reminds me a lot of sampling in the 80s, 90s and 2000s, when a lot of people were sampling classic songs,” he said. “This is an art. If you don’t protect the rights to their image, their voice, their body and their way of walking, then you can’t really protect humans from other activities.”