Film Industry Rendered

　

    Yu has been engaged in AI for years. In early February, his company, Hangzhou-based ChatMind AI, launched Clash of New Year’s Greetings, a popular AI game in China that simulates the stressful conversations young people face about marriage, work and other sensitive topics from relatives while home for the Chinese New Year holidays. 

　

    In the realm of AI content generation tools, Yu believed that text-based generators were the most mature, with image generators close behind. Text-tovideo seemed years away. 

　

    Like many in the industry, Yu never imagined Sora would arrive so soon. With a few lines of text, Sora can generate a 60-second video clip. In an official sample video presented by OpenAI, a stylish woman walks down a bustling Tokyo street at night amid blazing neon lights. Even the minutest of details – from the neon’s reflection on the wet street to the woman’s freckles – are realistically and exquisitely rendered. 

　

    Sora’s arrival has stirred discussions among content creators. Some wonder whether it is possible to feed Sora a screenplay and get a finished film. Long before Sora’s debut, Chinese filmmakers – sci-fi directors in particular – had expressed concerns over this issue. Now many are asking what Sora means for the film industry’s future.

　

    Sora emerged as a game changer. After carefully studying the sample videos, Yu Linfeng said that Sora can generate longer, more visually stable and consistent videos than its predecessors. Sora likely leverages OpenAI’s ChatGPT-4 to expand on text prompts with more detail, while the company’s text-to-image generator Dall-E3 may also play a part, Yu said. 

　

    “Dall-E3 has already displayed its powerful capability to understand language and ChatGPT-4 can provide more specific verbal details. That’s to say, when people use Sora, they don’t need to enter too many descriptive words. Sora can easily read people’s minds. Its efficiency and convenience make it quite accessible to anybody,” Yu told NewsChina.

    Li Dongdong is the associate director of the Specialized Committee of Science, Film and Television with the China Science Writers Association. Li told NewsChina that over their past few seminars, members have discussed the latest developments in generative AI technology. “We could all foresee the coming of video generators. But Sora came too soon, almost two years earlier than we imagined,” Li said. 

　

    ChatGPT-4 has amassed an enormous text database from users across the world that provides the “crude oil” to Sora. “Thus, Sora’s intelligence would experience an exponential rise at an accelerated speed, instead of rising linearly,” Li said. 

　

    Chinese filmmakers have long been concerned about the aggressive rise of AI. “Sora’s coming did shock everyone. But in fact, our most anxious time has passed. It was during late summer and early autumn last year when Chinese filmmakers’ anxiety reached fever pitch,” Li told NewsChina, a period when China’s film industry was still reeling from the Covid-19 pandemic. 

　

    In summer 2023, the Committee held an outdoor seminar that gathered many leading film producers and creators to discuss the impact of AI on the film industry. The conference was held behind a lake, which reminded them of the Chinese idiom “fight with one’s back to the water,” describing a lastditch effort with no possible retreat. “We all joked that this time was ‘a fight to win or die,’” Li said. 

　

    “Every time Guo Fan met with me, he reminded me repeatedly that AI has progressed even further,” said Ning Hao, a director known for his dark comedy films, at the Pingyao International Film Festival in December 2023. 

　

    “People are most anxious when they don’t know what to do,” Guo Fan told NewsChina. In 2023, Guo spent most of his spare time focusing on AI. He and his team visited 19 AI research institutions and tech companies in China, such as Huawei and Xiaomi, and in the US including Apple, Google and Meta, as well as world-leading universities. 

　

    After his survey of the industry, Guo gained his bearings on the future of AI and filmmaking and formed relationships with some AI-related institutions. “At least I found direction and know where and how to start,” he said. 

　

    “Not only Guo Fan, but also many young filmmakers are willing to embrace the new technology. They are no longer that anxious. As the saying goes, ‘If you can’t beat ’em, join ’em,’” Li Dongdong said. 

　

    Yu Gang, an assistant director on The Wandering Earth movie series, is among them. “Visual effects is the most expensive part of a sci-fi film budget,” he said. 

　

    “It would be easy to shoot a street scene in Tokyo. I can go to Tokyo with a movie camera and film without turning to Sora at all. But what about a cyberpunk city? Or a film about Mars? Anything we can’t shoot with a camera is the costliest part of filmmaking,” Yu Gang told NewsChina. 

　

    Another potential benefit of Sora is visual effects involving animals with realistic fur and whiskers, which pose some of the greatest challenges for effects designers, Yu Gang said. Director Ning Hao told media that the visual effects for his 2019 sci-fi comedy Crazy Alien – which included monkeys and aliens – cost him over 200 million (US$27.6m), about half of his overall budget. 

　

    “Rendering non-human creatures has long been a big limitation for domestic visual effects companies. Only a few top-notch ones can do it, and they charge very high prices,” Yu Gang noted. 

　

    “It would be a great convenience if Sora could create animals with minute details and all I have to do is prepare the background and insert the animals Sora made into it,” he added. 

　

    Zhang identified many bugs and artefacts in Sora’s videos: a woman whose head turns 180 degrees, a candle flame that doesn’t flicker and a man with six fingers on one hand. Even in the most-acclaimed video of the woman in Tokyo, the woman’s legs suddenly swap position twice. 

　

    Zhang said Sora’s biggest problem is precision. “You type in an abstract description and all you can get is an abstract outcome,” he said. 

　

    Li Dongdong illustrated this point with an example. “When you give Sora a prompt like ‘a baby bear doing homework by his desk,’ it will generate a video. When you enter the same prompt again, Sora creates a totally different video with different bear and different desk. It’s like a mystery box – you never know what it will spit out.” 

　

    For content creators, such uncontrollable abstraction is unacceptable. “After all, Sora is based on an algorithm. The current algorithm cannot achieve precise expression. Sora’s videos might be fine for people to share on social platforms, but it would be disastrous to directly apply them to a serious and professional cinematic project due to the countless errors they could produce,” Zhang told our reporter. 

　

    When text-to-image generator Midjourney first launched in March 2022, many believed it would disrupt the graphic design industry. But that is still not the case. As Li Dongdong observed, many advertising agencies are using Midjourney, mostly as a brainstorming aid, but never to produce a final product. 

　

    “It is because AI does not have a sense of aesthetic,” said Ah Wen, a graphic designer who has a million followers on Sina Weibo. Ah Wen started using AI tools to create graphics and videos last year. He told NewsChina that design professionals liken AIgenerated content to “picking a card from a deck,” as the results are usually too random. Its main advantage is to instantly provide ideas and starting points. 

　

    He stressed that the quality of video work hinges on creativity and aesthetics, aspects that AI does not possess as of yet. 

　

    “Some professional aerial photography teams are likely to find themselves out of work,” Li Dongdong said. Among the officially released Sora videos, scenes shot from above are the most realistic and precise. Often featured in crime and suspense genres, aerial photography is relatively difficult to shoot. 

　

    “In [Chinese] crime films and TV shows, stories are commonly set in a fictional city as most cities are unwilling to be depicted as crime infested. As the city is fictional, aerial shots should not be geographically recognizable. If Sora can create aerial scenes of a fictional city, then we don’t need to use drones to take aerial shots anymore,” Yu Gang told NewsChina. 

　

    AI has conspicuously boosted the efficiency of filmmaking. “In the past, artists spent a lot of time designing props. But now, AI tools can instantly generate hundreds of designs, among which the director can choose one for prop designers to refine and put into production,” Yu Gang noted. 

　

    New AI tools also help creative teams communicate ideas. “You can turn almost all the ideas in your mind into images without any effort. When we approach investors with a new project in the future, we might not need to make PowerPoints anymore. We can use Sora-made videos to show our ideas,” Yu Gang added. 

　

    Zhang Xiaobei argues that those who actively embrace new technologies will not be easily replaced. “In high-technology fields, people tend to overestimate the impact of breakthroughs in the short term but overlook the long-term changes. AI is such a case. We won’t see how drastically it changes the world in months or in a few years. But in the long run, the world’s great change has already unfolded,” Zhang said. 

　

    “The film industry in particular has historically gone through five or six major technological changes. Every time, some people were left behind while others survived. We should strive to be survivors,” he added.