Ubisoft, in partnership with AI company Convai, unveiled Project Neural Nexus, an advancement that uses AI to power real-time conversations with NPCs. Read on to learn more about the project and insights from its representatives.
Ubisoft's NEO NPC AI Technology Unveiled at GDC 2024
Simulating Real-Time Conversations with NPCs
During the kickoff of GDC 2024 yesterday, Ubisoft introduced Project Neural Nexus's NEO NPCs, a collaborative project that involves generative AI technology designed to simulate in-game conversations and interactions with non-playable characters (NPCs). The technology uses Behavior, a prerelease feature from Unity Muse, Unity’s AI-powered platform for real-time 3D and game development.
As discussed and demonstrated at the event, NEO NPCs use a large language model (LLM)* to instill such characters with unique backgrounds, personalities, and attitudes. Nvidia's Audio2Face technology is also used in the project to assist in enabling real-time animation of NEO NPCs.
- As defined, LLM is a type of AI algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate, and predict new content.
In its early stage of development, a playable demo at GDC showcased the capabilities of NEO NPCs. Participating reporters were engaged in three distinct scenarios, conversing directly with NEO NPCs and witnessing real-time responses tailored to their interactions. For instance, game scenarios involved players learning about a character named Bloom, sharing their own background, and obtaining information related to the game's objectives.
Another aspect of the NEO NPC technology is its integration of open-source voice recognition and AI engine, allowing NPCs to interpret and remember player dialogue. This functionality was said to enable dynamic interactions, with NPCs responding appropriately to player inquiries and emotions to shape the course of the game.
The demo culminated with an operation to infiltrate a building alongside another NPC named Iron. One reporter noted that while talking to Iron and asking her questions provided "more insight on the goals of the operation, it was more structured like a traditional dialogue tree: there were correct responses for the best outcome."
Ubisoft clarified that NEO NPCs are crafted by human writers, such as Narrative Director Virginie Mosser, who shape character backgrounds, dialogues, and traits. Project Senior Data Scientist Melanie Lopez Malet confirmed she works alongside the writers to ensure that the AI models accurately reflect the intended characterizations.
"The model’s task becomes: I must impersonate this character," explained Malet. " It is really important to us that it behaves like the character Virginie created. So, while we’re talking to it, we ask ourselves: ‘is this Lisa? Would Lisa say this?’ and if the answer is no, we need to go back and find out what happened within the model to make it stray from the vision Virginie had."
"Generative AI is a Tool. It Doesn't Create Games."
Ubisoft confirmed that its narrative team oversees the parameters for NEO NPCs to make sure dialogues align with the game's lore. Xavier Manzanares, producer on the NEO NPC project, spoke further on the philosophy behind the project that melds technology and game design.
"Something we keep in mind is every new tech that's on our table cannot create games by itself. It has to go in line with design and innovation as well. It's really important that GenAI is exactly the same thing," Manzanares said at the event. "GenAI is a tool, it is tech. It doesn't create games, it has to be connected to design and it has to be connected to a team that really wants to push something with that tech. The questions we always had within the team is, 'What's in for the creatives? What's in for the players?'"
Still A Long Way to Go Before It Can Be Implemented in a Game
The project is spearheaded by Ubisoft Paris, Montpellier, and Montreal studios, and comprises a team of approximately 30 individuals, including senior data scientist Melanie Lopez Malet and scientific director David Louapre. Ubisoft stated that NEO NPCs are still in the prototype phase and not slated for inclusion in upcoming games.
The project is described as a "creative experiment," between Ubisoft, Nvidia, and Inworld.ai. Malet emphasized that they are merely scratching the surface with the technology and that its implementation in actual games is not currently a priority. Ubisoft CEO Yves Guillemot drew parallels between this experimentation with generative AI and the company's previous ventures into emerging technologies like the Kinect and Wii.
At the event, CEO Guillemot addressed concerns regarding Ubisoft's foray into generative AI, noting that generative AI has long existed in gaming and is now seeing increased investment and usage across various sectors. "I think what you have to consider is all the new evolutions have to be considered and tested," the CEO said, according to news site VG247.
"Project NEO NPC is only a prototype, and there’s still a way to go before it can be implemented in a game. The end goal is to keep it as a flexible tool, as useful for smaller Ubisoft projects as it could be for AAA. The team are using GDC 2024 to present the project, but also as the checkpoint to take stock of industry feedback, and gather their learnings to date on what the technology means for them and their players in the future," stated Ubisoft on its website.
Source:
Ubisoft: How Ubisoft’s New Generative AI Prototype Changes the Narrative for NPCs
The Verge: Ubisoft let me actually speak with its new AI-powered video game NPCs
Eurogamer: Ubisoft unveils generative AI NEO NPCs
, and the spirit of Peter Molyneux's Milo lives on
VG247: Ubisoft CEO Yves Guillemot says generative AI will be “considered and tested”
