Imagine a futuristic world where the lines between gaming and artificial intelligence are blurred. A place where you no longer have to navigate through predefined dialog options, but can actually interact with characters. This vision is no longer a distant fantasy, it has been turned into reality by Nvidia.
At Computex 2023, Nvidia presented a fascinating demonstration of this collaborative gaming and AI world. In a cyberpunk ramen bar, players could press a button, say something in their own voice, and receive a response from a video game character. This revolutionary development could represent a major shift in the way we experience games.
The technology behind this impressive demonstration is a middleware suite called Nvidia ACE (Avatar Cloud Engine) for games. It’s designed to run both locally and in the cloud and includes a number of tools, including NeMo for delivering large language models (LLMs), Riva speech recognition, and text-to-speech.
The demo uses more than just these tools, however. It was created in Unreal Engine 5, making it visually stunning. So much so that the chatbot part seems almost unimpressive in comparison. Still, it’s important to point out that the AI’s ability to respond to natural language is an impressive advance.
Nvidia has stated that this technology could theoretically be scaled to multiple characters at once and even let NPCs talk to each other. However, it is not yet known if and when this feature will be tested in practice.
Some game developers are already starting to use parts of Nvidia’s ACE toolkit. For example, games like S.T.A.L.K.E.R. 2 Heart of Chernobyl and Fort Solis use “Omniverse Audio2Face,” which adapts the facial animation of a 3D character to the speech of the speaker.
We are at the dawn of a new era in gaming, where interaction with video game characters is taking on a whole new dimension. This future is exciting and full of possibilities. But as with any new technology, there is room for improvement. Perhaps in the next demo we’ll see even more compelling dialogue using more advanced tools like GPT-4 or Sudowrite. Only time will tell what Nvidia and other technology giants have in store next.
