Muse AI – a generative AI model designed for gameplay ideation. Officially named the World and Human Action Model (WHAM), Muse can generate game visuals, controller actions, or both. The research is published in Nature, highlighting a collaboration between Microsoft Research’s Game Intelligence and Teachable AI Experiences teams, along with Ninja Theory (Xbox Game Studios). In conjunction with the publication, Microsoft is open-sourcing the model weights, sample data, and the executable for the WHAM Demonstrator, a prototype with a visual interface for interacting with Muse.
The motivation behind Muse stemmed from the release of ChatGPT and the question of how similar transformer-based generative models could be applied to video games using large amounts of gameplay data. The team leveraged data from Ninja Theory’s Bleeding Edge, an online multiplayer game, for this purpose. Scaling model training presented significant challenges, requiring the team to utilize powerful GPU clusters. Early demonstrations showed the model “learning” over time, leading to further work exploring scaling laws.
The team involved users from the beginning, collaborating with game creatives to shape model capabilities and understand their needs. This multidisciplinary approach led to the development of the WHAM Demonstrator, which allows users to directly interact with the model. Key capabilities identified through this process include consistency (generating gameplay that respects game rules), diversity (generating a range of gameplay variants), and persistency (incorporating user modifications into generated sequences).
Features
- Muse AI is a new system from Microsoft that is able to understand gameplay and generate new gameplay sequences. It has been trained using data from games like Bleeding Edge
- Muse can generate consistent, diverse, and persistent gameplay. The AI is able to adapt to changes in a game world.
- Muse has the potential to revolutionize game development. This is done by rapid prototyping of levels and mechanics and creating dynamic, interactive game worlds.
- Muse has potential to revive classic games trapped on old hardware. By learning from recorded gameplay and rendering it on modern platforms.
- Muse is intended to be a tool to enhance human creativity. It is meant to automate tedious tasks rather than replacing developers.
- Microsoft has open-sourced Muse’s model weights, sample data, and a “Wham demonstrator.” This allows others to experiment with the tech.
Links
Official: muse.ai | The power of video made simple for you.
Announcement: Introducing Muse: Our first generative AI model designed for gameplay ideation – Microsoft Research
More Video news.
