In its paper, Nvidia discusses how it trained MineDojo to be able to play Minecraft. The company fed 730,000 Minecraft YouTube videos where it transcribed 2.2 billion words. Further training came from scraping 7,000 webpages from Minicraft Wiki, and 340,000 Reddit posts covering 6.6 million total comments. “While researchers have long trained autonomous AI agents in video-game environments such as Starcraft, Dota and Go, these agents are usually specialists in only a few tasks. So NVIDIA researchers turned to Minecraft, the world’s most popular game, to develop a scalable training framework for a generalist agent — one that can successfully execute a wide variety of open-ended tasks.”
Training
After learning from the data, MineDojo can execute commands and complete tasks. Nvidia uses a transformer model known as MineCLIP that can match video clips with specific activities within the game. “As a proof of concept, the researchers behind MineDojo created a large-scale foundation model, called MineCLIP, that learned to associate YouTube footage of Minecraft gameplay with the video’s transcript, in which the player typically narrates the onscreen action. Using MineCLIP, the team was able to train a reinforcement learning agent capable of performing several tasks in Minecraft without human intervention.” Tip of the day: Windows lets you use Cortana to translate sentences, words, or phrases, with the results read back to you automatically. This makes it particularly useful for group scenarios, but you can also type if you’re unsure about pronunciation. Cortana translation sports an impressive 40 languages and utilizes machine learning to provide natural results in many cases. Check our full guide to learn how to use Cortana for quick translations.
