Jin Lei from Concave Fei Temple qubit | qbitai
NVIDIA's latest big move is here - universal embodied agents.
The news was revealed by Jim Fan, a senior scientist at Nvidia, who said:
Together with my old teammate Yuke Zhu, I will form a new research group within NVIDIA, Gear, Universal Embodied Agent Research. We believe that in the future, every moving machine will be autonomous, and robots and analog agents will be as ubiquitous as iPhones. We're building a foundational agent: an AI with universal capabilities that can learn how to act skillfully in many virtual and real worlds.
Jim Fan also stressed that 2024 will be the year of robots, gaming AI, and simulation.
And with the skyrocketing market value of Nvidia in the past two days, Jim Fan also added an X and said:
We have the funds to tackle the robot base model, the game base model, and generative simulation all at once. Our team is probably the richest embodied intelligence lab in the world.
Well, rich, very trenched.
Demonstration of relevant achievements.
When Jim Fan released this new career news, he also took stock of the relevant work that NVIDIA had done in embodied intelligence before.
For example, Eureka, which was named one of the "Top 10 NVIDIA Projects in 2023".
Using GPT-4 to generate reward functions, the teaching robot completes more than 30 complex tasks: for example, quickly turning a pen, opening drawers and cabinets, throwing and catching balls.
Train with GPU-accelerated physics simulations up to 1000x faster than real-time!
Another example is Voyager, putting GPT-4 into Minecraft
The speed at which the tech tree is lit in the game is 15 of the previous method3 times, and the unique items obtained at the same time are the previous 33 times, the exploration range is 23 times.
What's more, Voyager relies entirely on in-game graphics, with all operations and feedback via text and the game's J**Ascript API.
It is also the first LLM-driven agent to be proficient in Minecraft.
In addition, MineDojo won the best of Neurips 2022.
This study proposes an "embodied GPT-3" consisting of 3 agents that can perceive and act in an infinite world.
MineDojo is an open framework that turns Minecraft into an AGI research playground.
The team collected 100,000 YouTube wiki pages and Reddit posts to train Minecraft agents.
There are also studies like VIMA: the first multimodal LLM with a robotic arm, introducing "multimodal prompts" for robot learning.
For more inventory of relevant achievements, you can click the link at the end of the article.
The embodied intelligence that was set on fire by Jensen Huang.
In fact, it is not very surprising that Jim Fan team leader is engaged in universal embodied agents.
Back last year, Huang publicly expressed his views on the next generation of artificial intelligence:
This new type of AI is called embodied AI, which is intelligent systems that can understand, reason, and interact with the physical world.
Since last year, whether it is universities or industries, research related to embodied intelligence has emerged in an endless stream.
The most typical is the housework robot of Stanford University, which stunned many netizens.
So what results will the Jim Fan team bring this year, it is worth looking forward to.
Reference link: [1].