Isaac GR00T N1: The First Open Humanoid Robot Foundation Model

E68 ISAAC GR00T N1_ THE FIRST OPEN HUMANOID ROBOT FOUNDATION MODEL

Transcript

Dalton Anderson (00:01.42) Welcome to Venture Step podcast, where we discuss entrepreneurship, industry trends, and the occasional book review. I spoke about robots and Nvidia figure 01 and some other robots. I I also touched on Boston Dynamics, and then I also touched on Tesla's Optimus robot. And in that episode, also touched on the infrastructure that Nvidia was building. And the episode, think is called

AI Butler's and NVIDIA's Superbrain, which I don't know why I called it that. It was a horrible name when I looking back, but I spoke about that in March, I think March 19th, 2024, about a year ago, or just over, it's over a year, but you know what I mean. Like it's, it's pretty close.

And in that episode, talked about the different programs that Nvidia was building and how they all pieced together into this seamless training experience for the robots and being able to easily put on some VR headsets and some gloves, control the robot, teach the robot how to do something, take that 3D image or

I guess 3D render or you could render it because you have the 3D cameras on the robot and sensors. So then you take that 3D rendering and the colors, throw that omniverse, and then there you go from that one or two hour training experience with the robot in person. Now you've trained years on that one item in the virtual world, and then you can transfer that back to your many robots on your fleet.

Dalton Anderson (01:52.078) quite dense, quite interesting. But then last month, NVIDIA had their NVIDIA AI day, I'm pretty sure. And during that conference, they spoke about one thing that I didn't get to touch on, because there's been so much stuff coming out. Everyone wants to make their announcement. But NVIDIA announced their group,

Isaac in one and that was their robotics open source model that you can use Omniverse to train new applications for. But it already starts out. It already has like a lot of things that are built in kind of like if you did chat to ET, but then you wanted to do it for health insurance or health or whatever it may be manufacturing, like it knows manufacturing, but it might need a little bit, a little bit of help in certain areas.

And so, Nvidia built this foundational model for companies to train on it.

And so instead of me explaining it, I'm gonna share my screen and we're gonna watch a quick video of the actual explanation of the video. So let's do this, this right here. Share my screen.

Dalton Anderson (03:20.596) Audio as well, of course. By the way, if you can't see my screen, you'll be able to hear the audio. But I wanted to call out how cracked my side panel of video suggestions are. VC, Nvidia, Robotics, MIT. I mean, just nonstop. We got this lo-fi girl thing going on. So just wanted to call that one out.

I'm doing it for everybody. Just messing around. All right, so let's go to theater mode. Maybe zoom in a bit. Maybe like 175. That might be too big.

Dalton Anderson (06:22.798) It's pretty cool, right? Yeah, I thought so myself. I think it's...

It's something similar to when Google was first starting. The more people that use the internet, the better, right? The more people that are building robots, better for Nvidia, because Nvidia is one, the person that's offering the chips for the robots, especially for smaller companies that don't have the infrastructure to build their own chips. And the only companies that have the infrastructure to build their own chips would be companies that are foundries or are companies

that can afford the R &D costs and overhead of those kind of expenses. And the companies that can afford to do that are very, very minimal. And the companies that are interested in that kind of thing is pretty much just Google and Meta. And then anyone else building chips, they build chips because that's what they do. So, Nvidia is always gonna be building chips.

Intel's gonna be building chips, AMD's gonna be building chips, but they're not competing with Nvidia, but Google has their TPUs or tensor processing units, and then Meta is working on a chip in-house, but I don't think that chip is gonna be.

I mean, it's supposed to be for AI, I'm not too familiar with Metastrip. know that they're just spending a lot of money to develop.

Dalton Anderson (08:00.32) But as I was saying, if you're any kind of startup or if you're not Meta or Google or some just massive trillion dollar market cap company or worth hundreds of billions, then your easiest bet is to use Nvidia's hardware, their GPUs, their infrastructure to do real time tele-operating or tele-operating and training.

and then use Omniverse and Cosmos. And it didn't explain it in that video, but Omniverse is a physics-based, like real-world engine, and it's used to simulate the world, and it's supposed to be as close to a real-world simulation as it can get. So that's what Omniverse is supposed to be doing.

There's all sorts of crazy things like simulating global warming or different events or if you drop a rock, how a meteor hits, how big are the waves or whatever it may be. And it also has a practical application for things like training robots. And that's what Nvidia is using. I think it took them like seven years to build the engine. Like it took a long time.

and using Omniverse and Cosmos, you're able to train the robot and...

Dalton Anderson (09:34.422) It's just overall sick. I don't know. So to emphasize how sick it is, I'm going to share this one minute video of NVIDIA and ONE X collaboration. And basically in this collaboration, and I didn't know this until earlier today, but I'd watched the demo from ONE X about their, Neo robot and the Neo robot was doing household tasks in a home.

with another human in the home, which is unusual because a lot of these robot demos, they're in controlled environments. They're in a lab. They're in a warehouse. They're not in somebody's home for the most part. mean, there's some videos, but like they seem very controlled. I think Tesla has a video of a robot in a house, but it didn't seem like people like live there, live there. Whereas the X

I can't say XAI because that's a different company, but One X, One X and Nvidia did a partnership and their engineers like live together or work together for like a week. Do you this all squared away? overall, once you have that background, you can see, okay, wow, they did all this in one week. It's pretty cool. So I'm going to share my screen again. So share this tab instead. And I'll play this video. Let me go in theater mode.

Dalton Anderson (12:32.512) option.

Dalton Anderson (12:37.838) So instead of telling you that this open source foundation model built by NVIDIA is a big deal, including the teleoperating and the omniverse piece for the training, I wanted to show you with those two videos, obviously when 1X AI, that's gonna confuse me because 1X and X AI, they sound very close. 1X AI spent a lot of time

on the engineering aspect of building the robot and getting the right sensors and getting the product in the right space and having it operational with this just everything that goes on with having a battery operated robot that can move and is textile, has textile sensors, understands objects in a way with your touch and your sensors because you can have this foundational model but if you can't link everything all together then it's kind of worthless.

But for them to be able to do everything they said in a week with just a little bit of collaboration and hard work, I think it emphasizes how big of a deal that is, where instead of everyone building their own model, there's this already pre-built model that people can just attach onto and then they can add or change things out from the model and

that will dramatically reduce the resources required to get started and allows people to focus on the things that they need to, like specified tasks for the robot that it doesn't come out of the box from the foundational model. Whereas before, people were all building their own models and it doesn't make much sense. Everything is closed source. And from NVIDIA's perspective,

people using the model, having it being open source and integrating very well with Nvidia products. It's a win-win for Nvidia because if you open source it and then that's the preferred model for your hardware to run these AI chips on, then it just makes everything more seamless. And you know, the kid at home who's in high school or middle school that dreams about robots, they're going to use the open source model, the model that's easiest to use.

Dalton Anderson (15:04.426) all of that stuff. And then eventually when that person grows up, they'll be using that same company probably. So it just, it really helps out. I like, I like when I started learning programming, I was in the Google developers club and that's a club that's sponsored by Google. And then they give you like hundreds of dollars of Google credits.

You build projects, you pitch things, you go out and you try to help charities in these different aspects. And you do that with using Google products. And you're able to do that because school makes it pretty much free. They give you a substantial amount of credits, free classes. And if you do well, they'll fly you out to California and you get to present your project.

And then you also, I think you get an internship, something like that. Like they set you up for sure. But even if they, even if none of that stuff happened, you still get a lot of opportunity to learn with all their stuff. So just an example of like how this thing can trickle down and over like a 10 year period. But yeah, it's a game changer because it, it removes the barriers to entry by a lot and

It also...

It also allows people to tinker and try things out. once you put something in the hands of millions and millions of people, people are going to build cool stuff. I mean, it may be from someone that doesn't have this established background and wouldn't be able to interact with these things unless they were working at that company and working with their close source model. And so I'm a big believer in open source simply because it gives everyone an opportunity, but that doesn't mean that

Dalton Anderson (17:02.542) An opportunity doesn't mean that there will be results. A lot of results are by determination and grit. And you can see that's true with a lot of these high end universities or highly esteemed universities like Harvard, Stanford, MIT, and the list goes on. They have open sourced all of their courses, their lectures, they put them all on.

these websites and they're all free, they're on YouTube or wherever you watch them. could also take the exams, all the coursework.

and people just don't do it. People just won't do it. And it's just such a weird thing. Just because things are open source and things are available doesn't mean people are going to themselves. But kind of on a tangent there, but really what I'm trying to say is that it's really cool and it's cool because it allows builders to build and

things are gonna become even more interesting. And it just seems like every month there's something crazy that comes out or every couple of weeks right now. And it's just hard to keep up with, hard to.

just understand the velocity of which the world is moving at the moment in terms of technology. There's just so much going on and it's moving so fast that it's exciting. It's so exciting. There is no doubt that like five years in route, you won't even know what that looks like anymore. It's just the trajectories of

Dalton Anderson (18:56.512) of how things are moving are just so much faster than what were anticipated like three to four years ago or five years ago. And then here we are into the present day. And I'm like, I don't even know what six months looks like from now, because six months from now could be like there could be like five monumental announcements all at one time. So so much. It's very exciting. So I'm thrilled with it. And I think that it's going to open up a lot of opportunities for

other shops to get into robotics. Like you could be like an engineering shop, but you don't necessarily have all the infrastructure to build robots. But now that you have this open source model, really all you need to do is integrate all the sensors and you could have like a crappy robot. Doesn't mean your robot's any good, but you can at least try to build something. I don't know.

Dalton Anderson (19:56.362) Anyways, so I wanted to then talk about two more robots. We're on a robot talk right now. We're talking about robots today. That's what I want to talk about. So was really interested. I was writing a paper on, or I wouldn't say paper. I was writing an article on one of my podcast episodes because I want to get all my podcast episodes in an articles format.

And from those article formats, I'm going to have them all online on a blog. And that blog will have all of the podcast episodes and an ability to watch them and a link to the website. And I'm doing that simply because. Who knows how long we have to live, and I would really like to have everything structured if I have to leave early, I don't know. So and was definitely.

inspired by Paul Graham's essays and read them throughout college and so and now still. And so I also want to start writing like essays, like kind of like a Paul Graham style essay, not to imitate him, but use him for inspiration. So I want to do something cool like that. And I need a website to put all my stuff there. So while I was writing this article.

I had remembered about the figure 01 robot and I was like, Oh, like I wonder how that company is doing. And I also wanted to link the video that I talked about in the podcast onto the article and behold, they have an announcement with this Helix X or Helix AI. They now have another kind of cool announcement that happened a month ago that I'd missed.

So I want to show you that and it's a real time reasoning model. think prior their reasoning model was a partnership with OpenAI. I don't know the whole background and architecture of the Helix AI model that they're talking about, but in a general sense, it seems pretty cool. And then they came up with a new robot about like 10 months ago that looks very sleek, but it looks strong. Like it's arms.

Dalton Anderson (22:20.354) I mean, it's got some serious forms and they're doing this one demo that I saw online a couple of days ago where they're pushing the robot around and the guy's pushing it with like a steel pipe with like padding on the end of it.

and the robot kind of like deflects the push a couple times and the steel meets the arm and man like it just those those robots are buff those are some buff robots and then so I have that and then there is also

the Boston Dynamics Atlas robot, which I think is kind of freaky. They're talking about how it's like the most efficient robot and it's got all these cool features. It could do somersaults, run, and it does all sorts of like crazy stuff that the other robots can't do because Boston Dynamics has that little foundation because they've been doing robots for like 20 years. But man, is it freaky. Like they're doing this demo and I'll show it later on in the episode.

where

the robot is picking up this part for manufacturing sequencing. And basically the robot takes the skew of a part and puts it in the right bin. And so like all your engine covers will go here and maybe your red engine covers on the bottom row or wherever they are. And then instead of like turning around, like I'm on video, so, but like instead of turning around,

Dalton Anderson (23:59.502) It does a full 180 with its head and then just starts walking backwards and then does like a 180 with its torso and it's freaky. then in the beginning of the video, they have the robot laying down and they turn it on.

crazy stuff. And the robot, like, I don't know how to describe it. It like this freaky, like, exorcist thing and, like, comes up but, like, flips its arms around and its legs around, like 360 to get up. Freaky stuff. Like, it just looks so, so weird. And the 180 head turn and torso turn is a bit unsettling.

I would say, like I feel more comfortable with the figure robot because the figure robot seems chill. But the 180 turn was just like, my goodness. And there's a top comment on that YouTube video is like, imagine. Imagine you say like, hey, what are you doing to the robot? And the robot does a 180 turn and just says like, what? What did you say to me?

You'll know what I'm talking about if you look up the video or if you're watching right now. But so let's share my screen. So let's the first one we'll do is Helix.

And...

Dalton Anderson (25:34.764) Yeah, let's do Helix. All right, so let's do this.

Dalton Anderson (25:42.68) Helix. A lot of sharing today. I'm feeling very, very thoughtful.

Dalton Anderson (25:51.266) Mother's Day.

Dalton Anderson (26:46.894) So there's not a lot of narration on this, but basically there's two robots in this home lab that are putting away the groceries. And it takes a lot longer because they're robots, but in a general sense, they're doing their job well and they're collaborating with each other and they're handing each other items. And one person is putting stuff in the refrigerator and the other person is putting away things like in the dry pantry.

Dalton Anderson (27:14.242) and they're thinking, they're walking around and.

Dalton Anderson (27:20.948) They both know where stuff goes. So if one person has a dry item that's nearby, the other robot's already reaching out their hand to be given that item.

So it's pretty neat.

Dalton Anderson (27:41.774) So the full video is two minutes long. We're not gonna watch the whole thing, because it's just them fumbling around to get these items in the refrigerator. And so it was around like nine or so items.

And for you to put away nine items, wouldn't take you two minutes. But if this was something that was

not only one task, but the dishes, putting away the groceries, doing the floor, and then you just have somebody turning out work all day, every day, forever, then it does save a lot of time. So I think, you know, it could take them 30 minutes, but then it might take you 15. And then those extra 15 minutes compiled over a whole day of different 15 minute tasks, you'll get hours back.

And maybe you don't do cooking every day or maybe you don't do grocery shopping every day. Like I don't go shop that often, but when I do grocery shop, it takes me like an hour to put away everything because I just buy hundreds of dollars with the food, vacuum seal it up, put it in the freezer, label it, put all my stuff away and pack it away like a squirrel. And then I'm good to go and I don't have to do anything for a bit. And I just have to pick up vegetables. So.

But this is the new robot and if you can't see my screen because you're watching in the car or wherever you are, these arms, these arms are buff. there are some big arms and they're just like solid steel arms. they just, yeah, that demo where they're pushing the robot around and it was deflecting it. my goodness. Like people in the comments are like, my gosh.

Dalton Anderson (29:35.074) This arm, those arms are, those arms are serious. Like if you get hit with those arms on accident, like it's gonna be bad. It's gonna be real bad.

But overall, have all the robots. I think I appreciate figures robots the most on a design perspective, not on application, just design. Like the design of the robots I think are beautiful. And there's a video of them walking around and it's just, it looks sick. Like it literally looks so sick. Not so much with the Atlas robot. Like I'm not getting the same like, this is awesome vibes. I'm more like, my goodness.

This is Black Mirror. Whereas figure seems more chill. And the design of the robots are just beautiful. Like the aesthetic.

like the glossy gray with the matte blacks and the lights and the lights in the right areas. And when it's thinking that little logo kind of just flutters around in the middle of screen and just the way it looks, it just looks sleek. It looks great, honestly.

Maybe me too, should just fast forward.

Dalton Anderson (30:52.482) Like right there, I those look, those look like some cool robots if you ask me. Loving the design of the robots and how they're approaching the humanoid robot.

Dalton Anderson (31:12.11) Yeah, it looks great.

Really, really big, really big fan. So search sharing this time. So this is, we're transitioning over to the Boston Dynamics robot and see how this is more like I'm searching on Mars and this, it's less chill and relaxing.

Dalton Anderson (31:39.662) But this robot video is about, it's titled Run, or it's just walk, run, crawl, R-L, fun. Boston Dynamics, Atlas.

Dalton Anderson (32:44.16) Okay, so in that 50 second video, the Atlas robot did a whole bunch of crazy stuff where the robot did a cartwheel, it did a handstand, and it did break dancing moves. I'm not a break dancer or an avid dancer, but I don't know what the move's called, but you've seen it all the time in those videos where the person's spinning one leg around and

they're alternating their hands and the leg is going all the way around their body. You know what I'm talking about. That's the move. That's the dirty dance move, baby. my goodness. I'm just making myself a fool. All right. So that was the Atlas. And then I've got one last video here. Many videos today. Show this time.

Dalton Anderson (34:04.686) So we don't need to watch the whole video. I just wanted to watch and show you the 180 contortions that the robot does. So freaky where instead of turning around, it just twists its whole torso.

and its head and then slowly re-ordinates.

orientates, reorientates its feet while it's walking and then the feet slowly reposition, but the head and the torso are like twisted around and then it's walking kind of like sideways-ish and then eventually turns into like a forward motion again. And they said that they did that to become more efficient. Instead of turning around, it's more efficient to just spin around, which I...

get that part, but man, is it unsettling. For sure, very unsettling. yeah, that's my main gripe with that. It's just, I don't like it. It makes me not so comfortable, so.

Dalton Anderson (35:24.174) That was a lot of robots. There was a lot of videos about robots. I think the main gist of it is that things are, things are heating up and all these announcements were made like last month. And there's a couple other robots that I'm not going to touch on that are working and they're Boston dynamics robots. Figure 01 also they're figure. sorry. So figure 01 and then now figure a two figure a two.

is doing a or completed a successful robot test with BMW on their manufacturing plant, putting together bumpers. I think it was the rear bumper. And basically to do that, normally you would have a human that has a big piece and then has the two corner pieces. So the mold is three pieces. And once those pieces are kind of put together by a human, then it's sent off

to this autonomous manufacturing piece, but the autonomous manufacturing line still needs someone to put the pieces together. So in the testing, figure 02 completed the testing or completed the assembly of the rear bumper for BMW's autonomous manufacturing line.

And so they did that over a couple of weeks and BMW said it was successful and they're looking forward to.

Dalton Anderson (36:59.758) of reevaluating the, I think they said like reevaluating the autonomous robot capacity on their assembly line or something like that. So they're looking into it, but they're obviously they're sensitive about it because they're gonna have to either move people or fire them. So people get nervous about that. And especially in a highly unionized.

industry, I could see how that could cause issues. So I'm sure they're just walking around the subject, but they're if they're doing the testing for it, they're definitely open to it. That's how I'd feel like you're not going to fly robots to Germany, test it in your plant. And then like, yeah, I was just thinking about it. I wasn't actually going to do it like it was just a thought. But come on, you know me, I would never.

Okay, all right. Yeah, sure, sure, sure, sure, sure. No worries. But, so that's figure two and then Boston Dynamics has Spot, which is like the little robot dog that costs 75 grand. And there's quite a few companies that have built additional sensors or capabilities on top of the robot and they made companies because Boston Dynamics was like, the robot's not ready. There's stuff wrong with it. But now Boston Dynamics is

like providing like actual value with the robot. And so basically what their thing and the robot's applications are for are with like monitoring and understanding the sensors or manufacturing of like power plants or industrial sites that are quite big. And then

Maybe they're building it out and they don't necessarily have the sensors yet. So SPOT can walk around and monitor everything and kind of have verification. if like a sensor says that there's some anomaly going on, instead of sending a human out there to walk all the way over to it, it would be sent by or it'd be monitored by SPOT. So SPOT would get a notification. SPOT would turn on from its doghouse type of thing where it's charging.

Dalton Anderson (39:21.578) and then Spotlight walk over, look at it, and then all of that data is integrated into a product that they built called Orbit. And then Orbit will give you a fleet monitoring system that is verified by Spot, and that data is integrated and uploaded to the cloud, and allows people to manage things from a robot perspective, like as a dashboard, the KPIs.

and the data is created by the robot's walking and monitoring the place. And it's pretty cool, the application. I know my father works in construction and he does large construction projects. And recently, one of his jobs that he's in charge of overseeing,

They're partnering with this AI company that scans and creates or translates the blueprints for the company instead of having the different trades come out and translate the results. So for these large construction companies, they have this CAD software that 3D prints, or it's not printed, but

It's a 3D model of the building, all the wires, where the walls are going, which walls are structural, which ones aren't going to be. And then you could look at it in different phases of the project. But the problem is when you're translating the blueprints over onto the actual flooring of the building, it could take weeks for each trade to do it. And then one trade reliant on another trade and then vice versa.

So if one is delayed, then the other person's delayed, and then since that person's delayed, then the two people after it are delayed. And so it causes a mess. And there's this robot company that prints out or understands the CAD model, and then from there prints out all the blueprints for the job site. And the only thing that the user needs to do is

Dalton Anderson (41:44.15) move the robot up and down the floors. Like the robot can't climb floors, but it's pretty good. So robots are becoming more more useful and practical in everyday life. And then I had another robot that I wanted to talk about, I'm not gonna share a video or go too deep, but it's a robot called the Stretcher. Man, now I'm blanking on the name. What is it called?

it's called stretch. I wanted to call it the stretcher for some reason. It's called stretch. And stretch is another robot by Boston Dynamics that is partnered with Gap and some other companies, but basically stretch is a robot that is non-stationary and can move around, but once it's starting a stop process, it doesn't necessarily move around as much. And the general sense of stretch is it's a robot that has a...

kind of that manufacturing arm that you've seen a lot of videos of. And then on that manufacturing arm, instead of a grippy, like where it grabs things, it's got these like large suction cups. And so it just like sucks onto a package, which has gotta be pretty strong, because suction cups don't work that well on cardboard, but that's besides the point. So it has these suction cups and it sucks onto the box, and then it's got this

AI scanning thing that allows the robot to understand where the package needs to be orientated onto the conveyor belt for their scanning software to work. Aware of their scanning software.

sensors are positioned, it will position that box to be the correct way so it can integrate into the company system. But basically, instead of somebody unloading and loading different

Dalton Anderson (43:46.318) trucks containers, different truck containers. Stretch, I wanted to say stretcher, stretch. Stretch is doing that for the workers. And in the video that I watched, it was saying that workers that would unload the truck, like each truck load or like per day, the average box for gap was 30 pounds and.

the people that would unload the truck, it was a bare minimum requirement to do 3,000 boxes per day. So that's 90,000 pounds of moving. Like that over...

That over five days is almost half a million pounds. And then...

Dalton Anderson (44:44.014) Let's do 90,000.

Dalton Anderson (44:52.206) let's do like 40, because maybe you have weeks off. That's 18 million pounds. Just from your job, it's a lot. So basically what the guy was saying is that it doesn't allow people to work that job too long because of how brutal it is on your back and your body. And eventually your body just kind of just gives out and stretch allows people to still work the job but not have to work with

that level of physical strain and then also work weekends with less people and overall make everyone more productive and happier. And then instead of the workers, and you're upscaling the workers too, because instead of them moving boxes, they're controlling and robot certified to control robots. And then,

move the robots into the truckload, operate it, get it to turn on, and then it runs on itself. But you still have to have someone who understands the robot and then eventually probably moving into two robotics repairs or something like that. You could learn that. It just provides just different information that you can obtain or make use of instead of just moving boxes.

So that was what I wanted to talk about today. I was on a robotics just marathon today with everything I shared with you and the stuff that I watched throughout the week. I was like, I to talk about robots this week. I've got to, I've got to do that. Eventually I'm going to be.

More centered around probably finance here shortly. That's what I'm probably going get into in the next couple of episodes. I'm just going to be deep down in finance, but

Dalton Anderson (46:47.726) More to come on that. More closing out here and I appreciate everyone listening and wherever you are in this world, have a good day. Either it be morning, night or the afternoon. Thank you for tuning in and I'll talk to you next week. Goodbye.