Toolchains vs. Monoliths: Apple's Illusion of Thinking

E72 TOOLCHAINS VS. MONOLITHS_ APPLE'S ILLUSION OF THINKING

Transcript

Dalton Anderson (00:01.292) Welcome to VentureStep podcast, where we discuss entrepreneurship, trends, and the occasional book review. Everyone's talking about Apple's new shiny UI, but their AI roadmap is suspiciously quiet. Not funny, haha, funny, weird. Today we're going to be discussing the recent AI paper that Apple came out with as of June. It's still June, yeah? Time's been flying by.

and it's titled the illusion of thinking and it's it's kind of a long long title but the illusion of thinking is a shortened one discussing that and the results of that model and it might be a little bit more smoke than fire and yeah it's overall an odd paper so wanted to review that and then talk about

Apple's new release or planned release of their liquid UI, which has been widely criticized and overall love Apple, but been disappointed with them as of late, unfortunately. So here it is, we're going to dive right into it. And I'm going to start off by doing a host intro. And then after that, we're going to read.

I'm going to read one paragraph from the paper, which is the conclusion, which I think is the most meaningful. OK, my name is Dalton Anderson. I'm your host. I've got a mix of background. I work in insurance. I've done data science. And in my free time, I like to run, build my side business, or read a good book.

All right, that's me. Okay, so let's transition over to the conclusion. So I'm to read this off here and hopefully everyone that's watching the video, bear with me. There's not a lot of stuff going on right now. I just moved and actually don't even have a desk for me to sit down. So when I work all day, I stand and there's a sweatshop out here right now. No desk, no way to mount my

Dalton Anderson (02:14.1) my laptop to anything. don't have a docking station, no monitors and just old school in it. I've got some shoe boxes and a dream. So just keep that in mind when you're like, what, why is this such a weird setup? Well, that's why. Okay. So this is the paper, the conclusion. So it says in this paper, we systematically examine frontier large reasoning models, LRMs through the lens of problem complexity using controllable

puzzle environments. Our findings reveal fundamental limitations in current models. Despite sophisticated self-reflection mechanisms, these models fail to develop generalizable reasoning capabilities beyond certain complexity thresholds. Gotta take a breath.

We identified three distinct reasoning regimes. Standard LLMs outperform LRMs at low complexity. LRMs excel at moderate complexity and both collapse at high complexity, particularly concerning as the counterintuitive reduction in reasoning effort as models approach critical complexity, suggesting an inherent compute scaling limit in LRMs.

Our detailed analysis of reasoning traces further expose complexity dependent reasoning patterns from inefficient overthinking on simpler problems to complete failure on complex ones. These insights challenge prevailing assumptions about LRM capabilities and suggest that current approaches may be encountering fundamental barriers to generalizable reasoning. Finally,

We have presented some surprising results in LRMs that have led to several open questions for future work. Most notably, we have observed their limitations in performing exact computation. For example, when we provided the solution algorithm for the Tower of Hano to the models, their performance on the puzzle did not improve. Moreover, investigating the first failure move of the models revealed surprising behaviors. For instance,

Dalton Anderson (04:27.544) They could form up to a hundred correct moves in the Tower of Hano, but failed to provide more than five correct moves in the river crossing puzzle. We believe our results can pave the way for future investigations into reasoning capabilities of these systems. So that was their conclusion of their paper. It was a 30 page paper of those 30 pages. It's only 11 of substance and then

there is about, you know.

Dalton Anderson (05:03.022) 19 pages of just references and acknowledgments.

So in the paper, they discuss a couple things. They talk about what the title is, the illusion of thinking. And that's a big overarching piece. And they're talking about generalization of reasoning or generalizable reasoning. Whereas I think it like there, you're fundamentally flawed on your thought process. And these people are way smarter than I am, but.

Hear me out here. So if you think about an analogy of someone trying to garden with a bulldozer and then they're like, like bulldozers aren't that good at, at gardening. I really need a shovel or some kind of rake or something like that. It's kind of the same gist where Apple is alluding to, Hey,

LRMs, which are large reasoning models and LLMs. LLMs are really good at simple problems. LRMs, large reasoning models are good at medium complexity problems. And then LRMs aren't very good at super complex problems. Okay. I see that thought. And I would take it to say, they're harping on LRMs not being able to solve simple problems as fast as

the simple models are. And they talk about the inefficiency of, of thought and these other things. But I want to touch on the first part where they're trying to put one model in a box. And I would think about it as tool chains versus monoliths. A monolith would just be this massive model that's good at everything. But you've seen that in architectures throughout these other technologies where that doesn't work.

Dalton Anderson (07:08.544) Like when have you seen one person be good at all sports like like a Barry Bonds or something like that or or whoever it is. You don't see those people very often like they're like once in every 60 years. And so. When have you seen somebody at work that's good at sales, good at. Good at product tech. Those people are founders, those people are CEOs. They don't happen very often. And the reason they don't is because.

those are difficult things to manage and they're rare and they're normally inefficient to house all that knowledge with one person. And so the same thing is true in this regard where, Hey, like you're trying to generalize a model and it's supposed to be good at simple problems. It's supposed to be good at medium complexity and large complexity. And they're like, well, it's not better than the simple model at simple things. And it's good. It's good in

it exceeds expectations at medium complexity, but for large complexity, seems like it just doesn't know what it's doing and it can't do it. So that being said, I would say that just at a high level that we don't have a model yet ready for large, very complex problems. And I think that people would agree with that, right? Like you can ask LLMs or LRMs things and they probably do a

pretty good job at medium complexity to low complexity. But when you ask it something super complicated, it's going to get choked up and it's, you might have to bring down the complexity and break it into parts and segment it and then reduce the amount of tokens that are needed to understand the problem.

Dalton Anderson (08:55.906) Very, very true. I think from my experience and people listening, they would think the same thing. And I just don't think what they're alluding to makes much sense. And I think it's counterproductive. And in my opinion, I think it's a cop out for their lack of innovation in this space and saying, there's fundamental issues with LRMs and LLMs. we...

we see this and we saw this and so that's why we're not too deep in AI right now.

Okay, sure. Like that's what that's how it seems to me. Like I see this company that's been on the forefront of innovation for years, my whole life. That's Apple has been that guy. And when you're that girl and you're outperforming everybody and then somebody, somebody new comes around and starts doing good things like you, like you point the finger and be like, Oh, well, you know that that's not scalable or

you know, that technology, the foundations aren't right. And it's it just seems like a cop out. Like it doesn't seem legitimate. Like I would feel better about it if like Google or Nvidia or OpenAI came out with this paper and said like, hey, like, you know, we did additional more additional research on this on this problem that we know is already a problem. Like what Apple is talking about and discussing. This isn't a ground

breaking theory. Like people know, I mean, I'm not an AI researcher. I'm just a podcaster. Like that's all I am. These people are way smarter than I am. And I even know it. And many people on the internet know that like LLMs and LRMs have fundamental issues. But each year it's getting less expensive, more efficient. The cost per tokens are decreasing year in, year out. And it's just like,

Dalton Anderson (10:55.832) The cost per token is decreasing so much, like within 18 months.

It's insane. So they're talking about how there's a lack of generalization, generalized. wow. I'm going to make someone that have somebody to generalize reasoning. And then they don't admit to, or they fail to bring up that the decline in token costs, the rising inference speed.

the growing developer access to these AI tools. And so it's still a groundbreaking technology where it hasn't been been around that long. mean, open AI has been at it for some time and Google has been at it for some time and Nvidia has been building infrastructure and Google has been building infrastructure and Amazon has as well and a little bit of Microsoft in there. So it's not something that didn't exist, but it's a level of usage of, of wide

adapted like, like this is no longer an innovator's technology. Like it's becoming, it's slowly transitioning from innovator to general public and people are using it all the time. And for that to happen, you've got to greatly reduce the costs. And I think that's true where the cost, the cost per token has substantially decreased. And an example would be that the cost per chat GPT 3.5 in 18 months decreased

280%.

Dalton Anderson (12:37.038) So I misspoke there. That was the inference, the inference level. So it's sped up by 280%. And then the actual reduction in costs have decreased quite a bit as well.

Let me pull that up because I messed up. It's not there.

Dalton Anderson (13:01.614) I'm getting all caught up.

Dalton Anderson (13:08.396) Okay, so the cost per token decrease at OpenAI from $36 per million to $4 a million. So 79 % decrease in one year. And then Stanford's AI index reports a 280x drop in chat GPT level costs since 2022.

Dalton Anderson (13:33.818) and A16Z is referencing and stating a 10x drop year over year. And so you see these things and you look around and you're like, wow, so there's all these incredible gains on efficiency. And a lot of the things that people were saying years ago was like, well, it's too expensive and it's not scalable. And

you know, how is this going to work long-term? And it requires too much power and all of the issues that people brought up, the naysayers. it's a lot, life is a lot better when you're optimistic and the optimists build stuff and those are the people that make the money. And the people that are very pessimistic about life and things, they don't do so well.

And that being said, that's what happened. Like everything dropped, the cost per inference or the cost per token dropped, the inference speed dropped or increased. But you know what I mean? Like the time it took to inference the tokens and output data to the user has substantially dropped. And they built, they built models purposely built for that, like Gemma or Mini. And so all of those things just

doesn't add up with what Apple is saying, where they're thinking monolithic, where everyone else is thinking purpose-built models that are not only purpose-built, but they're also multimodal, where one model doesn't serve all tasks. So within that model, they might have an image generating model, where a model that understands and encodes audio and a model that translates

whatever it may be. mean, they don't reveal every little nuance, but they do admit that their models are multimodal and not just a single model. And they use a mixture of experts as I talked about in previous research papers that we've right? Where I went over the meta paper, the open AI paper, they specifically emphasize the efficiency gains of mixture of experts. And that's what's important here is that Apple

Dalton Anderson (15:59.534) created this paper, The Illusion of Thinking. And as I mentioned, they're talking about

monolithic approach. But the architecture that companies are employing and the people that are fundamentally innovating in the space, they're not taking that direction. So why even think about that? The people that are at the forefront of innovation, they're not even going that route because they already tried it and it doesn't work. And so within these big models, are, they're multimodal as well.

So I don't know, the whole thing is quite confusing and I just, I just don't understand Apple. I really don't understand. Like I love Apple. They're awesome. Like the products that design I've got, I've got the M1 or M2 Mac. I mean, this thing's a beast. Like the cost, the cost of this Mac and the compute that it has, how good it is.

and just how uncompetitive other products are when you compare to other Mac, like MacBook Pros, especially M4, it's not even close. And so they make great products and they've consistently innovated for such a long time. But lately their releases with the iPhone, their lack of integration with these AI functions, they were the first people

in the AI space. Like Steve Jobs was amazed by Siri, like back in the day when Siri could like give you appointments and do stuff for you. Steve Jobs thought it was groundbreaking. He was like, what? This is insane. I've got to get my hands on. We need this. This is going to be a differentiator for us and we've got to lean in. This is what the customers are going to want this in the future.

Dalton Anderson (18:05.718) And so Siri was the first AI system, Like a legit, like you could ask questions, it was advanced, and there's others that exist, but it was nowhere near as good as Siri. And then Alexa came around and then Gemini, not Gemini, but Google as they called themselves.

But Siri was like at the forefront of all of this and they've got the data, they've got the users, they've got all the inferences. They have everything, right? Like they design their own chips in-house. They design their hardware.

They had an AI self-driving studio for the longest. They've got just a ton of money in cash. So I'm just scratching my head here. Like, what are you doing? What are you doing? Like, please do something, please. And yeah, I just.

I just struggle with it. I really like Apple, really like Google. I like all these companies. I want all the companies to push each other. that's, that's a, that's a general sense. if, if no one's pushing, if no one pushed Google to start releasing AI stuff, I bet they would, they haven't done it yet. And they would just house all the AI talent. But before OpenAI started pushing all this AI stuff and stockholders were getting concerned.

with Google's positioning with AI because for the longest in the marketplace, they're considered the AI group. Like this is the AI company. They were experts. And then open AI comes along and starts releasing all these models and Google had no counter. Bard was horrific. Their live demo cost billions in shareholder value on the stock price drop simply because Google got caught on its back foot and that woke Google up.

Dalton Anderson (20:08.096) And Google has just shipped an amazing amount of stuff in 24 months. Like out of any company that went from zero to something, it's Google and XAI. And both of those are fueled by rivalries where deep, deep, what is it? I want to say deep seek for some reason, but.

man, I'm blanking on the Google's AI research team. DeepMind, there we go. DeepMind is, before OpenAI housed, I think, like 85 plus percent of the AI researcher workforce. They had just an absolute domination on AI talent. And before...

Open AI, Google are the people that are innovating, but they're innovating in house. It wasn't yet ready for public. wasn't, it wasn't polished enough. There was concerns and this and that. And then open AI comes along and Google was like, well, we got to start, we got to start getting stuff out the door right now. Like people are concerned. And if we don't start doing that, then heads are rolled.

And heads did roll. People laughed. People got concerned. They didn't like how fast they were shipping stuff. But Google would have never done that if it wasn't OpenAI. And that's why I'm emphasizing Apple. You've got to push. You've got to push. All your friends are out there. They're all doing great things and you need to join them. You need to join them. You're part of that group and you've got to push them as well. And that's all I want. All I want is to have the best product ever.

And to do that, each company needs to push each other to the limit. For years and years and never stop. And that's and that's capitalism, baby. And. I think that's possible if they push, but if they don't, then that's fine. It is what it is. Eventually, people are going to emphasize AI so much. That there.

Dalton Anderson (22:26.126) product positioning is going to slowly decrease and it has been stagnating over the years. And then from there, people are going to switch to other products and then they are going to slowly get out the garden.

Dalton Anderson (22:48.166) And yeah, it's going to be a sad ordeal if that happens. I hope that Apple figures it out. But one recent thing that they've been focusing on is shipping. Is there liquid, I think liquid glass UI, which

I don't even know how to say it. Like it didn't look good. It didn't get, it didn't get very much praise. And that was like their hype for their WWDC.

They had no major AI vision. The stock dipped. And the biggest emphasis was like a UI revamp, which looked like Windows Visto back in the day. And you can't even see the UI. Like on a light background, all your apps and stuff are translucent. And so is like your notification center. So if you pull down your notification center, like your apps still bleed through and you can't even see what's going on. Like people were sending clips of the beta.

It's not publicly released yet and hopefully they fix it before they release it because people are not going to like that at all. And I was showing people and they're like, that's horrific. Like, please don't please don't make that a mandatory. Like, I don't want that. Please.

think that just shows a state of like the disconnect of like what they're deeming as important versus what is actually important. And that's a fundamental issue where leadership misunderstands what's critical at the time and then ships something like completely irrelevant, like a UI revamp. Okay. You can do that anytime. You could have done that 20 years from now, a year from now, five years from now.

Dalton Anderson (24:40.318) No one's anticipating a UI revamp. They're like, John, you see that new UI revamp? That was something else. Look at that. Look at that, John. I haven't seen a UI revamp like that since. nine. When they came out, the bibbidi bop cupcake. Like, like no one, like no one's really talking about it. People talk about that stuff when you mess up. Like people just see.

and consider Google and Apple to be like great at UI and software. And so that's the, that's the bar. Like you're not, you're not going to reinvent the bar. It's already so high. And so you can add new features to it or do whatever, but no one's going to be like, wow, the UI revamp. Like it's not, people care about AI right now. Like the market cares about AI.

Users care about it. And I'll say that there's there's been of some of my friends and I'm not like shilling for people to use Google or these other things. But if they ask me questions, I'll I'll answer them. And they love like simple features like the call blocking for scammers or transcribing my calls automatically when I request it. And then them sending the transcription to me. And then another thing they like is that

there's a AI chip within Google on the Google phone, the pixel, and people are running their little AIs within on device for like little things and more complex things are in the cloud obviously, but that gets people turning. They're like, wow, that's pretty, that's pretty neat. And then they like the AI stuff that you can do with photos and

People like to publish like weird stuff on, their Instagram or Tik TOK or whatever it may be. And Apple doesn't have much of any of that stuff. They have an integration that seems to be lackluster with open AI. last time I checked. And so they just don't have a good offering and other companies are expanding. Like Alexa is coming out with Alexa plus or something like that. It's in beta right now.

Dalton Anderson (27:04.174) Hopefully I can get my hands on access. I have to look into how to do that. I don't think I'm going to buy Alexa show just so I can see how it works, but that's supposed to be very conversational, advanced, and it's supposed to generate cool stuff. Like I think it could generate images and these other things and order stuff for you and handle complex tasks and do whatever. But Apple doesn't have that.

Yeah, it's not that I don't like Apple. I'm just disappointed. You know, I know Apple's meant for so much more than what they're doing. And I hope that.

they turn things around and maybe they listen to this random episode and they're like, you know what? This random guy on the internet, he's not so bad. We haven't we haven't been.

We haven't been performing at max output, but.

Dalton Anderson (28:16.172) This is an opportunity for others, right? Like why they stall, others can sprint. And so the longer that Apple stalls, maybe they do an acquisition. For how much money? have no idea because the acquisition needs to pay for everyone's salaries, their comp, then it needs to pay for their, their position in the market and their forecasted position in the market.

And as long as they could pay for the forecasted position of what they would be in the marketplace, then it would make sense. But that's going to be really expensive. There's there's only like a couple of players and of the ones that could be acquired. It's really like anthropic. Or the French company with DeepSeek, but there's not that many in the marketplace that are open to acquisition. And then.

If you do acquire them...

Like, what are you going to do? mean, it seems like your position at the company, at least, doesn't seem like it's emphasizing what you're buying. So what's the point? But yeah, in a general sense, takeaways is that modular AI wins. Tool sets versus monoliths, building systems, not Swiss Army knives, right?

Dalton Anderson (29:46.35) Cost curves are substantially decreasing and that allows for experimentation as you've been seeing by these new companies that are popping up and...

Dalton Anderson (30:01.996) Apple is showing a disconnected vision on what people want of them and what they are willing to commit to or have vision for.

Dalton Anderson (30:17.966) But hey, we all learn faster, slower than others. So give them a chance, right? But of course, I appreciate everybody listening to this episode. If you found it enjoyable, please like and subscribe. And if you want to leave a review, I'd love that as well. Recently got a nice review and I appreciate that. So.

Once again, listening to the next episode and I appreciate you listening in to this one and wherever you are in this world. Good morning, good afternoon, good evening. Thank you for listening and I hope you listen in next week. Thank you, goodbye.