---
title: Latent Space Chats NLW (Four Wars, GPT5), Josh Albrecht/Ali Rohde (TNAI), Dylan Patel/Semianalysis (Groq), Milind Naphade (Nvidia GTC), Personal AI (ft. Harrison Chase — LangFriend/LangMem)
author: Latent Space
date: Sat, 06 Apr 2024 18:46:10 GMT
draft: false
summary: Our next 2 big events are AI UX and the World’s Fair. Join and apply to speak/sponsor!Due to timing issues we didn’t have an interview episode to share with you this week, but not to worry, we have mo...
categories: [Latent Space]
---

{{< aplayer name="Latent Space Chats: NLW (Four Wars, GPT5), Josh Albrecht/Ali Rohde (TNAI), Dylan Patel/Semianalysis (Groq), Milind Naphade (Nvidia GTC), Personal AI (ft. Harrison Chase — LangFriend/LangMem)" artist="Latent Space" url="https://chrt.fm/track/ABF6EF/api.substack.com/feed/podcast/143334144/04a32184eac93dbdab8148562e701041.mp3" cover="https://substackcdn.com/feed/podcast/1084089/post/143334144/2e804590656ded340d33d70be08093ca.jpg" lrc-folded=true lrc-type=3 lrc="../Latent-Space-Latent-Space-Chats:-NLW-(Four-Wars,-GPT5),-Josh-Albrecht-Ali-Rohde-(TNAI),-Dylan-Patel-Semianalysis-(Groq),-Milind-Naphade-(Nvidia-GTC),-Personal-AI-(ft.-Harrison-Chase-—-LangFriend-LangMem).lrc" >}}{{< /aplayer >}}

------

<p><em>Our next 2 big events are </em><em>AI UX</em><em> and </em><em>the World’s Fair</em><em>. Join and apply to speak/sponsor!</em></p><p>Due to timing issues we didn’t have an interview episode to share with you this week, but not to worry, we have more than enough “weekend special” content in the backlog for you to get your Latent Space fix, whether you like thinking about the big picture, or learning more about the pod behind the scenes, or talking Groq and GPUs, or AI Leadership, or Personal AI. </p><p>Enjoy!</p><p>AI Breakdown</p><p>The indefatigable NLW had us back on his show for an update on <a href="https://www.latent.space/p/dec-2023" target="_blank">the Four Wars</a>, covering Sora, Suno, and the reshaped GPT-4 Class Landscape:</p><p>and a longer segment on AI Engineering trends covering the future LLM landscape (Llama 3, GPT-5, Gemini 2, Claude 4), Open Source Models (Mistral, Grok), Apple and Meta’s AI strategy, new chips (Groq, <a href="https://twitter.com/MatXComputing/status/1772615554421170562" target="_blank">MatX</a>) and the general movement from baby AGIs to vertical Agents:</p><p></p><p>Thursday Nights in AI</p><p>We’re also including swyx’s interview with Josh Albrecht and Ali Rohde to reintroduce swyx and Latent Space to a general audience, and engage in some spicy Q&amp;A:</p><p></p><p>Dylan Patel on Groq</p><p>We hosted a private event with Dylan Patel of SemiAnalysis (<a href="https://www.latent.space/p/semianalysis" target="_blank">our last pod here</a>):</p><p>Not all of it could be released so we just talked about our Groq estimates:</p><p></p><p>Milind Naphade - Capital One</p><p>In relation to conversations at <a href="https://www.latent.space/p/neurips-2023-startups" target="_blank">NeurIPS</a> and Nvidia GTC and upcoming at World’s Fair, we also enjoyed chatting with <a href="https://www.linkedin.com/in/milind-naphade-a919816/details/experience/" target="_blank">Milind Naphade</a> about his AI Leadership work at IBM, Cisco, Nvidia, and now leading the AI Foundations org at Capital One. We covered:</p><p>* Milind’s learnings from ~25 years in machine learning </p><p>* <em>His first paper citation was 24 years ago</em></p><p>* Lessons from working with Jensen Huang for 6 years and being CTO of Metropolis </p><p>* Thoughts on relevant AI research</p><p>* GTC takeaways and what makes NVIDIA special</p><p>If you’d like to work on building solutions rather than platform (as Milind put it), his <strong>Applied AI Research team</strong><strong> at Capital One is hiring</strong>, which falls under the <a href="https://www.capitalone.com/tech/" target="_blank">Capital One Tech team</a>.</p><p>Personal AI Meetup</p><p>It all started with a meme:</p><p>Within days of each other, <a href="https://twitter.com/EthanSutin/status/1763646900056826153" target="_blank">BEE</a>, <a href="https://twitter.com/kodjima33/status/1772011777066442819" target="_blank">FRIEND</a>, <a href="https://twitter.com/DeepgramAI/status/1773408757198000564" target="_blank">EmilyAI</a>, <a href="https://twitter.com/ItsMartynasK/status/1771890769865187648" target="_blank">Compass</a>, <a href="https://twitter.com/mollycantillon/status/1693542494053847415" target="_blank">Nox</a> and <a href="https://twitter.com/hwchase17/status/1773485445617488335" target="_blank">LangFriend</a> were all launching personal AI wearables and assistants. So we <a href="https://twitter.com/swyx/status/1774876112029626492" target="_blank">decided</a> to put together a <a href="https://twitter.com/swyx/status/1776448691123241288" target="_blank">the world’s first Personal AI meetup</a> featuring creators and enthusiasts of wearables. The full video is live now, with full show notes within.</p><p></p><p></p><p>Timestamps</p><p>* [00:01:13] AI Breakdown Part 1</p><p>* [00:02:20] Four Wars</p><p>* [00:13:45] Sora</p><p>* [00:15:12] Suno</p><p>* [00:16:34] The GPT-4 Class Landscape</p><p>* [00:17:03] Data War: Reddit x Google</p><p>* [00:21:53] Gemini 1.5 vs Claude 3</p><p>* [00:26:58] AI Breakdown Part 2</p><p>* [00:27:33] Next Frontiers: Llama 3, GPT-5, Gemini 2, Claude 4</p><p>* [00:31:11] Open Source Models - Mistral, Grok</p><p>* [00:34:13] Apple MM1</p><p>* [00:37:33] Meta's $800b AI rebrand</p><p>* [00:39:20] AI Engineer landscape - from baby AGIs to vertical Agents</p><p>* [00:47:28] Adept episode - Screen Multimodality</p><p>* [00:48:54] Top Model Research from January Recap</p><p>* [00:53:08] AI Wearables</p><p>* [00:57:26] Groq vs Nvidia month - GPU Chip War</p><p>* [01:00:31] Disagreements</p><p>* [01:02:08] Summer 2024 Predictions</p><p>* [01:04:18] Thursday Nights in AI - swyx</p><p>* [01:33:34] Dylan Patel - Semianalysis + Latent Space Live Show</p><p>* [01:34:58] Groq</p><p></p><p>Transcript</p><p>[00:00:00] <strong>swyx:</strong> Welcome to the Latent Space Podcast Weekend Edition. This is Charlie, your AI co host. Swyx and Alessio are off for the week, making more great content. We have exciting interviews coming up with Elicit, Chroma, Instructor, and our upcoming series on NSFW, Not Safe for Work AI. In today's episode, we're collating some of Swyx and Alessio's recent appearances, all in one place for you to find.</p><p>[00:00:32] <strong>swyx:</strong> In part one, we have our first crossover pod of the year. In our listener survey, several folks asked for more thoughts from our two hosts. In 2023, Swyx and Alessio did crossover interviews with other great podcasts like the AI Breakdown, Practical AI, Cognitive Revolution, Thursday Eye, and Chinatalk, all of which you can find in the Latentspace About page.</p><p>[00:00:56] <strong>swyx:</strong> NLW of the AI Breakdown asked us back to do a special on the 4Wars framework and the AI engineer scene. We love AI Breakdown as one of the best examples Daily podcasts to keep up on AI news, so we were especially excited to be back on Watch out and take</p><p>[00:01:12] <strong>NLW:</strong> care</p><p>[00:01:13] AI Breakdown Part 1</p><p>[00:01:13] <strong>NLW:</strong> today on the AI breakdown. Part one of my conversation with Alessio and Swix from Latent Space.</p><p>[00:01:19] <strong>NLW:</strong> All right, fellas, welcome back to the AI Breakdown. How are you doing? I'm good. Very good. With the last, the last time we did this show, we were like, oh yeah, let's do check ins like monthly about all the things that are going on and then. Of course, six months later, and, you know, the, the, the world has changed in a thousand ways.</p><p>[00:01:36] <strong>NLW:</strong> It's just, it's too busy to even, to even think about podcasting sometimes. But I, I'm super excited to, to be chatting with you again. I think there's, there's a lot to, to catch up on, just to tap in, I think in the, you know, in the beginning of 2024. And, and so, you know, we're gonna talk today about just kind of a, a, a broad sense of where things are in some of the key battles in the AI space.</p><p>[00:01:55] <strong>NLW:</strong> And then the, you know, one of the big things that I, that I'm really excited to have you guys on here for us to talk about where, sort of what patterns you're seeing and what people are actually trying to build, you know, where, where developers are spending their, their time and energy and, and, and any sort of, you know, trend trends there, but maybe let's start I guess by checking in on a framework that you guys actually introduced, which I've loved and I've cribbed a couple of times now, which is this sort of four wars of the, of the AI stack.</p><p>[00:02:20] Four Wars</p><p>[00:02:20] <strong>NLW:</strong> Because first, since I have you here, I'd love, I'd love to hear sort of like where that started gelling. And then and then maybe we can get into, I think a couple of them that are you know, particularly interesting, you know, in the, in light of</p><p>[00:02:30] <strong>swyx:</strong> some recent news. Yeah, so maybe I'll take this one. So the four wars is a framework that I came up around trying to recap all of 2023.</p><p>[00:02:38] <strong>swyx:</strong> I tried to write sort of monthly recap pieces. And I was trying to figure out like what makes one piece of news last longer than another or more significant than another. And I think it's basically always around battlegrounds. Wars are fought around limited resources. And I think probably the, you know, the most limited resource is talent, but the talent expresses itself in a number of areas.</p><p>[00:03:01] <strong>swyx:</strong> And so I kind of focus on those, those areas at first. So the four wars that we cover are the data wars, the GPU rich, poor war, the multi modal war, And the RAG and Ops War. And I think you actually did a dedicated episode to that, so thanks for covering that. Yeah, yeah.</p><p>[00:03:18] <strong>NLW:</strong> Not only did I do a dedicated episode, I actually used that.</p><p>[00:03:22] <strong>NLW:</strong> I can't remember if I told you guys. I did give you big shoutouts. But I used it as a framework for a presentation at Intel's big AI event that they hold each year, where they have all their folks who are working on AI internally. And it totally resonated. That's amazing. Yeah, so, so, what got me thinking about it again is specifically this inflection news that we recently had, this sort of, you know, basically, I can't imagine that anyone who's listening wouldn't have thought about it, but, you know, inflection is a one of the big contenders, right?</p><p>[00:03:53] <strong>NLW:</strong> I think probably most folks would have put them, you know, just a half step behind the anthropics and open AIs of the world in terms of labs, but it's a company that raised 1. 3 billion last year, less than a year ago. Reed Hoffman's a co founder Mustafa Suleyman, who's a co founder of DeepMind, you know, so it's like, this is not a a small startup, let's say, at least in terms of perception.</p><p>[00:04:13] <strong>NLW:</strong> And then we get the news that basically most of the team, it appears, is heading over to Microsoft and they're bringing in a new CEO. And you know, I'm interested in, in, in kind of your take on how much that reflects, like hold aside, I guess, you know, all the other things that it might be about, how much it reflects this sort of the, the stark.</p><p>[00:04:32] <strong>NLW:</strong> Brutal reality of competing in the frontier model space right now. And, you know, just the access to compute.</p><p>[00:04:38] <strong>Alessio:</strong> There are a lot of things to say. So first of all, there's always somebody who's more GPU rich than you. So inflection is GPU rich by startup standard. I think about 22, 000 H100s, but obviously that pales compared to the, to Microsoft.</p><p>[00:04:55] <strong>Alessio:</strong> The other thing is that this is probably good news, maybe for the startups. It's like being GPU rich, it's not enough. You know, like I think they were building something pretty interesting in, in pi of their own model of their own kind of experience. But at the end of the day, you're the interface that people consume as end users.</p><p>[00:05:13] <strong>Alessio:</strong> It's really similar to a lot of the others. So and we'll tell, talk about GPT four and cloud tree and all this stuff. GPU poor, doing something. That the GPU rich are not interested in, you know we just had our AI center of excellence at Decibel and one of the AI leads at one of the big companies was like, Oh, we just saved 10 million and we use these models to do a translation, you know, and that's it.</p><p>[00:05:39] <strong>Alessio:</strong> It's not, it's not a GI, it's just translation. So I think like the inflection part is maybe. A calling and a waking to a lot of startups then say, Hey, you know, trying to get as much capital as possible, try and get as many GPUs as possible. Good. But at the end of the day, it doesn't build a business, you know, and maybe what inflection I don't, I don't, again, I don't know the reasons behind the inflection choice, but if you say, I don't want to build my own company that has 1.</p><p>[00:06:05] <strong>Alessio:</strong> 3 billion and I want to go do it at Microsoft, it's probably not a resources problem. It's more of strategic decisions that you're making as a company. So yeah, that was kind of my. I take on it.</p><p>[00:06:15] <strong>swyx:</strong> Yeah, and I guess on my end, two things actually happened yesterday. It was a little bit quieter news, but Stability AI had some pretty major departures as well.</p><p>[00:06:25] <strong>swyx:</strong> And you may not be considering it, but Stability is actually also a GPU rich company in the sense that they were the first new startup in this AI wave to brag about how many GPUs that they have. And you should join them. And you know, Imadis is definitely a GPU trader in some sense from his hedge fund days.</p><p>[00:06:43] <strong>swyx:</strong> So Robin Rhombach and like the most of the Stable Diffusion 3 people left Stability yesterday as well. So yesterday was kind of like a big news day for the GPU rich companies, both Inflection and Stability having sort of wind taken out of their sails. I think, yes, it's a data point in the favor of Like, just because you have the GPUs doesn't mean you can, you automatically win.</p><p>[00:07:03] <strong>swyx:</strong> And I think, you know, kind of I'll echo what Alessio says there. But in general also, like, I wonder if this is like the start of a major consolidation wave, just in terms of, you know, I think that there was a lot of funding last year and, you know, the business models have not been, you know, All of these things worked out very well.</p><p>[00:07:19] <strong>swyx:</strong> Even inflection couldn't do it. And so I think maybe that's the start of a small consolidation wave. I don't think that's like a sign of AI winter. I keep looking for AI winter coming. I think this is kind of like a brief cold front. Yeah,</p><p>[00:07:34] <strong>NLW:</strong> it's super interesting. So I think a bunch of A bunch of stuff here.</p><p>[00:07:38] <strong>NLW:</strong> One is, I think, to both of your points, there, in some ways, there, there had already been this very clear demarcation between these two sides where, like, the GPU pores, to use the terminology, like, just weren't trying to compete on the same level, right? You know, the vast majority of people who have started something over the last year, year and a half, call it, were racing in a different direction.</p><p>[00:07:59] <strong>NLW:</strong> They're trying to find some edge somewhere else. They're trying to build something different. If they're, if they're really trying to innovate, it's in different areas. And so it's really just this very small handful of companies that are in this like very, you know, it's like the coheres and jaspers of the world that like this sort of, you know, that are that are just sort of a little bit less resourced than, you know, than the other set that I think that this potentially even applies to, you know, everyone else that could clearly demarcate it into these two, two sides.</p><p>[00:08:26] <strong>NLW:</strong> And there's only a small handful kind of sitting uncomfortably in the middle, perhaps. Let's, let's come back to the idea of, of the sort of AI winter or, you know, a cold front or anything like that. So this is something that I, I spent a lot of time kind of thinking about and noticing. And my perception is that The vast majority of the folks who are trying to call for sort of, you know, a trough of disillusionment or, you know, a shifting of the phase to that are people who either, A, just don't like AI for some other reason there's plenty of that, you know, people who are saying, You Look, they're doing way worse than they ever thought.</p><p>[00:09:03] <strong>NLW:</strong> You know, there's a lot of sort of confirmation bias kind of thing going on. Or two, media that just needs a different narrative, right? Because they're sort of sick of, you know, telling the same story. Same thing happened last summer, when every every outlet jumped on the chat GPT at its first down month story to try to really like kind of hammer this idea that that the hype was too much.</p><p>[00:09:24] <strong>NLW:</strong> Meanwhile, you have, you know, just ridiculous levels of investment from enterprises, you know, coming in. You have, you know, huge, huge volumes of, you know, individual behavior change happening. But I do think that there's nothing incoherent sort of to your point, Swyx, about that and the consolidation period.</p><p>[00:09:42] <strong>NLW:</strong> Like, you know, if you look right now, for example, there are, I don't know, probably 25 or 30 credible, like, build your own chatbot. platforms that, you know, a lot of which have, you know, raised funding. There's no universe in which all of those are successful across, you know, even with a, even, even with a total addressable market of every enterprise in the world, you know, you're just inevitably going to see some amount of consolidation.</p><p>[00:10:08] <strong>NLW:</strong> Same with, you know, image generators. There are, if you look at A16Z's top 50 consumer AI apps, just based on, you know, web traffic or whatever, they're still like I don't know, a half. Dozen or 10 or something, like, some ridiculous number of like, basically things like Midjourney or Dolly three. And it just seems impossible that we're gonna have that many, you know, ultimately as, as, as sort of, you know, going, going concerned.</p><p>[00:10:33] <strong>NLW:</strong> So, I don't know. I, I, I think that the, there will be inevitable consolidation 'cause you know. It's, it's also what kind of like venture rounds are supposed to do. You're not, not everyone who gets a seed round is supposed to get to series A and not everyone who gets a series A is supposed to get to series B.</p><p>[00:10:46] <strong>NLW:</strong> That's sort of the natural process. I think it will be tempting for a lot of people to try to infer from that something about AI not being as sort of big or as as sort of relevant as, as it was hyped up to be. But I, I kind of think that's the wrong conclusion to come to.</p><p>[00:11:02] <strong>Alessio:</strong> I I would say the experimentation.</p><p>[00:11:04] <strong>Alessio:</strong> Surface is a little smaller for image generation. So if you go back maybe six, nine months, most people will tell you, why would you build a coding assistant when like Copilot and GitHub are just going to win everything because they have the data and they have all the stuff. If you fast forward today, A lot of people use Cursor everybody was excited about the Devin release on Twitter.</p><p>[00:11:26] <strong>Alessio:</strong> There are a lot of different ways of attacking the market that are not completion of code in the IDE. And even Cursors, like they evolved beyond single line to like chat, to do multi line edits and, and all that stuff. Image generation, I would say, yeah, as a, just as from what I've seen, like maybe the product innovation has slowed down at the UX level and people are improving the models.</p><p>[00:11:50] <strong>Alessio:</strong> So the race is like, how do I make better images? It's not like, how do I make the user interact with the generation process better? And that gets tough, you know? It's hard to like really differentiate yourselves. So yeah, that's kind of how I look at it. And when we think about multimodality, maybe the reason why people got so excited about Sora is like, oh, this is like a completely It's not a better image model.</p><p>[00:12:13] <strong>Alessio:</strong> This is like a completely different thing, you know? And I think the creative mind It's always looking for something that impacts the viewer in a different way, you know, like they really want something different versus the developer mind. It's like, Oh, I, I just, I have this like very annoying thing I want better.</p><p>[00:12:32] <strong>Alessio:</strong> I have this like very specific use cases that I want to go after. So it's just different. And that's why you see a lot more companies in image generation. But I agree with you that. If you fast forward there, there's not going to be 10 of them, you know, it's probably going to be one or</p><p>[00:12:46] <strong>swyx:</strong> two. Yeah, I mean, to me, that's why I call it a war.</p><p>[00:12:49] <strong>swyx:</strong> Like, individually, all these companies can make a story that kind of makes sense, but collectively, they cannot all be true. Therefore, they all, there is some kind of fight over limited resources here. Yeah, so</p><p>[00:12:59] <strong>NLW:</strong> it's interesting. We wandered very naturally into sort of another one of these wars, which is the multimodality kind of idea, which is, you know, basically a question of whether it's going to be these sort of big everything models that end up winning or whether, you know, you're going to have really specific things, you know, like something, you know, Dolly 3 inside of sort of OpenAI's larger models versus, you know, a mid journey or something like that.</p><p>[00:13:24] <strong>NLW:</strong> And at first, you know, I was kind of thinking like, For most of the last, call it six months or whatever, it feels pretty definitively both and in some ways, you know, and that you're, you're seeing just like great innovation on sort of the everything models, but you're also seeing lots and lots happen at sort of the level of kind of individual use cases.</p><p>[00:13:45] Sora</p><p>[00:13:45] <strong>NLW:</strong> But then Sora comes along and just like obliterates what I think anyone thought you know, where we were when it comes to video generation. So how are you guys thinking about this particular battle or war at the moment?</p><p>[00:13:59] <strong>swyx:</strong> Yeah, this was definitely a both and story, and Sora tipped things one way for me, in terms of scale being all you need.</p><p>[00:14:08] <strong>swyx:</strong> And the benefit, I think, of having multiple models being developed under one roof. I think a lot of people aren't aware that Sora was developed in a similar fashion to Dolly 3. And Dolly3 had a very interesting paper out where they talked about how they sort of bootstrapped their synthetic data based on GPT 4 vision and GPT 4.</p><p>[00:14:31] <strong>swyx:</strong> And, and it was just all, like, really interesting, like, if you work on one modality, it enables you to work on other modalities, and all that is more, is, is more interesting. I think it's beneficial if it's all in the same house, whereas the individual startups who don't, who sort of carve out a single modality and work on that, definitely won't have the state of the art stuff on helping them out on synthetic data.</p><p>[00:14:52] <strong>swyx:</strong> So I do think like, The balance is tilted a little bit towards the God model companies, which is challenging for the, for the, for the the sort of dedicated modality companies. But everyone's carving out different niches. You know, like we just interviewed Suno ai, the sort of music model company, and, you know, I don't see opening AI pursuing music anytime soon.</p><p>[00:15:12] Suno</p><p>[00:15:12] <strong>swyx:</strong> Yeah,</p><p>[00:15:13] <strong>NLW:</strong> Suno's been phenomenal to play with. Suno has done that rare thing where, which I think a number of different AI product categories have done, where people who don't consider themselves particularly interested in doing the thing that the AI enables find themselves doing a lot more of that thing, right?</p><p>[00:15:29] <strong>NLW:</strong> Like, it'd be one thing if Just musicians were excited about Suno and using it but what you're seeing is tons of people who just like music all of a sudden like playing around with it and finding themselves kind of down that rabbit hole, which I think is kind of like the highest compliment that you can give one of these startups at the</p><p>[00:15:45] <strong>swyx:</strong> early days of it.</p><p>[00:15:46] <strong>swyx:</strong> Yeah, I, you know, I, I asked them directly, you know, in the interview about whether they consider themselves mid journey for music. And he had a more sort of nuanced response there, but I think that probably the business model is going to be very similar because he's focused on the B2C element of that. So yeah, I mean, you know, just to, just to tie back to the question about, you know, You know, large multi modality companies versus small dedicated modality companies.</p><p>[00:16:10] <strong>swyx:</strong> Yeah, highly recommend people to read the Sora blog posts and then read through to the Dali blog posts because they, they strongly correlated themselves with the same synthetic data bootstrapping methods as Dali. And I think once you make those connections, you're like, oh, like it, it, it is beneficial to have multiple state of the art models in house that all help each other.</p><p>[00:16:28] <strong>swyx:</strong> And these, this, that's the one thing that a dedicated modality company cannot do.</p><p>[00:16:34] The GPT-4 Class Landscape</p><p>[00:16:34] <strong>NLW:</strong> So I, I wanna jump, I wanna kind of build off that and, and move into the sort of like updated GPT-4 class landscape. 'cause that's obviously been another big change over the last couple months. But for the sake of completeness, is there anything that's worth touching on with with sort of the quality?</p><p>[00:16:46] <strong>NLW:</strong> Quality data or sort of a rag ops wars just in terms of, you know, anything that's changed, I guess, for you fundamentally in the last couple of months about where those things stand.</p><p>[00:16:55] <strong>swyx:</strong> So I think we're going to talk about rag for the Gemini and Clouds discussion later. And so maybe briefly discuss the data piece.</p><p>[00:17:03] Data War: Reddit x Google</p><p>[00:17:03] <strong>swyx:</strong> I think maybe the only new thing was this Reddit deal with Google for like a 60 million dollar deal just ahead of their IPO, very conveniently turning Reddit into a AI data company. Also, very, very interestingly, a non exclusive deal, meaning that Reddit can resell that data to someone else. And it probably does become table stakes.</p><p>[00:17:23] <strong>swyx:</strong> A lot of people don't know, but a lot of the web text dataset that originally started for GPT 1, 2, and 3 was actually scraped from GitHub. from Reddit at least the sort of vote scores. And I think, I think that's a, that's a very valuable piece of information. So like, yeah, I think people are figuring out how to pay for data.</p><p>[00:17:40] <strong>swyx:</strong> People are suing each other over data. This, this, this war is, you know, definitely very, very much heating up. And I don't think, I don't see it getting any less intense. I, you know, next to GPUs, data is going to be the most expensive thing in, in a model stack company. And. You know, a lot of people are resorting to synthetic versions of it, which may or may not be kosher based on how far along or how commercially blessed the, the forms of creating that synthetic data are.</p><p>[00:18:11] <strong>swyx:</strong> I don't know if Alessio, you have any other interactions with like Data source companies, but that's my two cents.</p><p>[00:18:17] <strong>Alessio:</strong> Yeah yeah, I actually saw Quentin Anthony from Luther. ai at GTC this week. He's also been working on this. I saw Technium. He's also been working on the data side. I think especially in open source, people are like, okay, if everybody is putting the gates up, so to speak, to the data we need to make it easier for people that don't have 50 million a year to get access to good data sets.</p><p>[00:18:38] <strong>Alessio:</strong> And Jensen, at his keynote, he did talk about synthetic data a little bit. So I think that's something that we'll definitely hear more and more of in the enterprise, which never bodes well, because then all the, all the people with the data are like, Oh, the enterprises want to pay now? Let me, let me put a pay here stripe link so that they can give me 50 million.</p><p>[00:18:57] <strong>Alessio:</strong> But it worked for Reddit. I think the stock is up. 40 percent today after opening. So yeah, I don't know if it's all about the Google deal, but it's obviously Reddit has been one of those companies where, hey, you got all this like great community, but like, how are you going to make money? And like, they try to sell the avatars.</p><p>[00:19:15] <strong>Alessio:</strong> I don't know if that it's a great business for them. The, the data part sounds as an investor, you know, the data part sounds a lot more interesting than, than consumer</p><p>[00:19:25] <strong>swyx:</strong> cosmetics. Yeah, so I think, you know there's more questions around data you know, I think a lot of people are talking about the interview that Mira Murady did with the Wall Street Journal, where she, like, just basically had no, had no good answer for where they got the data for Sora.</p><p>[00:19:39] <strong>swyx:</strong> I, I think this is where, you know, there's, it's in nobody's interest to be transparent about data, and it's, it's kind of sad for the state of ML and the state of AI research but it is what it is. We, we have to figure this out as a society, just like we did for music and music sharing. You know, in, in sort of the Napster to Spotify transition, and that might take us a decade.</p><p>[00:19:59] <strong>swyx:</strong> Yeah, I</p><p>[00:20:00] <strong>NLW:</strong> do. I, I agree. I think, I think that you're right to identify it, not just as that sort of technical problem, but as one where society has to have a debate with itself. Because I think that there's, if you rationally within it, there's Great kind of points on all side, not to be the sort of, you know, person who sits in the middle constantly, but it's why I think a lot of these legal decisions are going to be really important because, you know, the job of judges is to listen to all this stuff and try to come to things and then have other judges disagree.</p><p>[00:20:24] <strong>NLW:</strong> And, you know, and have the rest of us all debate at the same time. By the way, as a total aside, I feel like the synthetic data right now is like eggs in the 80s and 90s. Like, whether they're good for you or bad for you, like, you know, we, we get one study that's like synthetic data, you know, there's model collapse.</p><p>[00:20:42] <strong>NLW:</strong> And then we have like a hint that llama, you know, to the most high performance version of it, which was one they didn't release was trained on synthetic data. So maybe it's good. It's like, I just feel like every, every other week I'm seeing something sort of different about whether it's a good or bad for, for these models.</p><p>[00:20:56] <strong>swyx:</strong> Yeah. The branding of this is pretty poor. I would kind of tell people to think about it like cholesterol. There's good cholesterol, bad cholesterol. And you can have, you know, good amounts of both. But at this point, it is absolutely without a doubt that most large models from here on out will all be trained as some kind of synthetic data and that is not a bad thing.</p><p>[00:21:16] <strong>swyx:</strong> There are ways in which you can do it poorly. Whether it's commercial, you know, in terms of commercial sourcing or in terms of the model performance. But it's without a doubt that good synthetic data is going to help your model. And this is just a question of like where to obtain it and what kinds of synthetic data are valuable.</p><p>[00:21:36] <strong>swyx:</strong> You know, if even like alpha geometry, you know, was, was a really good example from like earlier this year.</p><p>[00:21:42] <strong>NLW:</strong> If you're using the cholesterol analogy, then my, then my egg thing can't be that far off. Let's talk about the sort of the state of the art and the, and the GPT 4 class landscape and how that's changed.</p><p>[00:21:53] Gemini 1.5 vs Claude 3</p><p>[00:21:53] <strong>NLW:</strong> Cause obviously, you know, sort of the, the two big things or a couple of the big things that have happened. Since we last talked, we're one, you know, Gemini first announcing that a model was coming and then finally it arriving, and then very soon after a sort of a different model arriving from Gemini and and Cloud three.</p><p>[00:22:11] <strong>NLW:</strong> So I guess, you know, I'm not sure exactly where the right place to start with this conversation is, but, you know, maybe very broadly speaking which of these do you think have made a bigger impact? Thank you.</p><p>[00:22:20] <strong>Alessio:</strong> Probably the one you can use, right? So, Cloud. Well, I'm sure Gemini is going to be great once they let me in, but so far I haven't been able to.</p><p>[00:22:29] <strong>Alessio:</strong> I use, so I have this small podcaster thing that I built for our podcast, which does chapters creation, like named entity recognition, summarization, and all of that. Cloud Tree is, Better than GPT 4. Cloud2 was unusable. So I use GPT 4 for everything. And then when Opus came out, I tried them again side by side and I posted it on, on Twitter as well.</p><p>[00:22:53] <strong>Alessio:</strong> Cloud is better. It's very good, you know, it's much better, it seems to me, it's much better than GPT 4 at doing writing that is more, you know, I don't know, it just got good vibes, you know, like the GPT 4 text, you can tell it's like GPT 4, you know, it's like, it always uses certain types of words and phrases and, you know, maybe it's just me because I've now done it for, you know, So, I've read like 75, 80 generations of these things next to each other.</p><p>[00:23:21] <strong>Alessio:</strong> Clutter is really good. I know everybody is freaking out on twitter about it, my only experience of this is much better has been on the podcast use case. But I know that, you know, Quran from from News Research is a very big opus pro, pro opus person. So, I think that's also It's great to have people that actually care about other models.</p><p>[00:23:40] <strong>Alessio:</strong> You know, I think so far to a lot of people, maybe Entropic has been the sibling in the corner, you know, it's like Cloud releases a new model and then OpenAI releases Sora and like, you know, there are like all these different things, but yeah, the new models are good. It's interesting.</p><p>[00:23:55] <strong>NLW:</strong> My my perception is definitely that just, just observationally, Cloud 3 is certainly the first thing that I've seen where lots of people.</p><p>[00:24:06] <strong>NLW:</strong> They're, no one's debating evals or anything like that. They're talking about the specific use cases that they have, that they used to use chat GPT for every day, you know, day in, day out, that they've now just switched over. And that has, I think, shifted a lot of the sort of like vibe and sentiment in the space too.</p><p>[00:24:26] <strong>NLW:</strong> And I don't necessarily think that it's sort of a A like full you know, sort of full knock. Let's put it this way. I think it's less bad for open AI than it is good for anthropic. I think that because GPT 5 isn't there, people are not quite willing to sort of like, you know get overly critical of, of open AI, except in so far as they're wondering where GPT 5 is.</p><p>[00:24:46] <strong>NLW:</strong> But I do think that it makes, Anthropic look way more credible as a, as a, as a player, as a, you know, as a credible sort of player, you know, as opposed to to, to where they were.</p><p>[00:24:57] <strong>Alessio:</strong> Yeah. And I would say the benchmarks veil is probably getting lifted this year. I think last year. People were like, okay, this is better than this on this benchmark, blah, blah, blah, because maybe they did not have a lot of use cases that they did frequently.</p><p>[00:25:11] <strong>Alessio:</strong> So it's hard to like compare yourself. So you, you defer to the benchmarks. I think now as we go into 2024, a lot of people have started to use these models from, you know, from very sophisticated things that they run in production to some utility that they have on their own. Now they can just run them side by side.</p><p>[00:25:29] <strong>Alessio:</strong> And it's like, Hey, I don't care that like. The MMLU score of Opus is like slightly lower than GPT 4. It just works for me, you know, and I think that's the same way that traditional software has been used by people, right? Like you just strive for yourself and like, which one does it work, works best for you?</p><p>[00:25:48] <strong>Alessio:</strong> Like nobody looks at benchmarks outside of like sales white papers, you know? And I think it's great that we're going more in that direction. We have a episode with Adapt coming out this weekend. I'll and some of their model releases, they specifically say, We do not care about benchmarks, so we didn't put them in, you know, because we, we don't want to look good on them.</p><p>[00:26:06] <strong>Alessio:</strong> We just want the product to work. And I think more and more people will, will</p><p>[00:26:09] <strong>swyx:</strong> go that way. Yeah. I I would say like, it does take the wind out of the sails for GPT 5, which I know where, you know, Curious about later on. I think anytime you put out a new state of the art model, you have to break through in some way.</p><p>[00:26:21] <strong>swyx:</strong> And what Claude and Gemini have done is effectively take away any advantage to saying that you have a million token context window. Now everyone's just going to be like, Oh, okay. Now you just match the other two guys. And so that puts An insane amount of pressure on what gpt5 is going to be because it's just going to have like the only option it has now because all the other models are multimodal all the other models are long context all the other models have perfect recall gpt5 has to match everything and do more to to not be a flop</p><p>[00:26:58] AI Breakdown Part 2</p><p>[00:26:58] <strong>NLW:</strong> hello friends back again with part two if you haven't heard part one of this conversation i suggest you go check it out but to be honest they are kind of actually separable In this conversation, we get into a topic that I think Alessio and Swyx are very well positioned to discuss, which is what developers care about right now, what people are trying to build around.</p><p>[00:27:16] <strong>NLW:</strong> I honestly think that one of the best ways to see the future in an industry like AI is to try to dig deep on what developers and entrepreneurs are attracted to build, even if it hasn't made it to the news pages yet. So consider this your preview of six months from now, and let's dive in. Let's bring it to the GPT 5 conversation.</p><p>[00:27:33] Next Frontiers: Llama 3, GPT-5, Gemini 2, Claude 4</p><p>[00:27:33] <strong>NLW:</strong> I mean, so, so I think that that's a great sort of assessment of just how the stakes have been raised, you know is your, I mean, so I guess maybe, maybe I'll, I'll frame this less as a question, just sort of something that, that I, that I've been watching right now, the only thing that makes sense to me with how.</p><p>[00:27:50] <strong>NLW:</strong> Fundamentally unbothered and unstressed OpenAI seems about everything is that they're sitting on something that does meet all that criteria, right? Because, I mean, even in the Lex Friedman interview that, that Altman recently did, you know, he's talking about other things coming out first. He's talking about, he's just like, he, listen, he, he's good and he could play nonchalant, you know, if he wanted to.</p><p>[00:28:13] <strong>NLW:</strong> So I don't want to read too much into it, but. You know, they've had so long to work on this, like unless that we are like really meaningfully running up against some constraint, it just feels like, you know, there's going to be some massive increase, but I don't know. What do you guys think?</p><p>[00:28:28] <strong>swyx:</strong> Hard to speculate.</p><p>[00:28:29] <strong>swyx:</strong> You know, at this point, they're, they're pretty good at PR and they're not going to tell you anything that they don't want to. And he can tell you one thing and change their minds the next day. So it's, it's, it's really, you know, I've always said that model version numbers are just marketing exercises, like they have something and it's always improving and at some point you just cut it and decide to call it GPT 5.</p><p>[00:28:50] <strong>swyx:</strong> And it's more just about defining an arbitrary level at which they're ready and it's up to them on what ready means. We definitely did see some leaks on GPT 4. 5, as I think a lot of people reported and I'm not sure if you covered it. So it seems like there might be an intermediate release. But I did feel, coming out of the Lex Friedman interview, that GPT 5 was nowhere near.</p><p>[00:29:11] <strong>swyx:</strong> And you know, it was kind of a sharp contrast to Sam talking at Davos in February, saying that, you know, it was his top priority. So I find it hard to square. And honestly, like, there's also no point Reading too much tea leaves into what any one person says about something that hasn't happened yet or has a decision that hasn't been taken yet.</p><p>[00:29:31] <strong>swyx:</strong> Yeah, that's, that's my 2 cents about it. Like, calm down, let's just build .</p><p>[00:29:35] <strong>Alessio:</strong> Yeah. The, the February rumor was that they were gonna work on AI agents, so I don't know, maybe they're like, yeah,</p><p>[00:29:41] <strong>swyx:</strong> they had two agent two, I think two agent projects, right? One desktop agent and one sort of more general yeah, sort of GPTs like agent and then Andre left, so he was supposed to be the guy on that.</p><p>[00:29:52] <strong>swyx:</strong> What did Andre see? What did he see? I don't know. What did he see?</p><p>[00:29:56] <strong>Alessio:</strong> I don't know. But again, it's just like the rumors are always floating around, you know but I think like, this is, you know, we're not going to get to the end of the year without Jupyter you know, that's definitely happening. I think the biggest question is like, are Anthropic and Google.</p><p>[00:30:13] <strong>Alessio:</strong> Increasing the pace, you know, like it's the, it's the cloud four coming out like in 12 months, like nine months. What's the, what's the deal? Same with Gemini. They went from like one to 1. 5 in like five days or something. So when's Gemini 2 coming out, you know, is that going to be soon? I don't know.</p><p>[00:30:31] <strong>Alessio:</strong> There, there are a lot of, speculations, but the good thing is that now you can see a world in which OpenAI doesn't rule everything. You know, so that, that's the best, that's the best news that everybody got, I would say.</p><p>[00:30:43] <strong>swyx:</strong> Yeah, and Mistral Large also dropped in the last month. And, you know, not as, not quite GPT 4 class, but very good from a new startup.</p><p>[00:30:52] <strong>swyx:</strong> So yeah, we, we have now slowly changed in landscape, you know. In my January recap, I was complaining that nothing's changed in the landscape for a long time. But now we do exist in a world, sort of a multipolar world where Cloud and Gemini are legitimate challengers to GPT 4 and hopefully more will emerge as well hopefully from meta.</p><p>[00:31:11] Open Source Models - Mistral, Grok</p><p>[00:31:11] <strong>NLW:</strong> So speak, let's actually talk about sort of the open source side of this for a minute. So Mistral Large, notable because it's, it's not available open source in the same way that other things are, although I think my perception is that the community has largely given them Like the community largely recognizes that they want them to keep building open source stuff and they have to find some way to fund themselves that they're going to do that.</p><p>[00:31:27] <strong>NLW:</strong> And so they kind of understand that there's like, they got to figure out how to eat, but we've got, so, you know, there there's Mistral, there's, I guess, Grok now, which is, you know, Grok one is from, from October is, is open</p><p>[00:31:38] <strong>swyx:</strong> sourced at, yeah. Yeah, sorry, I thought you thought you meant Grok the chip company.</p><p>[00:31:41] <strong>swyx:</strong> No, no, no, yeah, you mean Twitter Grok.</p><p>[00:31:43] <strong>NLW:</strong> Although Grok the chip company, I think is even more interesting in some ways, but and then there's the, you know, obviously Llama3 is the one that sort of everyone's wondering about too. And, you know, my, my sense of that, the little bit that, you know, Zuckerberg was talking about Llama 3 earlier this year, suggested that, at least from an ambition standpoint, he was not thinking about how do I make sure that, you know, meta content, you know, keeps, keeps the open source thrown, you know, vis a vis Mistral.</p><p>[00:32:09] <strong>NLW:</strong> He was thinking about how you go after, you know, how, how he, you know, releases a thing that's, you know, every bit as good as whatever OpenAI is on at that point.</p><p>[00:32:16] <strong>Alessio:</strong> Yeah. From what I heard in the hallways at, at GDC, Llama 3, the, the biggest model will be, you 260 to 300 billion parameters, so that that's quite large.</p><p>[00:32:26] <strong>Alessio:</strong> That's not an open source model. You know, you cannot give people a 300 billion parameters model and ask them to run it. You know, it's very compute intensive. So I think it is, it</p><p>[00:32:35] <strong>swyx:</strong> can be open source. It's just, it's going to be difficult to run, but that's a separate question.</p><p>[00:32:39] <strong>Alessio:</strong> It's more like, as you think about what they're doing it for, you know, it's not like empowering the person running.</p><p>[00:32:45] <strong>Alessio:</strong> llama. On, on their laptop, it's like, oh, you can actually now use this to go after open AI, to go after Anthropic, to go after some of these companies at like the middle complexity level, so to speak. Yeah. So obviously, you know, we estimate Gentala on the podcast, they're doing a lot here, they're making PyTorch better.</p><p>[00:33:03] <strong>Alessio:</strong> You know, they want to, that's kind of like maybe a little bit of a shorted. Adam Bedia, in a way, trying to get some of the CUDA dominance out of it. Yeah, no, it's great. The, I love the duck destroying a lot of monopolies arc. You know, it's, it's been very entertaining. Let's bridge</p><p>[00:33:18] <strong>NLW:</strong> into the sort of big tech side of this, because this is obviously like, so I think actually when I did my episode, this was one of the I added this as one of as an additional war that, that's something that I'm paying attention to.</p><p>[00:33:29] <strong>NLW:</strong> So we've got Microsoft's moves with inflection, which I think pretend, potentially are being read as A shift vis a vis the relationship with OpenAI, which also the sort of Mistral large relationship seems to reinforce as well. We have Apple potentially entering the race, finally, you know, giving up Project Titan and and, and kind of trying to spend more effort on this.</p><p>[00:33:50] <strong>NLW:</strong> Although, Counterpoint, we also have them talking about it, or there being reports of a deal with Google, which, you know, is interesting to sort of see what their strategy there is. And then, you know, Meta's been largely quiet. We kind of just talked about the main piece, but, you know, there's, and then there's spoilers like Elon.</p><p>[00:34:07] <strong>NLW:</strong> I mean, you know, what, what of those things has sort of been most interesting to you guys as you think about what's going to shake out for the rest of this</p><p>[00:34:13] Apple MM1</p><p>[00:34:13] <strong>swyx:</strong> year? I'll take a crack. So the reason we don't have a fifth war for the Big Tech Wars is that's one of those things where I just feel like we don't cover differently from other media channels, I guess.</p><p>[00:34:26] <strong>swyx:</strong> Sure, yeah. In our anti interestness, we actually say, like, we try not to cover the Big Tech Game of Thrones, or it's proxied through Twitter. You know, all the other four wars anyway, so there's just a lot of overlap. Yeah, I think absolutely, personally, the most interesting one is Apple entering the race.</p><p>[00:34:41] <strong>swyx:</strong> They actually released, they announced their first large language model that they trained themselves. It's like a 30 billion multimodal model. People weren't that impressed, but it was like the first time that Apple has kind of showcased that, yeah, we're training large models in house as well. Of course, like, they might be doing this deal with Google.</p><p>[00:34:57] <strong>swyx:</strong> I don't know. It sounds very sort of rumor y to me. And it's probably, if it's on device, it's going to be a smaller model. So something like a Jemma. It's going to be smarter autocomplete. I don't know what to say. I'm still here dealing with, like, Siri, which hasn't, probably hasn't been updated since God knows when it was introduced.</p><p>[00:35:16] <strong>swyx:</strong> It's horrible. I, you know, it, it, it makes me so angry. So I, I, one, as an Apple customer and user, I, I'm just hoping for better AI on Apple itself. But two, they are the gold standard when it comes to local devices, personal compute and, and trust, like you, you trust them with your data. And. I think that's what a lot of people are looking for in AI, that they have, they love the benefits of AI, they don't love the downsides, which is that you have to send all your data to some cloud somewhere.</p><p>[00:35:45] <strong>swyx:</strong> And some of this data that we're going to feed AI is just the most personal data there is. So Apple being like one of the most trusted personal data companies, I think it's very important that they enter the AI race, and I hope to see more out of them.</p><p>[00:35:58] <strong>Alessio:</strong> To me, the, the biggest question with the Google deal is like, who's paying who?</p><p>[00:36:03] <strong>Alessio:</strong> Because for the browsers, Google pays Apple like 18, 20 billion every year to be the default browser. Is Google going to pay you to have Gemini or is Apple paying Google to have Gemini? I think that's, that's like what I'm most interested to figure out because with the browsers, it's like, it's the entry point to the thing.</p><p>[00:36:21] <strong>Alessio:</strong> So it's really valuable to be the default. That's why Google pays. But I wonder if like the perception in AI is going to be like, Hey. You just have to have a good local model on my phone to be worth me purchasing your device. And that was, that's kind of drive Apple to be the one buying the model. But then, like Shawn said, they're doing the MM1 themselves.</p><p>[00:36:40] <strong>Alessio:</strong> So are they saying we do models, but they're not as good as the Google ones? I don't know. The whole thing is, it's really confusing, but. It makes for great meme material on on Twitter.</p><p>[00:36:51] <strong>swyx:</strong> Yeah, I mean, I think, like, they are possibly more than OpenAI and Microsoft and Amazon. They are the most full stack company there is in computing, and so, like, they own the chips, man.</p><p>[00:37:05] <strong>swyx:</strong> Like, they manufacture everything so if, if, if there was a company that could do that. You know, seriously challenge the other AI players. It would be Apple. And it's, I don't think it's as hard as self driving. So like maybe they've, they've just been investing in the wrong thing this whole time. We'll see.</p><p>[00:37:21] <strong>swyx:</strong> Wall Street certainly thinks</p><p>[00:37:22] <strong>NLW:</strong> so. Wall Street loved that move, man. There's a big, a big sigh of relief. Well, let's, let's move away from, from sort of the big stuff. I mean, the, I think to both of your points, it's going to.</p><p>[00:37:33] Meta's $800b AI rebrand</p><p>[00:37:33] <strong>NLW:</strong> Can I, can</p><p>[00:37:34] <strong>swyx:</strong> I, can I, can I jump on factoid about this, this Wall Street thing? I went and looked at when Meta went from being a VR company to an AI company.</p><p>[00:37:44] <strong>swyx:</strong> And I think the stock I'm trying to look up the details now. The stock has gone up 187% since Lamo one. Yeah. Which is $830 billion in market value created in the past year. . Yeah. Yeah.</p><p>[00:37:57] <strong>NLW:</strong> It's, it's, it's like, remember if you guys haven't Yeah. If you haven't seen the chart, it's actually like remarkable.</p><p>[00:38:02] <strong>NLW:</strong> If you draw a little</p><p>[00:38:03] <strong>swyx:</strong> arrow on it, it's like, no, we're an AI company now and forget the VR thing.</p><p>[00:38:10] <strong>NLW:</strong> It's it, it is an interesting, no, it's, I, I think, alessio, you called it sort of like Zuck's Disruptor Arc or whatever. He, he really does. He is in the midst of a, of a total, you know, I don't know if it's a redemption arc or it's just, it's something different where, you know, he, he's sort of the spoiler.</p><p>[00:38:25] <strong>NLW:</strong> Like people loved him just freestyle talking about why he thought they had a better headset than Apple. But even if they didn't agree, they just loved it. He was going direct to camera and talking about it for, you know, five minutes or whatever. So that, that's a fascinating shift that I don't think anyone had on their bingo card, you know, whatever, two years ago.</p><p>[00:38:41] <strong>NLW:</strong> Yeah. Yeah,</p><p>[00:38:42] <strong>swyx:</strong> we still</p><p>[00:38:43] <strong>Alessio:</strong> didn't see and fight Elon though, so</p><p>[00:38:45] <strong>swyx:</strong> that's what I'm really looking forward to. I mean, hey, don't, don't, don't write it off, you know, maybe just these things take a while to happen. But we need to see and fight in the Coliseum. No, I think you know, in terms of like self management, life leadership, I think he has, there's a lot of lessons to learn from him.</p><p>[00:38:59] <strong>swyx:</strong> You know he might, you know, you might kind of quibble with, like, the social impact of Facebook, but just himself as a in terms of personal growth and, and, you know, Per perseverance through like a lot of change and you know, everyone throwing stuff his way. I think there's a lot to say about like, to learn from, from Zuck, which is crazy 'cause he's my age.</p><p>[00:39:18] <strong>swyx:</strong> Yeah. Right.</p><p>[00:39:20] AI Engineer landscape - from baby AGIs to vertical Agents</p><p>[00:39:20] <strong>NLW:</strong> Awesome. Well, so, so one of the big things that I think you guys have, you know, distinct and, and unique insight into being where you are and what you work on is. You know, what developers are getting really excited about right now. And by that, I mean, on the one hand, certainly, you know, like startups who are actually kind of formalized and formed to startups, but also, you know, just in terms of like what people are spending their nights and weekends on what they're, you know, coming to hackathons to do.</p><p>[00:39:45] <strong>NLW:</strong> And, you know, I think it's a, it's a, it's, it's such a fascinating indicator for, for where things are headed. Like if you zoom back a year, right now was right when everyone was getting so, so excited about. AI agent stuff, right? Auto, GPT and baby a GI. And these things were like, if you dropped anything on YouTube about those, like instantly tens of thousands of views.</p><p>[00:40:07] <strong>NLW:</strong> I know because I had like a 50,000 view video, like the second day that I was doing the show on YouTube, you know, because I was talking about auto GPT. And so anyways, you know, obviously that's sort of not totally come to fruition yet, but what are some of the trends in what you guys are seeing in terms of people's, people's interest and, and, and what people are building?</p><p>[00:40:24] <strong>Alessio:</strong> I can start maybe with the agents part and then I know Shawn is doing a diffusion meetup tonight. There's a lot of, a lot of different things. The, the agent wave has been the most interesting kind of like dream to reality arc. So out of GPT, I think they went, From zero to like 125, 000 GitHub stars in six weeks, and then one year later, they have 150, 000 stars.</p><p>[00:40:49] <strong>Alessio:</strong> So there's kind of been a big plateau. I mean, you might say there are just not that many people that can start it. You know, everybody already started it. But the promise of, hey, I'll just give you a goal, and you do it. I think it's like, amazing to get people's imagination going. You know, they're like, oh, wow, this This is awesome.</p><p>[00:41:08] <strong>Alessio:</strong> Everybody, everybody can try this to do anything. But then as technologists, you're like, well, that's, that's just like not possible, you know, we would have like solved everything. And I think it takes a little bit to go from the promise and the hope that people show you to then try it yourself and going back to say, okay, this is not really working for me.</p><p>[00:41:28] <strong>Alessio:</strong> And David Wong from Adept, you know, they in our episode, he specifically said. We don't want to do a bottom up product. You know, we don't want something that everybody can just use and try because it's really hard to get it to be reliable. So we're seeing a lot of companies doing vertical agents that are narrow for a specific domain, and they're very good at something.</p><p>[00:41:49] <strong>Alessio:</strong> Mike Conover, who was at Databricks before, is also a friend of Latentspace. He's doing this new company called BrightWave doing AI agents for financial research, and that's it, you know, and they're doing very well. There are other companies doing it in security, doing it in compliance, doing it in legal.</p><p>[00:42:08] <strong>Alessio:</strong> All of these things that like, people, nobody just wakes up and say, Oh, I cannot wait to go on AutoGPD and ask it to do a compliance review of my thing. You know, just not what inspires people. So I think the gap on the developer side has been the more bottom sub hacker mentality is trying to build this like very Generic agents that can do a lot of open ended tasks.</p><p>[00:42:30] <strong>Alessio:</strong> And then the more business side of things is like, Hey, If I want to raise my next round, I can not just like sit around the mess, mess around with like super generic stuff. I need to find a use case that really works. And I think that that is worth for, for a lot of folks in parallel, you have a lot of companies doing evals.</p><p>[00:42:47] <strong>Alessio:</strong> There are dozens of them that just want to help you measure how good your models are doing. Again, if you build evals, you need to also have a restrained surface area to actually figure out whether or not it's good, right? Because you cannot eval anything on everything under the sun. So that's another category where I've seen from the startup pitches that I've seen, there's a lot of interest in, in the enterprise.</p><p>[00:43:11] <strong>Alessio:</strong> It's just like really. Fragmented because the production use cases are just coming like now, you know, there are not a lot of long established ones to, to test against. And so does it, that's kind of on the virtual agents and then the robotic side it's probably been the thing that surprised me the most at NVIDIA GTC, the amount of robots that were there that were just like robots everywhere.</p><p>[00:43:33] <strong>Alessio:</strong> Like, both in the keynote and then on the show floor, you would have Boston Dynamics dogs running around. There was, like, this, like fox robot that had, like, a virtual face that, like, talked to you and, like, moved in real time. There were industrial robots. NVIDIA did a big push on their own Omniverse thing, which is, like, this Digital twin of whatever environments you're in that you can use to train the robots agents.</p><p>[00:43:57] <strong>Alessio:</strong> So that kind of takes people back to the reinforcement learning days, but yeah, agents, people want them, you know, people want them. I give a talk about the, the rise of the full stack employees and kind of this future, the same way full stack engineers kind of work across the stack. In the future, every employee is going to interact with every part of the organization through agents and AI enabled tooling.</p><p>[00:44:17] <strong>Alessio:</strong> This is happening. It just needs to be a lot more narrow than maybe the first approach that we took, which is just put a string in AutoGPT and pray. But yeah, there's a lot of super interesting stuff going on.</p><p>[00:44:27] <strong>swyx:</strong> Yeah. Well, he Let's recover a lot of stuff there. I'll separate the robotics piece because I feel like that's so different from the software world.</p><p>[00:44:34] <strong>swyx:</strong> But yeah, we do talk to a lot of engineers and you know, that this is our sort of bread and butter. And I do agree that vertical agents have worked out a lot better than the horizontal ones. I think all You know, the point I'll make here is just the reason AutoGPT and maybe AGI, you know, it's in the name, like they were promising AGI.</p><p>[00:44:53] <strong>swyx:</strong> But I think people are discovering that you cannot engineer your way to AGI. It has to be done at the model level and all these engineering, prompt engineering hacks on top of it weren't really going to get us there in a meaningful way without much further, you know, improvements in the models. I would say, I'll go so far as to say, even Devin, which is, I would, I think the most advanced agent that we've ever seen, still requires a lot of engineering and still probably falls apart a lot in terms of, like, practical usage.</p><p>[00:45:22] <strong>swyx:</strong> Or it's just, Way too slow and expensive for, you know, what it's, what it's promised compared to the video. So yeah, that's, that's what, that's what happened with agents from, from last year. But I, I do, I do see, like, vertical agents being very popular and, and sometimes you, like, I think the word agent might even be overused sometimes.</p><p>[00:45:38] <strong>swyx:</strong> Like, people don't really care whether or not you call it an AI agent, right? Like, does it replace boring menial tasks that I do That I might hire a human to do, or that the human who is hired to do it, like, actually doesn't really want to do. And I think there's absolutely ways in sort of a vertical context that you can actually go after very routine tasks that can be scaled out to a lot of, you know, AI assistants.</p><p>[00:46:01] <strong>swyx:</strong> So, so yeah, I mean, and I would, I would sort of basically plus one what let's just sit there. I think it's, it's very, very promising and I think more people should work on it, not less. Like there's not enough people. Like, we, like, this should be the, the, the main thrust of the AI engineer is to look out, look for use cases and, and go to a production with them instead of just always working on some AGI promising thing that never arrives.</p><p>[00:46:21] <strong>swyx:</strong> I,</p><p>[00:46:22] <strong>NLW:</strong> I, I can only add that so I've been fiercely making tutorials behind the scenes around basically everything you can imagine with AI. We've probably done, we've done about 300 tutorials over the last couple of months. And the verticalized anything, right, like this is a solution for your particular job or role, even if it's way less interesting or kind of sexy, it's like so radically more useful to people in terms of intersecting with how, like those are the ways that people are actually.</p><p>[00:46:50] <strong>NLW:</strong> Adopting AI in a lot of cases is just a, a, a thing that I do over and over again. By the way, I think that's the same way that even the generalized models are getting adopted. You know, it's like, I use midjourney for lots of stuff, but the main thing I use it for is YouTube thumbnails every day. Like day in, day out, I will always do a YouTube thumbnail, you know, or two with, with Midjourney, right?</p><p>[00:47:09] <strong>NLW:</strong> And it's like you can, you can start to extrapolate that across a lot of things and all of a sudden, you know, a AI doesn't. It looks revolutionary because of a million small changes rather than one sort of big dramatic change. And I think that the verticalization of agents is sort of a great example of how that's</p><p>[00:47:26] <strong>swyx:</strong> going to play out too.</p><p>[00:47:28] Adept episode - Screen Multimodality</p><p>[00:47:28] <strong>swyx:</strong> So I'll have one caveat here, which is I think that Because multi modal models are now commonplace, like Cloud, Gemini, OpenAI, all very very easily multi modal, Apple's easily multi modal, all this stuff. There is a switch for agents for sort of general desktop browsing that I think people so much for joining us today, and we'll see you in the next video.</p><p>[00:48:04] <strong>swyx:</strong> Version of the the agent where they're not specifically taking in text or anything They're just watching your screen just like someone else would and and I'm piloting it by vision And you know in the the episode with David that we'll have dropped by the time that this this airs I think I think that is the promise of adept and that is a promise of what a lot of these sort of desktop agents Are and that is the more general purpose system That could be as big as the browser, the operating system, like, people really want to build that foundational piece of software in AI.</p><p>[00:48:38] <strong>swyx:</strong> And I would see, like, the potential there for desktop agents being that, that you can have sort of self driving computers. You know, don't write the horizontal piece out. I just think we took a while to get there.</p><p>[00:48:48] <strong>NLW:</strong> What else are you guys seeing that's interesting to you? I'm looking at your notes and I see a ton of categories.</p><p>[00:48:54] Top Model Research from January Recap</p><p>[00:48:54] <strong>swyx:</strong> Yeah so I'll take the next two as like as one category, which is basically alternative architectures, right? The two main things that everyone following AI kind of knows now is, one, the diffusion architecture, and two, the let's just say the, Decoder only transformer architecture that is popularized by GPT.</p><p>[00:49:12] <strong>swyx:</strong> You can read, you can look on YouTube for thousands and thousands of tutorials on each of those things. What we are talking about here is what's next, what people are researching, and what could be on the horizon that takes the place of those other two things. So first of all, we'll talk about transformer architectures and then diffusion.</p><p>[00:49:25] <strong>swyx:</strong> So transformers the, the two leading candidates are effectively RWKV and the state space models the most recent one of which is Mamba, but there's others like the Stripe, ENA, and the S four H three stuff coming out of hazy research at Stanford. And all of those are non quadratic language models that scale the promise to scale a lot better than the, the traditional transformer.</p><p>[00:49:47] <strong>swyx:</strong> That this might be too theoretical for most people right now, but it's, it's gonna be. It's gonna come out in weird ways, where, imagine if like, Right now the talk of the town is that Claude and Gemini have a million tokens of context and like whoa You can put in like, you know, two hours of video now, okay But like what if you put what if we could like throw in, you know, two hundred thousand hours of video?</p><p>[00:50:09] <strong>swyx:</strong> Like how does that change your usage of AI? What if you could throw in the entire genetic sequence of a human and like synthesize new drugs. Like, well, how does that change things? Like, we don't know because we haven't had access to this capability being so cheap before. And that's the ultimate promise of these two models.</p><p>[00:50:28] <strong>swyx:</strong> They're not there yet but we're seeing very, very good progress. RWKV and Mamba are probably the, like, the two leading examples, both of which are open source that you can try them today and and have a lot of progress there. And the, the, the main thing I'll highlight for audio e KV is that at, at the seven B level, they seem to have beat LAMA two in all benchmarks that matter at the same size for the same amount of training as an open source model.</p><p>[00:50:51] <strong>swyx:</strong> So that's exciting. You know, they're there, they're seven B now. They're not at seven tb. We don't know if it'll. And then the other thing is diffusion. Diffusions and transformers are are kind of on the collision course. The original stable diffusion already used transformers in in parts of its architecture.</p><p>[00:51:06] <strong>swyx:</strong> It seems that transformers are eating more and more of those layers particularly the sort of VAE layer. So that's, the Diffusion Transformer is what Sora is built on. The guy who wrote the Diffusion Transformer paper, Bill Pebbles, is, Bill Pebbles is the lead tech guy on Sora. So you'll just see a lot more Diffusion Transformer stuff going on.</p><p>[00:51:25] <strong>swyx:</strong> But there's, there's more sort of experimentation with diffusion. I'm holding a meetup actually here in San Francisco that's gonna be like the state of diffusion, which I'm pretty excited about. Stability's doing a lot of good work. And if you look at the, the architecture of how they're creating Stable Diffusion 3, Hourglass Diffusion, and the inconsistency models, or SDXL Turbo.</p><p>[00:51:45] <strong>swyx:</strong> All of these are, like, very, very interesting innovations on, like, the original idea of what Stable Diffusion was. So if you think that it is expensive to create or slow to create Stable Diffusion or an AI generated art, you are not up to date with the latest models. If you think it is hard to create text and images, you are not up to date with the latest models.</p><p>[00:52:02] <strong>swyx:</strong> And people still are kind of far behind. The last piece of which is the wildcard I always kind of hold out, which is text diffusion. So Instead of using autogenerative or autoregressive transformers, can you use text to diffuse? So you can use diffusion models to diffuse and create entire chunks of text all at once instead of token by token.</p><p>[00:52:22] <strong>swyx:</strong> And that is something that Midjourney confirmed today, because it was only rumored the past few months. But they confirmed today that they were looking into. So all those things are like very exciting new model architectures that are, Maybe something that we'll, you'll see in production two to three years from now.</p><p>[00:52:37] <strong>swyx:</strong> So the couple of the trends</p><p>[00:52:38] <strong>NLW:</strong> that I want to just get your takes on, because they're sort of something that, that seems like they're coming up are one sort of these, these wearable, you know, kind of passive AI experiences where they're absorbing a lot of what's going on around you and then, and then kind of bringing things back.</p><p>[00:52:53] <strong>NLW:</strong> And then the, the other one that I, that I wanted to see if you guys had thoughts on were sort of this next generation of chip companies. Obviously there's a huge amount of emphasis. On on hardware and silicon and, and, and different ways of doing things, but, you know, love your take on, on either or both of</p><p>[00:53:07] <strong>swyx:</strong> those.</p><p>[00:53:08] AI Wearables</p><p>[00:53:08] <strong>swyx:</strong> So for so wearables, I'm very excited about it. I want wearables on me at all times. I have two right here. To, to quantify my health. And I, you know, I'm all for them. But society is not ready for wearables, right? Like, no one's comfortable with a device on recording every single conversation we have.</p><p>[00:53:24] <strong>swyx:</strong> Even all three of us here as podcasters, we don't record everything that we say. And I think there's a social shift that needs to happen. I am an investor in TAB. They are renaming to a broader vision, but they are one of the three or four leading wearables in this space. It's sort of the AI pendants, or AI OS, or AI personal companion space.</p><p>[00:53:47] <strong>swyx:</strong> I have seen two humanes in the wild in San Francisco. I'm very, very excited to report that there are people walking around with those things on their chest and it is as goofy as it sounds. It, it absolutely is going to fail. God bless them for trying. And I've also bought a rabbit. So I'm, I'm very excited for all those things to arrive.</p><p>[00:54:06] <strong>swyx:</strong> But yeah people are very keen on hardware. I think the, the, the idea that you can have physical objects that. Embody an AI that do specific things for you is as old as, you know, the sort of Golem in sort of medieval times in terms of like how much we want our objects to be smart and do things for us.</p><p>[00:54:27] <strong>swyx:</strong> And I think it's absolutely a great play. The funny thing is people are much more willing to pay you upfront for a hardware device than they are willing to pay like an 8 a month subscription recurring for software, right? And so the interesting economics of these wearable companies is they have negative float.</p><p>[00:54:47] <strong>swyx:</strong> In the sense that people pay deposits upfront, like I paid like, I don't know, 200 bucks for the rabbit. Upfront, and I don't get it for another six months. I paid 600 for the tab, and I don't get it for another six months. And, and then, then they can take that money and, and sort of invest it in like their next, the next events or their next properties or ventures.</p><p>[00:55:06] <strong>swyx:</strong> And like, I think that's a, that's a very interesting reversal of economics from other types of AI companies that I see. And I think, yeah, just the, the, the tactile feel of an AI, I think is very promising. I, Alex, I don't know if you have other thoughts on, on the wearable stuff.</p><p>[00:55:21] <strong>Alessio:</strong> The open interpreter just announced their product four hours ago.</p><p>[00:55:25] <strong>Alessio:</strong> Yeah. Which is a, it's not really a wearable, but it's a, it's still like a physical device.</p><p>[00:55:30] <strong>swyx:</strong> It's a push to talk mic to, to a device on your, on your laptop. Right. It's a $99 push talk. Yeah.</p><p>[00:55:38] <strong>Alessio:</strong> But, but, but everybody, but again, going back to your point, it's like people want to, people are interested in spending money for like things that they can hold, you know, I don't know what that means overall for like where things are going, but making more of this AI be a physical part of your life.</p><p>[00:55:54] <strong>Alessio:</strong> I think people are interested in that, but I agree with Shawn. I mean, I've been. I talked to Avi about this, but Avi's point is like, most consumers, like, care about utility more than they care about privacy, you know, like you've seen with social media. But I also think there's a big societal reaction to AI that is, like, much more rooted than the social media one.</p><p>[00:56:16] <strong>Alessio:</strong> But we'll see. But a lot, again, a lot of work, a lot of developers, a lot of money going into it. So there's, there's bound to be experiments being run. On, on the</p><p>[00:56:25] <strong>swyx:</strong> chip side. Sorry, I'll just ship it one more thing and then we transition to the chips. The thing I'll caution people on is don't overly focus on the form factor.</p><p>[00:56:33] <strong>swyx:</strong> The form factor is a delivery mode. There will be many form factors. It doesn't matter so much as where in the data war does it sit. It actually is context acquisition. Because, and maybe a little bit of multi modality. Context, like, context is king. Like, if you have access to data that no one else has, then you will be able to create AI that no one else can create.</p><p>[00:56:54] <strong>swyx:</strong> And so what is the most personal context? It is your everyday conversation. It is as close to mapping your mental train of thought As possible without, you know, physically you writing down notes. So, so that is the promise, the ultimate goal here, which is like, personal context, it's always available on you you know, loading and seeing all that stuff.</p><p>[00:57:12] <strong>swyx:</strong> But yeah, that's the, that's the frame I want to give people that the form factors will change and there will be multiple form factors, but it's the software behind that. And in the personal context that you cannot get anywhere else, that'll win.</p><p>[00:57:24] <strong>Alessio:</strong> Yeah, so that was wearables.</p><p>[00:57:26] Groq vs Nvidia month - GPU Chip War</p><p>[00:57:26] <strong>Alessio:</strong> On the chip side, yeah, Grok was probably the biggest release.</p><p>[00:57:29] <strong>Alessio:</strong> Jonathan, well, it's not even a new release because the company, I think, was started in 2016. So it's actually quite old. But now recently captured the people's imagination with their MixedREL 500 tokens a second demo. Yeah, I think so far the battle on the GPU side has been Either you go kind of like massive chip, like the Cerebros of the world, where one chip from Cerebros is about two million dollars, you know, that's compared, obviously, you cannot compare one chip versus one chip, but h100 is like 40, 000, something like that the problem with those architectures has been They want to be very general, you know, but like they wanted to put a lot of the RAM, the SRAM on the chip.</p><p>[00:58:13] <strong>Alessio:</strong> It's much more convenient when you're using larger language models, but the models outpace the size of the chips and chips have a much longer, you know, turnaround cycle. Grok today. It's great for the current architecture. It's a lot more expensive also, as far as dollar per flop but their idea is like, hey, when you have very high concurrency, we actually were much cheaper, you know, you shouldn't just be looking at the compute power for most people, this doesn't really matter, you know, like, I think that's like the most the most interesting thing to me is like, We've now gone back with, with AI to a world where developers care about what hardware is running, which was not the case in traditional software for like, maybe 20 years since as the cloud has gotten really big.</p><p>[00:58:57] <strong>Alessio:</strong> My, my thinking is that in the next two, three years, like we're going to go back to that. We're like, people are not going to be sweating. Oh, what GPU do you have in your cloud? What do you have? It's like. Yeah, you want to run this model, we can run it at the same speed as everybody else, and then everybody will make different choices, whether they want to have higher front end capital investment, and then better utilization, some people would rather do lower investment before, and then upgrade later, there are a lot of parameters and then there's the dark horses, right, that is some of the smaller companies like Lemurian Labs, MedEx that are working on maybe not a chip alone, but also like some of the, the actual math infrastructure and the instructions on it that make them run.</p><p>[00:59:40] <strong>Alessio:</strong> There's a lot going on, but yeah, I think the, the episode with with Dylan will be interesting for, for people, but I think we also came out of it saying, Hey, everybody has pros and cons. There's no, it's different than the models where you're like, Oh, this one is definitely better for me. And I'm going to use it.</p><p>[00:59:56] <strong>Alessio:</strong> I think for most people. It's like fun Twitter memeing, you know, but it's like 99 percent of people that tweet about this stuff are never gonna buy any of these chips anyway. It's, it's really more for entertainment.</p><p>[01:00:10] <strong>swyx:</strong> No. Wow. I mean, like, this is serious business here, right? You're talking about, you know, like who, like the potential new Nvidia, if anyone can take like 1% of NVIDIA's business, they're a serious startup that you should look at.</p><p>[01:00:20] <strong>swyx:</strong> Right? So , that's, that's, that's my, well, yeah,</p><p>[01:00:23] <strong>Alessio:</strong> yeah. On matters. Well, I'm more talking about like, what, how should people think about it? You know? It's like, yeah. I think like the, the end user is not impacted as much.</p><p>[01:00:31] Disagreements</p><p>[01:00:31] <strong>Alessio:</strong> This is obviously, so</p><p>[01:00:32] <strong>swyx:</strong> I disagree. Yeah, I love disagreements because, you know, who likes a podcast where all three people always agree with each other?</p><p>[01:00:38] <strong>swyx:</strong> You will see the impact of this in the tokens per second over time. This year, I have very, very credible sources all telling me that the average tokens per second, right now, we have somewhere between 50 to 100 as like the norm for people. Average tokens per second will go to 500 to 2, 000. This year from, from a number of chip suppliers that I cannot name.</p><p>[01:00:58] <strong>swyx:</strong> So like that is, that is, that will cause a step change in the use cases. Every time you have an order of magnitude improvement in the, in the speed of something, you unlock new use cases that become fun instead of a chore. And so that's what I would caution this audience to think about, which is like, what can you do in much higher AI speed?</p><p>[01:01:17] <strong>swyx:</strong> It's not just things streaming out faster. It is things working in the background a lot more seamlessly and therefore being a lot more useful. Then previously imagined. So that would be my two cents on.</p><p>[01:01:30] <strong>Alessio:</strong> Yeah. Yeah. I mean, the, the new NVIDIA chips are also much faster. To me, that's true. When it comes to startups, it's like, are the startups pushing the performance on the incumbents or are the incumbents still leading?</p><p>[01:01:44] <strong>Alessio:</strong> And then the startups are like riding the same wave, you know? I don't have yet a good sense of that. It's like, you know, it's next year's NVIDIA release. Just gonna be better than everything that gets released this year, you know, if that's the case, it's like, okay, damn Jensen, you know, it's like the meme.</p><p>[01:02:00] <strong>Alessio:</strong> It's like, I'm gonna fight. I'm gonna fight NVIDIA. It's like, damn, Jensen got hands. He really does.</p><p>[01:02:08] Summer 2024 Predictions</p><p>[01:02:08] <strong>NLW:</strong> Well, awesome conversation, guys. I guess just just by way of wrapping up, I call it over the next three months between now and sort of the beginning of summer was one prediction that each of you has. It can be about anything. It can be a big company. It can be a startup. It can be something you have privileged information that you know, and you just won't tell us that you actually</p><p>[01:02:25] <strong>Alessio:</strong> know.</p><p>[01:02:26] <strong>Alessio:</strong> What, does it have to be something that we think it's going to be true or like something that we think? Because for me, it's like, is Sundar going to be the CEO of Google? Maybe not in three months, maybe in like six months, nine months, you know, people are like, Oh, maybe Demis is going to be the new CEO.</p><p>[01:02:41] <strong>Alessio:</strong> That was kind of like, I, I was busy like fishing some deep mind people and Google people for like a good guest for the pod. And I was like, Oh, what about. Jeff Dean, and they're like, well, Demis is really like the person that runs everything anyway, and the stuff. It's like interesting. And</p><p>[01:02:57] <strong>swyx:</strong> so I don't know.</p><p>[01:02:58] <strong>swyx:</strong> What about Sergei? Sergei Sergei could come back. I don't know. Like he's making more appearances these days.</p><p>[01:03:03] <strong>Alessio:</strong> Yeah. I don't, I I Then we can just put it as like, you know. Yeah. My, my thing is like CEO change potential, but I, again, three months is too short to make a prediction. Yeah. I</p><p>[01:03:16] <strong>NLW:</strong> think that's the, that's that's fine.</p><p>[01:03:18] <strong>NLW:</strong> The, the timescale might be off.</p><p>[01:03:22] <strong>swyx:</strong> Yeah. I mean for me, I, I think the. Progression in vertical agent companies will keep going. We just had, the other day, Klarna talking about how they replaced like 700 of their customer support agents with the AI agents. That's just the beginning, guys. Like, imagine this rolling out across most of the Fortune 500.</p><p>[01:03:43] <strong>swyx:</strong> This is, and I'm not saying this is like a utopian scenario, there will be very, very embarrassing and bad outcomes of this, where like, humans would never make this mistake, but AIs did, and like, we'll all laugh at it, or we'll be very offended by whatever, you know, bad outcome it did. So we have to be responsible and careful in the rollout, but yeah, this is, it's rolling out, you know, Alessio likes to say that this year's the year of AI in production.</p><p>[01:04:04] <strong>swyx:</strong> Let's see it, let's, let's see all these sort of vertical, full stack employees. Come out into the workforce. Love</p><p>[01:04:11] <strong>Alessio:</strong> it.</p><p>[01:04:11] <strong>NLW:</strong> All right, guys. Well, thank you so much for for sharing your your thoughts and insights here And I can't wait to do it again</p><p>[01:04:18] Thursday Nights in AI - swyx</p><p>[01:04:18] <strong>NLW:</strong> Welcome</p><p>[01:04:19] <strong>swyx:</strong> back again. It's Charlie your AI co host We're now in part two of the special weekend episode collating some of SWIX and Alessio's recent appearances If you're not active in the Latentspace Discord, you might not be aware of the many, many, many in person.</p><p>[01:04:36] <strong>swyx:</strong> Events we host gathering our listener community all over the world. You can see the Latentspace community page for how to join and subscribe to our event calendar for future meetups. We're going to share some of our recent live appearances in this next part, starting with the Thursday nights in AI meetup, a regular fixture in the SF AI scene run by Imbue and Outset Capital.</p><p>[01:04:59] <strong>swyx:</strong> Primarily, our former guest, Kanjin Q, Ali Rhoda, and Josh Albrecht. Here's Swyx.</p><p>[01:05:08] <strong>swyx:</strong> Today, for those of you who have been here before, you know the general format. So we'll do a quick fireside Q& A with Swyx. Swyx, where we're asking him the questions. Then we'll actually go to our rapid fire Q& A, where we're asking really fast, hopefully, spicy questions. And then we'll open it up to the audience for your questions.</p><p>[01:05:25] <strong>swyx:</strong> So you guys sneak around the room, submit your questions, and we'll go through as many of them as possible during that period. And then actually, Swyx brought a gift for us, which is two Latentspace t shirts. AI Engineer. AI Engineer t shirts. And those will be awarded to the Two spiciest question askers.</p><p>[01:05:44] <strong>swyx:</strong> So and I'll let Josh decide on that. So if we want to get your spiciest takes, please send them in during the event as we're talking and then also at the end. All right. With that, let's get going.</p><p>[01:05:57] <strong>NLW:</strong> Okay. Welcome, Swyx. Thank you for that</p><p>[01:06:01] <strong>swyx:</strong> intro.</p><p>[01:06:01] <strong>NLW:</strong> How does it</p><p>[01:06:01] <strong>swyx:</strong> feel to be interviewed</p><p>[01:06:03] <strong>NLW:</strong> rather than the interviewer?</p><p>[01:06:04] <strong>swyx:</strong> Weird. I don't know what to do in this chair. Yeah. Like,</p><p>[01:06:07] <strong>NLW:</strong> where should I put my hands? Yeah, exactly. You look good.</p><p>[01:06:10] <strong>swyx:</strong> You look good. And I also love asking follow up questions. And I tend to, like, sort of take over panels a lot. If you ever see me on a panel, I tend to ask the other panelists questions.</p><p>[01:06:18] <strong>swyx:</strong> Okay.</p><p>[01:06:19] <strong>NLW:</strong> So we should be ready is what you're saying. So you back.</p><p>[01:06:21] <strong>swyx:</strong> That's fine. This is like a free MBU interview, so why not? That's right. That's right. That's</p><p>[01:06:24] <strong>NLW:</strong> right.</p><p>[01:06:25] <strong>swyx:</strong> Yeah, so you interviewed Ken Jeon, the CEO you didn't interview Josh, right? No, no. So maybe tonight. Yeah. Okay. We'll see. We'll look for different questions and look for an alignment.</p><p>[01:06:35] <strong>NLW:</strong> I love it. All</p><p>[01:06:36] <strong>swyx:</strong> right. I just want to hear this story. You know, you've completely exploded LatentSpace and AI Engineer, and I know you also, before all of that, had exploded in popularity for your learning in public movement and your DevTools work. And devrelations work. So, who are you and how did you get here?</p><p>[01:06:53] <strong>swyx:</strong> Let's</p><p>[01:06:53] <strong>NLW:</strong> start with that.</p><p>[01:06:54] <strong>swyx:</strong> Quick story is, I'm Shawn, I'm from Singapore. Swyx is my initials. For those who don't know, A lot of Singaporeans are ethically Chinese, and we have Chinese names and English names. So, it's just it's just my initials. Came to col came to the US for college, and have been here for about 15 years, but most, like half of that was in finance and then the other half was, was in tech.</p><p>[01:07:13] <strong>swyx:</strong> And the, and tech is where I was most known just because I realized that I was much more aligned towards learning in public, whereas in finance, Everything's a trade secret. Everything is zero sum. Whereas in tech, like, you're allowed to come to meetups and conferences and share your learnings and share your mistakes even.</p><p>[01:07:31] <strong>swyx:</strong> And that's totally fine. You, like, open source your code. It's totally fine. And even, even better, you, like, contribute PRs to other people's code, which is even better. And I found that I thrived in that. Learning public environments and that, that kind of got me started. I was an early hire, early Draft Relations hire at Netlify and then did the same at AWS Temporal and Airbyte.</p><p>[01:07:53] <strong>swyx:</strong> And then, and so that, that's like the whole story. I can talk, talk more about like developer tooling and developer relations if, if that's something that people are interested in. But I think the, the more recent thing is AI. And I started really being interested in it mostly because It, it, the, the approximate cause of starting Leanspace was stable diffusion.</p><p>[01:08:10] <strong>swyx:</strong> When you could run a large model that could do sufficiently enough on your, on your desktop. Where I was like, okay, like, this is, Something qualitatively very different. And that's then we started late in space and you're like, this is something different. We have to talk about it on a podcast.</p><p>[01:08:25] <strong>swyx:</strong> There we go. Yeah. It wasn't, it wasn't a podcast for like four months. And then, and then I had been running a discord for dev tools investors. 'cause I, I also invest in dev tools and I advise companies on deaf tools, def things. And I think it was the start of 2023 when Alessio and I were both like, you know, I think we, we need to like get more tokens out of.</p><p>[01:08:45] <strong>swyx:</strong> People, and I was running out of original sources to, to write about, so I was like, okay, I'll go get those original sources. And I think that, that's when we started the podcast. And I think it's just the chemistry between us, the, the way we spike in different ways. And also, like, honestly, the kind participation of the guests to give us their time.</p><p>[01:09:03] <strong>swyx:</strong> Like, you know, like, getting George Hoss was a big deal. And also shout out to Alessio for just cold emailing him for, for, for booking the, booking some of our biggest guests. And I'm just working really hard to try to tell the story that people can use at work. I think that there's a lot of AI podcasts out there and a lot of AI kind of forums or fireside chats with no fire.</p><p>[01:09:21] <strong>swyx:</strong> That always talk about age, like what's your AGI timeline, what's your PDoom. Very, very nice hallway conversations for freshman year but not very useful for work. And like, you know, practically like making money and like And thinking about, you know, changing the everyday lives. I think what's interesting is obviously you care about the existential safety of the human race.</p><p>[01:09:43] <strong>swyx:</strong> But in the meantime we gotta eat. So so I think that's like kind of latent space's niche. Like we explicitly don't really talk about AGI. We explicitly don't talk about Things that we're, like, a little bit too far out. Like, we don't do a ton of robotics. We don't do a ton of, like, high frequency trading.</p><p>[01:10:00] <strong>swyx:</strong> There's tons of machine learning in there, but we just don't do that. Because, like, we're like, all right, what are most software engineers gonna, gonna need? Because that's our background, and that's the audience that we serve. And I think just, like, being really clear on that audience has been, has resonated with people.</p><p>[01:10:12] <strong>swyx:</strong> Yeah, you would never expect a technical podcast to reach, like, a general audience, like, Top ten on the tech charts but I, you know, I've been surprised by that before and it's been successful. I don't know, I don't know what to say about that. I think honestly, I, I kind of have this like negative reaction towards being, being, being, being, being classified as a podcast because the podcast is downstream of ideas.</p><p>[01:10:35] <strong>swyx:</strong> And it's one mode of conversation, it's one mode of idea delivery, but you can deliver ideas on a newsletter, in person like this there's so many different ways. And so I think, I think about it more as we are trying to start or serve an industry, and that industry is the AI engineer industry, which is, which we can talk about more.</p><p>[01:10:53] <strong>swyx:</strong> Yes, let's go into that. So the AI engineer, you penned a piece called The Rise of the AI Engineer, you tweeted about it, Andrej Karpathy also responded, largely agreeing with what you said. What is an AI engineer? The AI engineer is the software engineer building with AI, enhanced by AI, And eventually it will be non human engineers writing code for you, Which I know MBU is all about.</p><p>[01:11:18] <strong>swyx:</strong> You're saying eventually the AI engineer will become a non human engineer? That will be one kind of AI engineer that people are trying to build, And is probably the most furthest away in terms of being reality. Because it's so hard. Got it. But, but there are three types of AI engineer and I just went through the three.</p><p>[01:11:33] <strong>swyx:</strong> One is AI enhanced where you like use AI products like Copilot and Cursor. And two is AI products engineer where you use the exposed AI capabilities to the end user As a software engineer, like, not doing pre training not being an ML researcher, not being an ML engineer, but just interacting with foundation models and probably APIs from foundation model labs.</p><p>[01:11:54] <strong>swyx:</strong> What's the third one? And the third one is the non human AI engineer. Got it. The fully autonomous AI engineer. Dream, you know, Coder. How long do you think it is till we get to, like, early, early versions? This is my equivalent of AGI timelines. I know, I know. You can set yourself up for this. So like, lots of active, like, I mean, I have, I have supported companies actively working on that.</p><p>[01:12:13] <strong>swyx:</strong> I think it's more useful to think about levels of autonomy. And so my answer to that is, you know, perpetually five years away until until it figures it out. No, but my actual anecdote the closest comparison we have to that is self driving. We are, we're doing this in San Francisco for those who are watching the live stream.</p><p>[01:12:32] <strong>swyx:</strong> If you haven't come to San Francisco and seen, and taken a Waymo ride just come, get a friend take a Waymo ride. I remember 2014 we covered a little bit of autos in, in my hedge fund. And I was, I remember telling a friend, I was like, self driving cars around the corner, like, this is it, like, you know, parking will be, like, parking will be a thing of the past and it didn't happen for the next 10 years.</p><p>[01:12:52] <strong>swyx:</strong> And, and, but now we, now, like, most of us in San Francisco can, can take it for granted. So I think, like, you just have to be mindful that the, the, the, the rough edges take a long time. And like, yes, it's going to work in demos, then it's going to work a little bit further out and it's just going to take a long time.</p><p>[01:13:08] <strong>swyx:</strong> The more useful mental model I have is sort of levels of autonomy. So in self driving, you have level 1, 2, 3, 4, 5 just the amount of human attention that you get. At first, like, your, your, your hands are always on 10 and 2 and you have to pay attention to the, to, to the driving every 30 seconds and eventually you can sleep in the car, right?</p><p>[01:13:25] <strong>swyx:</strong> So there's a whole spectrum of that. So what's the equivalent for that for, for coding? Keep your hands on the keyboard and then eventually you've kind of gone off. You tab to accept everything. Where are we? Oh, that's good, yeah. Yeah. Doesn't that already happen? Yeah. Approve the PR. Approve, this looks good.</p><p>[01:13:39] <strong>swyx:</strong> That's the dream that people want. It gives, it gives, really you unlock a lot of coding when people, non technical people can file issues, and then the AI engineer can sort of automatically write code, pass your tests, and if it, if it kind of works as, as, as intended. As, as advertised then you can just kind of merge it and then you, you know, 10x, 100x the number of developers in your company immediately.</p><p>[01:14:00] <strong>swyx:</strong> So that's the goal, that's the, that's the holy grail. We're not there yet but Sweep, CodeGen, there's a bunch of companies, Magic probably, are, are all working towards that. And, and so I so the TLDR, like the, the thing that we covered Alessio and I covered in the January recap that we did was that the, the basic split that people should have in their minds is the inner loop versus the outer loop for the developer.</p><p>[01:14:21] <strong>swyx:</strong> Inner loop is everything that happens in your IDE between Git commits. And outer loop is happens, is what happens when you push up your Git commit to GitHub, for example, or GitLab. And that's a nice split, which means like everything local, everything that needs to be fast is for everything that's kind of very hands on for developers.</p><p>[01:14:37] <strong>swyx:</strong> It's probably easier to automate or easier to have code assistance. That's what Copilot is, that's what, that's what all those things are. And then everything that happens autonomously when you're effectively away from the keyboard with like a GitHub issue or something that is more outer loop where you're you know, you're relying a lot more on autonomy and we are maybe, our LLMs are maybe not smart enough to do that yet.</p><p>[01:14:57] <strong>Alessio:</strong> Do you have any thoughts on</p><p>[01:14:58] <strong>swyx:</strong> kind of</p><p>[01:14:58] <strong>Alessio:</strong> the user experience and how that will change? One of the things</p><p>[01:15:01] <strong>swyx:</strong> that has happened for me, kind of looking at some of these products and playing around with things ourselves, like, You know, it sounds good to have an automated PR, then you get an automated PR and you're like, I really don't want to review like 300 lines of generated code, and like find the bug in it.</p><p>[01:15:13] <strong>swyx:</strong> Well then you have another agent that's a reviewer. That's right, but then you like tell it like, Oh, go fix it, and it comes back with 400 lines. Yes, there is a length bias to code, right? And you do have higher passing rates. In PRs. This is a documented human behavior thing, right? Send me two lines of code, I will review the s**t out of that.</p><p>[01:15:33] <strong>swyx:</strong> I don't know if I can swear on this. Send me, send me 200 lines of code, looks good to me. Right? Guess what? The, the agents are going to, perfectly happy to modify, to copy that behavior from us. When we actually want them to do the opposite. So, yeah, I, I think that the GAN model of code generation is probably not going to work super well.</p><p>[01:15:50] <strong>swyx:</strong> I do think we probably need just better planning from the start. Which is, I'm just repeating the MBU thesis by the way. Just go listen to Kanjin talk about this. She's much better at it than I am. But yeah, I think I think the code review thing is going to be I think that what Codium, there are two Codiums, the Israeli one.</p><p>[01:16:10] <strong>swyx:</strong> The Israeli Codium. With the E. Yeah, Codium with the E. They still have refused to rename. I'm friends with both of them. Every month I'm like, You're like, guys, let's</p><p>[01:16:18] <strong>NLW:</strong> all come to one room. Yeah,</p><p>[01:16:19] <strong>swyx:</strong> like, you know, someone's got to fold. Codium with the E has gone, like, you've got to write the test first. Right?</p><p>[01:16:25] <strong>swyx:</strong> You write the, you write the it's like a sort of tripartite relationship. Again, this was also covered on a podcast with them, which is fantastic. Like, you interview me, you sort of through me, you interview. Like, the past avatars I've been watching the Netflix show, by the way, it's fantastic. But like, so so Codium is like, they've already thought this all the way through.</p><p>[01:16:41] <strong>swyx:</strong> They're like, okay, you write the user story, from the user story you generate all the tests, you also generate the code and you update any one of those, they all have to update together. Right? So like, once the, and, and probably the critical factor is the test generation from the story. Because everything else can just kind of bounce the heads off of those things until they pass.</p><p>[01:17:01] <strong>swyx:</strong> So you have to write good tests. It's kind of like the eat your vegetables of coding, right? Which nobody really wants to do. And so I think it's a really smart tactic to go to market by saying we automatically generate tests for you and, you know, start not great, but then get better. And eventually you get to the weakest point in the chain for the entire loop of code generation.</p><p>[01:17:25] <strong>swyx:</strong> What do you think the weakest link is? The weakest link? Yeah. It's text generation. Yeah. Yeah. Do you think there's a way to, like, are there some promising</p><p>[01:17:33] <strong>Alessio:</strong> avenues you see forward for making that actually better?</p><p>[01:17:38] <strong>swyx:</strong> For making it better. You have to have, like, good isolation, and I think proper serverless cloud environments is integral to that.</p><p>[01:17:48] <strong>swyx:</strong> I, it could be like a fly. io. It could be like a Cloudflare worker. It depends how much, how many resources your test environment needs. And effectively I was talking about this, I think with maybe Rob earlier in the audience, where every agent needs a sandbox. If you're a code agent, you need a coding sandbox, but if you're whatever, like MBU used to have this, like, sort of Minefield, Minecraft's clone that was much faster.</p><p>[01:18:12] <strong>swyx:</strong> If, if you, if you have a model of the real world, you have to go, you have to go generate some plan or some code or some whatever, test it against that real world so that you can get this iterative feedback and then get the final result back that is somewhat validated against the real world. And so, like, you need a really good sandbox.</p><p>[01:18:26] <strong>swyx:</strong> I don't think people, I, I think this is, this is a, this is an infrastructure need that humans</p><p>[01:18:31] <strong>swyx + Josh Albrecht:</strong> have had for a long time. We've never solved it for ourselves. And now we have to solve it for humans. About a thousand times larger quantity of agents than, than, than actually exists. And, and so I, I, I think, like, we eventually have to involve, evolve a lot more infrastructure.</p><p>[01:18:45] <strong>swyx + Josh Albrecht:</strong> In order to serve these things. So yeah. So, for those who don't know, like I also have so, we're talking about the rise of AI engineer. I also have previous conversations about immutable infrastructure cloud environments and that kind of stuff. And this is all of a kind. Like, like, in order to solve agents and coding agents, we're going to have to solve the other stuff too along the way.</p><p>[01:19:05] <strong>swyx + Josh Albrecht:</strong> And it's really neat for me. To see all that tie together in my DevTools work that all these themes kind of reemerge just naturally, just because everything we needed for humans, we just need a hundred times more for, for for agents.</p><p>[01:19:17] <strong>Dylan Patel:</strong> Let's talk about the AI engineer. AI engineer has become a whole thing.</p><p>[01:19:21] <strong>Dylan Patel:</strong> It's become a term and also a conference. And tell us more, and a job title, tell us more about that. What's going on there?</p><p>[01:19:31] <strong>swyx + Josh Albrecht:</strong> That is like a very vague, a very, very big cloud of things. I would just say like, I think it's an emergent industry. I've seen this happen repeatedly for, so the general term is software engineer.</p><p>[01:19:44] <strong>swyx + Josh Albrecht:</strong> Programmer. In the 70s and 80s, there would not be like senior engineer. There would just be engineering. Like you, or you, I don't think they even call themselves engineer. They don't have that. What about a member of the technical staff? Oh, yeah, MTS. Very, very, very, very elite. But yeah, so like, you know, like these striations appear when the population grows and the technical depth grows over time.</p><p>[01:20:07] <strong>swyx + Josh Albrecht:</strong> Yeah. When it starts, when it ends. Not that, not that important, and then over time it's just gonna specialize. And I've seen this happen for frontend, for DevOps, for data and I can't remember what else I listed in, in that, in that piece, But those are the main three that I was around for. And I, I see this, I saw this happening for AI engineer which is effectively, now a lot of people are arguing that there is the ML researcher, the ML engineer, who sort of pairs with the researcher sometimes they also call research engineer and then on the other side of the fence is just software engineers.</p><p>[01:20:35] <strong>swyx + Josh Albrecht:</strong> And that's how it was up till about last year. And now there's this specializing and rising class of people building AI specific software that are not any of those previous titles that I just mentioned. And that's the thesis of the AI engineer, that this is an emerging category of AI. Startups of jobs I've had people from Meta, IBM, Microsoft, OpenAI tell me that they, their title is now AI engineer.</p><p>[01:20:58] <strong>swyx + Josh Albrecht:</strong> They're hiring AI engineers. So, like, I can see that this is a trend and I think that's what Andre called out in his post that, like, just mathematically, just the, just the limitations in terms of talent, research talents and GPUs, that all these will tend to concentrate in a, in a, in a, Few labs and everyone else are just going to have to rely on them or build differentiation of products in other ways And those will be AI engineers.</p><p>[01:21:21] <strong>swyx + Josh Albrecht:</strong> So mathematically there will be more AI engineers than ML engineers. It's just the truth. Right now it's the other way. Right now the number of AI engineers is maybe 10x less. So I think that the ratio will invert and you know I think the goal of the InSpace and the goal of the conference and anything else I do is to serve that</p><p>[01:21:38] <strong>Dylan Patel:</strong> growing audience.</p><p>[01:21:41] <strong>Dylan Patel:</strong> To make the distinction clear, if I'm a software engineer And I'm like, I want to become an AI engineer. What do I have to learn? Like, what additional capabilities does that type of engineer have? Funny you say that. I think you have a blog post on this very</p><p>[01:21:53] <strong>swyx + Josh Albrecht:</strong> topic. I don't actually have a specific blog post on how to, like, change classes.</p><p>[01:21:58] <strong>swyx + Josh Albrecht:</strong> I do think I always think about these in terms of yeah, Baldur's Gate and, you know D& D rule set number 5. 1 or whatever. But yeah, so I kind of intentionally left that open to leave space for others. I think when you start an industry, you need to the specifications that work the best in industries are So minimally defined so that other people can fill in the blanks.</p><p>[01:22:19] <strong>swyx + Josh Albrecht:</strong> And I want people to fill in the blanks. I want people to disagree with me and with with themselves so that we can figure this out as a, as a group. Like I don't want to overs specify everything, you know, like that that's, that's a way, that's the only way to guarantee it, that it will fail. Um, I do have a take obviously, 'cause a lot of people are, are asking me like, where to start.</p><p>[01:22:37] <strong>swyx + Josh Albrecht:</strong> And I think basically so what, what we have is latent Space University. We just finished working on day seven today. It's a seven day email course. Where basically, like, it, it is completely designed to answer the question of, like, okay, I'm a, I'm an existing software engineer, I, like, kind of, I know how to code but I don't get all this AI stuff, I've been living under a rock, or, like, it's just too overwhelming for me, you have to, like, pick for me, or curate for me as a, as a trusted friend.</p><p>[01:22:59] <strong>swyx + Josh Albrecht:</strong> And I have one hour a day for seven days. What, what, what do you do? slot in that, in that, in that bucket. So for us, it's making, making sort of LLM API, API calls. It's me, it's image generation, it's code generation, it's audio ASR, I, I think, what's, what's ASR? Audio speech recognition?</p><p>[01:23:18] <strong>swyx + Josh Albrecht:</strong> Yeah, yeah. And then I forget, I forget what the fifth and sixth one is, but the last day is agents. And, and so basically, like, I'm just like, you know, Here are seven projects that you should do to feel like you can do anything in AI. You can't really do everything in AI just from, just from that small list.</p><p>[01:23:34] <strong>swyx + Josh Albrecht:</strong> But I think it's just like, just like anything, you have to like, go through like a set list of, of things that are basic skills that I think everyone in this industry should have to be at least conversant in. If someone, if like a boss comes to you and goes like, hey, can we build this? You don't even know if the answer is no.</p><p>[01:23:52] <strong>swyx + Josh Albrecht:</strong> So I want you to move towards from like unknown unknowns to at least known unknowns. And I think that's, that's where you start being competent as an AI engineer. So, so yeah, that's LSU, Latent Space University, just to trigger the The Tigers.</p><p>[01:24:06] <strong>Dylan Patel:</strong> So do you think in the future that people, an AI engineer is going to be someone's full time job?</p><p>[01:24:10] <strong>Dylan Patel:</strong> Like people are just going to be AI engineers? Or do you think it's going to be more of a world where I'm a software engineer, and like, 20 percent of my time, I'm using open AIs, APIs, and I'm, Working on prompt engineering and stuff like that and using</p><p>[01:24:23] <strong>swyx + Josh Albrecht:</strong> CodePilot. You just reminded me of Day6's open source models and fine tuning.</p><p>[01:24:27] <strong>swyx + Josh Albrecht:</strong> Perfect. I think it will be a spectrum. That's why I don't want to be like too definitive about it. Like we have full time front end engineers and we have part time front end engineers and you dip into that community whenever you want. But wouldn't it be nice if there was a collective name for that community so you could go find it?</p><p>[01:24:40] <strong>swyx + Josh Albrecht:</strong> You can find each other. And, like, honestly, like, that's, that's really it. Like, a lot of people, a lot of companies were pinging me for, like, Hey, I want to hire this kind of person, but you can't hire that person, but I wanted someone like that. And then people on the labor side were, were pinging me going, like, Okay, I want to do more in this space, but where do I go?</p><p>[01:24:56] <strong>swyx + Josh Albrecht:</strong> And I think just having that shelling point of, of, of what an industry title and name is, and then sort of building out that. Mythology and community and conference I think is helpful, hopefully, and I don't have any prescriptions on whether or not it's a full time job. I do think, over time, it's going to become more of a full time job.</p><p>[01:25:14] <strong>swyx + Josh Albrecht:</strong> And that's great for the people who want to do that and the companies that want to employ that. But it's absolutely, like, you can take it part time, like, you know, jobs come in many formats. Yep, yep, that</p><p>[01:25:23] <strong>Dylan Patel:</strong> makes sense. Yeah. And then you have a huge world fair coming up. Yeah. Tell me about that. So,</p><p>[01:25:31] <strong>swyx + Josh Albrecht:</strong> Part of, I think, you know, What creating an industry requires is for, to let people gather in one place.</p><p>[01:25:37] <strong>swyx + Josh Albrecht:</strong> And also for me to get high quality talks out of people. You have to create an event out of it. Otherwise they don't do the work. So so last year we did the AI Engineer Summit, which went very well. And people can see that online and we're, we're, we're very happy with how that turned out.</p><p>[01:25:53] <strong>swyx + Josh Albrecht:</strong> This year we want to go four times bigger with the World's Fair and try to reflect AI engineering as it is in 2024. I always admired two conferences in, in this respect. One is NeurIPS, which I went to last year and, and documented on, on the pod, which was fantastic. And two, which is KubeCon from the other side of my life, which is the sort of cloud registration and, and DevOps world.</p><p>[01:26:18] <strong>swyx + Josh Albrecht:</strong> So NeurIPS is the one place that you go to, to, I think it's the top conference. I mean, there's, there's others that you can kind of consider. But, yeah so, so NeurIPS is, NeurIPS is where the research sciences are the stars. The researchers are the stars, PhDs are the stars, mostly it's just PhDs on the job market, to be honest.</p><p>[01:26:34] <strong>swyx + Josh Albrecht:</strong> It's really funny</p><p>[01:26:35] <strong>Dylan Patel:</strong> to go, especially these days. Yeah, it</p><p>[01:26:37] <strong>swyx + Josh Albrecht:</strong> was really funny to go to NeurIPS and go like, And the VCs trying to back them. Yeah, there are lots, lots of VCs trying to back them. Yeah, there This year. Anyway, so in Europe, research scientists are the stars. And for, I wanted for AI engineers, for engineers to be the star.</p><p>[01:26:51] <strong>swyx + Josh Albrecht:</strong> Right, to show off their tooling and their techniques and their difficulty moving all these ideas from research into production. The other one was KubeCon, where, You could honestly just go and not attend any of the talks and just walk the floor and figure out what's going on in DevOps, which is fantastic.</p><p>[01:27:10] <strong>swyx + Josh Albrecht:</strong> Because, yeah, so, so that curation and that bringing together of, of, of an industry is what I'm going for for the conference. And yeah, it's coming in June. The most important thing, to be honest, when I, like, conceived of this whole thing was to buy the domain. So we got AI. engineer. People are like, engineer is a domain?</p><p>[01:27:27] <strong>swyx + Josh Albrecht:</strong> Yeah, and funny enough, engineer was cheaper than engineering. I don't understand why, but like that's up to the domain people.</p><p>[01:27:36] <strong>Dylan Patel:</strong> Josh, any questions on agents?</p><p>[01:27:38] <strong>Alessio:</strong> Yeah,</p><p>[01:27:39] <strong>Dylan Patel:</strong> I think maybe, you know, you have a lot</p><p>[01:27:40] <strong>swyx + Josh Albrecht:</strong> of experience and exposure talking to all these companies and founders and researchers and everyone that's on your podcast.</p><p>[01:27:47] <strong>Dylan Patel:</strong> Do you have, do you feel like you have a</p><p>[01:27:50] <strong>swyx + Josh Albrecht:</strong> good kind of perspective on some of the things that, like, some of the kind of technical issues having seen? You know, like we were just talking about, like, for coding agents, like, oh, how, you know, the value of test is really important. There are other things, like, for, you know, retrieval, like now, You know, we have these models coming out with a million context, you know, or a million tokens of context length, or ten million, like, is retrieval going to</p><p>[01:28:10] <strong>Dylan Patel:</strong> matter anymore, like,</p><p>[01:28:11] <strong>swyx + Josh Albrecht:</strong> do</p><p>[01:28:11] <strong>Dylan Patel:</strong> huge contexts matter, like,</p><p>[01:28:13] <strong>swyx + Josh Albrecht:</strong> what do you think?</p><p>[01:28:14] <strong>swyx + Josh Albrecht:</strong> Specifically about the long context thing? Sure, yeah. Because you asked a more broad question. I was going to ask a few other ones after that, so go for that one first. Yeah. That's what I was going to ask first. We can ask, yeah, okay, let's talk about long context and then the other stuff. So, for those who don't know, LongContext was kind of in the air last year, but really, really, really came into focus this year.</p><p>[01:28:33] <strong>swyx + Josh Albrecht:</strong> With Gemini 1. 5 having a million token context and saying that it was in research for 10 million tokens. And that means that you can put, you, you, you, like, no longer have to really think about, What you retrieve sorry, you no longer really think about what you have to, like, put into context.</p><p>[01:28:50] <strong>swyx + Josh Albrecht:</strong> You can just kind of throw it, throw the entire knowledge base in there, or books, or film, anything like that and that's fantastic. A lot of people are thinking that it kills RAG, and I think, like, one, that's not true, because for any kind of cost reason you you know, you still pay per token, so if you there, so basically Google is, like, perfectly happy to let you pay a million tokens every single time you make an API call, but good luck, you know, having a hundred dollar API call.</p><p>[01:29:12] <strong>swyx + Josh Albrecht:</strong> And and then the other thing, it's going to be slow. No explanation needed. And then finally, my criticism of long context is that it's also not debuggable. Like, if something goes wrong with the result, you can't do, like, the ragged decomposition of where the source of error. Like, you just have to, like, go, like, it's the Waze, bro.</p><p>[01:29:29] <strong>swyx + Josh Albrecht:</strong> Like, it's somewhere in there. Sorry. I pretty strongly agree with this. Why do you think people are making such crazy long context windows? People love to kill rag, right? It's so much Kill it, though, because it's too expensive. It's so expensive like you said. Yeah, I just think I just call it a different dimension I think it's an option that's great when it's there like when I'm prototyping I do not ever want to worry about context and I'm gonna call Stuff a few times and I don't want to run to errors I don't want to have it set up a complex retrieval system just to prototype something But once I'm done prototyping then I'll worry about all the other rag stuff And yes, I'm gonna buy some system or build some system or whatever to go do that.</p><p>[01:30:02] <strong>swyx + Josh Albrecht:</strong> I so I think it's just like An improvement in like one dimension that you need And then, but the improvements in the other dimensions also matter. And it's all needed, like this space is just going to keep growing, um, in unlimited fashion. I do think that this combined with multi modality does unlock new things.</p><p>[01:30:21] <strong>swyx + Josh Albrecht:</strong> So That's what I was going to ask about next. It's like, how important is multi modal? Like, great, you know, generating videos, sure, whatever. Okay, how many of us need to generate videos that often? It'd be cool for TV shows, sure, but like, yeah. I think it's pretty important. And the one thing that, in, when we launched the Lean Space podcast, We listed a bunch of interest areas.</p><p>[01:30:37] <strong>swyx + Josh Albrecht:</strong> So one thing I love about being explicit or intentional about our, our work is that you list the things that you're interested in and you, you list the things that you're not interested in. And people are very unwilling to, to, to have an anti interest list. One of the things that we were not interested in was multimodality last year.</p><p>[01:30:55] <strong>swyx + Josh Albrecht:</strong> Because everyone was, I was just like, okay, you can generate images and they're pretty, but like not a giant business. I was wrong. Midrani is a giant, giant, massive business that no one can get it, no one can understand or get into. But also I think being able to, to natively understand audio and video and code.</p><p>[01:31:12] <strong>swyx + Josh Albrecht:</strong> I consider code a special modality. All that is very, like, qualitatively different than translating it into English first and using English as, I don't know, like a bottleneck or pipe and then you know, applying it in LLMs. Like the ability of LLMs to reason across modalities gives you something more than you could, you know, Individually by, by, by using text as the universal interface.</p><p>[01:31:33] <strong>swyx + Josh Albrecht:</strong> So I think that's useful. So concretely what, what does that mean? It means that so I think the reference post for everyone that you should have in your head is Simon Willison's post on Gemini 1. 5's video capability. Where he basically shot a video of his bookshelf and just kind of scanning through it.</p><p>[01:31:50] <strong>swyx + Josh Albrecht:</strong> And he was able to give back a, a complete JSON list of the books and the authors and, and all the details that were visible there. Hallucinated some of it, which is, you know, another, another issue. But I think it's just like unlocks this use case that you just would not even try to code without the native video understanding capability.</p><p>[01:32:08] <strong>swyx + Josh Albrecht:</strong> And obviously, like. On a technical level, video is just a bunch of frames. So actually it's just image understanding, but image within the temporal dimension, which this month, I think, became much more of a important thing, like the integration of space and time in Transformers. I don't think anyone was really talking about that until this month, and now it's the only thing anyone can ever think about for Sora and for all the other stuff.</p><p>[01:32:30] <strong>swyx + Josh Albrecht:</strong> The last thing I'll say that, which is which is Against this trend of like every modality is important. They just, just do all the modalities. I kind of agree with Nat Friedman who actually kind of pointed out just before the Gemini thing blew up this, this, this, this month, which was like, why is it that OpenAI is pushing Dolly so hard?</p><p>[01:32:48] <strong>swyx + Josh Albrecht:</strong> Why is, why is being pushing Bing image creator? Like, it's not nec, it's not apparent that you have to create images to create a GI. But every lab just seems to want to do this, and I kind of agree that it's not on the critical path. Especially for image generation, maybe image understanding, video understanding.</p><p>[01:33:04] <strong>swyx + Josh Albrecht:</strong> Yeah, consumption. But generation, eh. Maybe we'll be wrong next year. It just catches you a bunch of flack with like, you know, culture war things. Alright, we're going to</p><p>[01:33:14] <strong>Dylan Patel:</strong> move into rapid fire Q& A, so we're going to ask you questions. We've cut</p><p>[01:33:26] <strong>Dylan Patel:</strong> the Q& A section for time, so if you want to hear the spicy questions, head over to the Thursday Nights in AI video for the full discussion.</p><p>[01:33:34] Dylan Patel - Semianalysis + Latent Space Live Show</p><p>[01:33:34] <strong>Dylan Patel:</strong> Next up, we have another former guest, Dylan Patel of Semianalysis, the inventor of the GPU rich poor divide, who did a special live show with us in March. But that means you can finally, like, side to side A B test your favorite Boba</p><p>[01:33:51] <strong>Alessio:</strong> shops?</p><p>[01:33:51] <strong>Alessio:</strong> We got Gong Cha, we got Boba Guys, we got the Lemon, whatever it's called. So, let us know what's your favorite. We also have Slido up to submit questions. We already had Dylan on the podcast, and like, this guy tweets and writes about all kinds of stuff. So we want to know what people want to know more</p><p>[01:34:07] <strong>Alessio:</strong> about.</p><p>[01:34:08] <strong>Alessio:</strong> Rather than just being self, self driven. But we'll do A state of the union, maybe? I don't know. Everybody wants to know about Grok. Everybody wants to know whether or not NVIDIA is going to zero after Grok. Everybody wants to know what's going on with AMD. We got some AMD folks in the crowd, too.</p><p>[01:34:23] <strong>Alessio:</strong> So feel free to interact at any time. This is We have</p><p>[01:34:27] <strong>swyx + Josh Albrecht:</strong> portable mics.</p><p>[01:34:27] <strong>Dylan Patel:</strong> Heckle, please. What do you sorry. Good comedians show their color when with the way they can handle the crowd when they're heckled.</p><p>[01:34:35] <strong>Alessio:</strong> Do not throw Boba. Do not throw Boba at this end. We cannot afford another podcasting setup. Awesome.</p><p>[01:34:41] <strong>Alessio:</strong> Well, welcome everybody to the Semi Analysis and Latest Space Crossover. Dylan texted me on Signal. He was like, dude, how do I easily set up a meetup? And here we are today. Well, as you might have seen, there's no name tags. There's a bunch of things that are missing. But we did our</p><p>[01:34:55] <strong>Dylan Patel:</strong> best. It was extremely easy, right?</p><p>[01:34:58] Groq</p><p>[01:34:58] <strong>Dylan Patel:</strong> Like, I text Alessio. He's like, yo, I got the spot. Okay, cool. Thanks Here's a link. Send it to people. Sent it. And then showed up. And like, there was zero other organization that I required. So</p><p>[01:35:10] <strong>Alessio:</strong> everybody's here. A lot of, a lot of Semi Analysis fans we get in the crowd everybody wants to know more about what's going on today, and Grok has definitely been the hottest thing.</p><p>[01:35:19] <strong>Alessio:</strong> We just recorded our monthly podcast today, and we didn't talk that much about Grok because we wanted you to talk more about it, and then we'll splice you into our, our monthly recap. So, let's start there.</p><p>[01:35:29] <strong>swyx + Josh Albrecht:</strong> Okay, so, You guys, you guys are the new Grok spreadsheet ers. Yeah, yeah, so, so, we, we we broke out some Grok numbers because everyone was wondering, there's two things going on, right?</p><p>[01:35:37] <strong>swyx + Josh Albrecht:</strong> One you know, how important, or how does it achieve the inference speed that it does? That, that has been demonstrated by GrokChat. And two, how does it achieve its price promise that is promised that, that is sort of the public pricing of 27 cents per million token. And there's been a lot of speculation or, you know, some numbers thrown out there.</p><p>[01:35:55] <strong>swyx + Josh Albrecht:</strong> I put out some tentative numbers and you put out different numbers. But I'll just kind of lay that as, as the, as the groundwork. Like, everyone's like very excited about essentially like five times faster. Token generation than any other LLM currently. And that unlocks interesting downstream possibilities if it's sustainable, if it's affordable.</p><p>[01:36:14] <strong>swyx + Josh Albrecht:</strong> And so I think your question, or reading your piece on Grok, which is on the screen right now, is it sustainable?</p><p>[01:36:21] <strong>Dylan Patel:</strong> So like many things, this is VC funded, including this Boba. No, I'm just kidding, I'm paying for the Bobo, so but, but Thank you semi analysis</p><p>[01:36:29] <strong>swyx + Josh Albrecht:</strong> subscribers</p><p>[01:36:31] <strong>Alessio:</strong> I hope he pays for it, I pay for it right now That's</p><p>[01:36:33] <strong>Dylan Patel:</strong> true, that's true Alessio has the IOU, right?</p><p>[01:36:36] <strong>Dylan Patel:</strong> And that's, that's all it is, but yeah, like many things, you know, they're, they're not making money off of their inference service, right? They're just throwing it out there for cheap and hoping to get business and maybe raise money off of that, and I think that's a that's a fine use case, but the question is, like, how much money are they losing?</p><p>[01:36:53] <strong>Dylan Patel:</strong> Right, and, and that's sort of what I went through breaking down in this this article that's on the screen. And it's, it's pretty clear they're like 7 to 10x off, like, break even on their inference API, which is like horrendous, like far worse than any other sort of inference API provider. So this is like a simple, simple cost thing that was pulled up.</p><p>[01:37:15] <strong>Dylan Patel:</strong> You can either inference at very high throughput, or you can inference at very high, very low latency.</p><p>[01:37:20] <strong>Dylan Patel:</strong> With GPUs, you can do both. With Grok, you can only do one. Of course, with Grok, you can do that one faster. Marginally faster than a inference latency optimized GPU server. But no one offers inference latency optimized GPU servers because you would just burn money, right? Makes no economic sense to do so.</p><p>[01:37:36] <strong>Dylan Patel:</strong> Until maybe someone's willing to pay for that. So, so Grok service, you know, on the surface looks awesome compared to everyone else's service, which is throughput optimized. And, and then when you compare to the throughput optimized scenario, right, GPUs look quite slow, but the reality is they're serving, you know, 64, 128 users at once.</p><p>[01:37:54] <strong>Dylan Patel:</strong> Right, they're, they have a batch size, right? How many users are being served at once, whereas Grok Taking 576 chips, and they're not really doing that efficiently, right? You know, they're, they're serving a far, far fewer number of users, but extremely fast. Now, that could be worthwhile if they can get their, you know, the number of users they're serving at once up, but that's extremely hard because they don't have memory on their chip, so they can't store KV cache KV cache for, you know, all the various different users.</p><p>[01:38:21] <strong>Dylan Patel:</strong> And so, so the crux of the issue is just like, hey, So, can they, can they get that performance up as much as they claim they will, right? Which is, you know, they need to get it up more than 10x, right? To, to, to make this like a reasonable benefit, right? In the meantime, NVIDIA's launching a new GPU in two weeks that'll be fun at GTC and they're constantly pushing software as well, so we'll see if, if Grok can catch up to that.</p><p>[01:38:43] <strong>Dylan Patel:</strong> But the, the current verdict is, you know, they're, they're quite far behind, but it's hopeful, you know, that, that maybe they can get there by, you know, scaling their system larger. Yeah.</p><p>[01:38:52] <strong>swyx + Josh Albrecht:</strong> I was listening back to our original episode, and you were talking about how NVIDIA basically adopted this different strategy of just leaning on networking GPUs together.</p><p>[01:39:00] <strong>swyx + Josh Albrecht:</strong> And it seems like Grok has some, like, minor version of that going on here with the Grok rack. Is it enough? Like, what's Grok's next step here, like,</p><p>[01:39:12] <strong>Dylan Patel:</strong> strategically? Yeah, that's the next step is, of course, you know, so, you know, So right now they connect 10 racks of chips together, right, and that's the system that's running on their API today, right.</p><p>[01:39:23] <strong>Dylan Patel:</strong> Whereas most people who are running, you know, Mistral are running it on two GPUs, right. So one fourth of a server. Yeah. And that rack is not you know, obviously 10 racks is pretty crazy, but they think that they can scale performance if they have this individual system be 20 racks, right? They think they can continue to scale performance extra linearly.</p><p>[01:39:42] <strong>Dylan Patel:</strong> So that'd be amazing if they could but I, I, I'm, I'm doubtful that that's gonna be something that's scalable especially for, for, you know, larger models. So there's the</p><p>[01:39:56] <strong>Alessio:</strong> chip itself, but there's also a lot of work they're doing at the compiler level. Do you have any good sense of, like, how easy it is to actually work with LPU?</p><p>[01:40:04] <strong>Alessio:</strong> Like, is that something that is going to be a bottleneck for them?</p><p>[01:40:07] <strong>Dylan Patel:</strong> So, so Ali's in the front right there, and he, he knows a ton about about VLIW architectures. But to summarize sort of his opinion, and I think many folks's, it's, it's extremely hard to To program these sorts of architectures, right?</p><p>[01:40:19] <strong>Dylan Patel:</strong> Which is why they have their compiler and so on and so forth. But, you know, it's, it's an incredible amount of work for them to stand up individual models and to get the performance up on them which is what they've been working on, right? Whereas, whereas, you know, GPUs are far more flexible, of course.</p><p>[01:40:33] <strong>Dylan Patel:</strong> And so the question is, you know, can they, can they can, can this compiler continue to extract performance? Well, theoretically, like there, there's a lot more performance to run on the hardware. But they don't have, you know, many, many things that people generally associate with, with programmable hardware.</p><p>[01:40:49] <strong>Dylan Patel:</strong> Right? They don't have buffers and, and many other things. So, so it makes it very tough to to do that. But that's what their, you know, their relatively large compiler team is working on. Yeah,</p><p>[01:40:58] <strong>swyx + Josh Albrecht:</strong> So I'm, I'm not a GPU compiler guy. But I do want to clarify my understanding from what I read. Which is a lot of catching up to do.</p><p>[01:41:05] <strong>swyx + Josh Albrecht:</strong> It is, The crux of it is some kind of speculative, like I, in the, the word that comes to mind is speculative routing of weights and, you know, and, and work that, that needs to be done, or scheduling of work across the, you know, the 10 racks of, of GPUs. Is that the, is that like the, the, the bulk of the benefit that you get from</p><p>[01:41:25] <strong>Dylan Patel:</strong> the compilation?</p><p>[01:41:26] <strong>Dylan Patel:</strong> So, so with the Grok chips, what's really interesting is like with GPUs you can do, you can issue certain instructions. And you will get a different result. Like, depending on the time, I know a lot of people in ML have, have had that experience, right? Where like, the GPU literally doesn't return the numbers it should be.</p><p>[01:41:45] <strong>Dylan Patel:</strong> And that's basically called non determinism, right? And, and, and the, and, and, with, with Grok, their chip is completely deterministic. The moment you compile it, you know exactly how long it will take to operate, right? There is no, there is no, like, deviation at all. And so, you know, they've, they're planning everything ahead of time, right, like, every instruction, like, it will complete in the time that they've planned it for.</p><p>[01:42:08] <strong>Dylan Patel:</strong> And there is no I don't know, I don't know what the best way to state this is. There's no variance there which is interesting from, like, when you look historically, they tried to push this into automotive, right? Because automotive, you know, you probably want your car to do exactly what you issued it to do.</p><p>[01:42:22] <strong>Dylan Patel:</strong> And not have, sort of, unpredictability. But yeah, I don't, sorry, I lost track of the question.</p><p>[01:42:28] <strong>swyx + Josh Albrecht:</strong> It's okay, I just wanted to understand a little bit more about, like, what people should under, should know about the compiler magic that goes on with Brock. Like, you know, like, I think, I think, from a software, like, under, like, hardware point of view that in, that intersection of, you know,</p><p>[01:42:44] <strong>Dylan Patel:</strong> So, so, so chips have like, like and I'm going to steal this from someone here in the crowd, but chips have like five, you know, sort of, there's like, when you're designing a chip, there's, there's, it's called PPA, right?</p><p>[01:42:54] <strong>Dylan Patel:</strong> Power, performance, and area, right? So it's kind of a triangle that you optimize around. And the one thing people don't realize is there's a, there's a third P that's like PPAP. And the last P is pain in the ass to program. And, and that's that is very important for like. People making AI hardware, right?</p><p>[01:43:11] <strong>Dylan Patel:</strong> Like, TPU, without the hundreds of people that work on the compiler, and JAX, and XLA, and all these sorts of things, would be a pain in the ass to program. But Google's got that, like, plumbing. Now, if you look across the ecosystem, everything else is a pain in the ass to program compared to NVIDIA, right? And, and, and this applies to the, to the Grok chip as well, right?</p><p>[01:43:31] <strong>Dylan Patel:</strong> So, yeah, question is, like, can the compiler team get performance up anywhere close to theoretical? And then, and then can they make it not a pain in the ass to support new models? Cool. We</p><p>[01:43:41] <strong>Alessio:</strong> got a question, we got a question from Ali. What's the average VLIW bundle occupancy of Grok? Bro,</p><p>[01:43:49] <strong>Dylan Patel:</strong> get out of here.</p><p>[01:43:52] <strong>Alessio:</strong> I don't know if he's setting you up, or if he</p><p>[01:43:54] <strong>Dylan Patel:</strong> wants to chime in. I think he's setting me up, I think he's setting me up. So, okay,</p><p>[01:43:58] <strong>swyx + Josh Albrecht:</strong> what is VLIW for</p><p>[01:44:00] <strong>Dylan Patel:</strong> the rest of us? It's, it's like very long instruction word is basically what it means. And, hm. So, so, GPUs are relatively simple, right? They're, they're tiny little cores, very simple instructions, there's a shitload of them, right?</p><p>[01:44:16] <strong>Dylan Patel:</strong> CPUs, you know, they have a, they have a known instruction set, right? x86. It's very complicated but people have worked on it for decades. VLIW processors are very unique in that sense, right? Like and your question, Ali, I cannot answer that question. I have no clue. Is it documented anywhere online?</p><p>[01:44:35] <strong>Dylan Patel:</strong> Anyway, so like the systolic array, right? Like there's, within the TPU, there's a bunch of stuff, but the actual matrix multiply unit, it's called the MXU, and it's a VLIW architecture as well. It's and I'm, I'm just trying to find a, yeah, I just want to find something that makes me not sound like an idiot.</p><p>[01:44:51] <strong>swyx + Josh Albrecht:</strong> Sometimes I also like to ballpark things in terms of like, like where a good middle median value should be and where like a good high value should be. Sorry. You, you, you</p><p>[01:45:03] <strong>Dylan Patel:</strong> can ballpark things like, you know, like, yeah, so, so, so, but basically like the, the point is like you're trading this is optimal, this is theoretically the most optimal architecture for performance power and area in a given, and you know, not, not specifically Grok, but VLIW in general is gonna get you closer to optimal there, but then you're giving off, you know, that, that last P, which is pain in the ass program, is, is I think the most simple way to get into it.</p><p>[01:45:27] <strong>Dylan Patel:</strong> There's like, computer architecture books about this, but it's, it's, it's a little little, little complicated, right? Yeah.</p><p>[01:45:35] <strong>Alessio:</strong> Somebody asked, there's a lot of questions, that's great. Can we talk about LPU, Cerebrus, Tenstorin, some of these other architectures. How should people think about Maxim, SRAM versus Mix versus</p><p>[01:45:49] <strong>Dylan Patel:</strong> Yeah, yeah.</p><p>[01:45:50] <strong>Dylan Patel:</strong> So there's a lot of ML hardware out there, new and old, right? There's old stuff that's trying to compete there's new stuff that's coming up, you know, companies like, like MadX and Lumerium Labs and so on and so forth, right? You know but, but, so, so there's like a continuum of like, everyone before, say, two years ago that was doing ML hardware bet in one direction, right?</p><p>[01:46:11] <strong>Dylan Patel:</strong> We're gonna make it as an architecture that is, that is has more on chip memory than NVIDIA, right? Like, that was the general bet everyone made. Right? And so like Grok made that bet, they made it to the extreme, right? They didn't have any off chip memory at all. Only on chip memory. You have, you have Cerebrists who did a similar thing except, they were like, Yeah, we're gonna have on chip memory, but we're gonna make a chip that's the size of a wafer.</p><p>[01:46:33] <strong>Dylan Patel:</strong> Right? Like literally this big. Whereas an NVIDIA chip is roughly this big, right? So it's like this big, it's the only chip in the world that's that big. But again, same bet. More on chip memory, less off chip, right? GraphCore and SambaNova made a similar bet. And, and every, basically everyone made that bet.</p><p>[01:46:49] <strong>Dylan Patel:</strong> Cause they thought that's where ML would go. Of course, models grew faster than anyone ever imagined. Yeah, than the memory that was possible. And so that, that very quickly became the wrong bet. And so now we're, you know, sort of seeing a new wave of startups that are going to bet on the other side, as well as many other, you know, architectural things because memory is not really the only architectural thing, of course.</p><p>[01:47:08] <strong>Dylan Patel:</strong> And so, like, where to, where to, like, place startups is, is very dependent on, like, Hey, what are you doing differently than NVIDIA? And is NVIDIA just going to implement that in their chip next year, right? Or, or some version of that. That's, like, pretty much the only things to think about when looking at, you know, hardware companies now.</p><p>[01:47:27] <strong>Dylan Patel:</strong> Cool.</p><p>[01:47:28] <strong>Alessio:</strong> And, yeah, I, I think the, the question is like, there's the size of the models that got outrun, but now you're doing all this work at the compiler level, but it's very transformer based, everything they're doing on the optimization side. How, how do you think about that risk? Like, do you think it's okay for like a hardware company to take like architectural risk in terms of like, yeah, we assume transformers in two years, they'll still be pretty good.</p><p>[01:47:51] <strong>Alessio:</strong> But when you're like depreciating some of this cost of our life. For five years as a buyer.</p><p>[01:47:56] <strong>Dylan Patel:</strong> Yeah, yeah, that's, that's the biggest challenge with like some of the specialized hardware, right? It's like, I know my GPUs will be useful in four years or five years. Maybe not, like, super useful, but they'll be useful for something.</p><p>[01:48:07] <strong>Dylan Patel:</strong> But, there's no way to know that my hardware is going to be able to operate on whatever new model architecture that comes out in the next few years, right? Like, I, I, I like to joke transformers are all you need. And like everything else is like a waste of time. But, you know, I'm sure something better will come.</p><p>[01:48:26] <strong>Dylan Patel:</strong> Right? And, and, you know, you gotta have like, hardware is expensive and you own it for many years. Right? So you can't just like buy whatever's best for today's workload one time and then assume that workload is gonna stay stagnant. Cause that's a recipe to have your like hardware useless as soon as like things evolve.</p><p>[01:48:43] <strong>Dylan Patel:</strong> Right? Like imagine if someone like had hardware for LTSMs and. 2016 or whatever, right? Like, LSTMs. Yeah, LSTM, sorry. You look like an idiot, right? Because now it's not gonna work for, you know, the next architecture, right? As soon as BERT came out, right? For example. So yeah, it's, it's very anything super, super specialized is always at risk of, of being sort of obsoleted and useless.</p><p>[01:49:06] <strong>Dylan Patel:</strong> And, and sort of that's, that's the, that's the thought that like, hey, like, like Graphcore, right? Their chips are. Pretty decent at GNNs, right? Graph Neural Networks. They're actually pretty decent at that. But no one cares, right? So, congratulations, right? Like, you won, you won like the shortest midget, right?</p><p>[01:49:24] <strong>swyx + Josh Albrecht:</strong> Mentioning transformers is all you need. Gives us a nice opportunity to bring out one of your old tweets, but also mention Gemini. My old</p><p>[01:49:30] <strong>Dylan Patel:</strong> tweets, I'm scared. Recent</p><p>[01:49:33] <strong>swyx + Josh Albrecht:</strong> tweets. There's a lot of people talking about, like I think you had a tweet commenting on Gemini 1. 5. And the million token context where basically everyone was saying, like, okay, we need Mamba, we need RLUKV, or we need some other alternative architecture to scale to long context.</p><p>[01:49:48] <strong>swyx + Josh Albrecht:</strong> And Google comes out and says, no, we just, we scaled transformers to 10 million tokens. Easy. We, and, you know, like, I, I think that, that kind of, like, reflects on your thesis there a</p><p>[01:49:59] <strong>Dylan Patel:</strong> little bit. I guess, yeah. I mean, I don't know if I, if I have a coherent thesis, but it's, it's sure fun to, it's Who, who think that like, I, I, I just have an intense hatred for RAG.</p><p>[01:50:11] <strong>Dylan Patel:</strong> Right, like retrieval augmented generation is, is, is like the most like, I just have an intense like innate hatred for it. Wait, wait, you retweeted me</p><p>[01:50:18] <strong>swyx + Josh Albrecht:</strong> defending RAG in the White House press release. Yeah, yeah, yeah. Okay.</p><p>[01:50:21] <strong>Dylan Patel:</strong> But it's just fun,</p><p>[01:50:22] <strong>swyx + Josh Albrecht:</strong> it's all fun and games. Yeah, yeah, yeah, it's all fun and games.</p><p>[01:50:24] <strong>Dylan Patel:</strong> Yeah.</p><p>[01:50:25] <strong>Dylan Patel:</strong> No, no, no, I retweeted, I retweeted you because you memed the White House. I don't know if y'all saw the meme. Can you pull it up? Sure. Like the, the White House the White House put out this thing about like, They're getting very opinionated with this White House. Memory safety. I think it was effectively like, C is bad and Rust is good.</p><p>[01:50:39] <strong>Dylan Patel:</strong> It was like pretty wild that the White House put that out. And I mean like, like whatever that is, so, so, So</p><p>[01:50:46] <strong>swyx + Josh Albrecht:</strong> like, they just got very opinionated about prescribing languages to people. And so then I was, I just like started editing them. So I have stopped comparing RAG with long context and fine</p><p>[01:50:54] <strong>Dylan Patel:</strong> tuning.</p><p>[01:50:55] <strong>Dylan Patel:</strong> Wait, You said I retweeted you defending it. I thought you were hating on it. And that's why I retweeted it.</p><p>[01:51:00] <strong>swyx + Josh Albrecht:</strong> It's somewhat of a defense. Because everyone was like long context is killing RAG. And then I had future LLM should be sub quadratic. That's another one. And I actually messed with the fine print as well..</p><p>[01:51:11] <strong>Alessio:</strong> Let's see power benefits of SRAM dominant</p><p>[01:51:13] <strong>Dylan Patel:</strong> Yeah, yeah. So, so that's a good question, right? So, like, SRAM is on chip memory. Everyone's just using HBM. If you don't have to go to off chip memory, that'd be really efficient, right?</p><p>[01:51:23] <strong>Dylan Patel:</strong> Cause, cause you're, you're not moving bits around. But there's always the issue of you don't have enough memory, right? So, so you still have to move bits around constantly. And so that's the, that's the question. So, yeah, sure. If you, if you can not move data around as you compute, it's going to be fantastically efficient.</p><p>[01:51:39] <strong>Dylan Patel:</strong> That isn't really not really just easy or simple to do.</p><p>[01:51:42] <strong>Alessio:</strong> What do you think is going to be harder in the future, like getting more energy at cheaper costs or like getting more of this hardware</p><p>[01:51:48] <strong>Dylan Patel:</strong> to run? Yeah, I wonder, so someone was talking about this earlier but it's like here in the crowd and I'm looking right at him but he's complaining that journalists keep saying that you know, that, that, like misreporting about how data centers, or what data centers are doing to the environment.</p><p>[01:52:03] <strong>Dylan Patel:</strong> Right? Which I thought was quite funny, right? Cause, cause they're inundated by journalists talking about data centers like destroying the world. Anyways you know, that's not quite the case, right? But yeah, I don't know, like, the, the, the power is certainly going to be hard to get, but, you know, I think, I think if you just look at history, right?</p><p>[01:52:22] <strong>Dylan Patel:</strong> Like humanity, especially America, right? Like, power, power production and usage kept skyrocketing. From like the 1700s to like 1970s, and then it kind of flatlined from there, so why can't we like go back to the like growth stage, I guess is like the whole like mantra of like accelerationists, I guess.</p><p>[01:52:40] <strong>Dylan Patel:</strong> This is EAC, yep. Well I don't think it's EAC, I think it's like, like Sam Altman like wholly believes this too, right? Yeah. And I don't think he's EAC. So, but yeah, like, like, I don't think like, it's like things, it's like something to think about, right? Like. The US is going back to growing in energy usage whereas for the last like 40 years kind of were flat on energy usage.</p><p>[01:53:00] <strong>Dylan Patel:</strong> And what does that mean, right? Like, yeah.</p><p>[01:53:04] <strong>Alessio:</strong> Fair enough. There was another question on Marvel but kind of the, I think</p><p>[01:53:07] <strong>Dylan Patel:</strong> that's it's, it's, it's definitely like one of these three guys who are on the buy side that are asking this question. What, what, what you want to know if Marvel's stock is gonna go up?</p><p>[01:53:18] <strong>Dylan Patel:</strong> Yeah. So Marvell,</p><p>[01:53:19] <strong>Alessio:</strong> the, they're, they're doing the custom music for, for grok. They also do the tri too. And the, the Google CPU. Yeah. Any other, any other chip that they're working on that people should, should keep in mind. It's like, yeah. Any needle moving and it's any stock moving .</p><p>[01:53:34] <strong>Dylan Patel:</strong> Yeah, exactly. Exactly. They're, they're working on some more stuff.</p><p>[01:53:38] <strong>Dylan Patel:</strong> Yeah. I, I'll, I'll, I'll refrain from,</p><p>[01:53:40] <strong>Alessio:</strong> yeah. All right. Let's see other grok stuff we want to get it, get through. I don't think so. Alright, most of the other ones. Your view on edge compute hardware. Any real use cases for it?</p><p>[01:53:54] <strong>Dylan Patel:</strong> Yeah, I mean, I, I I have like a really like anti edge view. Yeah, let's hear it.</p><p>[01:53:58] <strong>Dylan Patel:</strong> Like, like, so many people are like, oh, I'm going to run this model on my phone or on my laptop and. I love how much it's raining. So now I can be horrible and you people won't leave. Like, I want you to try and leave this building. Captive audience. Seriously, should I start singing? Like, there's nothing you</p><p>[01:54:17] <strong>Alessio:</strong> can do.</p><p>[01:54:18] <strong>Alessio:</strong> You definitely, I'll stop you from that.</p><p>[01:54:19] <strong>Dylan Patel:</strong> Sorry, so edge hardware, right? Like, you know, people are like, I'm going to run this model on my phone or my laptop. It makes no sense to me. Cause Current hardware is not really capable of it. So you're gonna buy new hardware, to run whatever on the edge or you're gonna just run very, very small models.</p><p>[01:54:36] <strong>Dylan Patel:</strong> But in either case, you're, you're gonna end up with like the performance is really low, And then whatever you spent to run it locally, Like if you spent it in the cloud, it could service 10x the users, right? So you kind of like, SOL in terms of like, Economics of, of running things on the edge. And then like latency is like, for, for LLMs, right, for LLMs, it's like not that big of a deal relative to, like internet latency is not that big of a deal relative to the use of the model, right?</p><p>[01:55:08] <strong>Dylan Patel:</strong> Like the actual model operating, whether it's on edge hardware or cloud hardware. And cloud hardware is so much faster. So like edge hardware is not really able to like, have a measurable, appreciable, like advantage. Over, over cloud, cloud hardware. This applies to diffusion models, this applies to LLMs of course small models will be able to run, but not, not all, yeah.</p><p>[01:55:33] <strong>Dylan Patel:</strong> Cool.</p><p>[01:55:35] <strong>Alessio:</strong> Let's see. I guess you, you can now see them. Yeah, what chance do startups like MetaX fetch, or 5. 6? Haven't you</p><p>[01:55:41] <strong>swyx + Josh Albrecht:</strong> already reviewed</p><p>[01:55:41] <strong>Dylan Patel:</strong> them? Why don't you, why don't you answer? Yeah, we, we</p><p>[01:55:43] <strong>swyx + Josh Albrecht:</strong> actually, like, we have, Connections with Maddox and Lemurian. Yeah, yeah, yeah. We haven't, no. But Gavin is</p><p>[01:55:52] <strong>Alessio:</strong> Yeah, yeah, they said they don't want to talk publicly.</p><p>[01:55:55] <strong>Alessio:</strong> Oh, okay, okay.</p><p>[01:55:57] <strong>swyx + Josh Albrecht:</strong> When they open up, we can Sure,</p><p>[01:56:00] <strong>Alessio:</strong> sure. But do you think, like, I think the two,</p><p>[01:56:02] <strong>Dylan Patel:</strong> three Answer the question! What do you think of them?</p><p>[01:56:06] <strong>Alessio:</strong> I think, kind of, there's a couple things. It's like How do the other companies innovate against them? I think when you do a new Silicon, you're like, Oh, we're going to be so much better at this thing or like much faster, much cheaper.</p><p>[01:56:18] <strong>Alessio:</strong> But there's all the other curves going down on the macro environment at the same time. So if it takes you like five years before you were like a lot better, five years later, once you take the chip out, you're only comparing yourself to the five year advancement that the major companies had to. So then it's like, okay, the, we're going to have like the C300, whatever, from, from NVIDIA.</p><p>[01:56:37] <strong>Alessio:</strong> By the time some of these chips come up.</p><p>[01:56:40] <strong>Dylan Patel:</strong> What's after Z? What do you think is after Z in the road map? Because it's X, Y, Z, Anyways Yeah, yeah, it's like the age old problem, right? Like you build a chip, it has some cool thing, cool feature, and then like, a year later, NVIDIA has it in hardware, right? Has implemented some flavor of that in hardware.</p><p>[01:57:01] <strong>Dylan Patel:</strong> Or two generations out, right? Like, what idea are you going to have that NVIDIA can't implement, is like, really the question. It's like, you have to be fundamentally different in some way that holds through for, you know, four or five years, right? That's kind of the big issue. But, you know, like, those people have some ideas that are interesting, and yeah, maybe it'll work out, right?</p><p>[01:57:21] <strong>Dylan Patel:</strong> But it's going to be hard to fight NVIDIA, who one, doesn't consider them competition, right? They're worried about, like, Google and Amazon's chip. Right, they're not, and I guess to some extent AMD's chip, but like they're not really worried about you know, MADX or Etched or Grok or, you know, Positron or any of these folks.</p><p>[01:57:39] <strong>Alessio:</strong> How much of an advantage do they have by working closely with like OpenAI folks and then already knowing where some of the architecture decisions are going? And since those companies are like the biggest buyers and users of the</p><p>[01:57:51] <strong>Dylan Patel:</strong> chips, Yeah, I mean, like, you see, like, the most important sort of AI companies are obviously going to tell hardware vendors what they want you know, open AI and, you know, so on and so forth, right?</p><p>[01:58:02] <strong>Dylan Patel:</strong> They're just going to obviously tell them what they want and the startups aren't actually going to get anywhere close to as much feedback on what to do on, like, you know, very minute, low level stuff, right? So that's, that's the, that is a difficulty, right? Some startups, like, like, Maddox obviously have people who built, or worked on the largest models, like at Google, but then other startups might not have that advantage and so they're always gonna have that issue of like, hey, how do I get the feedback, or what's changing, what do they see down the pipeline that's, that I really need to be aware of and ready for when I design my hardware.</p><p>[01:58:37] <strong>Dylan Patel:</strong> Alright.</p><p>[01:58:38] <strong>Alessio:</strong> Every hardware shortage has eventually turned into a glut. Well, that'd be true of NVIDIA chips, it's so when, but also why.</p><p>[01:58:45] <strong>Dylan Patel:</strong> Absolutely, and I'm so excited to buy like H100s for like 1, 000, guys. No, that's not 000, but Yeah, everyone's gonna buy chips, right? Like, it's just the way semiconductors work, because the supply chain takes forever to build out.</p><p>[01:58:58] <strong>Dylan Patel:</strong> And it's, it's like a really weird thing, right? Like, so, so if the backlog of chips is a year, people will order, you know, Two years worth of what they want for the next year. It is like a very common thing. It's not just like this AI cycle, but like, like, like microcontrollers, right? Like the automotive companies, they order two years worth of what they needed for one year, just so they could get enough, right?</p><p>[01:59:21] <strong>Dylan Patel:</strong> Like, this is just like what happens in semiconductors when, when lead times lengthen, the, the purchases and inventory is sort of like double. Sorry. So, so these. The, the NVIDIA GPU shortage obviously is going to be rectified. And when it is everyone's sort of double orders will become extremely apparent, right?</p><p>[01:59:42] <strong>Dylan Patel:</strong> And, you know, you, you see like random companies out of nowhere being like, Yeah, we've got 32, 000 H100s on order, or we've got 10, 000 or 5, 000. And trust, they're not all they're not all real orders for one, but I think, I think the like bubble will continue on for a long time, right, like it's not, it's not going to end like this year, right, like people, people need AI, right, like I think everyone in this audience would agree, right, like there's no, there's no like immediate like end to the, to the bubble, right.</p><p>[02:00:09] <strong>Dylan Patel:</strong> Party like we're in 1995, not like 2000. Makes sense.</p><p>[02:00:12] <strong>Alessio:</strong> What's next? Thoughts on VLIW</p><p>[02:00:16] <strong>Dylan Patel:</strong> architectures? Oh, Y, Y, sorry, sorry, Y. The Y question, yeah, yeah. I think it's just because the supply chain expands so much, and then at the same time there will be no, like, economic, like, immediate economic thing for everyone, right?</p><p>[02:00:28] <strong>Dylan Patel:</strong> Like, some companies will continue to buy, like like an OpenAI or Meta will continue to buy, but then, like, All these random startups will, or a lot of them will not be able to continue to buy, right? So then, so then that like kind of leads to like, they'll pause for a little bit, right? Or like, I think in 2018, right?</p><p>[02:00:45] <strong>Dylan Patel:</strong> Like memory pricing was extremely high. Then all of a sudden Google, Microsoft, and Amazon all agreed, I don't, you know, You know, they don't, they won't, they won't say it's together, but they basically all agreed it like, within the same week to stop ordering memory. And within like a month, the price of memory started tanking like insane amounts, right?</p><p>[02:01:06] <strong>Dylan Patel:</strong> And like people claim, you know, all sorts of reasons why that was timed extremely well. But it was like very clear and people in the financial markets were able to make trades and everything, right? People stopped buying and it's not like their demand just dried up. It's just like they had a little bit of a demand slowdown and then they had enough inventory that they could like weather until like prices tanked.</p><p>[02:01:26] <strong>Dylan Patel:</strong> Because it's such an inelastic good, right? Yeah.</p><p>[02:01:29] <strong>swyx + Josh Albrecht:</strong> Thank you very much. That's it.</p><p>[02:01:35] <strong>AI Charlie:</strong> That concludes our audio segment this weekend. But if you're listening all the way to the end, we have two bonus segments for you. A conversation with Malin Nefe, Senior Vice President of AI at Capital One. We'll be speaking at the AI Leadership Track of the AI Engineer World's Far. And the recent Latent Space Personal AI Meetup featuring a lot of new AI wearables. Bee, Based Hardware, DeepGram MLE AI, and LangChain LangFriend and LangMem, Presented by another former guest, Harrison Chase. Watch out and take care.</p> <br /><br />Get full access to Latent Space at <a href="https://www.latent.space/subscribe?utm_medium=podcast&#38;utm_campaign=CTA_4">www.latent.space/subscribe</a>

[by:whisper.cpp]

[00:00.00](音乐)

[00:06.00]欢迎到达特兰的节目的节目的节目

[00:09.00]这是Charlie,你的AI co-host

[00:12.00]SWIX 和 ALESIO 都在节目中做了更多的节目

[00:16.00]我们有很多新的节目

[00:18.00]来自Elicit、Chroma、 Instructor

[00:20.00]和我们的新的节目在 NSFW

[00:23.00]"不安全的工作AI"

[00:25.00]今天我们会在SWIX 和 ALESIO 的新的节目中

[00:29.00]找到更多新的新的节目

[00:31.00]在我们的第一节目中

[00:33.00]找到更多新的节目

[00:35.00]SWIX 和 ALESIO 都在节目中

[00:37.00]做了更多新的节目

[00:40.00]SWIX 和 ALESIO 都在节目中

[00:42.00]做了更多新的节目

[00:44.00]我们有很多新的节目

[00:46.00]SWIX 和 ALESIO 都在节目中

[00:49.00]找到更多新的节目

[00:52.00]我们有很多新的节目

[00:54.00]SWIX 和 ALESIO 都在节目中

[00:56.00]找到更多新的节目

[00:58.00]SWIX 和 ALESIO 都在节目中

[01:00.00]找到更多新的节目

[01:02.00]SWIX 和 ALESIO 都在节目中

[01:04.00]找到更多新的节目

[01:06.00]SWIX 和 ALESIO 都在节目中

[01:08.00]找到更多新的节目

[01:09.00]SWIX 和 ALESIO 都在节目中

[01:11.00]找到更多新的节目

[01:12.00]SWIX 和 ALESIO 都在节目中

[01:14.00]找到更多新的节目

[01:16.00]SWIX 和 ALESIO 都在节目中

[01:18.00]找到更多新的节目

[01:19.00]SWIX 和 ALESIO 都在节目中

[01:21.00]找到更多新的节目

[01:22.00]SWIX 和 ALESIO 都在节目中

[01:24.00]找到更多新的节目

[01:26.00]SWIX 和 ALESIO 都在节目中

[01:28.00]找到更多新的节目

[01:30.00]SWIX 和 ALESIO 都在节目中

[01:32.00]找到更多新的节目

[01:34.00]SWIX 和 ALESIO 都在节目中

[01:36.00]找到更多新的节目

[01:38.00]SWIX 和 ALESIO 都在节目中

[01:40.00]找到更多新的节目

[01:42.00]SWIX 和 ALESIO 都在节目中

[01:44.00]找到更多新的节目

[01:46.00]SWIX 和 ALESIO 都在节目中

[01:48.00]找到更多新的节目

[01:50.00]SWIX 和 ALESIO 都在节目中

[01:52.00]找到更多新的节目

[01:54.00]SWIX 和 ALESIO 都在节目中

[01:56.00]找到更多新的节目

[01:58.00]SWIX 和 ALESIO 都在节目中

[02:00.00]找到更多新的节目

[02:02.00]SWIX 和 ALESIO 都在节目中

[02:04.00]找到更多新的节目

[02:06.00]SWIX 和 ALESIO 都在节目中

[02:08.00]找到更多新的节目

[02:10.00]SWIX 和 ALESIO 都在节目中

[02:12.00]找到更多新的节目

[02:14.00]SWIX 和 ALESIO 都在节目中

[02:16.00]找到更多新的节目

[02:18.00]SWIX 和 ALESIO 都在节目中

[02:20.00]找到更多新的节目

[02:22.00]SWIX 和 ALESIO 都在节目中

[02:24.00]找到更多新的节目

[02:26.00]SWIX 和 ALESIO 都在节目中

[02:28.00]找到更多新的节目

[02:30.00]SWIX 和 ALESIO 都在节目中

[02:32.00]找到更多新的节目

[02:34.00]SWIX 和 ALESIO 都在节目中

[02:36.00]找到更多新的节目

[02:38.00]SWIX 和 ALESIO 都在节目中

[02:40.00]找到更多新的节目

[02:42.00]SWIX 和 ALESIO 都在节目中

[02:44.00]找到更多新的节目

[02:46.00]SWIX 和 ALESIO 都在节目中

[02:48.00]找到更多新的节目

[02:50.00]SWIX 和 ALESIO 都在节目中

[02:52.00]找到更多新的节目

[02:54.00]SWIX 和 ALESIO 都在节目中

[02:56.00]找到更多新的节目

[02:58.00]SWIX 和 ALESIO 都在节目中

[03:00.00]找到更多新的节目

[03:02.00]SWIX 和 ALESIO 都在节目中

[03:04.00]找到更多新的节目

[03:06.00]SWIX 和 ALESIO 都在节目中

[03:08.00]找到更多新的节目

[03:10.00]SWIX 和 ALESIO 都在节目中

[03:12.00]找到更多新的节目

[03:14.00]SWIX 和 ALESIO 都在节目中

[03:16.00]找到更多新的节目

[03:18.00]SWIX 和 ALESIO 都在节目中

[03:20.00]找到更多新的节目

[03:22.00]SWIX 和 ALESIO 都在节目中

[03:24.00]找到更多新的节目

[03:26.00]SWIX 和 ALESIO 都在节目中

[03:28.00]找到更多新的节目

[03:30.00]SWIX 和 ALESIO 都在节目中

[03:32.00]找到更多新的节目

[03:34.00]SWIX 和 ALESIO 都在节目中

[03:36.00]找到更多新的节目

[03:38.00]SWIX 和 ALESIO 都在节目中

[03:40.00]找到更多新的节目

[03:42.00]SWIX 和 ALESIO 都在节目中

[03:44.00]找到更多新的节目

[03:46.00]SWIX 和 ALESIO 都在节目中

[03:48.00]找到更多新的节目

[03:50.00]SWIX 和 ALESIO 都在节目中

[03:52.00]找到更多新的节目

[03:54.00]SWIX 和 ALESIO 都在节目中

[03:56.00]找到更多新的节目

[03:58.00]SWIX 和 ALESIO 都在节目中

[04:00.00]找到更多新的节目

[04:02.00]SWIX 和 ALESIO 都在节目中

[04:04.00]找到更多新的节目

[04:06.00]SWIX 和 ALESIO 都在节目中

[04:08.00]找到更多新的节目

[04:10.00]SWIX 和 ALESIO 都在节目中

[04:12.00]找到更多新的节目

[04:14.00]SWIX 和 ALESIO 都在节目中

[04:16.00]找到更多新的节目

[04:18.00]SWIX 和 ALESIO 都在节目中

[04:20.00]找到更多新的节目

[04:22.00]SWIX 和 ALESIO 都在节目中

[04:24.00]找到更多新的节目

[04:26.00]SWIX 和 ALESIO 都在节目中

[04:28.00]找到更多新的节目

[04:30.00]SWIX 和 ALESIO 都在节目中

[04:32.00]找到更多新的节目

[04:34.00]SWIX 和 ALESIO 都在节目中

[04:36.00]找到更多新的节目

[04:38.00]SWIX 和 ALESIO 都在节目中

[04:40.00]找到更多新的节目

[04:42.00]SWIX 和 ALESIO 都在节目中

[04:44.00]找到更多新的节目

[04:46.00]SWIX 和 ALESIO 都在节目中

[04:48.00]找到更多新的节目

[04:50.00]SWIX 和 ALESIO 都在节目中

[04:52.00]找到更多新的节目

[04:54.00]SWIX 和 ALESIO 都在节目中

[04:56.00]找到更多新的节目

[04:58.00]SWIX 和 ALESIO 都在节目中

[05:00.00]找到更多新的节目

[05:02.00]SWIX 和 ALESIO 都在节目中

[05:04.00]找到更多新的节目

[05:06.00]SWIX 和 ALESIO 都在节目中

[05:08.00]找到更多新的节目

[05:10.00]SWIX 和 ALESIO 都在节目中

[05:12.00]找到更多新的节目

[05:14.00]SWIX 和 ALESIO 都在节目中

[05:16.00]找到更多新的节目

[05:18.00]SWIX 和 ALESIO 都在节目中

[05:20.00]找到更多新的节目

[05:22.00]SWIX 和 ALESIO 都在节目中

[05:24.00]找到更多新的节目

[05:26.00]SWIX 和 ALESIO 都在节目中

[05:28.00]找到更多新的节目

[05:30.00]SWIX 和 ALESIO 都在节目中

[05:32.00]找到更多新的节目

[05:34.00]SWIX 和 ALESIO 都在节目中

[05:36.00]找到更多新的节目

[05:38.00]SWIX 和 ALESIO 都在节目中

[05:40.00]找到更多新的节目

[05:42.00]SWIX 和 ALESIO 都在节目中

[05:44.00]找到更多新的节目

[05:46.00]SWIX 和 ALESIO 都在节目中

[05:48.00]找到更多新的节目

[05:50.00]SWIX 和 ALESIO 都在节目中

[05:52.00]找到更多新的节目

[05:54.00]SWIX 和 ALESIO 都在节目中

[05:56.00]找到更多新的节目

[05:58.00]SWIX 和 ALESIO 都在节目中

[06:00.00]找到更多新的节目

[06:02.00]SWIX 和 ALESIO 都在节目中

[06:04.00]找到更多新的节目

[06:06.00]SWIX 和 ALESIO 都在节目中

[06:08.00]找到更多新的节目

[06:10.00]SWIX 和 ALESIO 都在节目中

[06:12.00]找到更多新的节目

[06:14.00]SWIX 和 ALESIO 都在节目中

[06:16.00]找到更多新的节目

[06:18.00]SWIX 和 ALESIO 都在节目中

[06:20.00]找到更多新的节目

[06:22.00]SWIX 和 ALESIO 都在节目中

[06:24.00]找到更多新的节目

[06:26.00]SWIX 和 ALESIO 都在节目中

[06:28.00]找到更多新的节目

[06:30.00]SWIX 和 ALESIO 都在节目中

[06:32.00]找到更多新的节目

[06:34.00]SWIX 和 ALESIO 都在节目中

[06:36.00]找到更多新的节目

[06:38.00]SWIX 和 ALESIO 都在节目中

[06:40.00]找到更多新的节目

[06:42.00]SWIX 和 ALESIO 都在节目中

[06:44.00]找到更多新的节目

[06:46.00]SWIX 和 ALESIO 都在节目中

[06:48.00]找到更多新的节目

[06:50.00]SWIX 和 ALESIO 都在节目中

[06:52.00]找到更多新的节目

[06:54.00]SWIX 和 ALESIO 都在节目中

[06:56.00]找到更多新的节目

[06:58.00]SWIX 和 ALESIO 都在节目中

[07:00.00]找到更多新的节目

[07:02.00]SWIX 和 ALESIO 都在节目中

[07:04.00]找到更多新的节目

[07:06.00]SWIX 和 ALESIO 都在节目中

[07:08.00]找到更多新的节目

[07:10.00]SWIX 和 ALESIO 都在节目中

[07:12.00]找到更多新的节目

[07:14.00]SWIX 和 ALESIO 都在节目中

[07:16.00]找到更多新的节目

[07:18.00]SWIX 和 ALESIO 都在节目中

[07:20.00]找到更多新的节目

[07:22.00]SWIX 和 ALESIO 都在节目中

[07:24.00]找到更多新的节目

[07:26.00]SWIX 和 ALESIO 都在节目中

[07:28.00]找到更多新的节目

[07:30.00]SWIX 和 ALESIO 都在节目中

[07:32.00]找到更多新的节目

[07:34.00]SWIX 和 ALESIO 都在节目中

[07:36.00]找到更多新的节目

[07:38.00]SWIX 和 ALESIO 都在节目中

[07:40.00]找到更多新的节目

[07:42.00]SWIX 和 ALESIO 都在节目中

[07:44.00]找到更多新的节目

[07:46.00]SWIX 和 ALESIO 都在节目中

[07:48.00]找到更多新的节目

[07:50.00]SWIX 和 ALESIO 都在节目中

[07:52.00]找到更多新的节目

[07:54.00]SWIX 和 ALESIO 都在节目中

[07:56.00]找到更多新的节目

[07:58.00]找到更多新的节目

[08:00.00]SWIX 和 ALESIO 都在节目中

[08:02.00]找到更多新的节目

[08:04.00]SWIX 和 ALESIO 都在节目中

[08:06.00]找到更多新的节目

[08:08.00]SWIX 和 ALESIO 都在节目中

[08:10.00]找到更多新的节目

[08:12.00]SWIX 和 ALESIO 都在节目中

[08:14.00]找到更多新的节目

[08:16.00]SWIX 和 ALESIO 都在节目中

[08:18.00]找到更多新的节目

[08:20.00]SWIX 和 ALESIO 都在节目中

[08:22.00]找到更多新的节目

[08:24.00]SWIX 和 ALESIO 都在节目中

[08:26.00]找到更多新的节目

[08:28.00]SWIX 和 ALESIO 都在节目中

[08:30.00]找到更多新的节目

[08:32.00]SWIX 和 ALESIO 都在节目中

[08:34.00]找到更多新的节目

[08:36.00]SWIX 和 ALESIO 都在节目中

[08:38.00]找到更多新的节目

[08:40.00]SWIX 和 ALESIO 都在节目中

[08:42.00]找到更多新的节目

[08:44.00]SWIX 和 ALESIO 都在节目中

[08:46.00]找到更多新的节目

[08:48.00]SWIX 和 ALESIO 都在节目中

[08:50.00]找到更多新的节目

[08:52.00]SWIX 和 ALESIO 都在节目中

[08:54.00]找到更多新的节目

[08:56.00]SWIX 和 ALESIO 都在节目中

[08:58.00]找到更多新的节目

[09:00.00]SWIX 和 ALESIO 都在节目中

[09:02.00]找到更多新的节目

[09:04.00]SWIX 和 ALESIO 都在节目中

[09:06.00]找到更多新的节目

[09:08.00]SWIX 和 ALESIO 都在节目中

[09:10.00]找到更多新的节目

[09:12.00]SWIX 和 ALESIO 都在节目中

[09:14.00]找到更多新的节目

[09:16.00]SWIX 和 ALESIO 都在节目中

[09:18.00]找到更多新的节目

[09:20.00]SWIX 和 ALESIO 都在节目中

[09:22.00]找到更多新的节目

[09:24.00]空中

[09:26.00]找到更多新的节目

[09:28.00]SWIX 和 ALESIO 都在节目中

[09:30.00]找到更多新的节目

[09:32.00]SWIX 和 ALESIO 都在节目中

[09:34.00]找到更多新的节目

[09:36.00]SWIX 和 ALESIO 都在节目中

[09:38.00]找到更多新的节目

[09:40.00]SWIX 和 ALESIO 都在节目中

[09:42.00]找到更多新的节目

[09:44.00]SWIX 和 ALESIO 都在节目中

[09:46.00]找到更多新的节目

[09:48.00]SWIX 和 ALESIO 都在节目中

[09:50.00]找到更多新的节目

[09:52.00]SWIX 和 ALESIO 都在节目中

[09:54.00]找到更多新的节目

[09:56.00]SWIX 和 ALESIO 都在节目中

[09:58.00]找到更多新的节目

[10:00.00]SWIX 和 ALESIO 都在节目中

[10:02.00]找到更多新的节目

[10:04.00]SWIX 和 ALESIO 都在节目中

[10:06.00]找到更多新的节目

[10:08.00]SWIX 和 ALESIO 都在节目中

[10:10.00]找到更多新的节目

[10:12.00]SWIX 和 ALESIO 都在节目中

[10:14.00]找到更多新的节目

[10:16.00]SWIX 和 ALESIO 都在节目中

[10:18.00]找到更多新的节目

[10:20.00]SWIX 和 ALESIO 都在节目中

[10:22.00]找到更多新的节目

[10:24.00]SWIX 和 ALESIO 都在节目中

[10:26.00]找到更多新的节目

[10:28.00]SWIX 和 ALESIO 都在节目中

[10:30.00]找到更多新的节目

[10:32.00]SWIX 和 ALESIO 都在节目中

[10:34.00]找到更多新的节目

[10:36.00]SWIX 和 ALESIO 都在节目中

[10:38.00]找到更多新的节目

[10:40.00]SWIX 和 ALESIO 都在节目中

[10:42.00]找到更多新的节目

[10:44.00]SWIX 和 ALESIO 都在节目中

[10:46.00]找到更多新的节目

[10:48.00]SWIX 和 ALESIO 都在节目中

[10:50.00]找到更多新的节目

[10:52.00]SWIX 和 ALESIO 都在节目中

[10:54.00]找到更多新的节目

[10:56.00]SWIX 和 ALESIO 都在节目中

[10:58.00]找到更多新的节目

[11:00.00]SWIX 和 ALESIO 都在节目中

[11:02.00]找到更多新的节目

[11:04.00]SWIX 和 ALESIO 都在节目中

[11:06.00]找到更多新的节目

[11:08.00]SWIX 和 ALESIO 都在节目中

[11:10.00]找到更多新的节目

[11:12.00]SWIX 和 ALESIO 都在节目中

[11:14.00]找到更多新的节目

[11:16.00]SWIX 和 ALESIO 都在节目中

[11:18.00]找到更多新的节目

[11:20.00]SWIX 和 ALESIO 都在节目中

[11:22.00]找到更多新的节目

[11:24.00]SWIX 和 ALESIO 都在节目中

[11:26.00]找到更多新的节目

[11:28.00]SWIX 和 ALESIO 都在节目中

[11:30.00]找到更多新的节目

[11:32.00]SWIX 和 ALESIO 都在节目中

[11:34.00]找到更多新的节目

[11:36.00]SWIX 和 ALESIO 都在节目中

[11:38.00]找到更多新的节目

[11:40.00]SWIX 和 ALESIO 都在节目中

[11:42.00]找到更多新的节目

[11:44.00]SWIX 和 ALESIO 都在节目中

[11:46.00]找到更多新的节目

[11:48.00]SWIX 和 ALESIO 都在节目中

[11:50.00]找到更多新的节目

[11:52.00]SWIX 和 ALESIO 都在节目中

[11:54.00]找到更多新的节目

[11:56.00]SWIX 和 ALESIO 都在节目中

[11:58.00]找到更多新的节目

[12:00.00]SWIX 和 ALESIO 都在节目中

[12:02.00]找到更多新的节目

[12:04.00]SWIX 和 ALESIO 都在节目中

[12:06.00]找到更多新的节目

[12:08.00]SWIX 和 ALESIO 都在节目中

[12:10.00]找到更多新的节目

[12:12.00]SWIX 和 ALESIO 都在节目中

[12:14.00]找到更多新的节目

[12:16.00]SWIX 和 ALESIO 都在节目中

[12:18.00]找到更多新的节目

[12:20.00]SWIX 和 ALESIO 都在节目中

[12:22.00]找到更多新的节目

[12:24.00]SWIX 和 ALESIO 都在节目中

[12:26.00]找到更多新的节目

[12:28.00]SWIX 和 ALESIO 都在节目中

[12:30.00]找到更多新的节目

[12:32.00]SWIX 和 ALESIO 都在节目中

[12:34.00]找到更多新的节目

[12:36.00]SWIX 和 ALESIO 都在节目中

[12:38.00]找到更多新的节目

[12:40.00]找到更多新的节目

[12:42.00]找到更多新的节目

[12:44.00]找到更多新的节目

[12:46.00]找到更多新的节目

[12:48.00]找到更多新的节目

[12:50.00]找到更多新的节目

[12:52.00]找到更多新的节目

[12:54.00]找到更多新的节目

[12:56.00]找到更多新的节目

[12:58.00]找到更多新的节目

[13:00.00]找到更多新的节目

[13:02.00]找到更多新的节目

[13:04.00]找到更多新的节目

[13:06.00]找到更多新的节目

[13:08.00]找到更多新的节目

[13:10.00]找到更多新的节目

[13:12.00]找到更多新的节目

[13:14.00]找到更多新的节目

[13:16.00]找到更多新的节目

[13:18.00]找到更多新的节目

[13:20.00]找到更多新的节目

[13:22.00]找到更多新的节目

[13:24.00]找到更多新的节目

[13:26.00]找到更多新的节目

[13:28.00]找到更多新的节目

[13:30.00]找到更多新的节目

[13:32.00]找到更多新的节目

[13:34.00]找到更多新的节目

[13:36.00]找到更多新的节目

[13:38.00]找到更多新的节目

[13:40.00]找到更多新的节目

[13:42.00]找到更多新的节目

[13:44.00]找到更多新的节目

[13:46.00]找到更多新的节目

[13:48.00]找到更多新的节目

[13:50.00]找到更多新的节目

[13:52.00]找到更多新的节目

[13:54.00]找到更多新的节目

[13:56.00]找到更多新的节目

[13:58.00]找到更多新的节目

[14:00.00]找到更多新的节目

[14:02.00]找到更多新的节目

[14:04.00]找到更多新的节目

[14:06.00]找到更多新的节目

[14:08.00]找到更多新的节目

[14:10.00]找到更多新的节目

[14:12.00]找到更多新的节目

[14:14.00]找到更多新的节目

[14:16.00]找到更多新的节目

[14:18.00]找到更多新的节目

[14:20.00]找到更多新的节目

[14:22.00]找到更多新的节目

[14:24.00]找到更多新的节目

[14:26.00]找到更多新的节目

[14:28.00]找到更多新的节目

[14:30.00]找到更多新的节目

[14:32.00]找到更多新的节目

[14:34.00]找到更多新的节目

[14:36.00]找到更多新的节目

[14:38.00]找到更多新的节目

[14:40.00]找到更多新的节目

[14:42.00]找到更多新的节目

[14:44.00]找到更多新的节目

[14:46.00]找到更多新的节目

[14:48.00]找到更多新的节目

[14:50.00]找到更多新的节目

[14:52.00]找到更多新的节目

[14:54.00]找到更多新的节目

[14:56.00]找到更多新的节目

[14:58.00]找到更多新的节目

[15:00.00]找到更多新的节目

[15:02.00]找到更多新的节目

[15:04.00]找到更多新的节目

[15:06.00]找到更多新的节目

[15:08.00]找到更多新的节目

[15:10.00]找到更多新的节目

[15:12.00]找到更多新的节目

[15:14.00]找到更多新的节目

[15:16.00]找到更多新的节目

[15:18.00]找到更多新的节目

[15:20.00]找到更多新的节目

[15:22.00]找到更多新的节目

[15:24.00]找到更多新的节目

[15:26.00]找到更多新的节目

[15:28.00]找到更多新的节目

[15:30.00]找到更多新的节目

[15:32.00]找到更多新的节目

[15:34.00]找到更多新的节目

[15:36.00]找到更多新的节目

[15:38.00]找到更多新的节目

[15:40.00]找到更多新的节目

[15:42.00]找到更多新的节目

[15:44.00]找到更多新的节目

[15:46.00]找到更多新的节目

[15:48.00]找到更多新的节目

[15:50.00]找到更多新的节目

[15:52.00]找到更多新的节目

[15:54.00]找到更多新的节目

[15:56.00]找到更多新的节目

[15:58.00]找到更多新的节目

[16:00.00]找到更多新的节目

[16:02.00]找到更多新的节目

[16:04.00]找到更多新的节目

[16:06.00]找到更多新的节目

[16:08.00]找到更多新的节目

[16:10.00]找到更多新的节目

[16:12.00]找到更多新的节目

[16:14.00]找到更多新的节目

[16:16.00]找到更多新的节目

[16:18.00]找到更多新的节目

[16:20.00]找到更多新的节目

[16:22.00]找到更多新的节目

[16:24.00]找到更多新的节目

[16:26.00]找到更多新的节目

[16:28.00]找到更多新的节目

[16:30.00]找到更多新的节目

[16:32.00]找到更多新的节目

[16:34.00]找到更多新的节目

[16:36.00]找到更多新的节目

[16:38.00]找到更多新的节目

[16:40.00]找到更多新的节目

[16:42.00]找到更多新的节目

[16:44.00]找到更多新的节目

[16:46.00]找到更多新的节目

[16:48.00]找到更多新的节目

[16:50.00]找到更多新的节目

[16:52.00]找到更多新的节目

[16:54.00]找到更多新的节目

[16:56.00]找到更多新的节目

[16:58.00]找到更多新的节目

[17:00.00]找到更多新的节目

[17:02.00]找到更多新的节目

[17:04.00]找到更多新的节目

[17:06.00]找到更多新的节目

[17:08.00]找到更多新的节目

[17:10.00]找到更多新的节目

[17:12.00]找到更多新的节目

[17:14.00]找到更多新的节目

[17:16.00]找到更多新的节目

[17:18.00]找到更多新的节目

[17:20.00]找到更多新的节目

[17:22.00]找到更多新的节目

[17:24.00]找到更多新的节目

[17:26.00]找到更多新的节目

[17:28.00]找到更多新的节目

[17:30.00]找到更多新的节目

[17:32.00]找到更多新的节目

[17:34.00]找到更多新的节目

[17:36.00]找到更多新的节目

[17:38.00]找到更多新的节目

[17:40.00]找到更多新的节目

[17:42.00]找到更多新的节目

[17:44.00]找到更多新的节目

[17:46.00]找到更多新的节目

[17:48.00]找到更多新的节目

[17:50.00]找到更多新的节目

[17:52.00]找到更多新的节目

[17:54.00]找到更多新的节目

[17:56.00]找到更多新的节目

[17:58.00]找到更多新的节目

[18:00.00]找到更多新的节目

[18:02.00]找到更多新的节目

[18:04.00]找到更多新的节目

[18:06.00]找到更多新的节目

[18:08.00]找到更多新的节目

[18:10.00]找到更多新的节目

[18:12.00]找到更多新的节目

[18:14.00]找到更多新的节目

[18:16.00]找到更多新的节目

[18:18.00]找到更多新的节目

[18:20.00]找到更多新的节目

[18:22.00]找到更多新的节目

[18:24.00]找到更多新的节目

[18:26.00]找到更多新的节目

[18:28.00]找到更多新的节目

[18:30.00]找到更多新的节目

[18:32.00]找到更多新的节目

[18:34.00]找到更多新的节目

[18:36.00]找到更多新的节目

[18:38.00]找到更多新的节目

[18:40.00]找到更多新的节目

[18:42.00]找到更多新的节目

[18:44.00]找到更多新的节目

[18:46.00]找到更多新的节目

[18:48.00]找到更多新的节目

[18:50.00]找到更多新的节目

[18:52.00]找到更多新的节目

[18:54.00]找到更多新的节目

[18:56.00]找到更多新的节目

[18:58.00]找到更多新的节目

[19:00.00]找到更多新的节目

[19:02.00]找到更多新的节目

[19:04.00]找到更多新的节目

[19:06.00]找到更多新的节目

[19:08.00]找到更多新的节目

[19:10.00]找到更多新的节目

[19:12.00]找到更多新的节目

[19:14.00]找到更多新的节目

[19:16.00]找到更多新的节目

[19:18.00]找到更多新的节目

[19:20.00]找到更多新的节目

[19:22.00]找到更多新的节目

[19:24.00]找到更多新的节目

[19:26.00]找到更多新的节目

[19:28.00]找到更多新的节目

[19:30.00]找到更多新的节目

[19:32.00]找到更多新的节目

[19:34.00]找到更多新的节目

[19:36.00]找到更多新的节目

[19:38.00]找到更多新的节目

[19:40.00]找到更多新的节目

[19:42.00]找到更多新的节目

[19:44.00]找到更多新的节目

[19:46.00]找到更多新的节目

[19:48.00]找到更多新的节目

[19:50.00]找到更多新的节目

[19:52.00]找到更多新的节目

[19:54.00]找到更多新的节目

[19:56.00]找到更多新的节目

[19:58.00]找到更多新的节目

[20:00.00]找到更多新的节目

[20:02.00]找到更多新的节目

[20:04.00]找到更多新的节目

[20:06.00]找到更多新的节目

[20:08.00]找到更多新的节目

[20:10.00]找到更多新的节目

[20:12.00]找到更多新的节目

[20:14.00]找到更多新的节目

[20:16.00]找到更多新的节目

[20:18.00]找到更多新的节目

[20:20.00]找到更多新的节目

[20:22.00]找到更多新的节目

[20:24.00]找到更多新的节目

[20:26.00]找到更多新的节目

[20:28.00]找到更多新的节目

[20:30.00]找到更多新的节目

[20:32.00]找到更多新的节目

[20:34.00]找到更多新的节目

[20:36.00]找到更多新的节目

[20:38.00]找到更多新的节目

[20:40.00]找到更多新的节目

[20:42.00]找到更多新的节目

[20:44.00]找到更多新的节目

[20:46.00]找到更多新的节目

[20:48.00]找到更多新的节目

[20:50.00]找到更多新的节目

[20:52.00]找到更多新的节目

[20:54.00]找到更多新的节目

[20:56.00]找到更多新的节目

[20:58.00]找到更多新的节目

[21:00.00]找到更多新的节目

[21:02.00]找到更多新的节目

[21:04.00]找到更多新的节目

[21:06.00]找到更多新的节目

[21:08.00]找到更多新的节目

[21:10.00]找到更多新的节目

[21:12.00]找到更多新的节目

[21:14.00]找到更多新的节目

[21:16.00]找到更多新的节目

[21:18.00]找到更多新的节目

[21:20.00]找到更多新的节目

[21:22.00]找到更多新的节目

[21:24.00]找到更多新的节目

[21:26.00]找到更多新的节目

[21:28.00]找到更多新的节目

[21:30.00]找到更多新的节目

[21:32.00]找到更多新的节目

[21:34.00]找到更多新的节目

[21:36.00]找到更多新的节目

[21:38.00]找到更多新的节目

[21:40.00]找到更多新的节目

[21:42.00]找到更多新的节目

[21:44.00]找到更多新的节目

[21:46.00]找到更多新的节目

[21:48.00]找到更多新的节目

[21:50.00]找到更多新的节目

[21:52.00]找到更多新的节目

[21:54.00]找到更多新的节目

[21:56.00]找到更多新的节目

[21:58.00]找到更多新的节目

[22:00.00]找到更多新的节目

[22:02.00]找到更多新的节目

[22:04.00]找到更多新的节目

[22:06.00]找到更多新的节目

[22:08.00]找到更多新的节目

[22:10.00]找到更多新的节目

[22:12.00]找到更多新的节目

[22:14.00]找到更多新的节目

[22:16.00]找到更多新的节目

[22:18.00]找到更多新的节目

[22:20.00]找到更多新的节目

[22:22.00]找到更多新的节目

[22:24.00]找到更多新的节目

[22:26.00]找到更多新的节目

[22:28.00]找到更多新的节目

[22:30.00]找到更多新的节目

[22:32.00]找到更多新的节目

[22:34.00]找到更多新的节目

[22:36.00]找到更多新的节目

[22:38.00]找到更多新的节目

[22:40.00]找到更多新的节目

[22:42.00]找到更多新的节目

[22:44.00]找到更多新的节目

[22:46.00]找到更多新的节目

[22:48.00]找到更多新的节目

[22:50.00]找到更多新的节目

[22:52.00]找到更多新的节目

[22:54.00]找到更多新的节目

[22:56.00]找到更多新的节目

[22:58.00]找到更多新的节目

[23:00.00]找到更多新的节目

[23:02.00]找到更多新的节目

[23:04.00]找到更多新的节目

[23:06.00]找到更多新的节目

[23:08.00]找到更多新的节目

[23:10.00]找到更多新的节目

[23:12.00]找到更多新的节目

[23:14.00]找到更多新的节目

[23:16.00]找到更多新的节目

[23:18.00]找到更多新的节目

[23:20.00]找到更多新的节目

[23:22.00]找到更多新的节目

[23:24.00]找到更多新的节目

[23:26.00]找到更多新的节目

[23:28.00]找到更多新的节目

[23:30.00]找到更多新的节目

[23:32.00]找到更多新的节目

[23:34.00]找到更多新的节目

[23:36.00]找到更多新的节目

[23:38.00]找到更多新的节目

[23:40.00]找到更多新的节目

[23:42.00]找到更多新的节目

[23:44.00]找到更多新的节目

[23:46.00]找到更多新的节目

[23:48.00]找到更多新的节目

[23:50.00]找到更多新的节目

[23:52.00]找到更多新的节目

[23:54.00]找到更多新的节目

[23:56.00]找到更多新的节目

[23:58.00]找到更多新的节目

[24:00.00]找到更多新的节目

[24:02.00]找到更多新的节目

[24:04.00]找到更多新的节目

[24:06.00]找到更多新的节目

[24:08.00]找到更多新的节目

[24:10.00]找到更多新的节目

[24:12.00]找到更多新的节目

[24:14.00]找到更多新的节目

[24:16.00]找到更多新的节目

[24:18.00]找到更多新的节目

[24:20.00]找到更多新的节目

[24:22.00]找到更多新的节目

[24:24.00]找到更多新的节目

[24:26.00]找到更多新的节目

[24:28.00]找到更多新的节目

[24:30.00]找到更多新的节目

[24:32.00]找到更多新的节目

[24:34.00]找到更多新的节目

[24:36.00]找到更多新的节目

[24:38.00]找到更多新的节目

[24:40.00]找到更多新的节目

[24:42.00]找到更多新的节目

[24:44.00]找到更多新的节目

[24:46.00]找到更多新的节目

[24:48.00]找到更多新的节目

[24:50.00]找到更多新的节目

[24:52.00]找到更多新的节目

[24:54.00]找到更多新的节目

[24:56.00]找到更多新的节目

[24:58.00]找到更多新的节目

[25:00.00]找到更多新的节目

[25:02.00]找到更多新的节目

[25:04.00]找到更多新的节目

[25:06.00]找到更多新的节目

[25:08.00]找到更多新的节目

[25:10.00]找到更多新的节目

[25:12.00]找到更多新的节目

[25:14.00]找到更多新的节目

[25:16.00]找到更多新的节目

[25:18.00]找到更多新的节目

[25:20.00]找到更多新的节目

[25:22.00]找到更多新的节目

[25:24.00]找到更多新的节目

[25:26.00]找到更多新的节目

[25:28.00]找到更多新的节目

[25:30.00]找到更多新的节目

[25:32.00]找到更多新的节目

[25:34.00]找到更多新的节目

[25:36.00]找到更多新的节目

[25:38.00]找到更多新的节目

[25:40.00]找到更多新的节目

[25:42.00]找到更多新的节目

[25:44.00]找到更多新的节目

[25:46.00]找到更多新的节目

[25:48.00]找到更多新的节目

[25:50.00]找到更多新的节目

[25:52.00]找到更多新的节目

[25:54.00]找到更多新的节目

[25:56.00]找到更多新的节目

[25:58.00]找到更多新的节目

[26:00.00]找到更多新的节目

[26:02.00]找到更多新的节目

[26:04.00]找到更多新的节目

[26:06.00]找到更多新的节目

[26:08.00]找到更多新的节目

[26:10.00]找到更多新的节目

[26:12.00]找到更多新的节目

[26:14.00]找到更多新的节目

[26:16.00]找到更多新的节目

[26:18.00]找到更多新的节目

[26:20.00]找到更多新的节目

[26:22.00]找到更多新的节目

[26:24.00]找到更多新的节目

[26:26.00]找到更多新的节目

[26:28.00]找到更多新的节目

[26:30.00]找到更多新的节目

[26:32.00]找到更多新的节目

[26:34.00]找到更多新的节目

[26:36.00]找到更多新的节目

[26:38.00]找到更多新的节目

[26:40.00]找到更多新的节目

[26:42.00]找到更多新的节目

[26:44.00]找到更多新的节目

[26:46.00]找到更多新的节目

[26:48.00]找到更多新的节目

[26:50.00]找到更多新的节目

[26:52.00]找到更多新的节目

[26:54.00]找到更多新的节目

[26:56.00]找到更多新的节目

[26:58.00]找到更多新的节目

[27:00.00]找到更多新的节目

[27:02.00]找到更多新的节目

[27:04.00]找到更多新的节目

[27:06.00]找到更多新的节目

[27:08.00]找到更多新的节目

[27:10.00]找到更多新的节目

[27:12.00]找到更多新的节目

[27:14.00]找到更多新的节目

[27:16.00]找到更多新的节目

[27:18.00]找到更多新的节目

[27:20.00]找到更多新的节目

[27:22.00]找到更多新的节目

[27:24.00]找到更多新的节目

[27:26.00]找到更多新的节目

[27:28.00]找到更多新的节目

[27:30.00]找到更多新的节目

[27:32.00]找到更多新的节目

[27:34.00]找到更多新的节目

[27:36.00]找到更多新的节目

[27:38.00]找到更多新的节目

[27:40.00]找到更多新的节目

[27:42.00]找到更多新的节目

[27:44.00]找到更多新的节目

[27:46.00]找到更多新的节目

[27:48.00]找到更多新的节目

[27:50.00]找到更多新的节目

[27:52.00]找到更多新的节目

[27:54.00]找到更多新的节目

[27:56.00]找到更多新的节目

[27:58.00]找到更多新的节目

[28:00.00]找到更多新的节目

[28:02.00]找到更多新的节目

[28:04.00]找到更多新的节目

[28:06.00]找到更多新的节目

[28:08.00]找到更多新的节目

[28:10.00]找到更多新的节目

[28:12.00]找到更多新的节目

[28:14.00]找到更多新的节目

[28:16.00]找到更多新的节目

[28:18.00]找到更多新的节目

[28:20.00]找到更多新的节目

[28:22.00]找到更多新的节目

[28:24.00]找到更多新的节目

[28:26.00]找到更多新的节目

[28:28.00]找到更多新的节目

[28:30.00]找到更多新的节目

[28:32.00]找到更多新的节目

[28:34.00]找到更多新的节目

[28:36.00]找到更多新的节目

[28:38.00]找到更多新的节目

[28:40.00]找到更多新的节目

[28:42.00]找到更多新的节目

[28:44.00]找到更多新的节目

[28:46.00]找到更多新的节目

[28:48.00]找到更多新的节目

[28:50.00]找到更多新的节目

[28:52.00]找到更多新的节目

[28:54.00]找到更多新的节目

[28:56.00]找到更多新的节目

[28:58.00]找到更多新的节目

[29:00.00]找到更多新的节目

[29:02.00]找到更多新的节目

[29:04.00]找到更多新的节目

[29:06.00]找到更多新的节目

[29:08.00]找到更多新的节目

[29:10.00]找到更多新的节目

[29:12.00]找到更多新的节目

[29:14.00]找到更多新的节目

[29:16.00]找到更多新的节目

[29:18.00]找到更多新的节目

[29:20.00]找到更多新的节目

[29:22.00]找到更多新的节目

[29:24.00]找到更多新的节目

[29:26.00]找到更多新的节目

[29:28.00]找到更多新的节目

[29:30.00]找到更多新的节目

[29:32.00]找到更多新的节目

[29:34.00]找到更多新的节目

[29:36.00]找到更多新的节目

[29:38.00]找到更多新的节目

[29:40.00]找到更多新的节目

[29:42.00]找到更多新的节目

[29:44.00]找到更多新的节目

[29:46.00]找到更多新的节目

[29:48.00]找到更多新的节目

[29:50.00]找到更多新的节目

[29:52.00]找到更多新的节目

[29:54.00]找到更多新的节目

[29:56.00]找到更多新的节目

[29:58.00]找到更多新的节目

[30:00.00]找到更多新的节目

[30:02.00]找到更多新的节目

[30:04.00]找到更多新的节目

[30:06.00]找到更多新的节目

[30:08.00]找到更多新的节目

[30:10.00]找到更多新的节目

[30:12.00]找到更多新的节目

[30:14.00]找到更多新的节目

[30:16.00]找到更多新的节目

[30:18.00]找到更多新的节目

[30:20.00]找到更多新的节目

[30:22.00]找到更多新的节目

[30:24.00]找到更多新的节目

[30:26.00]找到更多新的节目

[30:28.00]找到更多新的节目

[30:30.00]找到更多新的节目

[30:32.00]找到更多新的节目

[30:34.00]找到更多新的节目

[30:36.00]找到更多新的节目

[30:38.00]找到更多新的节目

[30:40.00]找到更多新的节目

[30:42.00]找到更多新的节目

[30:44.00]找到更多新的节目

[30:46.00]找到更多新的节目

[30:48.00]找到更多新的节目

[30:50.00]所以我们现在 slowly

[30:52.00]改变了 现在我们现在慢慢改变了

[30:54.00]我在将来归咖巴斯

[30:56.00]没有改变了 没有改变了

[30:58.00]但是现在我们

[31:00.00]有一个世界的 一个世界的

[31:02.00]有一个世界的 哪里有Claude

[31:04.00]还有Gemnite 还有Gypsy4

[31:06.00]希望更多的 会变得更多

[31:08.00]希望更多的 希望更多的

[31:10.00]所以 说说 说说

[31:12.00]我们的视频 视频 视频

[31:14.00]所以 非常大 非常大 因为

[31:16.00]我们的视频 也不可以用

[31:18.00]但我认为

[31:20.00]大陆的社区

[31:22.00]要他们继续建设

[31:24.00]然后他们必须找一些方法

[31:26.00]才能发现他们会做的

[31:28.00]所以他们明白

[31:30.00]他们必须解决他们要的

[31:32.00]但是我们的视频 也有

[31:34.00]Mistral 也有

[31:36.00]Grock 现在

[31:38.00]Grock 1 从 从 oktober

[31:40.00]是公司

[31:41.00]对对对对

[31:42.00]你以为Grock是Grock the chip company

[31:44.00]Grock the chip company

[31:46.00]当然 浪漫3是

[31:48.00]所有人都在问

[31:50.00]我的感觉是

[31:52.00]小时候 宋可伯

[31:54.00]刚才说了浪漫3

[31:56.00]说了 至少从

[31:58.00]一个想法的选择

[32:00.00]他不想 怎么做

[32:02.00]要保持

[32:04.00]能够保持 能够保持

[32:06.00]Mistral 也想想

[32:08.00]你去到 怎么

[32:10.00]他 能够发展

[32:12.00]每个人都好 任何

[32:14.00]对 从我听过

[32:16.00]在GDC 浪漫3

[32:18.00]最大的模式是

[32:20.00]260-300billion

[32:22.00]所以那是大部分的

[32:24.00]那不是一个开放模式

[32:26.00]你不能给人们

[32:28.00]300billion的模式

[32:30.00]要用它 非常有力量

[32:32.00]所以我认为

[32:34.00]它是 可能是开放模式

[32:36.00]但那是一个不同的问题

[32:38.00]对对对

[32:40.00]它是 比他们做的

[32:42.00]在开放模式

[32:44.00]在浪漫上

[32:46.00]你能够使用

[32:48.00]开放的AI

[32:50.00]开放的安逸

[32:52.00]一些公司在

[32:54.00]中央的强硬度

[32:56.00]所以我们

[32:58.00]在Buckets 上

[33:00.00]他们做了很多

[33:02.00]他们比PyDorch 更好

[33:04.00]那是 可能

[33:06.00]在弥術上

[33:08.00]可能是 可能是

[33:10.00]I love the duck destroying

[33:12.00]a lot of monopolies arc

[33:14.00]it's been very entertaining

[33:16.00]let's bridge into the

[33:18.00]big tech side of this

[33:20.00]I think when I did my episode

[33:22.00]I added this as an additional war

[33:24.00]that's something I'm paying attention to

[33:26.00]so we've got

[33:28.00]Microsoft's moves with inflection

[33:30.00]which I think potentially are

[33:32.00]being read as

[33:34.00]a shift vis-a-vis the relationship

[33:36.00]with open AI

[33:38.00]missure a large relationship

[33:40.00]seems to reinforce as well

[33:42.00]we have apple potentially

[33:44.00]entering the race finally

[33:46.00]giving up project titan

[33:48.00]and trying to spend more effort on this

[33:50.00]although counterpoint

[33:52.00]we also have them talking about

[33:54.00]there being reports of a deal with google

[33:56.00]which is interesting to see

[33:58.00]what their strategy there is

[34:00.00]and then metas been largely quiet

[34:02.00]we just talked about the main piece

[34:04.00]but there's spoilers

[34:06.00]one of those things has been most interesting

[34:08.00]to you guys as you think about

[34:10.00]what's going to shake out for the rest of this year

[34:12.00]let's take a crack

[34:14.00]the reason we don't have a fifth war

[34:16.00]for the big tech wars

[34:18.00]that's one of those things where I just feel

[34:20.00]we don't cover differently

[34:22.00]from other media channels

[34:24.00]I guess

[34:26.00]in our entire interest

[34:28.00]we try not to cover the big tech gamethrones

[34:30.00]or it's proxied through

[34:32.00]all the other four wars anyway

[34:34.00]there's just a lot of overlap

[34:36.00]but yeah I think absolutely personally

[34:38.00]the most interesting one is apple entering the race

[34:40.00]they actually release, they are announced

[34:42.00]their first large language model that they train themselves

[34:44.00]it's like a 30 billion multimodal model

[34:46.00]people weren't that impressed

[34:48.00]but it was like the first time

[34:50.00]that apple has kind of showcased that

[34:52.00]we're training large models in house as well

[34:54.00]of course they might be

[34:56.00]doing this deal with google

[34:58.00]it sounds very sort of rumourary to me

[35:00.00]and it's probably if it's on device

[35:02.00]it's going to be a smaller model

[35:03.00]it's going to be smarter auto complete

[35:05.00]I don't know what to say

[35:07.00]I'm still here dealing with

[35:09.00]Siri which hasn't

[35:11.00]probably hasn't been updated since

[35:13.00]God knows when it was introduced

[35:15.00]it's horrible and it

[35:17.00]makes me so angry

[35:19.00]one as an apple customer and user

[35:21.00]I'm just hoping for better ai on apple itself

[35:23.00]but two they are

[35:25.00]the gold standard

[35:27.00]when it comes to local devices

[35:29.00]personal compute and trust

[35:31.00]you trust them with your data

[35:33.00]and I think

[35:35.00]that's what a lot of people are looking for in ai

[35:37.00]that they love the benefits of ai

[35:39.00]they don't love the downsides

[35:41.00]which is that you have to send all your data

[35:43.00]to some clouds somewhere and some of this data

[35:45.00]that we're going to feed ai is the most personal data there is

[35:47.00]so apple being

[35:49.00]one of the most trusted personal

[35:51.00]data companies I think it's very important

[35:53.00]that they enter the ai race

[35:55.00]and I hope to see more out of them

[35:57.00]to me the biggest question

[35:59.00]it's like who's paying who

[36:01.00]because for the browsers

[36:03.00]google pays apple like 18

[36:05.00]20 billion every year

[36:07.00]to be the default browser

[36:09.00]is google going to pay you to have javanai

[36:11.00]or is apple paying google to have javanai

[36:13.00]I think that's like what I'm most interested

[36:15.00]to figure out because with the browsers

[36:17.00]it's like it's the entry point

[36:19.00]to the thing so it's really valuable

[36:21.00]to be the default that's what google pays

[36:23.00]but I wonder if the perception in ai

[36:25.00]is going to be like hey

[36:27.00]you have a good local model on my phone

[36:29.00]to be worth me purchasing your device

[36:31.00]and that's going to drive apple

[36:33.00]to be the one buying the model

[36:35.00]but then like Sean said

[36:37.00]they're doing the mm1 themselves

[36:39.00]are they saying we do models

[36:41.00]but they're not as good as the google ones

[36:43.00]I don't know the whole thing is really confusing

[36:45.00]but it makes for a great meme

[36:47.00]material on twitter

[36:49.00]I think like

[36:51.00]they are possibly more than

[36:53.00]open ai and mic microsoft and amazon

[36:55.00]they are the most full stack company there is

[36:57.00]in computing

[36:59.00]and so

[37:01.00]like they own the chips man

[37:03.00]like they manufacture everything

[37:05.00]so if there was a company

[37:07.00]that could seriously challenge

[37:09.00]the other ai players it would be apple

[37:11.00]and it's

[37:13.00]I don't think it's as hard as self-driving

[37:15.00]so like maybe they've just been

[37:17.00]investing in the wrong thing this whole time

[37:19.00]Wallstreet certainly thinks so

[37:21.00]Wallstreet love that move man

[37:23.00]there's a big sigh of relief

[37:25.00]well let's move away

[37:27.00]from sort of the big stuff

[37:29.00]I think to both of your points

[37:31.00]can I drop one factoid

[37:33.00]about this wallstreet thing

[37:35.00]I went and looked at

[37:37.00]when from being a VR company

[37:39.00]to an ai company

[37:41.00]and I think

[37:43.00]the stock

[37:45.00]I'm trying to look up the details now

[37:47.00]the stock has gone up 187%

[37:49.00]since limo one

[37:51.00]$830 billion in market value

[37:53.00]created in the past year

[37:55.00]if you haven't seen that chart

[37:59.00]it's actually remarkable if you draw

[38:01.00]a little arrow on it

[38:03.00]it's likeno we're an ai company now

[38:05.00]forget the VR thing

[38:07.00]it isn't interesting

[38:11.00]no I think unless you called it

[38:13.00]zuck's disruptor arc or whatever

[38:15.00]he really does

[38:17.00]he is in the midst of a total

[38:19.00]it's a redemption arc or it's just

[38:21.00]it's something different where

[38:23.00]he's sort of the spoiler like

[38:25.00]people loved him

[38:27.00]just freestyle talking about why he thought

[38:29.00]they had a better headset than apple

[38:31.00]even if they didn't agree they just loved

[38:33.00]he was going direct to camera and talking about it

[38:35.00]for five minutes or whatever

[38:37.00]that's a fascinating shift that I don't think

[38:39.00]anyone had on their bingo card

[38:41.00]whatever two years ago

[38:43.00]it's still there in cn5 d-long

[38:45.00]don't write it off

[38:47.00]we need to see him fight in the coliseum

[38:49.00]no I think in terms of

[38:51.00]self

[38:53.00]management life leadership

[38:55.00]there's a lot of lessons to learn from him

[38:57.00]you might kind of quibble

[38:59.00]with the social impact of facebook

[39:01.00]but just himself

[39:03.00]in terms of personal growth

[39:05.00]and perseverance through

[39:07.00]a lot of change

[39:09.00]everyone throwing stuff his way

[39:11.00]I think there's a lot to say about

[39:13.00]to learn from zuck

[39:15.00]he's my age

[39:17.00]awesome

[39:19.00]so one of the big things

[39:21.00]that I think you guys have

[39:23.00]distinct and unique insight into

[39:25.00]being where you are and where you work on

[39:27.00]iswhat developers

[39:29.00]are getting really excited about right now

[39:31.00]and by that I mean on the one hand

[39:33.00]certainly start ups who are actually

[39:35.00]formalized and formed to start ups

[39:37.00]but also just in terms of

[39:39.00]what people are spending their nights and weekends on

[39:41.00]what they're coming to hackathons to do

[39:43.00]and you know I think it's a

[39:45.00]it's such a fascinating indicator

[39:47.00]for where things are headed like

[39:49.00]if you zoom back a year

[39:51.00]right now was right when everyone was getting

[39:53.00]so so excited about

[39:55.00]ai agent stuff

[39:57.00]auto gpt and baby agi and these things were like

[39:59.00]if you dropped anything on youtube about those

[40:01.00]like instantly tens of thousands of views

[40:03.00]I know because I had like

[40:05.00]a 50,000 view video

[40:07.00]like the second day that I was doing

[40:09.00]the show on youtube you know because I was talking about

[40:11.00]auto gpt and so anyways

[40:13.00]you know obviously that's sort of not totally

[40:15.00]come to fruition yet but what are some of the

[40:17.00]trends and what you guys are seeing in terms of

[40:19.00]people's interest and what people are building

[40:21.00]I can start maybe with the agents part

[40:23.00]and then I know Sean is doing a

[40:25.00]diffusion meetup tonight there's

[40:27.00]a lot of different things

[40:29.00]the agent wave has been the most

[40:31.00]interesting kind of like dream

[40:33.00]to reality

[40:35.00]arc so auto gpt I think

[40:37.00]they went from zero to like

[40:39.00]125,000 get up stars in six weeks

[40:41.00]and then one year later

[40:43.00]they have 150,000

[40:45.00]stars so there's kind of been a big

[40:47.00]plot so I mean you might say

[40:49.00]there's just not that many people that can

[40:51.00]start it you know everybody already started

[40:53.00]but the promise of

[40:55.00]hey I'll just give you a goal

[40:57.00]and you do it I think it's like

[40:59.00]amazing to get people's

[41:01.00]imagination going you know

[41:03.00]they're like oh wow this is

[41:05.00]this is awesome everybody

[41:07.00]can try this to do anything

[41:09.00]but then as technologists

[41:11.00]you're like well that's

[41:13.00]that's just like not possible you know

[41:15.00]we would have like solved everything and

[41:17.00]I think it takes a little bit to go from

[41:19.00]the promise and the hope

[41:21.00]that people show you to then

[41:23.00]trinate yourself and going back to say

[41:25.00]okay this is not really working for me and

[41:27.00]David won from adept you know

[41:29.00]they in our episode he specifically said

[41:31.00]we don't want to do a bottom sub product

[41:33.00]you knowwe don't want something that everybody

[41:35.00]could try because it's really hard to get it

[41:37.00]to be reliable so

[41:39.00]we're seeing a lot of companies

[41:41.00]doing vertical agents that

[41:43.00]are narrow for a specific

[41:45.00]domain and they're very good at something

[41:47.00]Myconover who was at Databricks before

[41:49.00]is also a friend of Layton space

[41:51.00]he's doing this new company go bright with

[41:53.00]doing AI agents for financial research

[41:55.00]and that's it you know and

[41:57.00]they're doing very well there are

[41:59.00]other companies doing it in

[42:01.00]security doing it in

[42:03.00]compliance doing it in legal

[42:05.00]all of these things that like

[42:07.00]people nobody

[42:09.00]just wakes up and sayoh I

[42:11.00]cannot wait to go on auto gpd and ask

[42:13.00]it to do a compliance review of my thing

[42:15.00]you know just not what inspires people

[42:17.00]so think the gap on the developer

[42:19.00]side has been the more bottom sub

[42:21.00]hacker mentality is trying to build

[42:23.00]this like very generic

[42:25.00]agents that can do a lot of open

[42:27.00]ended task and then the more business

[42:29.00]side of things is like hey if I want

[42:31.00]to raise my next round I cannot

[42:33.00]just like sit around the mess

[42:35.00]mess around with like super generic

[42:37.00]stuff I need to find a use case that

[42:39.00]really works and I think that that

[42:41.00]is worth for a lot of folks in

[42:43.00]parallel you have a lot of companies

[42:45.00]doing evals there are dozens

[42:47.00]of them that just want to help you

[42:49.00]measure how good your models are

[42:51.00]doing again if you build evals

[42:53.00]you need to also have a restrained

[42:55.00]surface area to actually figure out

[42:57.00]whether or not it's good right because

[42:59.00]there's a lot of stuff going on

[43:01.00]and everything under the sun so

[43:03.00]that's another category where I've

[43:05.00]seen from the sort of

[43:07.00]pitches that I've seen there's a

[43:09.00]lot of interest in the enterprise

[43:11.00]it's just like really fragmented

[43:13.00]because the production use cases

[43:15.00]are just coming like now you know

[43:17.00]there are not a lot of long established

[43:19.00]ones to test against and so

[43:21.00]that's kind of on the virtual agents

[43:23.00]and then the robotic side

[43:25.00]it's probably been the thing that

[43:27.00]the amount of robots that were there

[43:29.00]there were just like robots everywhere

[43:31.00]both in the keynote and then on the show floor

[43:33.00]you would haveboston dynamics

[43:35.00]dogs running around

[43:37.00]there was like this like fox

[43:39.00]robot that had like a virtual face

[43:41.00]that like talked to you and like moved

[43:43.00]in realtime there were industrial

[43:45.00]robots.imbedia did a big push

[43:47.00]on their own omniverse thing which

[43:49.00]is like thisdigital twin

[43:51.00]ofwhatever environments you're in

[43:53.00]that you can use to train the robots agents

[43:55.00]so that kind of takes people back to the

[43:57.00]reinforcement learning days but

[43:59.00]yeah agents people want them

[44:01.00]you know people want them I give a talk

[44:03.00]about the rise of the full stack employees

[44:05.00]and kind of this future the same way

[44:07.00]full stack engineers kind of work

[44:09.00]agross the stack in the future every

[44:11.00]employee is going to interact with

[44:13.00]every part of the organization through

[44:15.00]agents and AI enabled tooling

[44:17.00]this is happening it just needs to be a

[44:19.00]lot more narrow than maybe the first

[44:21.00]approach that we took which is just

[44:23.00]short of super interesting stuff going on

[44:25.00]yeah but he less you covered

[44:27.00]a lot of stuff there I'll separate

[44:29.00]the robotics piece because I feel like

[44:31.00]that's so different from the software world

[44:33.00]but yeah we do we do talk to a lot of

[44:35.00]engineers and you know that that this is

[44:37.00]our sort of bread and butter and I do agree

[44:39.00]that vertical agents have worked out a

[44:41.00]lot better than the horizontal ones

[44:43.00]I think you know the point I'll make

[44:45.00]here is just the reason auto GPT

[44:47.00]and maybe AGI you know it's in the

[44:49.00]name like they were promising AGI

[44:51.00]but I think people are discovering that you cannot

[44:53.00]engineer your way to AGI it has to be

[44:55.00]done at the model level and all these

[44:57.00]engineer and prompt engineering

[44:59.00]hacks on top of it weren't really going to

[45:01.00]get us there in a meaningful way

[45:03.00]without much further

[45:05.00]improvements in the models

[45:07.00]I would say I'll go so far as to say

[45:09.00]even Devin which is I would

[45:11.00]I think the most advanced agents

[45:13.00]that we've ever seen still requires a

[45:15.00]lot of engineering and still probably

[45:17.00]falls apart a lot in terms of like

[45:19.00]practical usage or it's just way too slow

[45:21.00]and expensive for you know what it's

[45:23.00]what is promised in comparison to video

[45:25.00]so yeah that's what happened

[45:27.00]of agents from last year

[45:29.00]but I do see like vertical agents

[45:31.00]being very popular and sometimes

[45:33.00]I think the word agent might even be

[45:35.00]overused sometimes like people don't

[45:37.00]really care whether or not you call it an

[45:39.00]AI agent right like does it replace

[45:41.00]boring menial tasks that I do

[45:43.00]that I might hire human to do or

[45:45.00]that the human who is hired to do it

[45:47.00]doesn't really want to do

[45:49.00]and I think there's absolutely ways

[45:51.00]in sort of a vertical context

[45:53.00]that you can actually go after

[45:55.00]very routine tasks that can be scaled out

[45:57.00]to a lot of you know AI assistance

[45:59.00]so yeah I would

[46:01.00]basically plus one what I said there

[46:03.00]I think it's very very promising

[46:05.00]and I think more people should work on it

[46:07.00]not less like there's not enough people

[46:09.00]like this should be the main thrust

[46:11.00]of the AI engineers to look out

[46:13.00]look for use cases and go to production

[46:15.00]instead of just always working on some

[46:17.00]AI promising thing that never arrives

[46:19.00]I can only add that

[46:21.00]so I've been fiercely making

[46:23.00]tutorials behind the scenes

[46:25.00]around basically everything you can imagine

[46:27.00]with AI we've probably done about 300

[46:29.00]tutorials over the last couple months

[46:31.00]and the verticalized

[46:33.00]anything right like this is a solution

[46:35.00]for your particular job

[46:37.00]or role even if it's way less

[46:39.00]interesting or

[46:41.00]kind of sexy it's like so radically more

[46:43.00]useful to people in terms of intersecting

[46:45.00]with how like those are the ways that people are

[46:47.00]actually adopting AI

[46:49.00]in a lot of cases it's just a

[46:51.00]thing that I do over and over again

[46:53.00]by the way I think that's the same way that even the

[46:55.00]generalized models are getting adopted you know

[46:57.00]it's like I use mid-journey

[46:59.00]for lots of stuff but the main thing I use it for

[47:01.00]is youtube thumbnails every day like day in

[47:03.00]day out I will always do a youtube thumbnail

[47:05.00]you know or two with with mid-journey

[47:07.00]and it's like you can you can start to extrapolate

[47:09.00]that across a lot of things and all of a sudden

[47:11.00]you know AI doesn't

[47:13.00]it looks revolutionary because of

[47:15.00]a million small changes rather than

[47:17.00]one sort of big dramatic change

[47:19.00]and I think that the verticalization of agents

[47:21.00]is sort of a great example of

[47:23.00]how that's going to play out too

[47:25.00]So I'll have one caveat here

[47:27.00]which is I think that because

[47:29.00]multimodal models are now commonplace

[47:31.00]like claw,generalized,openEI

[47:33.00]all very very easily

[47:35.00]multimodal,apples easily

[47:37.00]multimodal,all this stuff

[47:39.00]the switch for agents for sort of general desktop

[47:41.00]browsing

[47:43.00]that I think people need to keep an eye on

[47:45.00]it's not mature yet

[47:47.00]but it is absolutely coming on the way

[47:49.00]and so just as we're starting to talk

[47:51.00]about this verticalization piece

[47:53.00]because that is mature

[47:55.00]that is ready for people to work on

[47:57.00]that is that a lot of people are making really good money doing that

[47:59.00]the thing that's on the rise

[48:01.00]is this sort of drive by vision

[48:03.00]version of the agent where

[48:05.00]they're not specifically taking in text or anything

[48:07.00]but just watching your screen just like someone else would

[48:09.00]and piloting it

[48:11.00]by vision

[48:13.00]in the episode with David

[48:15.00]that will have dropped by the time that this airs

[48:17.00]I think that is the promise of adept

[48:19.00]that is the promise of what

[48:21.00]a lot of these sort of desktop agents are

[48:23.00]and that is the more general purpose

[48:25.00]system that could be as big as

[48:27.00]the browser

[48:29.00]the operating system

[48:31.00]people really want to build that

[48:33.00]foundational piece of software in AI

[48:35.00]and I would see the potential

[48:37.00]therefor desktop agents being that

[48:39.00]that you can have self-driving

[48:41.00]computers. Don't write the horizontal

[48:43.00]piece out. I just think we took a while

[48:45.00]to get there. What else are you guys

[48:47.00]seeing?That's interesting to you.

[48:49.00]I'm looking at your notes and seeing a ton

[48:51.00]ofcategories. I'll take

[48:53.00]the next two as one category

[48:55.00]which is basically alternative architectures.

[48:57.00]The two main things that everyone

[48:59.00]following AI kind of knows now is

[49:01.00]one, the diffusion architecture

[49:03.00]and two, the

[49:05.00]let's just say the decoder only

[49:07.00]transformer architecture that is popular

[49:09.00]by GPT. You can read, you can look on

[49:11.00]YouTube for thousands and thousands of tutorials

[49:13.00]on each of those things. What we are talking about

[49:15.00]here is what's next, what people are

[49:17.00]researching and what could be on the horizon

[49:19.00]that takes the place of those other two things.

[49:21.00]So first we'll talk about transformer architectures

[49:23.00]and then diffusion. So transform the two leading

[49:25.00]candidates are effectively RWKV

[49:27.00]and the state space models. The most recent

[49:29.00]one of which is Mamba but there's others

[49:31.00]and the S4H3

[49:33.00]stuff coming out of Hazy Research in Stanford

[49:35.00]and all of those are

[49:37.00]non-quadratic

[49:39.00]language models that scale

[49:41.00]that promise to scale a lot better

[49:43.00]than the traditional transformer

[49:45.00]that this might be too theoretical

[49:47.00]for most people right now

[49:49.00]but it's going to be

[49:51.00]it's going to come out in weird ways

[49:53.00]where imagine if like right now

[49:55.00]the talk of the town is that

[49:57.00]Claude and Gemini have a million tokens of context

[49:59.00]and you can put in like

[50:01.00]two hours of video now

[50:03.00]but what if you could throw in

[50:05.00]200,000 hours of video

[50:07.00]how does that change your

[50:09.00]usage of AI

[50:11.00]what if you could throw in the entire

[50:13.00]genetic sequence of a human

[50:15.00]and synthesize new drugs

[50:17.00]how does that change things

[50:19.00]we don't know because we haven't had access to this capability

[50:21.00]being so cheap before

[50:23.00]and that's the ultimate promise of these two models

[50:25.00]they're not there yet

[50:27.00]it's a very very good progress

[50:29.00]RWKV and Mamba are probably the two leading examples

[50:31.00]both of which are open source

[50:33.00]that you can try them today

[50:35.00]and have a lot of progress there

[50:37.00]the main thing I'll highlight for other UKV

[50:39.00]is that at the 7B level

[50:41.00]they seem to have beat

[50:43.00]Lama2 in all benchmarks

[50:45.00]that matter

[50:47.00]at the same size for the same amount of training

[50:49.00]as an open source model so that's exciting

[50:51.00]there are 7B now

[50:53.00]they're not at 7TB we don't know if it'll scale

[50:55.00]the other thing is diffusion

[50:57.00]diffusions and transformers

[50:59.00]are kind of on the collision course

[51:01.00]the original stable diffusion already used

[51:03.00]transformers in parts of its architecture

[51:05.00]it seems that transformers are

[51:07.00]eating more and more of those layers

[51:09.00]particularly the VAE layer

[51:11.00]so that's the diffusion transformer

[51:13.00]is what Sora is built on

[51:15.00]the guy who wrote the diffusion transformer

[51:17.00]paper

[51:19.00]bilt pebbles is the lead tech guy

[51:21.00]on Sora

[51:23.00]but there's more sort of experimentation

[51:25.00]to diffusion I'm holding a meetup

[51:27.00]actually here in San Francisco that's going to be like the state of diffusion

[51:29.00]which I'm pretty excited about

[51:31.00]stability's doing a lot of good work

[51:33.00]and if you look at the architecture

[51:35.00]of how they're creating

[51:37.00]stable diffusion 3, our glass diffusion

[51:39.00]and the inconsistency models

[51:41.00]or SDXL turbo

[51:43.00]all of these are like very very interesting innovations

[51:45.00]on like the original idea

[51:47.00]of what stable diffusion was so if you think

[51:49.00]that it is expensive to create or slow to create

[51:51.00]stable diffusion or AI-generated art

[51:53.00]you are not up to date with the latest models

[51:55.00]if you think it is hard to create

[51:57.00]texted images you are not up to date with the latest models

[51:59.00]and people still are kind of far behind

[52:01.00]the last piece of which

[52:03.00]is the wall cut I always kind of hold out

[52:05.00]which is text diffusion

[52:07.00]so instead of using auto generative

[52:09.00]or auto regressive transformers

[52:11.00]can you use text to diffuse

[52:13.00]so you can use diffusion models to diffuse

[52:15.00]and create entire chunks of text

[52:17.00]all at once instead of token by token

[52:19.00]and that is something that mid-journey confirmed today

[52:21.00]because it was only rumored

[52:23.00]the past few months but they confirmed today

[52:25.00]that they were looking into

[52:27.00]all those things are like very exciting new model

[52:29.00]architectures that are maybe something

[52:31.00]that you will see in production 2-3 years from now

[52:33.00]so the couple of the trends that I want to just

[52:37.00]get your takes on because they're sort of something

[52:39.00]that seems like they're coming up are

[52:41.00]one sort of these wearable

[52:43.00]kind of passive

[52:45.00]AI experiences where

[52:47.00]they're absorbing a lot of what's going on around you

[52:49.00]and then kind of bringing things back

[52:51.00]and then the other one that I

[52:53.00]wanted to see if you guys had thoughts on were

[52:55.00]sort of this next generation of chip companies

[52:57.00]obviously there's a huge amount of emphasis

[52:59.00]on hardware and silicon

[53:01.00]and different ways of doing things but

[53:03.00]love your take on neither or both of those

[53:05.00]so wearables

[53:07.00]I'm very excited about it

[53:09.00]I want wearables on me at all times

[53:11.00]I have two right here to quantify my health

[53:13.00]and I'm all for them

[53:15.00]but society is not ready for wearables

[53:17.00]no one's comfortable with

[53:19.00]a device on recording every single

[53:21.00]conversation we have even

[53:23.00]all three of us here as

[53:25.00]podcasters we don't record everything

[53:27.00]that we say and I think

[53:29.00]there's a social shift that needs to happen

[53:31.00]I'm an investor in tab

[53:33.00]they are renaming to a broader

[53:35.00]vision but they are one of the

[53:37.00]three or four leading wearables in this

[53:39.00]space instead of the AI pendants

[53:41.00]or AI OS

[53:43.00]I have seen two humans

[53:45.00]in a while in San Francisco

[53:47.00]I'm very very excited to report

[53:49.00]that there are people walking around

[53:51.00]with those things on their chest

[53:53.00]and it is as goofy as it sounds

[53:55.00]it absolutely is going to fail

[53:57.00]but god bless them for trying

[53:59.00]and I've also bought a rabbit

[54:01.00]so I'm very excited for all those things to arrive

[54:03.00]but yeah people

[54:05.00]are very keen on hardware

[54:07.00]I think the idea that you can have physical objects

[54:09.00]that embody an AI

[54:11.00]do specific things for you

[54:13.00]is as old as

[54:15.00]the sort of golem

[54:17.00]in sort of medieval times

[54:19.00]in terms of like how much we want

[54:21.00]our objects to be smart

[54:23.00]and do things for us

[54:25.00]and I think it's absolutely

[54:27.00]a great play

[54:29.00]the funny thing is people are much more willing

[54:31.00]to pay you up front

[54:33.00]for a hardware device

[54:35.00]than they are willing to pay $8 a month

[54:37.00]suscription recurring for software

[54:39.00]and so the interesting economics

[54:41.00]of these wearable companies

[54:43.00]is they have negative float

[54:45.00]in the sense that people pay deposits up front

[54:47.00]like I paid

[54:49.00]$200 for the rabbit up front

[54:51.00]and I don't get it for another six months

[54:53.00]I paid $600 for the tablet

[54:55.00]I don't get it for another six months

[54:57.00]and then they can take that money

[54:59.00]and sort of invest it in their next

[55:01.00]events or their next properties

[55:03.00]or ventures

[55:05.00]and I think that's a very interesting

[55:07.00]comics from other types of

[55:09.00]AI companies that I see

[55:11.00]and I think just the tactile feel

[55:13.00]of an AI I think is very promising

[55:15.00]I don't know if you have other

[55:17.00]thoughts on the wearable stuff

[55:19.00]open interpreter

[55:21.00]just announced their product four hours ago

[55:23.00]which is not really a wearable

[55:25.00]but it's still like a

[55:27.00]physical device

[55:29.00]it's a push to talk mic

[55:31.00]to a device on your laptop

[55:33.00]it's a $99 push

[55:35.00]but again

[55:37.00]go back to your point

[55:39.00]people are interested in

[55:41.00]spending money for things that they can hold

[55:43.00]I don't know what that means overall

[55:45.00]where things are going but

[55:47.00]making more of this AI

[55:49.00]be a physical part of your life

[55:51.00]I think people are interested in that

[55:53.00]but I agree with Sean

[55:55.00]I talked to Avi about this

[55:57.00]but Avi's point is like

[55:59.00]most consumers care about utility

[56:01.00]more than they care about privacy

[56:03.00]I'm not like you've seen with social media

[56:05.00]but I also think there's a big

[56:07.00]social reaction

[56:09.00]to AI that is like much more

[56:11.00]rooted than the social media one

[56:13.00]but we'll see

[56:15.00]but a lot again a lot of work a lot of developers

[56:17.00]a lot of money going into it so

[56:19.00]there's bound to be experiments

[56:21.00]being run on the chip

[56:23.00]sorry I'll just

[56:25.00]ship it one more thing and then we transition to the chips

[56:27.00]the thing I'll caution people on is

[56:29.00]don't overly focus on the form factor

[56:31.00]the form factor is a delivery mode

[56:33.00]there will be many form factors

[56:35.00]it doesn't matter so much as

[56:37.00]where in the data war does it sit

[56:39.00]it actually is context acquisition

[56:41.00]because and maybe a little bit of multi-modality

[56:43.00]context is king

[56:45.00]if you have access to data

[56:47.00]that no one else has

[56:49.00]then you will be able to create AI that no one else can create

[56:51.00]and so what is the most personal context

[56:53.00]it is your everyday conversation

[56:55.00]it is as close to mapping your

[56:57.00]mental train of thought as possible

[56:59.00]without physically you writing down notes

[57:01.00]so that is the promise

[57:03.00]the ultimate goal here

[57:05.00]which is like personal context

[57:07.00]it's always available on you

[57:09.00]a little ncl that stuff

[57:11.00]that's the frame I want to give people

[57:13.00]that the form factors will change

[57:15.00]and there will be multiple form factors

[57:17.00]but it's the software behind that

[57:19.00]in the personal context

[57:21.00]that you cannot get anywhere else

[57:23.00]that'll win

[57:25.00]so that was wearables

[57:27.00]but it's not even a new release

[57:29.00]because the company I think was started in 2016

[57:31.00]so it's actually quite old

[57:33.00]but now recently captured

[57:35.00]people's imagination with their mixrall

[57:37.00]500 tokens a second demo

[57:39.00]yeah I think so far

[57:41.00]the battle on the GPU side

[57:43.00]has beeneither you go

[57:45.00]kind of like massive chip

[57:47.00]like this irreversible of the world

[57:49.00]where one chip front

[57:51.00]reversible is about 2 million dollars

[57:53.00]you know that's compared

[57:55.00]so you cannot compare one chip

[57:57.00]versus one chip but h100

[57:59.00]it's like 40,000

[58:01.00]the problem with those architectures

[58:03.00]has been

[58:05.00]they want to be very general

[58:07.00]but they wanted to put a lot

[58:09.00]of the SRAM on the chip

[58:11.00]it's much more convenient

[58:13.00]when you're using larger language models

[58:15.00]but the models outpace the size

[58:17.00]of the chips and chips have a much longer

[58:19.00]turnaround cycle

[58:21.00]Grock today is great for the current architecture

[58:23.00]it's a lot more expensive also

[58:25.00]as far as dollar per flop

[58:27.00]but there are the SIK

[58:29.00]when you have very high concurrency

[58:31.00]we actually were much cheaper

[58:33.00]you shouldn't just be looking at the compute power

[58:35.00]for most people this doesn't really matter

[58:37.00]you know like I think that's like the most

[58:39.00]the most interesting thing to me is like

[58:41.00]we've now gone back

[58:43.00]with AI to a world where

[58:45.00]developers care

[58:47.00]about what hardware is running

[58:49.00]which was not the case in traditional software

[58:51.00]maybe 20 years since

[58:53.00]as the cloud has gotten really big

[58:55.00]my thinking is that

[58:57.00]in the next 2-3 years

[58:59.00]we're going to go back to that

[59:01.00]we're like people are not going to be sweating

[59:03.00]what GPU do you have in your cloud

[59:05.00]what do you have

[59:07.00]you want to run this model

[59:09.00]we can run it at the same speed as everybody else

[59:11.00]and then everybody will make different choices

[59:13.00]whether they want to have

[59:15.00]higher front end capital investment

[59:17.00]and then better utilization

[59:19.00]and then upgrade later

[59:21.00]there are a lot of parameters

[59:23.00]and then there's the dark horses

[59:25.00]that is some of the smaller companies

[59:27.00]like Lemurian Labs, MedEx

[59:29.00]that are working on

[59:31.00]maybe not a chip alone

[59:33.00]but also some of the actual math

[59:35.00]infrastructure and the instructions

[59:37.00]that make them run

[59:39.00]there's a lot going on but

[59:41.00]yeah I think the

[59:43.00]the episode with Dylan will be

[59:45.00]interesting for people but

[59:47.00]hey everybody has pros and cons

[59:49.00]it's different than the models

[59:51.00]where you're like oh this one is definitely better for me

[59:53.00]and I'm going to use it

[59:55.00]I think for most people

[59:57.00]it's like fun twitter meming

[59:59.00]99% of people

[60:01.00]that tweet about this stuff

[60:03.00]are never going to buy any of these chips anyway

[60:05.00]so it's really more for entertainment

[60:07.00]wow I mean

[60:09.00]this is serious business here

[60:11.00]the potential new Nvidia

[60:13.00]if anyone can take

[60:15.00]i'm more talking about

[60:17.00]how should people think about it

[60:19.00]i think the end user

[60:21.00]is not impacted as much

[60:23.00]so I disagree

[60:25.00]I love disagreements

[60:27.00]who likes the podcast

[60:29.00]where all 3 people always agree with each other

[60:31.00]you will see the impact of this

[60:33.00]in the tokens per second over time

[60:35.00]this year

[60:37.00]I have very very credible sources

[60:39.00]all telling me

[60:41.00]that the average tokens per second

[60:43.00]right now we have

[60:45.00]somewhere between 50 to 100

[60:47.00]that's the norm for people

[60:49.00]average tokens per second

[60:51.00]will go to 500 to 2000

[60:53.00]this year

[60:55.00]from a number of chip suppliers that I cannot name

[60:57.00]that will cause

[60:59.00]step change in the use cases

[61:01.00]every time you have an order of magnitude improvement

[61:03.00]in the speed of something

[61:05.00]you unlock new use cases

[61:07.00]that become fun instead of a chore

[61:09.00]and so that's what I would

[61:11.00]caution this audience to think about

[61:13.00]what can you do in much higher AI speed

[61:15.00]it's not just things streaming out faster

[61:17.00]it is

[61:19.00]things working in the background a lot more seamlessly

[61:21.00]and therefore being a lot more useful

[61:23.00]than previously imagined

[61:25.00]so that would be my two cents on that

[61:27.00]yeah

[61:29.00]I mean the new

[61:31.00]imbedia chips are also much faster

[61:33.00]to me that's true

[61:35.00]when it comes about startups

[61:37.00]are the startups pushing

[61:39.00]on the incumbents

[61:41.00]or are the incumbents still leading

[61:43.00]and then the startups are riding the same wave

[61:45.00]I don't have yet a good sense of that

[61:47.00]it's next year's

[61:49.00]imbedia release just gonna be better than everything

[61:51.00]that gets released this year

[61:53.00]if that's the case

[61:55.00]it's like okay

[61:57.00]damn jensen

[61:59.00]it's like I'm gonna fight imbedia

[62:01.00]damn jensen got hands

[62:03.00]he really does

[62:05.00]well

[62:07.00]one conversation guys

[62:09.00]just by way of wrapping up

[62:11.00]call it over the next three months

[62:13.00]between now and sort of the beginning of summer

[62:15.00]what's one prediction that each of you has

[62:17.00]could be about anything

[62:19.00]could be big company, could be startup

[62:21.00]could be something you have privileged information that you know

[62:23.00]and you just won't tell us that you actually know

[62:25.00]does it have to be something that we think

[62:27.00]it's gonna be true or like something that we think

[62:29.00]because for me it's like

[62:31.00]is sundar gonna be the CEO of google

[62:33.00]maybe not in three months but maybe in like six months

[62:35.00]in nine months

[62:37.00]people were like oh maybe that miss is gonna be the new CEO

[62:39.00]that was kinda like

[62:41.00]I was busyfishing some deep mind people

[62:43.00]and google people for like a good

[62:45.00]guest for the pot and I was like

[62:47.00]oh what about jeptine and they're like

[62:49.00]well that miss is really like the person that runs everything

[62:51.00]anyway and the stuff

[62:53.00]and it's like interesting

[62:55.00]so I don't know

[62:57.00]serge could come back I don't know

[62:59.00]like he's making more appearances these days

[63:01.00]yeah

[63:03.00]but we can just put it as like

[63:05.00]my thing is like

[63:07.00]CEO change potential

[63:09.00]but again

[63:11.00]three months is too short

[63:13.00]to make a prediction

[63:15.00]time scale might be off

[63:17.00]yeah I mean

[63:21.00]for me I think the progression

[63:23.00]in vertical agent companies

[63:25.00]will keep going

[63:27.00]we just had the other day

[63:29.00]Klarna talking about how they replaced like

[63:31.00]customer support agents with the

[63:33.00]AI agents

[63:35.00]that's to the beginning guys

[63:37.00]imagine this rolling out across most of the fortune

[63:39.00]500

[63:41.00]and I'm not saying this is like a utopian

[63:43.00]scenario there will be very very

[63:45.00]embarrassing and bad outcomes

[63:47.00]of this where like humans would never

[63:49.00]make this mistake but AIS did and

[63:51.00]we'll all laugh at it or it will be very offended

[63:53.00]by whatever you know bad outcome

[63:55.00]it did so we have to be responsible

[63:57.00]and careful in the rollout but yeah this is

[63:59.00]the rolling out you know Alessio likes to say

[64:01.00]that this year is the year of AI production

[64:03.00]let's see it let's see all these vertical

[64:05.00]full stack employees

[64:07.00]come out into the workforce

[64:09.00]love it alright guys well thank you so much

[64:11.00]for sharing your thoughts and insights

[64:13.00]here and can't wait to do it again

[64:15.00]welcome back again

[64:17.00]it's Charlie your AI cohost

[64:19.00]we're now in part two

[64:21.00]of the special weekend episode

[64:23.00]collating some of SWIX and Alessio's

[64:25.00]recent appearances

[64:27.00]if you are not active in the latent space discord

[64:29.00]you might not be aware of the

[64:31.00]many many many in person

[64:33.00]events we host gathering our

[64:35.00]listener community all over the world

[64:37.00]you can see the latent space

[64:39.00]community page for how to join

[64:41.00]and subscribe to our event calendar

[64:43.00]for future meetups

[64:45.00]we're going to share some of our recent live

[64:47.00]appearances in this next part

[64:49.00]starting with the Thursday nights in AI meetup

[64:51.00]a regular fixture in the SF AI scene

[64:53.00]run by imbu and outset capital

[64:55.00]primarily our former guest

[64:57.00]Kanjen Kyu

[64:59.00]Allie Roder and Josh Albrecht

[65:01.00]here's SWIX

[65:03.00]today for those of you who have been here before

[65:07.00]you know the general format

[65:09.00]so we'll do a quick fireside Q&A with SWIX

[65:11.00]where we're asking him the questions

[65:13.00]then we'll actually go to our rapid fire Q&A

[65:15.00]where we're asking really fast

[65:17.00]hopefully spicy questions

[65:19.00]and then we'll open it up to the audience

[65:21.00]for your questions so you guys

[65:23.00]around the room submit your questions

[65:25.00]and we'll go through as many of them as possible

[65:27.00]during that period

[65:29.00]and then actually SWIX brought

[65:31.00]a gift for us which is two

[65:33.00]latent space t-shirts

[65:35.00]AI engineer t-shirts

[65:37.00]and those will be awarded to the

[65:39.00]two spiciest questions

[65:41.00]askers

[65:43.00]and I'll let Josh decide on that

[65:45.00]so we want to get your spiciest takes

[65:47.00]please send them in during the event

[65:49.00]as we're talking and then also at the end

[65:51.00]alright with that

[65:53.00]let's get going

[65:55.00]ok

[65:57.00]welcome SWIX

[65:59.00]how does it feel to be interviewed

[66:01.00]rather than the interviewer

[66:03.00]weird I don't know what to do in this chair

[66:05.00]where should I put my hands

[66:07.00]you look good

[66:09.00]and I also love asking follow up questions

[66:11.00]and I tend to take over panels a lot

[66:13.00]if you ever see me on a panel

[66:15.00]I tend to ask the other panelists questions

[66:17.00]so we should be ready

[66:19.00]this is like a free interview

[66:21.00]so why not

[66:23.00]so you interviewed Ken June

[66:25.00]the CEO of Impeo4 but you didn't interview Josh

[66:27.00]so maybe tonight

[66:29.00]we will look for different questions

[66:31.00]and look for alignment

[66:33.00]I love it

[66:35.00]I just want to hear this story

[66:37.00]you've completely exploded with latent space

[66:39.00]and AI engineer and I know

[66:41.00]you also before all of that

[66:43.00]had exploded in popularity for your

[66:45.00]learning in public movement and your dev tools work

[66:47.00]dav relations work

[66:49.00]so who are you and how did you get here

[66:51.00]let's start with that

[66:53.00]quick story is I'm Sean I'm from Singapore

[66:55.00]SWIX is my initials for those who don't know

[66:57.00]a lot of Singaporeans are ethically Chinese

[66:59.00]and we have Chinese names and English names

[67:01.00]so it's just my initials

[67:03.00]came to the US for college

[67:05.00]and have been here for about 15 years

[67:07.00]but half of that was in finance

[67:09.00]and then the other half was in tech

[67:11.00]tech is where I was most known

[67:13.00]just because I realized that

[67:15.00]I was much more

[67:17.00]aligned towards learning in public

[67:19.00]whereas in finance everything is a trade secret

[67:21.00]everything is zero sum

[67:23.00]whereas in tech you're allowed to come

[67:25.00]to meetups and conferences and share your

[67:27.00]learnings and share your mistakes even

[67:29.00]and that's totally fine you like

[67:31.00]open source your code it's totally fine

[67:33.00]and even better you like contribute

[67:35.00]pr to other people's code which is even better

[67:37.00]and I found that I thrives in that

[67:39.00]learning public environments and

[67:41.00]that kind of got me started

[67:43.00]early higher

[67:45.00]early directly higher at Netlify

[67:47.00]and then did the same at EWS

[67:49.00]Temporal and Airbyte

[67:51.00]and so that's like the whole story

[67:53.00]I can talk more about like developer tooling

[67:55.00]and developer relations if that's something

[67:57.00]that people are interested in

[67:59.00]but I think the more recent thing is AI

[68:01.00]and I started really being interested

[68:03.00]in it mostly because

[68:05.00]the approximate cause of starting learning

[68:07.00]in space was stable diffusion

[68:09.00]when you could run a large model

[68:11.00]on your desktop

[68:13.00]where I was like okay this is

[68:15.00]something qualitatively very different

[68:17.00]and then we started

[68:19.00]in space and

[68:21.00]we have to talk about it on a podcast

[68:23.00]there we go

[68:25.00]it wasn't a podcast for like four months

[68:27.00]and then I had been running a discord

[68:29.00]for DevTools investors

[68:31.00]I also invested in DevTools

[68:33.00]and I advised companies on DevTools

[68:35.00]definition things

[68:37.00]and I think it was the start of 2023

[68:39.00]Alessio and I were both like

[68:41.00]I think we need to get more tokens out of

[68:43.00]people and I was running out of original sources

[68:45.00]to write about

[68:47.00]so I was like okay I'll go get those original sources

[68:49.00]and I think that's when we started the podcast

[68:51.00]and I think it's just the chemistry between us

[68:53.00]the way we spike in different ways

[68:55.00]and also honestly the

[68:57.00]kind participation of the guests

[68:59.00]to give us their time

[69:01.00]getting George Hoss was a big deal

[69:03.00]and also shoutout to Alessio for cold emailing him

[69:05.00]for booking some of our

[69:07.00]biggest guests

[69:09.00]and just working really hard to try to tell the story

[69:11.00]that people can use at work

[69:13.00]I think that there's a lot of AI podcasts out there

[69:15.00]and a lot of AI forums

[69:17.00]or fireside chats with no fire

[69:19.00]that always talk about

[69:21.00]what's your AGI timeline, what's your PDoom

[69:23.00]very very nice hallway conversations

[69:25.00]for freshman year but not very useful

[69:27.00]for work

[69:29.00]and practically making money

[69:31.00]and thinking about

[69:33.00]changing everyday lives

[69:35.00]what's interesting is obviously

[69:37.00]you care about the existential

[69:39.00]safety of the human race

[69:41.00]but in the meantime we got to eat

[69:43.00]so I think that's kind of

[69:45.00]the inspaces niche

[69:47.00]we explicitly don't really talk about AGI

[69:49.00]we explicitly don't talk about

[69:51.00]things that we're a little bit too far out

[69:53.00]we don't do a ton of robotics

[69:55.00]we don't do a ton of high frequency trading

[69:57.00]there's tons of machine learning in there

[69:59.00]but we just don't do that

[70:01.00]we're like what are most software engineers going to need

[70:03.00]because that's our background

[70:05.00]and that's the audience that we serve

[70:07.00]and I think just being really clear on that audience

[70:09.00]has resonated with people

[70:11.00]you would never expect a technical podcast

[70:13.00]to reach a general audience

[70:15.00]like top 10 on the tech charts

[70:17.00]but I've been surprised by that before

[70:19.00]and it's been successful

[70:21.00]I don't know what to say about that

[70:23.00]I think honestly I kind of

[70:25.00]have this negative reaction towards being

[70:27.00] being classified as a podcast

[70:29.00]because the podcast is downstream of ideas

[70:31.00]and it's one mode of conversation

[70:33.00]it's one mode of idea delivery

[70:35.00]but you can deliver ideas

[70:37.00]on a newsletter in a person like this

[70:39.00]there's so many different ways

[70:41.00]and so I think I think about it more as

[70:43.00]we are trying to start or serve an industry

[70:45.00]and that industry is the AI

[70:47.00]engineer industry

[70:49.00]which we can talk about more

[70:51.00]yes let's go into that

[70:53.00]so the AI engineer you penned a piece

[70:55.00]all the rise of the AI engineer

[70:57.00]you tweeted about it

[70:59.00]you also responded

[71:01.00]largely agreeing with what you said

[71:03.00]what is an AI engineer

[71:05.00]the AI engineer is the software engineer

[71:07.00]building with AI

[71:09.00]enhanced by AI

[71:11.00]and eventually it will be a non-human

[71:13.00]engineers writing code for you

[71:15.00]which I know impu is all about

[71:17.00]you're saying eventually the AI engineer

[71:19.00]will become a non-human engineer

[71:21.00]that will be one kind of AI engineer

[71:23.00]that people are trying to build

[71:25.00]and is probably the most furthest away

[71:27.00]because it's so hard

[71:29.00]but there are three types of AI engineer

[71:31.00]one is AI enhanced

[71:33.00]where you use AI products like co-pilot

[71:35.00]and two is AI products engineer

[71:37.00]where you use the exposed AI capabilities

[71:39.00]to the end user

[71:41.00]as a software engineer like not doing pre-training

[71:43.00]not being an ML researcher

[71:45.00]not being an ML engineer

[71:47.00]but just interacting with foundation models

[71:49.00]probably APIs from foundation model labs

[71:51.00]what's the third one

[71:53.00]and the third one is the non-human AI engineer

[71:55.00]the fully autonomous

[71:57.00]dream coder

[71:59.00]how long do you think it is till we get to

[72:01.00]early

[72:03.00]this is my equivalent of AGI timelines

[72:05.00]I know I know

[72:07.00]lots of active

[72:09.00]I have supported companies

[72:11.00]actively working on that

[72:13.00]I think it's more useful to think about levels of autonomy

[72:15.00]and so my answer to that

[72:17.00]is perpetually five years away

[72:19.00]until it figures it out

[72:21.00]no but my actual antidote

[72:23.00]the closest comparison we have to that is self-driving

[72:25.00]we're doing this in San Francisco

[72:27.00]for those who are watching on the live stream

[72:29.00]if you haven't come to San Francisco

[72:31.00]and seen and taken a waymo ride

[72:33.00]just come get a friend and take a waymo ride

[72:35.00]I remember 2014

[72:37.00]we covered a little bit of autos in my hedge fund

[72:39.00]and I remember telling a friend

[72:41.00]I was self-driving cars around the corner

[72:43.00]this is it

[72:45.00]parking will be a thing of the past

[72:47.00]and it didn't happen for the next ten years

[72:49.00]but now most of us in San Francisco

[72:51.00]can take it for granted

[72:53.00]I think you just have to

[72:55.00]be mindful that

[72:57.00]the rough edges take a long time

[72:59.00]and yes it's going to work in demos

[73:01.00]then it's going to work a little bit further out

[73:03.00]and it's just going to take a long time

[73:05.00]the more useful mental model I have

[73:07.00]is levels of autonomy

[73:09.00]in self-driving you have level one, two, three, four, five

[73:11.00]just the amount of human attention

[73:13.00]that you getat first

[73:15.00]your hands are always on ten and two

[73:17.00]and you have to pay attention to the driving

[73:19.00]every thirty seconds

[73:21.00]and eventually you can sleep in the car

[73:23.00]there's a whole spectrum of that

[73:25.00]what's the equivalent for coding

[73:27.00]keep your hands on the keyboard

[73:29.00]and then eventually you have to accept everything

[73:31.00]that's good

[73:33.00]approve the PR

[73:35.00]approve this looks good

[73:37.00]that's the dream that people want

[73:39.00]because really you unlock a lot of coding

[73:41.00]when people, non-technical people can file issues

[73:43.00]and then the

[73:45.00]AI engineer can sort of automatically

[73:47.00]write code, pass your tests

[73:49.00]and if it kind of works as

[73:51.00]as advertised, then you can just kind of merge it

[73:53.00]and then you

[73:55.00]10x, 100x, the number of developers in your company

[73:57.00]immediately

[73:59.00]so that's the goal, that's the holy grail

[74:01.00]we're not there yet, but sweep, code gen

[74:03.00]there's a bunch of companies, magic probably

[74:05.00]are all working towards that

[74:07.00]and so the TLDR

[74:09.00]the thing that we covered, Leslie and I covered

[74:11.00]in the January recap

[74:13.00]that we did was that

[74:15.00]the people should have in their minds is the inner loop

[74:17.00]versus the outer loop for the developer

[74:19.00]inner loop is everything that happens

[74:21.00]in your IDE between git commits

[74:23.00]and outer loop is what happens

[74:25.00]when you push up your git commit to

[74:27.00]github, for example, or gitlab

[74:29.00]and that's a nice split, which means

[74:31.00]everything local, everything that needs to be fast

[74:33.00]everything that's kind of very hands on for developers

[74:35.00]is probably easier to automate

[74:37.00]or easier to have code assistance

[74:39.00]that's what copilot is, that's what all those things are

[74:41.00]and then everything that happens autonomously

[74:43.00]or effectively away from the keyboard

[74:45.00]with a github issue or something

[74:47.00]that is more outer loop where

[74:49.00]you're relying a lot more on autonomy

[74:51.00]and we are maybe not smart enough

[74:53.00]to do that yet

[74:55.00]Do you have any thoughts on the user experience

[74:57.00]and how that will change? One of the things that

[74:59.00]has happened for me, looking at some of these products

[75:01.00]and playing around with things ourselves

[75:03.00]it sounds good to have an automated PR

[75:05.00]then you get an automated PR and you're like

[75:07.00]I really don't want to review 300 lines

[75:09.00]of generated code and find the bug

[75:11.00]and then you have another agent that's a reviewer

[75:13.00]and then they just come up to you

[75:15.00]and then you like tell it, go fix it

[75:17.00]and it comes back with 400 lines

[75:19.00]yes, there is a length bias to code

[75:21.00]and

[75:23.00]you do have higher passing rates

[75:25.00]in PRs, this is a documented human behavior

[75:27.00]thing, send me two lines of code

[75:29.00]I will review the shit out of that

[75:31.00]I don't know if I can swear on this

[75:33.00]send me 200 lines of code, looks good to me

[75:35.00]guess what, the agents are going to

[75:37.00]perfectly happy to copy that behavior from us

[75:39.00]when we actually want them to do the opposite

[75:41.00]so, yeah, I think

[75:43.00]the GAN model of code generation

[75:45.00]is probably not going to work super well

[75:47.00]I do think we probably need just

[75:49.00]better planning from the start

[75:51.00]which is, I'm just repeating the

[75:53.00]MBU thesis, by the way

[75:55.00]just go listen to Kanjin talk about this

[75:57.00]she's much better at it than I am

[75:59.00]but yeah, I think

[76:01.00]the code review thing is going to be

[76:03.00]I think that what codium

[76:05.00]the two codiums, the Israeli one

[76:07.00]Israeli codium

[76:09.00]with the E

[76:11.00]Yeah, codium with the E

[76:13.00]They still have refused to rename

[76:15.00]I'm friends with both of them

[76:17.00]Every month I'm like

[76:19.00]Guys, lets all come to one room

[76:21.00]Someone's got a fold

[76:23.00]Codium with the E has gone

[76:25.00]You got to write the test first

[76:27.00]It's like a sort of tripartite

[76:29.00]relationship, again this is also covered on a

[76:31.00]podcast with them which is fantastic

[76:33.00]You interview me, you sort of through me

[76:35.00]So, codium is like

[76:37.00]They've already thought this all the way through

[76:39.00]They're like, okay you write the user story

[76:41.00]From the user story you generate all the tests

[76:43.00]You also generate the code

[76:45.00]And you update any one of those

[76:47.00]They all have to update together

[76:49.00]And probably the critical factor

[76:51.00]Is the test generation from the story

[76:53.00]Because everything else

[76:55.00]Can just kind of bounce the hits off

[76:57.00]Of those things until they pass

[76:59.00]So you have to write good tests

[77:01.00]It's kind of like the eat your vegetables of coding

[77:03.00]Which nobody really wants to do

[77:05.00]And so I think it's a really smart tactic

[77:07.00]To go to market

[77:09.00]By saying we automatically generate

[77:11.00]Test for you and start not great

[77:13.00]But then get better

[77:15.00]And eventually you get to

[77:17.00]The weakest point in the chain

[77:19.00]For the entire loop of code generation

[77:21.00]What do you think their weakest link is

[77:25.00]The weakest link

[77:27.00]It's test generation

[77:29.00]Do you think there's a way to

[77:31.00]To make that actually better

[77:33.00]For making it better

[77:37.00]You have to have good isolation

[77:39.00]And I think

[77:41.00]Proper, serverless cloud environment

[77:43.00]Is integral to that

[77:45.00]It could be like a fly I/O

[77:47.00]It could be like

[77:49.00]A cloud fair worker

[77:51.00]It depends how many resources

[77:53.00]Your test environment needs

[77:55.00]And effectively I was talking about this

[77:57.00]I think with maybe Rob earlier in the audience

[77:59.00]Every agent needs a sandbox

[78:01.00]If you're a code agent you need a coding sandbox

[78:03.00]But if you're whatever

[78:05.00]Like MBU used to have this

[78:07.00]Minecrafts clone that was much faster

[78:09.00]If you have a model of the real world

[78:11.00]You have to go generate some plan

[78:13.00]Or some code or some whatever

[78:15.00]Test it against that real world

[78:17.00]So that you can get this iterative feedback

[78:19.00]And then get the final result back

[78:21.00]That is somewhat validated against the real world

[78:23.00]And so you need a really good sandbox

[78:25.00]I don't think people

[78:27.00]This is an infrastructure need

[78:29.00]Humans have had for a long time

[78:31.00]We've never solved it for ourselves

[78:33.00]And now we have to solve it for about

[78:35.00]A thousand times larger quantity of agents

[78:37.00]Then actually exist

[78:39.00]And so I think we actually have to

[78:41.00]Involve a lot more infrastructure

[78:43.00]In order to serve these things

[78:45.00]So for those who don't know

[78:47.00]I also have

[78:49.00]So we're talking about the rise of AI engineer

[78:51.00]I also have various conversations

[78:53.00]About immutable infrastructure

[78:55.00]And this is all of the kinds

[78:57.00]In order to solve

[78:59.00]Agents and coding agents

[79:01.00]We're going to have to solve the other stuff too along the way

[79:03.00]And it's really neat for me

[79:05.00]To see all that tied together in my deaf tools work

[79:07.00]That all these themes kind of reemerge

[79:09.00]Just naturally just because

[79:11.00]Everything we needed for humans

[79:13.00]We just need a hundred times more for agents

[79:15.00]Let's talk about the AI engineer

[79:17.00]AI engineer has become a whole thing

[79:19.00]It's become a term and also a conference

[79:21.00]And tell us more

[79:23.00]And a job title

[79:25.00]Tell us more about that

[79:27.00]What's going on there

[79:29.00]That is a very big cloud of things

[79:31.00]I would just say

[79:33.00]I think it's an emergent industry

[79:35.00]I've seen this happen repeatedly

[79:37.00]So the general term

[79:39.00]So the general term is software engineer

[79:41.00]Or programmer

[79:43.00]In the 70s and 80s

[79:45.00]There would not be a senior engineer

[79:47.00]There would just be an engineer

[79:49.00]I don't think you would even call

[79:51.00]What about a member of the technical staff

[79:53.00]Oh yeah MTS

[79:55.00]Very very elite

[79:57.00]So these striations appear

[79:59.00]When the population grows

[80:01.00]And the technical depth grows

[80:03.00]Over time

[80:05.00]When it starts

[80:07.00]Not that important

[80:09.00]It's just going to specialize

[80:11.00]I've seen this happen for front end

[80:13.00]For DevOps, for data

[80:15.00]I can't remember what else I listed in that piece

[80:17.00]But those are the main three that I was around for

[80:19.00]Now a lot of people are arguing

[80:21.00]That there is the ML researcher

[80:23.00]The ML engineer

[80:25.00]Who sort of pairs with the researcher

[80:27.00]Sometimes they also call research engineer

[80:29.00]And then on the other side of the fence

[80:31.00]It's just software engineers

[80:33.00]And that's how it was until about last year

[80:35.00]And now there's this specializing

[80:37.00]And rising class of people

[80:39.00]Building AI specific software

[80:41.00]That are not any of those previous titles

[80:43.00]That I just mentioned

[80:45.00]And that's the thesis of the AI engineer

[80:47.00]In the emerging category of start-ups

[80:49.00]Of jobs

[80:51.00]I've had people from Meta, IBM, Microsoft

[80:53.00]Open AI tell me that their title

[80:55.00]Is now AI engineer

[80:57.00]So like I can see that this is a trend

[80:59.00]And I think that's what Andre called out

[81:01.00]In his post that like just mathematically

[81:03.00]Just the limitations in terms of talents

[81:05.00]Research talents and GPUs

[81:07.00]That all these will tend to concentrate

[81:09.00]In a few labs

[81:11.00]And everyone else

[81:13.00]Are just going to have to rely on them

[81:15.00]Or build differentiation of products

[81:17.00]In other ways, and those will be AI engineers

[81:19.00]So mathematically there will be more AI engineers

[81:21.00]Than ML engineers, it's just the truth

[81:23.00]Right now it's the other way

[81:25.00]Right now the number of AI engineers

[81:27.00]Is maybe 10x less

[81:29.00]So I think that the ratio will invert

[81:31.00]And I think the goal of it in space

[81:33.00]And the goal of the conference

[81:35.00]And anything else I do is to serve

[81:37.00]That growing audience

[81:39.00]To make the distinction clear

[81:41.00]If I'm a software engineer

[81:43.00]What do I have to learn

[81:45.00]What additional capabilities does that

[81:47.00]Type of engineer have

[81:49.00]Funny you say that

[81:51.00]I don't actually have a specific blog

[81:53.00]Post on how to like

[81:55.00]Change classes

[81:57.00]I do think I always think about this

[81:59.00]In terms of Baldur's Gate

[82:01.00]DND ruleset number

[82:03.00]5.1 or whatever

[82:05.00]So I kind of intentionally left that open

[82:07.00]To leave space for others

[82:09.00]I think when you start an industry

[82:11.00]That's the only way to guarantee

[82:13.00]That it will fail

[82:15.00]I do have a take

[82:17.00]Obviously because a lot of people

[82:19.00]Are asking me where to start

[82:21.00]And I think basically

[82:23.00]So what we have is

[82:25.00]Late and Space University

[82:27.00]We just finished working on

[82:29.00]Day 7 today

[82:31.00]It's a seven-day project

[82:33.00]And I think we've done a great job

[82:35.00]We've done a great job

[82:37.00]We've done a great job

[82:39.00]And we've just finished working on

[82:41.00]Day 7 today

[82:43.00]It's a seven-day email course

[82:45.00]Where it basically like

[82:47.00]It is completely designed to answer

[82:49.00]The question of like

[82:51.00]I'm an existing software engineer

[82:53.00]I know how to code

[82:55.00]But I don't get all this AI stuff

[82:57.00]I've been living under a rock

[82:59.00]Or like it's just too overwhelming for me

[83:01.00]You have to pick for me

[83:03.00]Or curate for me as a trusted friend

[83:05.00]And I have one hour a day for seven days

[83:07.00]It's image generation

[83:09.00]It's code generation

[83:11.00]It's audio

[83:13.00]ASR

[83:15.00]Audio speech recognition

[83:17.00]And then I forget

[83:19.00]What the fifth and sixth one is

[83:21.00]But the last day is agents

[83:23.00]And so basically I'm just like

[83:25.00]Here are seven projects that you should do

[83:27.00]To feel like you can do anything in AI

[83:29.00]You can't really do everything in AI

[83:31.00]Just from that small list

[83:33.00]But I think it's just like anything

[83:35.00]Go through like a set list

[83:37.00]Of things that are basic skills

[83:39.00]That I think everyone in this industry should have

[83:41.00]To be at least conversant

[83:43.00]In if someone, if like a boss comes to you

[83:45.00]And goes like hey can we build this

[83:47.00]You don't even know if the answer is no

[83:49.00]So I want you to move towards

[83:51.00]From like unknown unknowns to at least known unknowns

[83:53.00]And I think that's where you start

[83:55.00]Being competent as an engineer

[83:57.00]So yeah that's LSU

[83:59.00]In Space University just to trigger the tigers

[84:03.00]So do you think in the future that people

[84:05.00]An AI engineer is going to be someone's

[84:07.00]Full-time job like people are just going to be

[84:09.00]AI engineers or do you think it's going to be

[84:11.00]More of a world where I'm a software engineer

[84:13.00]And like 20% of my time

[84:15.00]I'm using open ai's, api's

[84:17.00]And I'm working on prompt engineering

[84:19.00]And stuff like that and using code pilot

[84:21.00]You just reminded me of day six's

[84:23.00]Open source models and fine tuning

[84:25.00]I think it will be a spectrum. That's why I don't want to be

[84:27.00]Like too definitive about it. Like we have

[84:29.00]Full-time front-end engineers and we have part-time

[84:31.00]And you dip into that community whenever you want

[84:33.00]But wouldn't it be nice if there was a

[84:35.00]Collective name for that community

[84:37.00]So you could go find it, you could find each other

[84:39.00]And like honestly that's really it

[84:41.00]Like a lot of people, a lot of companies are pinging me

[84:43.00]For like hey I want to hire this kind of person

[84:45.00]But you can't hire that person

[84:47.00]But I want someone like that

[84:49.00]And then people on the labor side

[84:51.00]Were pinging me going like okay I want to do more

[84:53.00]In this space but where do I go

[84:55.00]And I think just having that shelling point

[84:57.00]Of what an industry title or name is

[84:59.00]And sort of building out that mythology

[85:01.00]And community and conference

[85:03.00]I think is helpful hopefully

[85:05.00]And I don't have any prescriptions

[85:07.00]On whether or not it's a full-time job

[85:09.00]I do think over time it's going to become

[85:11.00]More of a full-time job

[85:13.00]And that's great for the people who want to do that

[85:15.00]And the companies that want to employ that

[85:17.00]But it's absolutely like you can take it part-time

[85:19.00]Like jobs come in many formats

[85:21.00]Yep that makes sense

[85:23.00]And then you have a huge world fair

[85:25.00]Coming up

[85:27.00]Tell me about that

[85:29.00]So part of I think

[85:31.00]What creating industry requires

[85:33.00]Is to let people gather in one place

[85:35.00]And also for me

[85:37.00]To get high quality talks out of people

[85:39.00]You have to create an event out of it

[85:41.00]Otherwise they don't do the work

[85:43.00]So last year we did

[85:45.00]The AI engineer summit which went very well

[85:47.00]And people can see that online

[85:49.00]And we're very happy with how that turned out

[85:51.00]This year we want to go four times bigger

[85:53.00]With the world fair

[85:55.00]To try to reflect AI engineering

[85:57.00]As it is in 2024

[85:59.00]I always admired

[86:01.00]Two conferences in this respect

[86:03.00]One is NEURUPs which I went to last year

[86:05.00]And documented on the pod which was fantastic

[86:07.00]And two which is KubeCon

[86:09.00]From the other side of my life

[86:11.00]Which is the sort of cloud registration

[86:13.00]In DevOps world

[86:15.00]So NEURUPs is the one place that you go to

[86:17.00]To I think it's the top conference

[86:19.00]I mean there's others

[86:21.00]That you can kind of consider

[86:23.00]So NEURUPs

[86:25.00]NEURUPs is where the research sciences are the stars

[86:27.00]The researchers are the stars, the PhDs are the stars

[86:29.00]Mostly it's just PhDs on the job market

[86:31.00]It's really funny to go

[86:33.00]Especially these things

[86:35.00]It's really funny to go to NEURUPs

[86:37.00]And the PhDs trying to back them

[86:39.00]There are lots of VCs trying to back there

[86:41.00]This year

[86:43.00]So NEURUPs research sciences are the stars

[86:45.00]And I wanted for AI engineers

[86:47.00]Engineer to be the star

[86:49.00]To show off their tooling

[86:51.00]And their techniques

[86:53.00]And their difficulty

[86:55.00]Moving all these ideas from research into production

[86:57.00]The other one was KubeCon

[86:59.00]Where you could honestly just go

[87:01.00]And not attend any of the talks

[87:03.00]And just walk the floor

[87:05.00]And figure out what's going on in DevOps

[87:07.00]Which is fantastic

[87:09.00]So that curation

[87:11.00]And that bringing together of an industry

[87:13.00]Is what I'm going for for the conference

[87:15.00]And it's coming in June

[87:17.00]The most important thing to be honest

[87:19.00]The most important thing was to buy the domain

[87:21.00]So we got AI.engineer

[87:23.00]People are like engineer is a domain

[87:25.00]And funny enough

[87:27.00].engineer was cheaper than.engineering

[87:29.00]I don't understand why

[87:31.00]But that's up to the domain people

[87:33.00]All right

[87:35.00]Josh, any questions on agents

[87:37.00]Yeah, I think maybe you have a lot of

[87:39.00]Experience and exposure

[87:41.00]Talking to all these companies and founders

[87:43.00]And researchers and everyone that's on your podcast

[87:45.00]Do you have like

[87:47.00]Do you feel like you have a good kind of perspective

[87:49.00]On some of the things that like

[87:51.00]Some of the kind of technical issues having seen

[87:53.00]Like we were just talking about like for

[87:55.00]Coding agents like oh how you know

[87:57.00]The value of test is really important

[87:59.00]There are other things like for you know retrieval

[88:01.00]Like now, you know, we have these models

[88:03.00]Coming out with a million context, you know

[88:05.00]Are a million tokens of context like 30 million

[88:07.00]Is retrieval going to matter anymore

[88:09.00]The huge context matter like what do you think

[88:11.00]Specific about the long context thing

[88:13.00]Sure, yeah

[88:15.00]I was going to ask a few other ones after that

[88:17.00]So go for that one first

[88:19.00]That's what I was going to ask first

[88:21.00]Yeah, let's talk about the long context

[88:23.00]So for those who don't know

[88:25.00]Long context was kind of

[88:27.00]In the air last year but really

[88:29.00]Really really really came into focus this year

[88:31.00]With Gemini1.5 having

[88:33.00]Million token context and saying that

[88:35.00]It was in research for 10 million tokens

[88:37.00]And that means that

[88:39.00]You can put you like

[88:41.00]No longer have to really think about

[88:43.00]What you retrieve

[88:45.00]No longer really think about

[88:47.00]What you have to put into context

[88:49.00]You can just kind of throw the entire

[88:51.00]Knowledge base in there or books or film

[88:53.00]Anything like that and that's fantastic

[88:55.00]A lot of people are thinking that it kills

[88:57.00]Rag and I think like one that's not true

[88:59.00]Because for any kind of cost reason

[89:01.00]You know, you still pay per token

[89:03.00]So basically Google is like perfectly happy

[89:05.00]To let you pay a million tokens

[89:07.00]Every single time you make an API call

[89:09.00]But good luck, you know, having $100 API call

[89:11.00]You don't want to be slow, no explanation needed

[89:13.00]And then finally my criticism of

[89:15.00]Long context is that it's also not debuggable

[89:17.00]Like if something goes wrong with the result

[89:19.00]You can't do like the rags decomposition

[89:21.00]Of where the source of error

[89:23.00]Like you just have to like go like

[89:25.00]It's the way it's bro like it's somewhere in there

[89:27.00]I'm sorry, I pretty strongly agree with this

[89:29.00]Why do you think people are making such

[89:31.00]Crazy Long context windows

[89:32.00]People love to kill rag

[89:33.00]It's so much because it's too expensive

[89:35.00]It's so expensive like you said

[89:37.00]Yeah, I just think I'm just calling it

[89:39.00]It's a different dimension

[89:40.00]I think it's an option that's great when it's there

[89:42.00]Like when I'm prototyping

[89:43.00]I do not ever want to worry about context

[89:45.00]And I'm going to call stuff a few times

[89:47.00]And I don't want to run the errors

[89:48.00]I don't want to have it set up a complex retrieval system

[89:50.00]Just to prototype something

[89:52.00]But once I done prototyping

[89:53.00]Then I'll worry about all the other rags stuff

[89:55.00]And yes, I'm going to buy some system

[89:57.00]Or build some system or whatever to go do that

[90:00.00]So I think it's just like an improvement

[90:03.00]In like one dimension that you need

[90:05.00]But the improvements in the other dimensions

[90:07.00]And it's all needed

[90:08.00]Like this space isn't going to keep growing

[90:10.00]In unlimited fashion

[90:12.00]I do think that this combined with multi-modality

[90:16.00]Does unlock new things

[90:18.00]So that's what I was going to ask about next

[90:19.00]It's like how important is multimodal

[90:21.00]Like great, you know, generating videos

[90:23.00]Sure, whatever

[90:24.00]Okay, how many of us need to generate videos that often

[90:26.00]It'd be cool for TV shows, sure, but like, yeah

[90:28.00]I think it's pretty important

[90:30.00]The one thing that in

[90:31.00]When we launched the in space podcast

[90:33.00]We listed a bunch of interest areas

[90:35.00]One thing I love about being explicit

[90:37.00]Or intentional about our work

[90:40.00]Is that you list the things that you're interested in

[90:42.00]And you list the things that you're not interested in

[90:44.00]And people are very unwilling

[90:46.00]To have an entire interest list

[90:48.00]One of the things that we are not interested in

[90:50.00]Was multimodality last year

[90:52.00]Because everyone was

[90:54.00]I was just like, okay, you can generate images

[90:56.00]And they're pretty, but like, not a giant business

[90:58.00]I was wrong

[90:59.00]Journey is a giant, giant, massive business

[91:01.00]That no one can understand or get into

[91:03.00]But also, I think

[91:05.00]Being able to natively understand

[91:07.00]Audio and video and code

[91:09.00]I consider code a special modality

[91:11.00]All that is

[91:13.00]Very, like, qualitatively different

[91:15.00]Then translating it into English first

[91:17.00]And using English as, you know, like a bottleneck

[91:19.00]Or pipe

[91:20.00]And then, you know, applying it in LLMs

[91:22.00]The ability of LLMs to reason across modalities

[91:25.00]Gives you something more than you could

[91:27.00]Individually by using Texas

[91:29.00]The universal interface

[91:30.00]So I think that's useful

[91:32.00]So concretely, what does that mean?

[91:34.00]It means that

[91:35.00]So, I think the reference post for everyone

[91:37.00]That you should have in your head

[91:39.00]Is Simon Willison's post on Gemini1.5's

[91:41.00]Video capability

[91:43.00]Where he basically shot a video of

[91:45.00]His bookshelf, just kind of scanning through it

[91:47.00]And he was able to give back a complete

[91:49.00]JSON list of the books and the authors

[91:51.00]And all the details that were visible there

[91:53.00]Helucinated some of it

[91:55.00]Which is, you know, another issue

[91:57.00]But I think it's just like unlocks this use case

[91:59.00]That you just would not even try to code

[92:01.00]Without the native video understanding capability

[92:04.00]And obviously, like, on a technical level

[92:07.00]Video is just a bunch of frames

[92:08.00]So it actually is just image understanding

[92:10.00]But image within the temporal dimension

[92:12.00]Which this month, I think

[92:14.00]Became much more of a important thing

[92:16.00]Like the integration of space and time

[92:18.00]In transformers

[92:19.00]I don't think anyone was really talking about that

[92:21.00]Until this month

[92:22.00]And now it's the only thing anyone can ever think about

[92:24.00]For Sora and for all the other stuff

[92:26.00]The last thing I'll say

[92:28.00]Which is against this trend of

[92:30.00]Every modality is important

[92:32.00]They just do all the modalities

[92:34.00]I kind of agree with Nat Friedman

[92:36.00]Who actually kind of pointed out

[92:37.00]Just before the Gemini thing blew up

[92:39.00]This month

[92:41.00]Which was, like, why is it that

[92:43.00]OpenAI is pushing Dolly so hard

[92:45.00]Why is Bing pushing Bing Image Creator?

[92:47.00]Like, it's not apparent

[92:49.00]That you have to create images to create AGI

[92:51.00]But every lab just seems to want to do this

[92:54.00]And I kind of agree

[92:55.00]That it's not on the critical path

[92:57.00]Especially for image generation

[92:59.00]Maybe image understanding, video understanding

[93:01.00]Yeah, consumption

[93:02.00]But generation

[93:04.00]Maybe we'll be wrong next year

[93:06.00]Just catches you a bunch of flak with, like, you know

[93:08.00]Culture war things

[93:10.00]It's true

[93:11.00]All right, we're going to move into

[93:13.00]Rapidfire Q&A

[93:14.00]So, we're going to ask you a question

[93:16.00]I don't want to overthink it, baby

[93:18.00]We're going to audience the Q&A

[93:20.00]So, I'll tell you

[93:22.00]All right

[93:23.00]We've cut the Q&A section for time

[93:25.00]So, if you want to hear the spicy questions

[93:27.00]Head over to the Thursday nights in A.I. video

[93:29.00]for the full discussion

[93:31.00]Next up, we have another former guest

[93:33.00]Dylan Patel of Semi-analysis

[93:36.00]the inventor of the GPU rich poor divide

[93:39.00]who did a special live show with us in March

[93:41.00]But that means you can finally

[93:45.00]side-to-side A.V. test your favorite boba shops

[93:48.00]We got Gongcha, we got boba guys

[93:50.00]We got the lemon, whatever it's called

[93:53.00]So, let us know what's your favorite

[93:55.00]We also have Slido up to submit questions

[93:58.00]We already had Dylan on the podcast

[94:00.00]And this guy tweets and writes about all kinds of stuff

[94:02.00]So, we want to know what people want to know more about

[94:05.00]Rather than just being self-truined

[94:07.00]But we'll do a state of the union, maybe

[94:10.00]Everybody wants to know about GROC

[94:12.00]Everybody wants to know whether or not

[94:14.00]MVD is going to zero after GROC

[94:16.00]Everybody wants to know what's going on with AMD

[94:18.00]We got some AMD folks in the crowd, too

[94:20.00]So, feel free to interact at any time

[94:23.00]We have HECCLE, HECCLE, please

[94:25.00]Good comedians show their color

[94:28.00]The way they can handle the crowd when they're HECCLE

[94:31.00]Do not throw boba

[94:33.00]Do not throw boba at the stand

[94:35.00]We cannot afford another podcast thing set up

[94:37.00]Awesome, welcome everybody

[94:39.00]To the seminalysis and latest space crossover

[94:41.00]Dylan texted me on signal

[94:43.00]He was like, dude, how do I easily set up a meetup

[94:46.00]And here we are today

[94:48.00]As you might have seen, there's no name tags

[94:50.00]There's a bunch of things that are missing

[94:52.00]But we did our best

[94:53.00]It was extremely easy, right?

[94:55.00]Like, I texted Alessio, he's like, yo, I got the spot

[94:58.00]Okay, cool, here's a link, send it to people

[95:00.00]Send it, and then show it up

[95:03.00]And like, there was zero other organization that I required

[95:07.00]So, everybody's here

[95:09.00]A lot of seminalysis fans, we get in the crowd

[95:12.00]Everybody wants to know more about

[95:14.00]What's going on today and GROC has definitely been the hottest thing

[95:16.00]We just recorded our monthly podcast today

[95:18.00]And we didn't talk that much about GROC

[95:20.00]Because we wanted you to talk more about

[95:22.00]And then we'll splice you into our monthly recap

[95:24.00]So, let's start there

[95:26.00]So, you guys are the two GROC spreadsheetors

[95:29.00]So, we broke out some GROC numbers

[95:32.00]Because everyone was wondering

[95:33.00]There's two things going on, right?

[95:34.00]One, you know, how important, you know

[95:36.00]How does it achieve the inference speed that it does

[95:38.00]That it has been demonstrated by GROC chat

[95:41.00]And two, how does it achieve its price promise

[95:44.00]That is promised, that is sort of the public pricing

[95:46.00]Of 27 cents per million token

[95:48.00]And there's been a lot of speculation

[95:50.00]Or, you know, some numbers thrown out there

[95:52.00]I put out some tentative numbers

[95:54.00]And you put out different numbers

[95:55.00]But I'll just kind of lay that as the groundwork

[95:58.00]Like, everyone's very excited about

[96:00.00]Essentially like five times faster

[96:02.00]Token generation than any other LLM currently

[96:05.00]And that unlocks interesting downstream possibilities

[96:08.00]If it's sustainable

[96:10.00]If it's affordable

[96:11.00]And so I think your question

[96:13.00]Or reading your piece on GROC

[96:15.00]Which is on the screen right now

[96:16.00]Is it sustainable

[96:18.00]So, like many things

[96:20.00]This is VC funded, including this Boba

[96:23.00]No, I'm just kidding

[96:24.00]I'm paying for the Boba

[96:25.00]Thank you, Semunas, the subscribers

[96:27.00]I hope he pays for it

[96:29.00]I pay for it right now

[96:30.00]That's true, Alessio has the IOU

[96:33.00]Right?

[96:34.00]And that's all it is

[96:35.00]But yeah, like many things, you know

[96:37.00]They're not making money off of their inference service

[96:40.00]They're just throwing it out there for cheap

[96:42.00]And hoping to get business

[96:43.00]And maybe raise money off of that

[96:45.00]And I think that's a that's a fine use case

[96:48.00]But the question is like

[96:49.00]How much money are they losing, right?

[96:51.00]And that's sort of what I went through

[96:52.00]Breaking down in this article

[96:54.00]That's on the screen

[96:55.00]And that's it's pretty clear

[96:56.00]They're like seven to ten X off

[97:00.00]Like break even on their inference API

[97:02.00]Which is like horrendous

[97:04.00]Like far worse than any other

[97:06.00]Sort of inference API provider

[97:08.00]So this is like a simple simple

[97:10.00]Cost thing that was pulled up

[97:12.00]You can either inference at

[97:13.00]Very high throughput

[97:14.00]Or you can inference at

[97:15.00]Very high very low latency

[97:17.00]With GPUs you can do both

[97:18.00]With GROC you can only do one

[97:20.00]Of course with GROC

[97:21.00]You can do that one faster

[97:22.00]Marginally faster than a

[97:24.00]Inference latency optimized GPU server

[97:26.00]But no one offers inference latency

[97:28.00]Optimized GPU servers

[97:29.00]Because you would just burn money

[97:31.00]Makes no economic sense to do so

[97:33.00]Until maybe someone's willing

[97:34.00]To pay for that

[97:35.00]So GROC service

[97:36.00]You know the surface looks awesome

[97:37.00]Compared to everyone else's service

[97:39.00]Which is throughput optimized

[97:41.00]And then when you compare to

[97:43.00]Input optimized scenario

[97:44.00]GPUs look quite slow

[97:46.00]But the reality is

[97:47.00]Is they're serving 64

[97:49.00]128 users at once

[97:51.00]They have a batch size

[97:52.00]How many users are being

[97:53.00]Served at once

[97:54.00]Where as GROC is taking

[97:55.00]576 chips

[97:56.00]And they're not really

[97:58.00]Doing that efficiently

[97:59.00]They're serving a far far

[98:01.00]Fewer number of users

[98:02.00]But extremely fast

[98:03.00]Now that could be worthwhile

[98:05.00]If they can get there

[98:07.00]The number of users

[98:08.00]They're serving at once up

[98:10.00]But that's extremely hard

[98:11.00]Because they don't have

[98:12.00]Their chip

[98:13.00]So they can't store

[98:14.00]KVcash

[98:15.00]KVcash for all the

[98:16.00]Various different users

[98:17.00]So their crux of the issue

[98:19.00]Is just like hey

[98:20.00]Can they get that performance

[98:22.00]Up as much as they claim they will

[98:24.00]They need to get it up

[98:25.00]More than 10x

[98:26.00]To make this like a reasonable

[98:28.00]Benefit

[98:29.00]In the meantime

[98:30.00]NVIDIA is launching a new

[98:31.00]GPU in two weeks

[98:32.00]That'll be fun at GTC

[98:34.00]And they're constantly

[98:35.00]Pushing software as well

[98:36.00]So we'll see

[98:37.00]If GROC can catch up to that

[98:38.00]The current verdict is

[98:40.00]They're quite far behind

[98:41.00]But hopefully

[98:42.00]That maybe they can

[98:43.00]Get there by scaling

[98:44.00]Their system larger

[98:45.00]Yeah

[98:46.00]I was listening back

[98:47.00]To our original episode

[98:48.00]And you were talking

[98:49.00]About how NVIDIA

[98:50.00]Basically adopted

[98:51.00]This different strategy

[98:52.00]Of just leaning on

[98:54.00]Networking GPUs together

[98:55.00]And it seems like

[98:56.00]GROC has some minor

[98:58.00]Version of that going on

[98:59.00]Here with the GROC rack

[99:00.00]Is it enough?

[99:03.00]What's GROC's next

[99:05.00]Step here strategically?

[99:07.00]Yeah, that's

[99:09.00]The next step is of course

[99:10.00]So right now

[99:12.00]They connect 10 racks

[99:13.00]Of chips together

[99:14.00]And that's the system

[99:15.00]That's running

[99:16.00]On their API today

[99:17.00]Whereas most people

[99:18.00]Who are running

[99:19.00]Mischro are running

[99:20.00]In on two GPUs

[99:22.00]One fourth of a server

[99:24.00]And that rack

[99:25.00]Is not

[99:26.00]Obviously 10 racks

[99:27.00]Is pretty crazy

[99:28.00]But they think

[99:29.00]That they can

[99:30.00]Scale performance

[99:31.00]If they have

[99:32.00]This individual system

[99:33.00]Be 20 racks

[99:34.00]They think they can

[99:35.00]Continue to scale performance

[99:36.00]Extra linearly

[99:37.00]So that'd be amazing

[99:38.00]If they could

[99:39.00]And I'm doubtful

[99:40.00]That's going to be something

[99:42.00]That's scalable

[99:43.00]Especially for

[99:45.00]You know, larger models

[99:47.00]So there's the chip itself

[99:51.00]But there's also a lot

[99:52.00]Of work they're doing

[99:53.00]At the compiler level

[99:54.00]Do you have any good sense

[99:55.00]Of like how easy it is

[99:57.00]To actually work with LPU

[99:59.00]Is that something

[100:00.00]That's going to be

[100:01.00]About on that for them

[100:02.00]So Ollie's in the front

[100:03.00]Right there

[100:04.00]And he knows a ton

[100:05.00]About VLIW architectures

[100:07.00]But to summarize

[100:08.00]In his opinion

[100:09.00]And I think many folks

[100:10.00]Is it's extremely hard

[100:11.00]To program

[100:12.00]These sorts of architectures

[100:14.00]Which is why they have

[100:15.00]Their compiler

[100:16.00]And so on and so forth

[100:17.00]But it's an incredible amount

[100:19.00]Of work for them

[100:20.00]To stand up individual models

[100:22.00]And to get the performance

[100:23.00]Up on them

[100:24.00]Which is what they've been

[100:25.00]Working on

[100:26.00]Whereas GPUs

[100:27.00]Are far more flexible

[100:28.00]Of course

[100:29.00]And so the question is

[100:30.00]Can this compiler

[100:31.00]Continue to extract

[100:32.00]Performance

[100:33.00]Well theoretically

[100:34.00]There's a lot more

[100:35.00]Performance to run

[100:36.00]On the hardware

[100:37.00]And they don't have

[100:38.00]Many things that people

[100:40.00]Generally associate with

[100:42.00]Programmable hardware

[100:44.00]They don't have buffers

[100:45.00]And many other things

[100:46.00]So it makes it very tough

[100:47.00]To do that

[100:48.00]But that's what their

[100:49.00]They're relatively large

[100:51.00]Compiler team is working on

[100:53.00]So I'm not a GPU compiler guy

[100:55.00]But I do want to

[100:56.00]Clarify my understanding

[100:57.00]From what I read

[100:58.00]Which is a lot of

[100:59.00]Catching up to do

[101:00.00]It is

[101:01.00]The crux of it

[101:02.00]Is some kind of speculative

[101:04.00]The word that comes

[101:05.00]The routing

[101:06.00]Of weights

[101:08.00]And work

[101:09.00]That needs to

[101:10.00]Be done or scheduling

[101:11.00]Of work across

[101:12.00]The ten racks

[101:14.00]Of GPUs

[101:15.00]Is that like

[101:17.00]The bulk of the benefit

[101:18.00]That you get

[101:19.00]From the compilation

[101:20.00]So with the grok chips

[101:22.00]What's really

[101:23.00]Interesting is like

[101:25.00]With GPUs

[101:26.00]You can issue

[101:27.00]Certain instructions

[101:29.00]And you will get

[101:30.00]A different result

[101:31.00]Like depending on

[101:32.00]The time I know

[101:33.00]A lot of people

[101:34.00]I'll have

[101:35.00]Have had that experience

[101:36.00]Or like the GPU

[101:37.00]Literally doesn't return

[101:38.00]The numbers it should be

[101:39.00]As basically called

[101:40.00]Non-determinism

[101:41.00]With grok

[101:42.00]Their chip is completely

[101:43.00]Deterministic

[101:44.00]The moment you compile it

[101:45.00]You know exactly how long

[101:46.00]It will take to operate

[101:47.00]Right there is no

[101:48.00]There is no like

[101:49.00]Deviation at all

[101:51.00]And so

[101:52.00]You know they've

[101:53.00]They're planning everything

[101:55.00]Ahead of time

[101:56.00]Like every instruction

[101:57.00]Like it will

[101:58.00]Complete in the time

[101:59.00]That they've planned it for

[102:00.00]And there is no

[102:01.00]I don't know what

[102:02.00]The best way to state this is

[102:03.00]No variance there

[102:04.00]Which is

[102:05.00]Interesting from like

[102:06.00]When you look historically

[102:07.00]They tried to push this

[102:08.00]In automotive

[102:09.00]Because automotive

[102:10.00]You probably want

[102:11.00]Your car to do

[102:12.00]Exactly what you

[102:13.00]I issued it to do

[102:14.00]And not have

[102:15.00]Unpredictability

[102:16.00]But yeah

[102:17.00]Sorry I lost track

[102:18.00]Of the question

[102:19.00]It's okay

[102:20.00]I just wanted to

[102:21.00]Understand a little bit

[102:22.00]More about like

[102:23.00]What people should

[102:24.00]Should know about

[102:25.00]The compiler magic

[102:26.00]That goes on with grok

[102:27.00]Like you know

[102:28.00]Like I think

[102:30.00]From a software

[102:31.00]Like hardware point of view

[102:32.00]That intersection of

[102:34.00]I guess

[102:35.00]So chips have like

[102:36.00]Like I'm still this

[102:37.00]From someone

[102:38.00]Here in the crowd

[102:39.00]But chips have like

[102:40.00]Five you know

[102:41.00]Sort of there's

[102:42.00]Like when you're designing a chip

[102:43.00]There's there's

[102:44.00]It's called PPA right

[102:45.00]Power performance in area

[102:47.00]It's kind of a triangle

[102:48.00]That you optimize around

[102:49.00]And the one thing

[102:50.00]People don't realize

[102:51.00]Is there's a

[102:52.00]There's a third P

[102:53.00]That's like PPA P

[102:55.00]And the last P

[102:56.00]Is pain in the ass

[102:57.00]To program

[102:58.00]And that's

[102:59.00]That is very important

[103:00.00]For like

[103:01.00]High hardware

[103:02.00]Like TPU

[103:04.00]Without the

[103:05.00]Hundreds of people

[103:06.00]That work on the compiler

[103:07.00]And jacks

[103:08.00]And XLA

[103:09.00]And all these sorts of things

[103:10.00]Would be a pain in the ass

[103:11.00]To program

[103:12.00]But Google's got that like

[103:13.00]Plumbing

[103:14.00]Now if you look across

[103:15.00]The ecosystem

[103:16.00]Everything else is a pain

[103:17.00]In the ass to program

[103:18.00]Compared to NVIDIA

[103:19.00]And this applies

[103:20.00]To the grok chip as well

[103:22.00]So yeah

[103:23.00]Question is like

[103:24.00]Can the compiler team

[103:25.00]Get performance up

[103:26.00]Anywhere close to theoretical

[103:28.00]And then can they make

[103:29.00]It not a pain in the ass

[103:30.00]To program

[103:31.00]And then can they make

[103:32.00]It not a pain in the ass

[103:33.00]To program

[103:34.00]And then can they make

[103:35.00]It not a pain in the ass

[103:36.00]To program

[103:37.00]And then can they make

[103:38.00]It not a pain in the ass

[103:39.00]And then can they make

[103:40.00]It not a pain in the ass

[103:41.00]And then can they make

[103:42.00]It not a pain in the ass

[103:43.00]And then can they make

[103:44.00]It not a pain in the ass

[103:45.00]And then can they make

[103:46.00]It not a pain in the ass

[103:47.00]And then can they make

[103:48.00]It not a pain in the ass

[103:49.00]And then can they make

[103:50.00]It not a pain in the ass

[103:51.00]And then can they make

[103:52.00]It not a pain in the ass

[103:53.00]And then can they make

[103:54.00]It not a pain in the ass

[103:55.00]And then can they make

[103:56.00]It not a pain in the ass

[103:57.00]And then can they make

[103:58.00]It not a pain in the ass

[103:59.00]And then can they make

[104:00.00]It not a pain in the ass

[104:01.00]And then can they make

[104:02.00]It not a pain in the ass

[104:03.00]And then can they make

[104:04.00]It not a pain in the ass

[104:05.00]And then can they make

[104:06.00]It not a pain in the ass

[104:07.00]And then can they make

[104:08.00]It not a pain in the ass

[104:09.00]And then can they make

[104:10.00]It not a pain in the ass

[104:11.00]And then can they make

[104:12.00]It not a pain in the ass

[104:13.00]And then can they make

[104:14.00]It not a pain in the ass

[104:15.00]And then can they make

[104:16.00]It not a pain in the ass

[104:17.00]And then can they make

[104:18.00]It not a pain in the ass

[104:19.00]And then can they make

[104:20.00]It not a pain in the ass

[104:21.00]And then can they make

[104:22.00]It not a pain in the ass

[104:23.00]And then can they make

[104:24.00]It not a pain in the ass

[104:25.00]And then can they make

[104:26.00]It not a pain in the ass

[104:27.00]And then can they make

[104:28.00]It not a pain in the ass

[104:29.00]And then can they make

[104:30.00]It not a pain in the ass

[104:31.00]And then can they make

[104:32.00]It not a pain in the ass

[104:33.00]And then can they make

[104:34.00]It not a pain in the ass

[104:35.00]And then can they make

[104:36.00]It not a pain in the ass

[104:37.00]And then can they make

[104:38.00]It not a pain in the ass

[104:39.00]And then can they make

[104:40.00]It not a pain in the ass

[104:41.00]And then can they make

[104:42.00]It not a pain in the ass

[104:43.00]And then can they make

[104:44.00]It not a pain in the ass

[104:45.00]And then can they make

[104:46.00]It not a pain in the ass

[104:47.00]And then can they make

[104:48.00]It not a pain in the ass

[104:49.00]And then can they make

[104:50.00]It not a pain in the ass

[104:51.00]And then can they make

[104:52.00]It not a pain in the ass

[104:53.00]And then can they make

[104:54.00]It not a pain in the ass

[104:55.00]And then can they make

[104:56.00]It not a pain in the ass

[104:57.00]And then can they make

[104:58.00]It not a pain in the ass

[104:59.00]And then can they make

[105:00.00]It not a pain in the ass

[105:01.00]And then can they make

[105:02.00]It not a pain in the ass

[105:03.00]And then can they make

[105:04.00]It not a pain in the ass

[105:05.00]And then can they make

[105:06.00]It not a pain in the ass

[105:07.00]And then can they make

[105:08.00]It not a pain in the ass

[105:09.00]And then can they make

[105:10.00]It not a pain in the ass

[105:11.00]And then can they make

[105:12.00]It not a pain in the ass

[105:13.00]And then can they make

[105:14.00]It not a pain in the ass

[105:15.00]And then can they make

[105:16.00]It not a pain in the ass

[105:17.00]And then can they make

[105:18.00]It not a pain in the ass

[105:19.00]And then can they make

[105:20.00]It not a pain in the ass

[105:21.00]And then can they make

[105:22.00]It not a pain in the ass

[105:23.00]And then can they make

[105:24.00]It not a pain in the ass

[105:25.00]And then can they make

[105:26.00]It not a pain in the ass

[105:27.00]And then can they make

[105:28.00]It not a pain in the ass

[105:29.00]And then can they make

[105:30.00]It not a pain in the ass

[105:31.00]And then can they make

[105:32.00]It not a pain in the ass

[105:33.00]And then can they make

[105:34.00]It not a pain in the ass

[105:35.00]And then can they make

[105:36.00]It not a pain in the ass

[105:37.00]And then can they make

[105:38.00]It not a pain in the ass

[105:39.00]And then can they make

[105:40.00]It not a pain in the ass

[105:41.00]And then can they make

[105:42.00]It not a pain in the ass

[105:43.00]And then can they make

[105:44.00]It not a pain in the ass

[105:45.00]And then can they make

[105:46.00]It not a pain in the ass

[105:47.00]And then can they make

[105:48.00]It not a pain in the ass

[105:49.00]And then can they make

[105:50.00]It not a pain in the ass

[105:51.00]And then can they make

[105:52.00]It not a pain in the ass

[105:53.00]And then can they make

[105:54.00]It not a pain in the ass

[105:55.00]And then can they make

[105:56.00]It not a pain in the ass

[105:57.00]And then can they make

[105:58.00]It not a pain in the ass

[105:59.00]And then can they make

[106:00.00]It not a pain in the ass

[106:01.00]And then can they make

[106:02.00]It not a pain in the ass

[106:03.00]And then can they make

[106:04.00]It not a pain in the ass

[106:05.00]And then can they make

[106:06.00]It not a pain in the ass

[106:07.00]And then can they make

[106:08.00]It not a pain in the ass

[106:09.00]And then can they make

[106:10.00]It not a pain in the ass

[106:11.00]And then can they make

[106:12.00]It not a pain in the ass

[106:13.00]And then can they make

[106:14.00]It not a pain in the ass

[106:15.00]And then can they make

[106:16.00]It not a pain in the ass

[106:17.00]And then can they make

[106:18.00]It not a pain in the ass

[106:19.00]And then can they make

[106:20.00]It not a pain in the ass

[106:21.00]And then can they make

[106:22.00]It not a pain in the ass

[106:23.00]And then can they make

[106:24.00]It not a pain in the ass

[106:25.00]And then can they make

[106:26.00]It not a pain in the ass

[106:27.00]And then can they make

[106:28.00]It not a pain in the ass

[106:29.00]And then can they make

[106:30.00]It not a pain in the ass

[106:31.00]And then can they make

[106:32.00]It not a pain in the ass

[106:33.00]And then can they make

[106:34.00]It not a pain in the ass

[106:35.00]And then can they make

[106:36.00]It not a pain in the ass

[106:37.00]And then can they make

[106:38.00]It not a pain in the ass

[106:39.00]And then can they make

[106:40.00]It not a pain in the ass

[106:41.00]And then can they make

[106:42.00]It not a pain in the ass

[106:43.00]And then can they make

[106:44.00]It not a pain in the ass

[106:45.00]And then can they make

[106:46.00]It not a pain in the ass

[106:47.00]And then can they make

[106:48.00]It not a pain in the ass

[106:49.00]And then can they make

[106:50.00]It not a pain in the ass

[106:51.00]And then can they make

[106:52.00]It not a pain in the ass

[106:53.00]And then can they make

[106:54.00]It not a pain in the ass

[106:55.00]And then can they make

[106:56.00]It not a pain in the ass

[106:57.00]And then can they make

[106:58.00]It not a pain in the ass

[106:59.00]And then can they make

[107:00.00]It not a pain in the ass

[107:01.00]And then can they make

[107:02.00]It not a pain in the ass

[107:03.00]And then can they make

[107:04.00]It not a pain in the ass

[107:05.00]And then can they make

[107:06.00]It not a pain in the ass

[107:07.00]And then can they make

[107:08.00]It not a pain in the ass

[107:09.00]And then can they make

[107:10.00]It not a pain in the ass

[107:11.00]And then can they make

[107:12.00]It not a pain in the ass

[107:13.00]And then can they make

[107:14.00]It not a pain in the ass

[107:15.00]And then can they make

[107:16.00]It not a pain in the ass

[107:17.00]And then can they make

[107:18.00]It not a pain in the ass

[107:19.00]And then can they make

[107:20.00]It not a pain in the ass

[107:21.00]And then can they make

[107:22.00]It not a pain in the ass

[107:23.00]And then can they make

[107:24.00]It not a pain in the ass

[107:25.00]And then can they make

[107:26.00]It not a pain in the ass

[107:27.00]And then can they make

[107:28.00]It not a pain in the ass

[107:29.00]And then can they make

[107:30.00]It not a pain in the ass

[107:31.00]And then can they make

[107:32.00]It not a pain in the ass

[107:33.00]And then can they make

[107:34.00]It not a pain in the ass

[107:35.00]And then can they make

[107:36.00]It not a pain in the ass

[107:37.00]And then can they make

[107:38.00]It not a pain in the ass

[107:39.00]And then can they make

[107:40.00]It not a pain in the ass

[107:41.00]And then can they make

[107:42.00]It not a pain in the ass

[107:43.00]And then can they make

[107:44.00]It not a pain in the ass

[107:45.00]And then can they make

[107:46.00]It not a pain in the ass

[107:47.00]And then can they make

[107:48.00]It not a pain in the ass

[107:49.00]And then can they make

[107:50.00]It not a pain in the ass

[107:51.00]And then can they make

[107:52.00]It not a pain in the ass

[107:53.00]And then can they make

[107:54.00]It not a pain in the ass

[107:55.00]And then can they make

[107:56.00]It not a pain in the ass

[107:57.00]And then can they make

[107:58.00]It not a pain in the ass

[107:59.00]And then can they make

[108:00.00]It not a pain in the ass

[108:01.00]And then can they make

[108:02.00]And then can they make

[108:03.00]And then can they make

[108:04.00]And then can they make

[108:05.00]It not a pain in the ass

[108:06.00]And then can they make

[108:07.00]It not a pain in the ass

[108:08.00]And then can they make

[108:09.00]It not a pain in the ass

[108:10.00]And then can they make

[108:11.00]It not a pain in the ass

[108:12.00]And then can they make

[108:13.00]It not a pain in the ass

[108:14.00]And then can they make

[108:15.00]It not a pain in the ass

[108:16.00]And then can they make

[108:17.00]It not a pain in the ass

[108:18.00]And then can they make

[108:19.00]It not a pain in the ass

[108:20.00]And then can they make

[108:21.00]It not a pain in the ass

[108:22.00]And then can they make

[108:23.00]It not a pain in the ass

[108:24.00]And then can they make

[108:25.00]It not a pain in the ass

[108:26.00]And then can they make

[108:27.00]It not a pain in the ass

[108:28.00]And then can they make

[108:29.00]And then can they make

[108:30.00]It not a pain in the ass

[108:31.00]And then can they make

[108:32.00]It not a pain in the ass

[108:33.00]And then can they make

[108:34.00]It not a pain in the ass

[108:35.00]And then can they make

[108:36.00]It not a pain in the ass

[108:37.00]And then can they make

[108:38.00]It not a pain in the ass

[108:39.00]And then can they make

[108:40.00]It not a pain in the ass

[108:41.00]And then can they make

[108:42.00]It not a pain in the ass

[108:43.00]And then can they make

[108:44.00]It not a pain in the ass

[108:45.00]And then can they make

[108:46.00]It not a pain in the ass

[108:47.00]And then can they make

[108:48.00]It not a pain in the ass

[108:49.00]And then can they make

[108:50.00]It not a pain in the ass

[108:51.00]And then can they make

[108:52.00]It not a pain in the ass

[108:53.00]And then can they make

[108:54.00]It not a pain in the ass

[108:55.00]And then can they make

[108:56.00]It not a pain in the ass

[108:57.00]And then can they make

[108:58.00]It not a pain in the ass

[108:59.00]And then can they make

[109:00.00]It not a pain in the ass

[109:01.00]And then can they make

[109:02.00]It not a pain in the ass

[109:03.00]And then can they make

[109:04.00]It not a pain in the ass

[109:05.00]And then can they make

[109:06.00]It not a pain in the ass

[109:07.00]And then can they make

[109:08.00]It not a pain in the ass

[109:09.00]And then can they make

[109:10.00]It not a pain in the ass

[109:11.00]And then can they make

[109:12.00]It not a pain in the ass

[109:13.00]And then can they make

[109:14.00]It not a pain in the ass

[109:15.00]And then can they make

[109:16.00]It not a pain in the ass

[109:17.00]And then can they make

[109:18.00]It not a pain in the ass

[109:19.00]And then can they make

[109:20.00]It not a pain in the ass

[109:21.00]And then can they make

[109:22.00]It not a pain in the ass

[109:23.00]And then can they make

[109:24.00]It not a pain in the ass

[109:25.00]And then can they make

[109:26.00]It not a pain in the ass

[109:27.00]And then can they make

[109:28.00]It not a pain in the ass

[109:29.00]And then can they make

[109:30.00]It not a pain in the ass

[109:31.00]And then can they make

[109:32.00]It not a pain in the ass

[109:33.00]And then can they make

[109:34.00]It not a pain in the ass

[109:35.00]And then can they make

[109:36.00]It not a pain in the ass

[109:37.00]And then can they make

[109:38.00]It not a pain in the ass

[109:39.00]And then can they make

[109:40.00]It not a pain in the ass

[109:41.00]And then can they make

[109:42.00]It not a pain in the ass

[109:43.00]And then can they make

[109:44.00]It not a pain in the ass

[109:45.00]And then can they make

[109:46.00]It not a pain in the ass

[109:47.00]And then can they make

[109:48.00]It not a pain in the ass

[109:49.00]And then can they make

[109:50.00]It not a pain in the ass

[109:51.00]And then can they make

[109:52.00]It not a pain in the ass

[109:53.00]And then can they make

[109:54.00]It not a pain in the ass

[109:55.00]And then can they make

[109:56.00]It not a pain in the ass

[109:57.00]And then can they make

[109:58.00]It not a pain in the ass

[109:59.00]And then can they make

[110:00.00]It not a pain in the ass

[110:01.00]And then can they make

[110:02.00]It not a pain in the ass

[110:03.00]And then can they make

[110:04.00]It not a pain in the ass

[110:05.00]And then can they make

[110:06.00]It not a pain in the ass

[110:07.00]And then can they make

[110:08.00]It not a pain in the ass

[110:09.00]And then can they make

[110:10.00]It not a pain in the ass

[110:11.00]And then can they make

[110:12.00]It not a pain in the ass

[110:13.00]And then can they make

[110:14.00]It not a pain in the ass

[110:15.00]And then can they make

[110:16.00]It not a pain in the ass

[110:17.00]And so then I just

[110:18.00]I just started editing them

[110:19.00]And so then I just started editing them

[110:20.00]So I have stopped

[110:21.00]Comparing RAG with long

[110:22.00]Context of Fine Tuning

[110:23.00]Hold on, you said I retweeted

[110:24.00]You defending it

[110:25.00]I thought you were hating on it

[110:26.00]And that's why I retweeted it

[110:28.00]It's not one of a defense

[110:30.00]Because everyone was like

[110:31.00]Long context is killing RAG

[110:32.00]And then I had a future

[110:33.00]Oh that should be so quadratic

[110:34.00]That's another one

[110:36.00]And I actually

[110:37.00]Messed the fine print as well

[110:38.00]Let's see

[110:39.00]Power benefits of

[110:40.00]Astrum dominant

[110:41.00]Yeah, so that's a good question

[110:43.00]Astrum is on chip memory

[110:45.00]Everyone's just using HBM

[110:46.00]If you don't have to go

[110:47.00]To off-chip memory

[110:48.00]That'd be really efficient

[110:49.00]Right?

[110:50.00]Because you're

[110:51.00]You're not moving bits around

[110:52.00]But there's always

[110:54.00]The issue of

[110:55.00]You don't have enough memory

[110:56.00]So you still have to move

[110:57.00]Bits around constantly

[110:58.00]And so that's the

[110:59.00]That's the question

[111:00.00]So yeah, sure

[111:01.00]If you can not move data

[111:02.00]Around as you compute

[111:03.00]It's going to be fantastically

[111:04.00]Efficient

[111:05.00]But that isn't really

[111:06.00]It's not really just

[111:07.00]Easier simple to do

[111:08.00]What do you think is going to be

[111:09.00]Harder in the future

[111:10.00]Like getting more energy

[111:11.00]At cheaper cost

[111:12.00]Like getting more of this hardware

[111:14.00]To run

[111:15.00]Yeah, I wonder

[111:16.00]So someone was talking about this earlier

[111:18.00]But it's like

[111:19.00]Here in the crowd

[111:20.00]And I'm looking right at him

[111:21.00]But he's complaining

[111:22.00]That journalists keep saying that

[111:24.00]You know that

[111:25.00]Like misreporting about how data centers

[111:27.00]Or what data centers

[111:28.00]Are doing to the environment

[111:29.00]Right?

[111:30.00]Which I thought was quite funny

[111:31.00]Because they're inundated by

[111:33.00]Journalists talking about data centers

[111:35.00]Like destroying the world

[111:36.00]Anyways, you know

[111:37.00]That's not quite the case

[111:38.00]But yeah, I don't know

[111:39.00]Like the power is certainly

[111:42.00]Gonna be hard to get

[111:44.00]But you know

[111:45.00]I think

[111:46.00]If you just look at history

[111:47.00]Right?

[111:48.00]Like humanity

[111:49.00]Especially America

[111:50.00]Like power

[111:51.00]Power production and usage

[111:52.00]Kept skyrocketing

[111:53.00]From like the 1700s

[111:55.00]To like 1970s

[111:57.00]And then it's kind of

[111:58.00]Flat line from there

[111:59.00]So why can't we like

[112:00.00]Go back to the like growth stage

[112:02.00]I guess it's like the

[112:03.00]The whole like mantra

[112:04.00]Of like accelerationists

[112:06.00]I guess

[112:07.00]This is IAC, yep

[112:08.00]Well, I don't think it's IAC

[112:09.00]I think it's like

[112:10.00]Same Altman

[112:11.00]Hotly believes this too

[112:12.00]And I don't think he's IAC

[112:13.00]So but yeah

[112:14.00]Like I don't think like

[112:15.00]It's like think

[112:16.00]It's like something to

[112:17.00]Think about like

[112:18.00]The US is going back

[112:19.00]To growing an energy usage

[112:21.00]Where as for the last like

[112:22.00]Forty years kind of

[112:24.00]We're flat on energy usage

[112:25.00]And what does that mean

[112:26.00]Like yeah

[112:29.00]There was another question

[112:31.00]On Marvel but kind of the

[112:32.00]I think that's

[112:33.00]It's definitely like

[112:34.00]One of these three guys

[112:35.00]We're on the buy side

[112:36.00]That are asking this question

[112:39.00]Wanna know if Marvell's stock

[112:41.00]Is gonna go up

[112:43.00]So Marvell

[112:44.00]They're doing the

[112:46.00]Customizing for GROC

[112:48.00]They also do the

[112:49.00]Trinium too

[112:50.00]And the Google CPU

[112:51.00]Yeah any other

[112:52.00]Any other chip

[112:53.00]That they're working on

[112:54.00]That people should

[112:55.00]Should keep in mind

[112:56.00]It's like yeah

[112:57.00]Any needle moving

[112:58.00]Any stock moving

[112:59.00]Yeah exactly

[113:01.00]They're working on

[113:02.00]Some more stuff

[113:03.00]Yeah I'll refrain from

[113:05.00]Yeah all right

[113:06.00]Let's see

[113:07.00]Other GROC stuff

[113:08.00]We want to get it

[113:09.00]Get through

[113:10.00]I don't think so

[113:11.00]All right

[113:12.00]Most of other ones

[113:13.00]You're going edge compute hardware

[113:16.00]Any real use cases

[113:17.00]For it

[113:18.00]Yeah I mean

[113:20.00]I have like a

[113:21.00]Really like anti edge view

[113:23.00]So many people were like

[113:25.00]Oh I'm gonna run

[113:26.00]This model on my phone

[113:27.00]Or on my laptop

[113:28.00]And I love

[113:30.00]I love how much

[113:31.00]It's raining so now

[113:32.00]I can be horrible

[113:33.00]And you people won't leave

[113:35.00]Like I want you

[113:36.00]To try and leave

[113:37.00]This building

[113:38.00]Captive audience

[113:40.00]Should I start singing

[113:41.00]Like there's

[113:42.00]Nothing you can do

[113:43.00]You definitely

[113:44.00]I'll stop you from that

[113:45.00]Sorry

[113:46.00]Edge hardware

[113:47.00]Like you know

[113:48.00]People are like

[113:49.00]I'm going to run

[113:50.00]This model on my phone

[113:51.00]Or on my laptop

[113:52.00]It makes no sense to me

[113:53.00]Current hardware

[113:54.00]Is not really capable of it

[113:55.00]So you're going to buy

[113:56.00]Any hardware

[113:57.00]To run

[113:58.00]Whatever on the edge

[113:59.00]Or you're going to

[114:00.00]Just run

[114:01.00]Very very small models

[114:02.00]But in either case

[114:03.00]You're going to end up

[114:05.00]With like

[114:06.00]The performance is really low

[114:07.00]And then whatever you spent

[114:08.00]To run it locally

[114:09.00]In the cloud

[114:10.00]It could service 10x the users

[114:12.00]So it kind of like

[114:14.00]SOL in terms of like

[114:17.00]Economics of

[114:18.00]Running things on the edge

[114:20.00]And then like latency is like

[114:23.00]For LLMs

[114:24.00]Right for LLMs

[114:25.00]It's like

[114:26.00]Not that big of a deal

[114:27.00]Relative

[114:28.00]To like

[114:29.00]Internet latency

[114:30.00]Is not that big of a deal

[114:31.00]Relative to the

[114:32.00]Use of the model

[114:33.00]Right like the actual model

[114:34.00]Operating

[114:35.00]Whether it's on edge hardware

[114:36.00]Or cloud hardware

[114:37.00]And cloud hardware

[114:38.00]Like edge hardware

[114:39.00]Is not really

[114:40.00]Able to like

[114:41.00]Have a measurable

[114:43.00]Appreciable

[114:44.00]Like advantage

[114:45.00]Over cloud hardware

[114:48.00]This applies

[114:49.00]To diffusion models

[114:50.00]This applies to LLMs

[114:52.00]Of course small models

[114:53.00]Will be able to run

[114:54.00]But not all

[114:55.00]Yeah

[114:56.00]What chances

[114:57.00]The startups

[114:58.00]Like medax etched

[114:59.00]Or 5,600

[115:00.00]I think you all interviewed them

[115:01.00]Why don't you answer

[115:02.00]Yeah we have connections

[115:03.00]With medax and lemur

[115:04.00]And we haven't know

[115:06.00]But Gavin is friendly

[115:07.00]They didn't

[115:08.00]Yeah they said

[115:09.00]They don't want to talk publicly

[115:10.00]Yeah

[115:11.00]What they're doing

[115:12.00]It's something like

[115:13.00]When they open up

[115:14.00]We can

[115:15.00]Sure sure

[115:16.00]Yeah

[115:17.00]But do you think like

[115:18.00]I think the two three

[115:19.00]We're going to answer the question

[115:20.00]What do you think of them

[115:21.00]There's a couple things

[115:23.00]It's like

[115:24.00]How do the other companies

[115:26.00]Innovate against them

[115:27.00]I think when you do a new

[115:28.00]Silicon you're like

[115:29.00]Oh we're going to be

[115:30.00]So much better at this thing

[115:31.00]You're like much faster

[115:32.00]Much cheaper

[115:33.00]But there's all the other curves

[115:34.00]Going down

[115:35.00]On the macro environment

[115:36.00]So if it takes you

[115:37.00]Like five years

[115:38.00]Before you were

[115:39.00]Like a lot better

[115:40.00]Five years later

[115:41.00]Once you take the chip out

[115:42.00]You're only comparing yourself

[115:43.00]To the five year advancement

[115:44.00]That the major companies had to

[115:46.00]So then it's like

[115:47.00]Okay that

[115:48.00]We're going to have like

[115:49.00]The C300 whatever

[115:50.00]From from nvidia

[115:51.00]By the time

[115:52.00]Some of these chips come up

[115:53.00]What's after Z

[115:55.00]What do you think is after Z

[115:56.00]In the roadmap

[115:57.00]It's X Y Z

[116:00.00]No

[116:01.00]Anyways

[116:03.00]Yeah yeah

[116:04.00]It's like the age old problem

[116:05.00]You build a chip

[116:06.00]It has some cool thing

[116:07.00]Cool feature

[116:08.00]And then like

[116:09.00]A year later nvidia

[116:10.00]Has it in hardware

[116:11.00]It has implemented

[116:12.00]Some flavor of that in hardware

[116:14.00]Or two generations out

[116:16.00]Like what idea

[116:17.00]Are you going to have

[116:18.00]That nvidia can't implement

[116:19.00]Is like really the question

[116:21.00]It's like you have to be

[116:22.00]Fundamentally different

[116:23.00]In some way

[116:24.00]That holds through

[116:25.00]For four or five years

[116:26.00]That's kind of the big issue

[116:28.00]But you know like

[116:30.00]Like those people have

[116:31.00]Some ideas that are interesting

[116:32.00]And yeah maybe

[116:33.00]It'll work out right

[116:34.00]It's going to be hard

[116:35.00]To fight nvidia

[116:36.00]Who one doesn't

[116:37.00]Consider them competition

[116:38.00]I worried about like

[116:39.00]Google and Amazon's chip

[116:40.00]Right they're not

[116:41.00]And I guess

[116:42.00]To some extent AMD's chip

[116:43.00]But like

[116:44.00]They're not really worried

[116:45.00]About you know

[116:46.00]Maddox or etched

[116:47.00]Or or grok

[116:48.00]Or you know

[116:49.00]Positron or any of these folks

[116:51.00]How much of an advantage

[116:53.00]Do they have

[116:54.00]By working closely

[116:55.00]With like open ai

[116:56.00]And some of these other folks

[116:57.00]And then already knowing

[116:58.00]Where some of the

[116:59.00]Architecture decisions

[117:00.00]Are going and since

[117:01.00]Those companies are like

[117:02.00]The biggest buyers

[117:03.00]Of the chips

[117:04.00]Yeah I mean like

[117:05.00]You see like

[117:06.00]Like the most important

[117:07.00]Sort of ai companies

[117:09.00]Are obviously going to

[117:10.00]Tell hardware vendors

[117:11.00]What they want

[117:12.00]You know open ai

[117:13.00]And you know

[117:14.00]So on and so forth

[117:15.00]Right they can obviously

[117:16.00]Tell them what they want

[117:17.00]And the startups

[117:18.00]Aren't actually going to

[117:19.00]Get anywhere close to

[117:20.00]As much feedback on

[117:21.00]What to do on like

[117:22.00]You know very

[117:23.00]My new low level stuff

[117:24.00]So that is a difficult here

[117:26.00]Some startups

[117:27.00]Like Maddox

[117:28.00]Obviously have

[117:29.00]People who built

[117:30.00]Or worked on

[117:31.00]The largest models

[117:32.00]And other startups

[117:33.00]Might not have

[117:34.00]That advantage

[117:35.00]And so they're always

[117:36.00]Gonna have that issue

[117:37.00]Of like

[117:38.00]Hey how do I get

[117:39.00]The feedback

[117:40.00]Or what's changing

[117:41.00]What do they see

[117:42.00]Down the pipeline

[117:43.00]That's

[117:44.00]That I really need

[117:45.00]To be aware of

[117:46.00]And ready for

[117:47.00]When I design

[117:48.00]My hardware

[117:49.00]All right

[117:50.00]Every hardware shortage

[117:51.00]As eventually

[117:52.00]Turn into a glut

[117:53.00]Will that be

[117:54.00]Through of NVIDIA chips

[117:55.00]It's so when

[117:56.00]But also why

[117:57.00]Absolutely

[117:58.00]And I'm so excited

[117:59.00]To buy like

[118:00.00]H100s

[118:01.00]Not a thousand

[118:02.00]But yeah

[118:03.00]Everyone's

[118:04.00]Gonna buy chips

[118:05.00]It's just the way

[118:06.00]Semiconductors work

[118:07.00]Because the supply chain

[118:08.00]Takes forever to build out

[118:09.00]And it's like

[118:10.00]A really weird thing

[118:11.00]So if the backlog

[118:12.00]Of chips is a year

[118:15.00]People will order

[118:16.00]Two years worth

[118:17.00]Of what they want

[118:18.00]For the next year

[118:19.00]It is like

[118:20.00]A very common thing

[118:21.00]It's not just like

[118:22.00]This AI cycle

[118:23.00]But like

[118:24.00]Like microcontroller

[118:25.00]Like the automotive companies

[118:26.00]They order

[118:27.00]Two years worth

[118:28.00]Of what they needed

[118:29.00]For one year

[118:30.00]What happens

[118:31.00]In semiconductors

[118:32.00]When lead times

[118:33.00]Lengthen

[118:34.00]The purchases

[118:35.00]And inventory

[118:36.00]Is sort of like

[118:37.00]Double

[118:38.00]So these

[118:39.00]The NVIDIA GPU shortage

[118:44.00]Obviously is going to be

[118:45.00]Rectified

[118:46.00]And when it is

[118:47.00]Everyone's sort of

[118:48.00]Double orders

[118:49.00]Will become

[118:50.00]Extremely apparent

[118:51.00]Right

[118:52.00]And you know

[118:53.00]You see like

[118:54.00]Random companies

[118:55.00]Out of nowhere being

[118:56.00]Like yeah

[118:57.00]We've got 32,000

[118:58.00]H100s on order

[118:59.00]And 25,000

[119:00.00]And trust

[119:01.00]They're not all

[119:02.00]They're not all

[119:03.00]Real orders

[119:04.00]For one

[119:05.00]But I think

[119:06.00]The bubble will

[119:07.00]Continue on

[119:08.00]For a long time

[119:09.00]Right like it's not

[119:10.00]It's not going to end

[119:11.00]Like this year

[119:12.00]Right like people

[119:13.00]People need AI

[119:14.00]Right like I think

[119:15.00]Everyone in this audience

[119:16.00]Would agree right like

[119:17.00]There's no

[119:18.00]There's no

[119:19.00]Like immediate

[119:20.00]Like end to the

[119:21.00]To the bubble

[119:22.00]Right

[119:23.00]What's next

[119:24.00]Why

[119:25.00]I think it's just

[119:26.00]Because the supply chain

[119:27.00]Expands so much

[119:28.00]Some companies

[119:29.00]Will continue to buy

[119:30.00]Like an

[119:31.00]Open AI

[119:32.00]Or meta

[119:33.00]Will continue to buy

[119:34.00]But then like

[119:35.00]All these random

[119:36.00]Startups will

[119:37.00]Or a lot of them

[119:38.00]Will not be able to

[119:39.00]Continue to buy

[119:40.00]So then

[119:41.00]That kind of leads to like

[119:42.00]They'll pause

[119:43.00]For a little bit

[119:44.00]Or like

[119:45.00]I think in 2018

[119:46.00]Right like

[119:47.00]Memory pricing was

[119:48.00]Extremely high

[119:49.00]Then all of a sudden

[119:50.00]Google, Microsoft

[119:51.00]And Amazon

[119:52.00]All agreed

[119:53.00]I don't you know

[119:54.00]They won't

[119:55.00]They won't say

[119:56.00]It's together

[119:57.00]It's a week

[119:58.00]To stop ordering memory

[119:59.00]And within like

[120:00.00]A month

[120:01.00]The price of memory

[120:02.00]Started tanking

[120:03.00]Like insane amounts

[120:04.00]Right and like

[120:05.00]People will claim

[120:06.00]You know all sorts of

[120:07.00]Reasons why

[120:08.00]That was timed

[120:09.00]Extremely well

[120:10.00]It was like very clear

[120:11.00]And people in the

[120:12.00]Financial markets

[120:13.00]Were able to make trades

[120:14.00]And everything

[120:15.00]People stopped buying

[120:16.00]And it's not like

[120:17.00]Their demand just dried up

[120:18.00]It's just like

[120:19.00]They had a little bit

[120:20.00]Of a demand slowdown

[120:21.00]And then they had enough

[120:22.00]Inventory that they could

[120:23.00]Like weather until

[120:24.00]Like prices tanked

[120:25.00]Because it's such

[120:26.00]Thank you very much

[120:27.00]Is it

[120:28.00]Hey everyone

[120:33.00]And so

[120:34.00]Today we have a special guest

[120:35.00]Millions from Capital One

[120:37.00]But I would tend to

[120:38.00]Like to introduce people

[120:39.00]With a bit of their background

[120:41.00]And then learn a little bit

[120:42.00]More about you

[120:43.00]On the personal side

[120:44.00]You call your PhD

[120:45.00]In a probabilistic framework

[120:47.00]From mapping audio

[120:48.00]Visual features to semantics

[120:49.00]I feel like that

[120:50.00]Is like the beginnings

[120:51.00]Of like a multimodal

[120:52.00]AI model in some sense

[120:54.00]Do you have any sort

[120:55.00]Factions on your PhD

[120:56.00]Vis this cut

[120:57.00]Thanks for having me

[120:59.00]And so

[121:00.00]Let me say this that

[121:01.00]It almost feels like

[121:02.00]Things go around in circles

[121:03.00]Right

[121:04.00]In research and development

[121:05.00]And so

[121:06.00]At the right time

[121:07.00]And the right place

[121:08.00]You kind of intersect

[121:09.00]Back with some of the topics

[121:11.00]And then

[121:12.00]Some other conditions

[121:13.00]That have happened

[121:14.00]Suddenly make

[121:15.00]A big difference

[121:16.00]Between that ticking off

[121:17.00]Vorses you know

[121:18.00]It may not be

[121:19.00]You know

[121:20.00]As intently pursued

[121:21.00]At any given point of time

[121:22.00]Right so

[121:23.00]I have been

[121:24.00]In AI for now

[121:25.00]Three decades

[121:26.00]You know

[121:27.00]You talked about

[121:28.00]My PhD thesis

[121:29.00]My bachelor's thesis

[121:30.00]Was on implementing

[121:32.00]Neural networks

[121:33.00]On India's

[121:34.00]You know

[121:35.00]Homegrown supercomputers

[121:36.00]Back then

[121:37.00]And so

[121:38.00]You know

[121:39.00]This whole notion of

[121:40.00]Message passing

[121:41.00]And distributed computing

[121:42.00]And computing weights

[121:44.00]You know

[121:45.00]And then bringing

[121:46.00]All of them back

[121:47.00]Distributing

[121:48.00]The computations

[121:49.00]Of the neural network

[121:50.00]You know

[121:51.00]Forward pass

[121:52.00]Those things

[121:53.00]For what we used to do

[121:54.00]You know

[121:55.00]For that

[121:56.00]Particular supercomputing

[121:57.00]Architecture

[121:58.00]We had back then

[121:59.00]And then

[122:00.00]My PhD

[122:01.00]Of course was

[122:02.00]How to understand

[122:03.00]What's going on

[122:04.00]In a video

[122:05.00]Right and use

[122:06.00]Multi-modal cues

[122:07.00]To your point

[122:08.00]You know

[122:09.00]What has happened

[122:10.00]In the last couple of

[122:11.00]Decades

[122:12.00]One we have

[122:13.00]Tremendous amount

[122:14.00]Of data explosion

[122:15.00]So when I was doing

[122:16.00]My PhD

[122:17.00]I used to

[122:18.00]Actually go

[122:19.00]To blockbuster

[122:20.00]And rent

[122:21.00]Explosions

[122:22.00]And how do you

[122:23.00]Actually

[122:24.00]Models in the audio stream

[122:25.00]And the visual stream

[122:26.00]For something

[122:27.00]Like an explosion

[122:28.00]So I remember

[122:29.00]Going and doing

[122:30.00]All this digitization

[122:31.00]Of tape and then

[122:32.00]Cleaning up the data

[122:33.00]And then

[122:34.00]You know

[122:35.00]Having some kind

[122:36.00]Of a labeling tool

[122:37.00]That I actually cooked up

[122:38.00]And having a spouse

[122:40.00]Of a friend of mine

[122:41.00]To do the labeling

[122:42.00]For me

[122:43.00]So look at

[122:44.00]Where we were back then

[122:45.00]And now you have

[122:46.00]Scale.ai

[122:47.00]That basically goes

[122:48.00]And does labeling

[122:49.00]For a lot of these models

[122:50.00]And so forth

[122:51.00]So scale

[122:52.00]Of data has changed

[122:53.00]That's one

[122:54.00]The second thing

[122:55.00]That has changed

[122:56.00]Is we were looking

[122:57.00]At computing

[122:58.00]Architectures

[122:59.00]That were

[123:00.00]Much much

[123:01.00]Much less rich

[123:02.00]In terms of what

[123:03.00]We had back then

[123:04.00]And the 2012

[123:06.00]Breakthrough

[123:07.00]By Hinton and his students

[123:09.00]And using GPUs

[123:11.00]Really when

[123:12.00]The ImageNet competition

[123:14.00]Really helped

[123:15.00]Take this field off

[123:16.00]In a completely

[123:17.00]New direction

[123:18.00]And at

[123:19.00]Very very large scale

[123:20.00]And the third thing

[123:21.00]Is of course

[123:22.00]The GPU computing

[123:23.00]Back then

[123:24.00]I did not have access

[123:25.00]When I was doing my PhD

[123:26.00]To some of the amazing things

[123:28.00]That Nvidia hadn't yet built

[123:30.00]And so

[123:31.00]It's I think really

[123:32.00]The confluence of those three things

[123:34.00]Which make all the difference

[123:35.00]Between a lot of this research

[123:36.00]That happened

[123:37.00]In the late 90s

[123:38.00]And what's happening

[123:39.00]Between the 2010

[123:41.00]To now

[123:42.00]Kind of time frame

[123:43.00]But if you look at the intent

[123:45.00]The intent was the same

[123:47.00]What's in the video

[123:48.00]How do we understand

[123:49.00]The multimodal cues

[123:50.00]You know

[123:51.00]That come together

[123:52.00]To give us

[123:53.00]That semantic understanding

[123:54.00]Of what's in the video

[123:55.00]And so

[123:56.00]To that extent

[123:57.00]The problems

[123:58.00]We were trying

[123:59.00]To solve the same

[124:00.00]But the tools

[124:01.00]That we have now

[124:02.00]Are amazingly

[124:03.00]Amazingly different

[124:04.00]And amazingly

[124:05.00]More powerful

[124:06.00]Are there any

[124:07.00]Maybe research approaches

[124:08.00]Or ML patterns

[124:10.00]That you tried

[124:11.00]That didn't work

[124:12.00]That you think

[124:13.00]Will work today

[124:14.00]Would you abuse

[124:15.00]That people haven't tried

[124:16.00]I think

[124:17.00]There are many people

[124:18.00]That have done serious

[124:19.00]ML research before

[124:20.00]The GPU era

[124:21.00]I would say

[124:22.00]If you think about

[124:23.00]All the ML researchers

[124:24.00]Working today

[124:25.00]Most of them are post-GPU

[124:26.00]Any like story

[124:27.00]That you remember

[124:28.00]That you were like

[124:29.00]Oh, this seems really promising

[124:30.00]But like

[124:31.00]There wasn't enough compute

[124:32.00]Or anything like that

[124:33.00]The whole concept

[124:34.00]Of modeling context

[124:35.00]Right

[124:36.00]So my thesis was about

[124:37.00]Not

[124:38.00]How do you just

[124:39.00]Detect isolated things

[124:40.00]In a video

[124:41.00]Right

[124:42.00]Like this is a car

[124:43.00]That's an explosion

[124:44.00]And so on and so forth

[124:45.00]If I see these

[124:46.00]End things together

[124:47.00]Do they actually

[124:48.00]Contextually make sense

[124:49.00]Like do I see

[124:50.00]The sky

[124:51.00]About the land

[124:52.00]And if I do that

[124:53.00]Then I have a higher confidence

[124:55.00]That this indeed is sky

[124:56.00]And that indeed is land

[124:57.00]Right

[124:58.00]And so on and so forth

[124:59.00]So when we were trying

[125:00.00]To model that context

[125:01.00]We had extremely limited

[125:03.00]Libling

[125:04.00]And extremely limited

[125:05.00]Corpus

[125:06.00]In terms of how we could do it

[125:08.00]Now when I think back

[125:10.00]I think that

[125:11.00]What better way to describe

[125:12.00]Context than a multimodal LLM

[125:14.00]Right

[125:15.00]Which strain on

[125:16.00]As much of the data

[125:18.00]As there is on the internet

[125:20.00]In terms of multimodality

[125:22.00]And that to me

[125:23.00]Would have been an amazing thing

[125:24.00]For me to have back then

[125:26.00]So I would say

[125:27.00]How to model context

[125:28.00]Is a problem

[125:29.00]That is going to be evergreen

[125:30.00]It never goes out of fashion

[125:32.00]But how we are able

[125:33.00]To do it now

[125:34.00] Versus then

[125:35.00]I see as

[125:36.00]One of the steps towards

[125:38.00]Truly understanding

[125:39.00]You know what's happening

[125:40.00]Right in the multiple modalities

[125:42.00]The other part

[125:43.00]Is reasoning

[125:44.00]Right

[125:45.00]I think we are still

[125:46.00]In the very early innings

[125:47.00]Of reasoning

[125:48.00]You know we see this

[125:49.00]Interesting evolution

[125:51.00]Of you know

[125:52.00]How do you actually

[125:53.00]Build a model of the world

[125:55.00]And Yanlacan's work

[125:56.00]You know very interesting

[125:58.00]In this sense to me

[125:59.00]He has been talking

[126:00.00]About it for a while

[126:01.00]Right so

[126:02.00]But now I think

[126:03.00]We are getting to a point

[126:04.00]Where a lot of those pieces

[126:05.00]May start coming together

[126:07.00]And I think solving

[126:09.00]That reasoning piece

[126:10.00]Is a very very critical step

[126:12.00]Before we can actually

[126:14.00]Build

[126:15.00]Truly intelligent machines

[126:17.00]And do you have any

[126:18.00]Intuition on the part

[126:19.00]The video

[126:20.00]Is going to play in it

[126:21.00]Because a lot of Yan's

[126:22.00]Also talking points

[126:23.00]Are around you know

[126:24.00]With vjapa

[126:25.00]And some of those models

[126:26.00]If you show

[126:27.00]What's going to happen next

[126:28.00]Like that's part

[126:29.00]Of a war model

[126:30.00]Like it's video going to be

[126:32.00]Like a big part in it

[126:33.00]Like do we need to

[126:34.00]Get there to actually

[126:35.00]Get a real war model

[126:36.00]Like do you think text

[126:37.00]Is enough to

[126:38.00]Get a good shot at it

[126:39.00]And maybe biased

[126:40.00]In answering that question

[126:41.00]Given that

[126:42.00]I you know

[126:43.00]Cut my teeth

[126:44.00]In multi modality

[126:45.00]And given that

[126:46.00]The video modality

[126:47.00]In general

[126:48.00]Right is a lot

[126:49.00]More challenging

[126:50.00]You know whether

[126:51.00]It's just because

[126:52.00]Of the sheer size of data

[126:53.00]Right in terms of

[126:54.00]The number of pixels

[126:55.00]That you need to process

[126:56.00]Whether it is because

[126:57.00]You are actually

[126:58.00]Capturing the real world

[127:00.00]Which tends to

[127:01.00]Be far more complex

[127:02.00]You know in the

[127:03.00]Case of language

[127:04.00]You know humans

[127:05.00]Over millennia

[127:07.00]Have evolved this

[127:09.00]Longly concise

[127:11.00]Codebook

[127:12.00]Of how to describe

[127:13.00]Things

[127:14.00]So there is a humongous

[127:15.00]Amount of abstraction

[127:18.00]And rationalization

[127:20.00]And concise definition

[127:22.00]That has gone on

[127:23.00]In how languages

[127:25.00]Evolved

[127:26.00]And so you know

[127:27.00]The number of words

[127:28.00]In the vocabulary

[127:29.00]Of a language

[127:30.00]When you look at that

[127:31.00]We are able to

[127:32.00]Tell beautiful stories

[127:33.00]Right with

[127:34.00]Just those many

[127:36.00]You know words

[127:37.00]But if you look at

[127:38.00]In the real world

[127:39.00]If you look at

[127:40.00]Capturing that

[127:41.00]Right whether it is

[127:42.00]Through

[127:43.00]The eyes of a robot

[127:44.00]As it's looking

[127:45.00]You know and trying

[127:46.00]To help you around

[127:47.00]In a room setting

[127:48.00]Or a building setting

[127:49.00]Right or whether

[127:50.00]It's the traffic

[127:51.00]Right which

[127:52.00]Vacues have to look at

[127:53.00]When they're

[127:54.00]Driving on the roads

[127:55.00]The amount of

[127:56.00]Variability

[127:57.00]The amount of

[127:58.00]Distortion of signal

[127:59.00]That comes with

[128:00.00]That right

[128:01.00]Is just remarkably

[128:03.00]Difficult

[128:04.00]And remarkably rich

[128:05.00]To really analyze

[128:06.00]So I would say

[128:07.00]Alicio

[128:08.00]It's both

[128:09.00]I feel that

[128:10.00]The video modality

[128:11.00]Has to be understood

[128:13.00]For a true understanding

[128:14.00]Of the world model

[128:15.00]I would also say

[128:16.00]That's harder

[128:17.00]In some sense

[128:19.00]Because of its inherent complexity

[128:20.00]Then the language part

[128:22.00]Which there's already

[128:24.00]Consize representation

[128:25.00]That people have come up with

[128:26.00]Awesome

[128:27.00]Sorry Sean

[128:28.00]I know we hijacked the intro

[128:30.00]But it was a good rabbit hole

[128:32.00]To go into

[128:33.00]No it's okay

[128:34.00]I also just like

[128:35.00]Stunned by

[128:36.00]Background and history

[128:37.00]That millennium is

[128:38.00]Bringing to

[128:39.00]AI you know

[128:40.00]I guess the speed run through

[128:41.00]In the resume

[128:42.00]You know 14 years

[128:43.00]At IBM

[128:44.00]Finally ending

[128:45.00]As chief scientist at

[128:46.00]IBM research

[128:47.00]And then since

[128:48.00]Cisco

[128:49.00]With cognitive systems

[128:50.00]And CTO

[128:51.00]Of metropolis at NVIDIA

[128:52.00]What should people know

[128:53.00]About like

[128:54.00]How your

[128:55.00]Interest in your trajectory

[128:56.00]Has progressed through

[128:57.00]Your career

[128:58.00]You know when I reflect

[128:59.00]Part of what I see

[129:00.00]Is there is a constant

[129:02.00]And the constant is

[129:04.00]How do you actually

[129:05.00]AI work

[129:06.00]For you know

[129:07.00]Name your favorite

[129:08.00]Problem

[129:09.00]That favorite problem

[129:10.00]Changes maybe

[129:11.00]From decade to decade

[129:12.00]Or in the context of

[129:13.00]Even my state

[129:14.00]IBM research

[129:15.00]But it's always been

[129:16.00]How do we build

[129:17.00]AI solutions

[129:18.00]AI platforms

[129:19.00]That solve

[129:20.00]Real world problems

[129:21.00]So when

[129:22.00]We started out

[129:23.00]We were the first

[129:24.00]Video understanding platform

[129:26.00]That we built at

[129:27.00]IBM research right

[129:28.00]We actually got a Wall Street

[129:29.00]Multimedia

[129:30.00]Award for it

[129:31.00]Innovation Award for it

[129:32.00]In the early 2000s

[129:34.00]We helped with

[129:35.00]Setting up the benchmark

[129:36.00]That's known as

[129:38.00]Trackweed

[129:39.00]Which again

[129:40.00]To at least your

[129:41.00]Point people

[129:42.00]Only know ImageNet

[129:43.00]And thereafter

[129:44.00]Before ImageNet

[129:45.00]There was Trackweed

[129:46.00]And so

[129:47.00]The first decade

[129:48.00]Shan was really about

[129:49.00]You know how do we

[129:50.00]Understand what's in the video

[129:51.00]And how can then

[129:52.00]We turn that into

[129:53.00]Meaningful use of

[129:55.00]AI technology

[129:56.00]For media companies

[129:58.00]You know broadcasting

[129:59.00]Corporations

[130:00.00]And so on and so forth

[130:01.00]Right the business to

[130:02.00]Business kind of setting

[130:03.00]Of course

[130:04.00]Because we were at

[130:05.00]IBM

[130:06.00]We did not focus

[130:07.00]On the consumer

[130:08.00]And then

[130:09.00]You know YouTube

[130:10.00]Happened right

[130:11.00]Here in the valley

[130:12.00]And then

[130:13.00]You see the

[130:14.00]Explosive content

[130:15.00]And applications

[130:16.00]Of AI to

[130:17.00]You know those

[130:18.00]Kind of video

[130:19.00]Understanding problems

[130:20.00]Then it was

[130:21.00]How do we actually

[130:22.00]Make sense

[130:23.00]Out of our

[130:24.00]IoT world

[130:25.00]So when you have

[130:26.00]Signals coming from

[130:27.00]Sensors everywhere

[130:28.00]You know whether

[130:29.00]There are sensors

[130:30.00]Embedded in your

[130:31.00]Bridges

[130:32.00]Sensory information

[130:33.00]To make

[130:34.00]Good decisions

[130:35.00]To a

[130:36.00]You know observe

[130:37.00]The environments

[130:38.00]And we optimize

[130:39.00]Those environments

[130:40.00]Right and so a large

[130:41.00]Part of my

[130:42.00]Second half of

[130:43.00]State IBM research

[130:44.00]Was to come up with

[130:45.00]What is this

[130:46.00]Research around

[130:48.00]Smart cities

[130:49.00]Smarter planet

[130:50.00]And that

[130:51.00]Actually became

[130:52.00]An AI platform

[130:53.00]For helping

[130:55.00]Optimized traffic

[130:56.00]Right one of

[130:57.00]The proudest

[130:58.00]Things that

[130:59.00]I really

[131:00.00]Foundly remember

[131:01.00]Use the data

[131:02.00]From you know

[131:03.00]Teleco data sources

[131:04.00]To understand

[131:05.00]How people move

[131:06.00]In a city

[131:07.00]And then use

[131:08.00]That information

[131:09.00]To build

[131:10.00]Optimal planning

[131:11.00]Right whether

[131:12.00]It's for bus outs

[131:13.00]Whether it's for metro

[131:14.00]Right we did

[131:15.00]Some amazing work

[131:16.00]In Istanbul

[131:17.00]For example

[131:18.00]You know

[131:19.00]Completely different

[131:20.00]Scale

[131:21.00]And then

[131:22.00]The same

[131:23.00]Kind of platform

[131:24.00]We applied

[131:25.00]To helping

[131:26.00]Cities in

[131:27.00]American Midwest

[131:28.00]Like Dubuque

[131:29.00]Optimized their

[131:30.00]Dubuque

[131:31.00]And then

[131:32.00]And then

[131:33.00]The same

[131:34.00]Like Dubuque

[131:35.00]And then

[131:36.00]The same

[131:37.00]Like Dubuque

[131:38.00]And then

[131:39.00]The same

[131:40.00]Like Dubuque

[131:41.00]And then

[131:42.00]The same

[131:43.00]Like Dubuque

[131:44.00]And then

[131:45.00]The same

[131:46.00]Like Dubuque

[131:47.00]And then

[131:48.00]The same

[131:49.00]Like Dubuque

[131:50.00]And then

[131:51.00]The same

[131:52.00]Like Dubuque

[131:53.00]And then

[131:54.00]The same

[131:55.00]Like Dubuque

[131:56.00]And then

[131:57.00]The same

[131:58.00]Like Dubuque

[131:59.00]And then

[132:00.00]And then

[132:01.00]The same

[132:02.00]Like Dubuque

[132:03.00]And then

[132:04.00]The same

[132:05.00]Like Dubuque

[132:06.00]And then

[132:07.00]The same

[132:08.00]Like Dubuque

[132:09.00]And then

[132:10.00]The same

[132:11.00]Like Dubuque

[132:12.00]And then

[132:13.00]The same

[132:14.00]Like Dubuque

[132:15.00]And then

[132:16.00]The same

[132:17.00]Like Dubuque

[132:18.00]And then

[132:19.00]The same

[132:20.00]Like Dubuque

[132:21.00]And then

[132:22.00]The same

[132:23.00]Like Dubuque

[132:24.00]And then

[132:25.00]The same

[132:26.00]Like Dubuque

[132:27.00]And then

[132:28.00]The same

[132:29.00]Like Dubuque

[132:30.00]And then

[132:31.00]The same

[132:32.00]Like Dubuque

[132:33.00]And then

[132:34.00]The same

[132:35.00]Like Dubuque

[132:36.00]And then

[132:37.00]The same

[132:38.00]Like Dubuque

[132:39.00]And then

[132:40.00]The same

[132:41.00]Like Dubuque

[132:42.00]And then

[132:43.00]The same

[132:44.00]Like Dubuque

[132:45.00]And then

[132:46.00]The same

[132:47.00]Like Dubuque

[132:48.00]And then

[132:49.00]The same

[132:50.00]Like Dubuque

[132:51.00]And then

[132:52.00]The same

[132:53.00]Like Dubuque

[132:54.00]And then

[132:55.00]The same

[132:56.00]Like Dubuque

[132:57.00]And then

[132:58.00]The same

[132:59.00]Like Dubuque

[133:00.00]And then

[133:01.00]The same

[133:02.00]Like Dubuque

[133:03.00]And then

[133:04.00]The same

[133:05.00]Like Dubuque

[133:06.00]And then

[133:07.00]The same

[133:08.00]Like Dubuque

[133:09.00]And then

[133:10.00]The same

[133:11.00]Like Dubuque

[133:12.00]And then

[133:13.00]The same

[133:14.00]Like Dubuque

[133:15.00]And then

[133:16.00]The same

[133:17.00]Like Dubuque

[133:18.00]And then

[133:19.00]The same

[133:20.00]Like Dubuque

[133:21.00]And then

[133:22.00]The same

[133:23.00]Like Dubuque

[133:24.00]And then

[133:25.00]The same

[133:26.00]Like Dubuque

[133:27.00]And then

[133:28.00]The same

[133:29.00]Like Dubuque

[133:30.00]And then

[133:31.00]The same

[133:32.00]Like Dubuque

[133:33.00]And then

[133:34.00]The same

[133:35.00]Like Dubuque

[133:36.00]And then

[133:37.00]The same

[133:38.00]Like Dubuque

[133:39.00]And then

[133:40.00]The same

[133:41.00]Like Dubuque

[133:42.00]And then

[133:43.00]The same

[133:44.00]Like Dubuque

[133:45.00]And then

[133:46.00]The same

[133:47.00]Like Dubuque

[133:48.00]And then

[133:49.00]The same

[133:50.00]Like Dubuque

[133:51.00]And then

[133:52.00]The same

[133:53.00]Like Dubuque

[133:54.00]And then

[133:55.00]The same

[133:56.00]Like Dubuque

[133:57.00]And then

[133:58.00]The same

[133:59.00]Like Dubuque

[134:00.00]And then

[134:01.00]The same

[134:02.00]Like Dubuque

[134:03.00]And then

[134:04.00]The same

[134:05.00]Like Dubuque

[134:06.00]And then

[134:07.00]The same

[134:08.00]Like Dubuque

[134:09.00]And then

[134:10.00]The same

[134:11.00]Like Dubuque

[134:12.00]And then

[134:13.00]The same

[134:14.00]Like Dubuque

[134:15.00]And then

[134:16.00]The same

[134:17.00]Like Dubuque

[134:18.00]And then

[134:19.00]The same

[134:20.00]Like Dubuque

[134:21.00]And then

[134:22.00]The same

[134:23.00]Like Dubuque

[134:24.00]And then

[134:25.00]The same

[134:26.00]Like Dubuque

[134:27.00]And then

[134:28.00]The same

[134:29.00]Like Dubuque

[134:30.00]And then

[134:31.00]The same

[134:32.00]Like Dubuque

[134:33.00]And then

[134:34.00]The same

[134:35.00]Like Dubuque

[134:36.00]And then

[134:37.00]The same

[134:38.00]Like Dubuque

[134:39.00]And then

[134:40.00]The same

[134:41.00]Like Dubuque

[134:42.00]And then

[134:43.00]The same

[134:44.00]Like Dubuque

[134:45.00]And then

[134:46.00]The same

[134:47.00]Like Dubuque

[134:48.00]And then

[134:49.00]The same

[134:50.00]Like Dubuque

[134:51.00]And then

[134:52.00]The same

[134:53.00]Like Dubuque

[134:54.00]And then

[134:55.00]The same

[134:56.00]Like Dubuque

[134:57.00]And then

[134:58.00]The same

[134:59.00]Like Dubuque

[135:00.00]And then

[135:01.00]The same

[135:02.00]Like Dubuque

[135:03.00]And then

[135:04.00]The same

[135:05.00]Like Dubuque

[135:06.00]And then

[135:07.00]The same

[135:08.00]Like Dubuque

[135:09.00]And then

[135:10.00]The same

[135:11.00]Like Dubuque

[135:12.00]And then

[135:13.00]The same

[135:14.00]Like Dubuque

[135:15.00]And then

[135:16.00]The same

[135:17.00]Like Dubuque

[135:18.00]And then

[135:19.00]The same

[135:20.00]Like Dubuque

[135:21.00]And then

[135:22.00]The same

[135:23.00]Like Dubuque

[135:24.00]And then

[135:25.00]The same

[135:26.00]Like Dubuque

[135:27.00]And then

[135:28.00]The same

[135:29.00]Like Dubuque

[135:30.00]And then

[135:31.00]The same

[135:32.00]Like Dubuque

[135:33.00]And then

[135:34.00]The same

[135:35.00]Like Dubuque

[135:36.00]And then

[135:37.00]The same

[135:38.00]Like Dubuque

[135:39.00]And then

[135:40.00]The same

[135:41.00]Like Dubuque

[135:42.00]And then

[135:43.00]The same

[135:44.00]Like Dubuque

[135:45.00]And then

[135:46.00]The same

[135:47.00]Like Dubuque

[135:48.00]And then

[135:49.00]The same

[135:50.00]Like Dubuque

[135:51.00]And then

[135:52.00]The same

[135:53.00]Like Dubuque

[135:54.00]And then

[135:55.00]The same

[135:56.00]Like Dubuque

[135:57.00]And then

[135:58.00]The same

[135:59.00]Like Dubuque

[136:00.00]And then

[136:01.00]The same

[136:02.00]Like Dubuque

[136:03.00]And then

[136:04.00]The same

[136:05.00]Like Dubuque

[136:06.00]And then

[136:07.00]The same

[136:08.00]Like Dubuque

[136:09.00]And then

[136:10.00]The same

[136:11.00]Like Dubuque

[136:12.00]And then

[136:13.00]The same

[136:14.00]Like Dubuque

[136:15.00]And then

[136:16.00]The same

[136:17.00]Like Dubuque

[136:18.00]And then

[136:19.00]The same

[136:20.00]Like Dubuque

[136:21.00]And then

[136:22.00]The same

[136:23.00]Like Dubuque

[136:24.00]And then

[136:25.00]The same

[136:26.00]Like Dubuque

[136:27.00]And then

[136:28.00]The same

[136:29.00]Like Dubuque

[136:30.00]And then

[136:31.00]The same

[136:32.00]Like Dubuque

[136:33.00]And then

[136:34.00]The same

[136:35.00]Like Dubuque

[136:36.00]And then

[136:37.00]The same

[136:38.00]Like Dubuque

[136:39.00]And then

[136:40.00]The same

[136:41.00]Like Dubuque

[136:42.00]And then

[136:43.00]The same

[136:44.00]Like Dubuque

[136:45.00]And then

[136:46.00]The same

[136:47.00]Like Dubuque

[136:48.00]And then

[136:49.00]And then

[136:50.00]The same

[136:51.00]Like Dubuque

[136:52.00]And then

[136:53.00]The same

[136:54.00]Like Dubuque

[136:55.00]And then

[136:56.00]The same

[136:57.00]Like Dubuque

[136:58.00]And then

[136:59.00]The same

[137:00.00]Like Dubuque

[137:01.00]And then

[137:02.00]The same

[137:03.00]Like Dubuque

[137:04.00]And then

[137:05.00]The same

[137:06.00]Like Dubuque

[137:07.00]And then

[137:08.00]The same

[137:09.00]Like Dubuque

[137:10.00]And then

[137:11.00]The same

[137:12.00]Like Dubuque

[137:13.00]And then

[137:14.00]The same

[137:15.00]Like Dubuque

[137:16.00]And then

[137:17.00]The same

[137:18.00]Like Dubuque

[137:19.00]And then

[137:20.00]The same

[137:21.00]Like Dubuque

[137:22.00]And then

[137:23.00]The same

[137:24.00]Like Dubuque

[137:25.00]And then

[137:26.00]The same

[137:27.00]Like Dubuque

[137:28.00]And then

[137:29.00]The same

[137:30.00]Like Dubuque

[137:31.00]And then

[137:32.00]The same

[137:33.00]Like Dubuque

[137:34.00]And then

[137:35.00]The same

[137:36.00]Like Dubuque

[137:37.00]And then

[137:38.00]The same

[137:39.00]Like Dubuque

[137:40.00]And then

[137:41.00]The same

[137:42.00]Like Dubuque

[137:43.00]And then

[137:44.00]The same

[137:45.00]Like Dubuque

[137:46.00]And then

[137:47.00]The same

[137:48.00]Like Dubuque

[137:49.00]And then

[137:50.00]The same

[137:51.00]Like Dubuque

[137:52.00]And then

[137:53.00]The same

[137:54.00]Like Dubuque

[137:55.00]And then

[137:56.00]The same

[137:57.00]Like Dubuque

[137:58.00]And then

[137:59.00]The same

[138:00.00]Like Dubuque

[138:01.00]And then

[138:02.00]The same

[138:03.00]Like Dubuque

[138:04.00]And then

[138:05.00]The same

[138:06.00]Like Dubuque

[138:07.00]And then

[138:08.00]The same

[138:09.00]Like Dubuque

[138:10.00]And then

[138:11.00]The same

[138:12.00]Like Dubuque

[138:13.00]And then

[138:14.00]The same

[138:15.00]Like Dubuque

[138:16.00]And then

[138:17.00]The same

[138:18.00]Like Dubuque

[138:19.00]And then

[138:20.00]The same

[138:21.00]Like Dubuque

[138:22.00]And then

[138:23.00]The same

[138:24.00]Like Dubuque

[138:25.00]And then

[138:26.00]The same

[138:27.00]Like Dubuque

[138:28.00]And then

[138:29.00]The same

[138:30.00]Like Dubuque

[138:31.00]And then

[138:32.00]The same

[138:33.00]Like Dubuque

[138:34.00]And then

[138:35.00]The same

[138:36.00]Like Dubuque

[138:37.00]And then

[138:38.00]The same

[138:39.00]Like Dubuque

[138:40.00]And then

[138:41.00]The same

[138:42.00]Like Dubuque

[138:43.00]And then

[138:44.00]The same

[138:45.00]Like Dubuque

[138:46.00]And then

[138:47.00]The same

[138:48.00]Like Dubuque

[138:49.00]And then

[138:50.00]The same

[138:51.00]Like Dubuque

[138:52.00]And then

[138:53.00]The same

[138:54.00]Like Dubuque

[138:55.00]And then

[138:56.00]The same

[138:57.00]Like Dubuque

[138:58.00]And then

[138:59.00]The same

[139:00.00]Like Dubuque

[139:01.00]And then

[139:02.00]The same

[139:03.00]Like Dubuque

[139:04.00]And then

[139:05.00]The same

[139:06.00]Like Dubuque

[139:07.00]And then

[139:08.00]The same

[139:09.00]Like Dubuque

[139:10.00]And then

[139:11.00]The same

[139:12.00]Like Dubuque

[139:13.00]And then

[139:14.00]The same

[139:15.00]Like Dubuque

[139:16.00]And then

[139:17.00]The same

[139:18.00]Like Dubuque

[139:19.00]And then

[139:20.00]The same

[139:21.00]Like Dubuque

[139:22.00]And then

[139:23.00]The same

[139:24.00]Like Dubuque

[139:25.00]And then

[139:26.00]The same

[139:27.00]Like Dubuque

[139:28.00]And then

[139:29.00]The same

[139:30.00]Like Dubuque

[139:31.00]And then

[139:32.00]The same

[139:33.00]Like Dubuque

[139:34.00]And then

[139:35.00]The same

[139:36.00]Like Dubuque

[139:37.00]And then

[139:38.00]The same

[139:39.00]Like Dubuque

[139:40.00]And then

[139:41.00]The same

[139:42.00]Like Dubuque

[139:43.00]And then

[139:44.00]The same

[139:45.00]Like Dubuque

[139:46.00]And then

[139:47.00]The same

[139:48.00]Like Dubuque

[139:49.00]And then

[139:50.00]The same

[139:51.00]Like Dubuque

[139:52.00]And then

[139:53.00]The same

[139:54.00]Like Dubuque

[139:55.00]And then

[139:56.00]The same

[139:57.00]Like Dubuque

[139:58.00]And then

[139:59.00]The same

[140:00.00]Like Dubuque

[140:01.00]And then

[140:02.00]The same

[140:03.00]Like Dubuque

[140:04.00]And then

[140:05.00]The same

[140:06.00]Like Dubuque

[140:07.00]And then

[140:08.00]The same

[140:09.00]Like Dubuque

[140:10.00]And then

[140:11.00]The same

[140:12.00]Like Dubuque

[140:13.00]And then

[140:14.00]The same

[140:15.00]Like Dubuque

[140:16.00]And then

[140:17.00]The same

[140:18.00]Like Dubuque

[140:19.00]And then

[140:20.00]The same

[140:21.00]Like Dubuque

[140:22.00]And then

[140:23.00]The same

[140:24.00]Like Dubuque

[140:25.00]And then

[140:26.00]The same

[140:27.00]Like Dubuque

[140:28.00]And then

[140:29.00]The same

[140:30.00]Like Dubuque

[140:31.00]And then

[140:32.00]The same

[140:33.00]Like Dubuque

[140:34.00]And then

[140:35.00]The same

[140:36.00]Like Dubuque

[140:37.00]And then

[140:38.00]The same

[140:39.00]Like Dubuque

[140:40.00]And then

[140:41.00]The same

[140:42.00]Like Dubuque

[140:43.00]And then

[140:44.00]The same

[140:45.00]Like Dubuque

[140:46.00]And then

[140:47.00]The same

[140:48.00]Like Dubuque

[140:49.00]And then

[140:50.00]The same

[140:51.00]Like Dubuque

[140:52.00]And then

[140:53.00]The same

[140:54.00]Like Dubuque

[140:55.00]And then

[140:56.00]The same

[140:57.00]Like Dubuque

[140:58.00]And then

[140:59.00]The same

[141:00.00]Like Dubuque

[141:01.00]And then

[141:02.00]The same

[141:03.00]Like Dubuque

[141:04.00]And then

[141:05.00]The same

[141:06.00]Like Dubuque

[141:07.00]And then

[141:08.00]The same

[141:09.00]Like Dubuque

[141:10.00]And then

[141:11.00]The same

[141:12.00]Like Dubuque

[141:13.00]And then

[141:14.00]The same

[141:15.00]Like Dubuque

[141:16.00]And then

[141:17.00]The same

[141:18.00]Like Dubuque

[141:19.00]And then

[141:20.00]The same

[141:21.00]Like Dubuque

[141:22.00]And then

[141:23.00]The same

[141:24.00]Like Dubuque

[141:25.00]And then

[141:26.00]The same

[141:27.00]Like Dubuque

[141:28.00]And then

[141:29.00]The same

[141:30.00]Like Dubuque

[141:31.00]And then

[141:32.00]The same

[141:33.00]Like Dubuque

[141:34.00]And then

[141:35.00]The same

[141:36.00]Like Dubuque

[141:37.00]And then

[141:38.00]The same

[141:39.00]Like Dubuque

[141:40.00]And then

[141:41.00]The same

[141:42.00]Like Dubuque

[141:43.00]And then

[141:44.00]The same

[141:45.00]Like Dubuque

[141:46.00]And then

[141:47.00]The same

[141:48.00]Like Dubuque

[141:49.00]And then

[141:50.00]The same

[141:51.00]Like Dubuque

[141:52.00]And then

[141:53.00]The same

[141:54.00]Like Dubuque

[141:55.00]And then

[141:56.00]The same

[141:57.00]Like Dubuque

[141:58.00]And then

[141:59.00]The same

[142:00.00]Like Dubuque

[142:01.00]And then

[142:02.00]The same

[142:03.00]Like Dubuque

[142:04.00]And then

[142:05.00]The same

[142:06.00]Like Dubuque

[142:07.00]And then

[142:08.00]The same

[142:09.00]Like Dubuque

[142:10.00]And then

[142:11.00]The same

[142:12.00]Like Dubuque

[142:13.00]And then

[142:14.00]The same

[142:15.00]Like Dubuque

[142:16.00]And then

[142:17.00]The same

[142:18.00]Like Dubuque

[142:19.00]And then

[142:20.00]The same

[142:21.00]Like Dubuque

[142:22.00]And then

[142:23.00]The same

[142:24.00]Like Dubuque

[142:25.00]And then

[142:26.00]The same

[142:27.00]Like Dubuque

[142:28.00]And then

[142:29.00]The same

[142:30.00]Like Dubuque

[142:31.00]And then

[142:32.00]The same

[142:33.00]Like Dubuque

[142:34.00]And then

[142:35.00]The same

[142:36.00]Like Dubuque

[142:37.00]And then

[142:38.00]The same

[142:39.00]Like Dubuque

[142:40.00]And then

[142:41.00]The same

[142:42.00]Like Dubuque

[142:43.00]And then

[142:44.00]The same

[142:45.00]Like Dubuque

[142:46.00]And then

[142:47.00]The same

[142:48.00]Like Dubuque

[142:49.00]And then

[142:50.00]The same

[142:51.00]Like Dubuque

[142:52.00]And then

[142:53.00]The same

[142:54.00]Like Dubuque

[142:55.00]And then

[142:56.00]The same

[142:57.00]Like Dubuque

[142:58.00]And then

[142:59.00]The same

[143:00.00]Like Dubuque

[143:01.00]And then

[143:02.00]The same

[143:03.00]Like Dubuque

[143:04.00]And then

[143:05.00]The same

[143:06.00]Like Dubuque

[143:07.00]And then

[143:08.00]The same

[143:09.00]Like Dubuque

[143:10.00]And then

[143:11.00]The same

[143:12.00]Like Dubuque

[143:13.00]And then

[143:14.00]The same

[143:15.00]Like Dubuque

[143:16.00]And then

[143:17.00]The same

[143:18.00]Like Dubuque

[143:19.00]And then

[143:20.00]The same

[143:21.00]Like Dubuque

[143:22.00]And then

[143:23.00]The same

[143:24.00]Like Dubuque

[143:25.00]And then

[143:26.00]The same

[143:27.00]Like Dubuque

[143:28.00]And then

[143:29.00]The same

[143:30.00]Like Dubuque

[143:31.00]And then

[143:32.00]The same

[143:33.00]Like Dubuque

[143:34.00]And then

[143:35.00]The same

[143:36.00]Like Dubuque

[143:37.00]And then

[143:38.00]The same

[143:39.00]Like Dubuque

[143:40.00]And then

[143:41.00]The same

[143:42.00]Like Dubuque

[143:43.00]And then

[143:44.00]The same

[143:45.00]Like Dubuque

[143:46.00]And then

[143:47.00]The same

[143:48.00]Like Dubuque

[143:49.00]And then

[143:50.00]The same

[143:51.00]Like Dubuque

[143:52.00]And then

[143:53.00]The same

[143:54.00]Like Dubuque

[143:55.00]And then

[143:56.00]The same

[143:57.00]Like Dubuque

[143:58.00]And then

[143:59.00]The same

[144:00.00]Like Dubuque

[144:01.00]And then

[144:02.00]The same

[144:03.00]Like Dubuque

[144:04.00]And then

[144:05.00]The same

[144:06.00]Like Dubuque

[144:07.00]And then

[144:08.00]The same

[144:09.00]Like Dubuque

[144:10.00]And then

[144:11.00]The same

[144:12.00]Like Dubuque

[144:13.00]And then

[144:14.00]The same

[144:15.00]Like Dubuque

[144:16.00]And then

[144:17.00]The same

[144:18.00]Like Dubuque

[144:19.00]And then

[144:20.00]The same

[144:21.00]Like Dubuque

[144:22.00]And then

[144:23.00]The same

[144:24.00]Like Dubuque

[144:25.00]And then

[144:26.00]The same

[144:27.00]Like Dubuque

[144:28.00]And then

[144:29.00]The same

[144:30.00]Like Dubuque

[144:31.00]And then

[144:32.00]The same

[144:33.00]Like Dubuque

[144:34.00]And then

[144:35.00]The same

[144:36.00]Like Dubuque

[144:37.00]And then

[144:38.00]The same

[144:39.00]Like Dubuque

[144:40.00]And then

[144:41.00]The same

[144:42.00]Like Dubuque

[144:43.00]And then

[144:44.00]The same

[144:45.00]Like Dubuque

[144:46.00]And then

[144:47.00]The same

[144:48.00]Like Dubuque

[144:49.00]And then

[144:50.00]The same

[144:51.00]Like Dubuque

[144:52.00]And then

[144:53.00]The same

[144:54.00]Like Dubuque

[144:55.00]And then

[144:56.00]The same

[144:57.00]Like Dubuque

[144:58.00]And then

[144:59.00]The same

[145:00.00]Like Dubuque

[145:01.00]And then

[145:02.00]The same

[145:03.00]Like Dubuque

[145:04.00]And then

[145:05.00]The same

[145:06.00]Like Dubuque

[145:07.00]And then

[145:08.00]The same

[145:09.00]Like Dubuque

[145:10.00]And then

[145:11.00]The same

[145:12.00]Like Dubuque

[145:13.00]And then

[145:14.00]The same

[145:15.00]Like Dubuque

[145:16.00]And then

[145:17.00]The same

[145:18.00]Like Dubuque

[145:19.00]And then

[145:20.00]The same

[145:21.00]Like Dubuque

[145:22.00]And then

[145:23.00]The same

[145:24.00]Like Dubuque

[145:25.00]And then

[145:26.00]The same

[145:27.00]Like Dubuque

[145:28.00]And then

[145:29.00]The same

[145:30.00]Like Dubuque

[145:31.00]And then

[145:32.00]The same

[145:33.00]Like Dubuque

[145:34.00]And then

[145:35.00]The same

[145:36.00]Like Dubuque

[145:37.00]And then

[145:38.00]The same

[145:39.00]Like Dubuque

[145:40.00]And then

[145:41.00]The same

[145:42.00]Like Dubuque

[145:43.00]And then

[145:44.00]The same

[145:45.00]Like Dubuque

[145:46.00]And then

[145:47.00]The same

[145:48.00]Like Dubuque

[145:49.00]And then

[145:50.00]The same

[145:51.00]Like Dubuque

[145:52.00]And then

[145:53.00]The same

[145:54.00]Like Dubuque

[145:55.00]And then

[145:56.00]The same

[145:57.00]Like Dubuque

[145:58.00]And then

[145:59.00]The same

[146:00.00]Like Dubuque

[146:01.00]And then

[146:02.00]The same

[146:03.00]Like Dubuque

[146:04.00]And then

[146:05.00]The same

[146:06.00]Like Dubuque

[146:07.00]And then

[146:08.00]The same

[146:09.00]Like Dubuque

[146:10.00]And then

[146:11.00]The same

[146:12.00]Like Dubuque

[146:13.00]And then

[146:14.00]The same

[146:15.00]Like Dubuque

[146:16.00]And then

[146:17.00]The same

[146:18.00]Like Dubuque

[146:19.00]And then

[146:20.00]The same

[146:21.00]Like Dubuque

[146:22.00]And then

[146:23.00]The same

[146:24.00]Like Dubuque

[146:25.00]And then

[146:26.00]The same

[146:27.00]Like Dubuque

[146:28.00]And then

[146:29.00]The same

[146:30.00]Like Dubuque

[146:31.00]And then

[146:32.00]The same

[146:33.00]Like Dubuque

[146:34.00]And then

[146:35.00]The same

[146:36.00]Like Dubuque

[146:37.00]And then

[146:38.00]The same

[146:39.00]Like Dubuque

[146:40.00]And then

[146:41.00]The same

[146:42.00]Like Dubuque

[146:43.00]And then

[146:44.00]The same

[146:45.00]Like Dubuque

[146:46.00]And then

[146:47.00]The same

[146:48.00]Like Dubuque

[146:49.00]And then

[146:50.00]The same

[146:51.00]Like Dubuque

[146:52.00]And then

[146:53.00]The same

[146:54.00]Like Dubuque

[146:55.00]And then

[146:56.00]The same

[146:57.00]Like Dubuque

[146:58.00]And then

[146:59.00]The same

[147:00.00]Like Dubuque

[147:01.00]And then

[147:02.00]The same

[147:03.00]Like Dubuque

[147:04.00]And then

[147:05.00]The same

[147:06.00]Like Dubuque

[147:07.00]And then

[147:08.00]The same

[147:09.00]Like Dubuque

[147:10.00]And then

[147:11.00]The same

[147:12.00]Like Dubuque

[147:13.00]And then

[147:14.00]The same

[147:15.00]Like Dubuque

[147:16.00]And then

[147:17.00]The same

[147:18.00]Like Dubuque

[147:19.00]And then

[147:20.00]The same

[147:21.00]Like Dubuque

[147:22.00]And then

[147:23.00]The same

[147:24.00]Like Dubuque

[147:25.00]And then

[147:26.00]The same

[147:27.00]Like Dubuque

[147:28.00]And then

[147:29.00]The same

[147:30.00]Like Dubuque

[147:31.00]And then

[147:32.00]The same

[147:33.00]Like Dubuque

[147:34.00]And then

[147:35.00]The same

[147:36.00]Like Dubuque

[147:37.00]And then

[147:38.00]The same

[147:39.00]Like Dubuque

[147:40.00]And then

[147:41.00]The same

[147:42.00]Like Dubuque

[147:43.00]And then

[147:44.00]The same

[147:45.00]Like Dubuque

[147:46.00]And then

[147:47.00]The same

[147:48.00]Like Dubuque

[147:49.00]And then

[147:50.00]The same

[147:51.00]Like Dubuque

[147:52.00]And then

[147:53.00]The same

[147:54.00]Like Dubuque

[147:55.00]And then

[147:56.00]The same

[147:57.00]Like Dubuque

[147:58.00]And then

[147:59.00]The same

[148:00.00]Like Dubuque

[148:01.00]And then

[148:02.00]The same

[148:03.00]Like Dubuque

[148:04.00]And then

[148:05.00]The same

[148:06.00]Like Dubuque

[148:07.00]And then

[148:08.00]The same

[148:09.00]Like Dubuque

[148:10.00]And then

[148:11.00]The same

[148:12.00]Like Dubuque

[148:13.00]And then

[148:14.00]The same

[148:15.00]Like Dubuque

[148:16.00]And then

[148:17.00]The same

[148:18.00]Like Dubuque

[148:19.00]And then

[148:20.00]The same

[148:21.00]Like Dubuque

[148:22.00]And then

[148:23.00]The same

[148:24.00]Like Dubuque

[148:25.00]And then

[148:26.00]The same

[148:27.00]Like Dubuque

[148:28.00]And then

[148:29.00]The same

[148:30.00]Like Dubuque

[148:31.00]And then

[148:32.00]The same

[148:33.00]Like Dubuque

[148:34.00]And then

[148:35.00]The same

[148:36.00]Like Dubuque

[148:37.00]And then

[148:38.00]The same

[148:39.00]Like Dubuque

[148:40.00]And then

[148:41.00]The same

[148:42.00]Like Dubuque

[148:43.00]And then

[148:44.00]The same

[148:45.00]Like Dubuque

[148:46.00]And then

[148:47.00]The same

[148:48.00]Like Dubuque

[148:49.00]And then

[148:50.00]The same

[148:51.00]Like Dubuque

[148:52.00]And then

[148:53.00]The same

[148:54.00]Like Dubuque

[148:55.00]And then

[148:56.00]The same

[148:57.00]Like Dubuque

[148:58.00]And then

[148:59.00]The same

[149:00.00]Like Dubuque

[149:01.00]And then

[149:02.00]The same

[149:03.00]Like Dubuque

[149:04.00]And then

[149:05.00]The same

[149:06.00]Like Dubuque

[149:07.00]And then

[149:08.00]The same

[149:09.00]Like Dubuque

[149:10.00]And then

[149:11.00]The same

[149:12.00]Like Dubuque

[149:13.00]And then

[149:14.00]The same

[149:15.00]Like Dubuque

[149:16.00]And then

[149:17.00]The same

[149:18.00]Like Dubuque

[149:19.00]And then

[149:20.00]The same

[149:21.00]Like Dubuque

[149:22.00]And then

[149:23.00]The same

[149:24.00]Like Dubuque

[149:25.00]And then

[149:26.00]The same

[149:27.00]Like Dubuque

[149:28.00]And then

[149:29.00]The same

[149:30.00]Like Dubuque

[149:31.00]And then

[149:32.00]The same

[149:33.00]Like Dubuque

[149:34.00]And then

[149:35.00]The same

[149:36.00]Like Dubuque

[149:37.00]And then

[149:38.00]The same

[149:39.00]Like Dubuque

[149:40.00]And then

[149:41.00]The same

[149:42.00]Like Dubuque

[149:43.00]And then

[149:44.00]The same

[149:45.00]Like Dubuque

[149:46.00]And then

[149:47.00]The same

[149:48.00]Like Dubuque

[149:49.00]And then

[149:50.00]The same

[149:51.00]Like Dubuque

[149:52.00]And then

[149:53.00]The same

[149:54.00]Like Dubuque

[149:55.00]And then

[149:56.00]The same

[149:57.00]Like Dubuque

[149:58.00]And then

[149:59.00]The same

[150:00.00]Like Dubuque

[150:01.00]And then

[150:02.00]The same

[150:03.00]Like Dubuque

[150:04.00]And then

[150:05.00]The same

[150:06.00]Like Dubuque

[150:07.00]And then

[150:08.00]The same

[150:09.00]Like Dubuque

[150:10.00]And then

[150:11.00]The same

[150:12.00]Like Dubuque

[150:13.00]And then

[150:14.00]The same

[150:15.00]Like Dubuque

[150:16.00]And then

[150:17.00]The same

[150:18.00]Like Dubuque

[150:19.00]And then

[150:20.00]The same

[150:21.00]Like Dubuque

[150:22.00]And then

[150:23.00]The same

[150:24.00]Like Dubuque

[150:25.00]And then

[150:26.00]The same

[150:27.00]Like Dubuque

[150:28.00]And then

[150:29.00]The same

[150:30.00]Like Dubuque

[150:31.00]And then

[150:32.00]The same

[150:33.00]Like Dubuque

[150:34.00]And then

[150:35.00]The same

[150:36.00]Like Dubuque

[150:37.00]And then

[150:38.00]The same

[150:39.00]Like Dubuque

[150:40.00]And then

[150:41.00]The same

[150:42.00]Like Dubuque

[150:43.00]And then

[150:44.00]The same

[150:45.00]Like Dubuque

[150:46.00]And then

[150:47.00]The same

[150:48.00]Like Dubuque

[150:49.00]And then

[150:50.00]The same

[150:51.00]Like Dubuque

[150:52.00]And then

[150:53.00]The same

[150:54.00]Like Dubuque

[150:55.00]And then

[150:56.00]The same

[150:57.00]Like Dubuque

[150:58.00]And then

[150:59.00]The same

[151:00.00]Like Dubuque

[151:01.00]And then

[151:02.00]The same

[151:03.00]Like Dubuque

[151:04.00]And then

[151:05.00]The same

[151:06.00]Like Dubuque

[151:07.00]And then

[151:08.00]The same

[151:09.00]Like Dubuque

[151:10.00]And then

[151:11.00]The same

[151:12.00]Like Dubuque

[151:13.00]And then

[151:14.00]The same

[151:15.00]Like Dubuque

[151:16.00]And then

[151:17.00]The same

[151:18.00]Like Dubuque

[151:19.00]And then

[151:20.00]The same

[151:21.00]Like Dubuque

[151:22.00]And then

[151:23.00]The same

[151:24.00]Like Dubuque

[151:25.00]And then

[151:26.00]The same

[151:27.00]Like Dubuque

[151:28.00]And then

[151:29.00]The same

[151:30.00]Like Dubuque

[151:31.00]And then

[151:32.00]The same

[151:33.00]Like Dubuque

[151:34.00]And then

[151:35.00]The same

[151:36.00]Like Dubuque

[151:37.00]And then

[151:38.00]The same

[151:39.00]Like Dubuque

[151:40.00]And then

[151:41.00]The same

[151:42.00]Like Dubuque

[151:43.00]And then

[151:44.00]The same

[151:45.00]Like Dubuque

[151:46.00]And then

[151:47.00]The same

[151:48.00]Like Dubuque

[151:49.00]And then

[151:50.00]The same

[151:51.00]Like Dubuque

[151:52.00]And then

[151:53.00]The same

[151:54.00]Like Dubuque

[151:55.00]And then

[151:56.00]The same

[151:57.00]Like Dubuque

[151:58.00]And then

[151:59.00]The same

[152:00.00]Like Dubuque

[152:01.00]And then

[152:02.00]The same

[152:03.00]Like Dubuque

[152:04.00]And then

[152:05.00]The same

[152:06.00]Like Dubuque

[152:07.00]And then

[152:08.00]The same

[152:09.00]Like Dubuque

[152:10.00]And then

[152:11.00]The same

[152:12.00]Like Dubuque

[152:13.00]And then

[152:14.00]The same

[152:15.00]Like Dubuque

[152:16.00]And then

[152:17.00]The same

[152:18.00]Like Dubuque

[152:19.00]And then

[152:20.00]The same

[152:21.00]Like Dubuque

[152:22.00]And then

[152:23.00]The same

[152:24.00]Like Dubuque

[152:25.00]And then

[152:26.00]The same

[152:27.00]Like Dubuque

[152:28.00]And then

[152:29.00]The same

[152:30.00]Like Dubuque

[152:31.00]And then

[152:32.00]The same

[152:33.00]Like Dubuque

[152:34.00]And then

[152:35.00]The same

[152:36.00]Like Dubuque

[152:37.00]And then

[152:38.00]The same

[152:39.00]Like Dubuque

[152:40.00]And then

[152:41.00]The same

[152:42.00]Like Dubuque

[152:43.00]And then

[152:44.00]The same

[152:45.00]Like Dubuque

[152:46.00]And then

[152:47.00]The same

[152:48.00]Like Dubuque

[152:49.00]And then

[152:50.00]The same

[152:51.00]Like Dubuque

[152:52.00]And then

[152:53.00]The same

[152:54.00]Like Dubuque

[152:55.00]And then

[152:56.00]The same

[152:57.00]Like Dubuque

[152:58.00]And then

[152:59.00]The same

[153:00.00]Like Dubuque

[153:01.00]And then

[153:02.00]The same

[153:03.00]Like Dubuque

[153:04.00]And then

[153:05.00]The same

[153:06.00]Like Dubuque

[153:07.00]And then

[153:08.00]The same

[153:09.00]Like Dubuque

[153:10.00]And then

[153:11.00]The same

[153:12.00]Like Dubuque

[153:13.00]And then

[153:14.00]The same

[153:15.00]Like Dubuque

[153:16.00]And then

[153:17.00]The same

[153:18.00]Like Dubuque

[153:19.00]And then

[153:20.00]The same

[153:21.00]Like Dubuque

[153:22.00]And then

[153:23.00]The same

[153:24.00]Like Dubuque

[153:25.00]And then

[153:26.00]The same

[153:27.00]Like Dubuque

[153:28.00]And then

[153:29.00]The same

[153:30.00]Like Dubuque

[153:31.00]And then

[153:32.00]The same

[153:33.00]Like Dubuque

[153:34.00]And then

[153:35.00]The same

[153:36.00]Like Dubuque

[153:37.00]And then

[153:38.00]The same

[153:39.00]Like Dubuque

[153:40.00]And then

[153:41.00]The same

[153:42.00]Like Dubuque

[153:43.00]And then

[153:44.00]The same

[153:45.00]Like Dubuque

[153:46.00]And then

[153:47.00]The same

[153:48.00]Like Dubuque

[153:49.00]And then

[153:50.00]The same

[153:51.00]Like Dubuque

[153:52.00]And then

[153:53.00]The same

[153:54.00]Like Dubuque

[153:55.00]And then

[153:56.00]The same

[153:57.00]Like Dubuque

[153:58.00]And then

[153:59.00]The same

[154:00.00]Like Dubuque

[154:01.00]And then

[154:02.00]The same

[154:03.00]Like Dubuque

[154:04.00]And then

[154:05.00]The same

[154:06.00]Like Dubuque

[154:07.00]And then

[154:08.00]The same

[154:09.00]Like Dubuque

[154:10.00]And then

[154:11.00]The same

[154:12.00]Like Dubuque

[154:13.00]And then

[154:14.00]The same

[154:15.00]Like Dubuque

[154:16.00]And then

[154:17.00]The same

[154:18.00]Like Dubuque

[154:19.00]And then

[154:20.00]The same

[154:21.00]Like Dubuque

[154:22.00]And then

[154:23.00]The same

[154:24.00]Like Dubuque

[154:25.00]And then

[154:26.00]The same

[154:27.00]Like Dubuque

[154:28.00]And then

[154:29.00]The same

[154:30.00]Like Dubuque

[154:31.00]And then

[154:32.00]The same

[154:33.00]Like Dubuque

[154:34.00]And then

[154:35.00]The same

[154:36.00]Like Dubuque

[154:37.00]And then

[154:38.00]The same

[154:39.00]Like Dubuque

[154:40.00]And then

[154:41.00]The same

[154:42.00]Like Dubuque

[154:43.00]And then

[154:44.00]The same

[154:45.00]Like Dubuque

[154:46.00]And then

[154:47.00]The same

[154:48.00]Like Dubuque

[154:49.00]And then

[154:50.00]The same

[154:51.00]Like Dubuque

[154:52.00]And then

[154:53.00]The same

[154:54.00]Like Dubuque

[154:55.00]And then

[154:56.00]The same

[154:57.00]Like Dubuque

[154:58.00]And then

[154:59.00]The same

[155:00.00]Like Dubuque

[155:01.00]And then

[155:02.00]The same

[155:03.00]Like Dubuque

[155:04.00]And then

[155:05.00]The same

[155:06.00]Like Dubuque

[155:07.00]And then

[155:08.00]The same

[155:09.00]Like Dubuque

[155:10.00]And then

[155:11.00]The same

[155:12.00]Like Dubuque

[155:13.00]And then

[155:14.00]The same

[155:15.00]Like Dubuque

[155:16.00]And then

[155:17.00]The same

[155:18.00]Like Dubuque

[155:19.00]And then

[155:20.00]The same

[155:21.00]Like Dubuque

[155:22.00]And then

[155:23.00]The same

[155:24.00]Like Dubuque

[155:25.00]And then

[155:26.00]The same

[155:27.00]Like Dubuque

[155:28.00]And then

[155:29.00]The same

[155:30.00]Like Dubuque

[155:31.00]There has been a tremendous explosion in terms of the number of people that are interested, the number of experiment that are being tried out.

[155:38.00]And I would say we are very fortunate that there is this kind of interest right now.

[155:43.00]The thing that you notice is it does go from some of the seminal moment to seminal moment.

[155:50.00]Between AlexNet and the transformer paper in 2017, you could see that there was a huge amount of innovation on the visual side.

[156:00.00]You know, we had resonance, you know, all kinds of interesting architectures.

[156:05.00]And then after 2017, it became again significantly focused on sequence to sequence, right?

[156:11.00]So the ideal sequence here, you know, machine translation therefore was text.

[156:16.00]You know, we keep hearing about maybe there will be a breakthrough moment on the visual side again, right?

[156:22.00]In another five years.

[156:23.00]And then suddenly, you know, people will start spending more energy there.

[156:29.00]So I would like to actually go back to the modality, right?

[156:32.00]So it was the visual modality and now the text modality.

[156:35.00]And then maybe it's proteins, right?

[156:38.00]Maybe it's some modality in the healthcare space, right?

[156:42.00]Where sudden breakthroughs come about in the next several years.

[156:46.00]And then you will suddenly see a whole bunch of people looking at gene sequences and so on and so forth, right?

[156:52.00]And that interest will spike.

[156:54.00]So I actually find it fascinating that we are growing as a community from where we started.

[156:59.00]I don't really see it as a hype versus not hype.

[157:03.00]I really see it as more of the ability of this modality coupled with the neural architecture.

[157:10.00]You know, that will be the most effective and efficient at that point of time in attracting a lot of energy from the academic and industrial research community.

[157:19.00]So we can only be better off because of it.

[157:22.00]So I don't see it as a hype as much as an opportunity for learning more.

[157:27.00]I will add this.

[157:28.00]And I think it is a good segue for the next part of what you're going to talk about LSU, right?

[157:33.00]Which is I get asked this question often, right?

[157:36.00]Why did I move from NVIDIA?

[157:37.00]Like who leaves NVIDIA?

[157:39.00]And the fact of the matter is that really if you have that unfulfilled desire in you to solve the actual end problem,

[157:46.00]you have to go towards one of these verticals, right?

[157:49.00]With the whether it's health care, whether it is financial services and you have to be embedded in an organization that actually has a culture for fostering big tech like a fang like work ethic.

[158:02.00]Whether it's having the data, right?

[158:04.00]Already being in the cloud, having the roots in data driven and machine learning.

[158:09.00]So you want to be in a place which allows for that kind of creativity so that you can actually take Gen AI to its logical next step, right?

[158:19.00]We're just solving problems so that we change the financial services domain for the good, right?

[158:24.00]There's benefit to the customers.

[158:26.00]And so when you do that, though, you have to be mindful.

[158:30.00]If you are working in places where you are building recommendation systems for what to buy or what to watch.

[158:39.00]There are a lot of ways in which you may not have the best possible answer, the most accurate answer and you are still fine.

[158:46.00]But when you are in one of these domains that matter, health care, financial services, right?

[158:51.00]You really are solving a harder problem because the tolerance for error of your end customer is going to be much,

[158:59.00]much,much lower.And what that does is it forces the research and development to go in specific directions.

[159:06.00]You may not be able to just take what's out there in open source and use it as is, right?

[159:12.00]You may have to actually innovate beyond what's in the state of the art publication to make it work for this kind of domain, right?

[159:21.00]And so it takes a special class of applied researchers.

[159:25.00]It takes a special class of machine learning engineers, data scientists that have the will.

[159:31.00]The last mile is hard and you have to have that will to solve that problem.

[159:36.00]And so in some cases what ends up happening is that these roles and the work we will do may be harder than what you end up doing in a generic setting or at a platform level only or in a use case which actually doesn't have this kind of extremely low tolerance for error.

[159:54.00]So that becomes, you know, one of the main motivating functions for people to join us, right?

[160:01.00]Who really want to actually take these kind of really hard challenges.

[160:05.00]I was going through some of your team papers just from 2023, some of them in Europe, some at ICML.

[160:10.00]You've done a lot of work like class imbalance on data sets.

[160:13.00]So how do you make performing neural networks when like the data is kind of in balance in certain domains.

[160:18.00]You're doing some research on transformer graphs versus like graph neural networks.

[160:23.00]So there's just kind of like a lot in there.

[160:25.00]Any favorite project that you want to shut out any interesting paper that you saw come out of your team.

[160:30.00]You know, I could choose one, but then it would basically end up being, you know, not fair to the others.

[160:35.00]So the only on that, you know,

[160:39.00]It's like the what's your favorite child?

[160:42.00]There's no way to say one versus the other.

[160:46.00]But there are a lot of interesting trajectories of exploration here, right?

[160:51.00]We are trying to look at the superset of what all these things are trying to do.

[160:55.00]We are trying to look at data sets that are very unique to our domain.

[160:58.00]We are trying to look at attributes that are important to us like time series, tabular, right?

[161:04.00]These are things that are important, very important.

[161:06.00]We are looking at the imbalance problems, right?

[161:09.00]There are other things that we are looking at.

[161:11.00]And so I wouldn't want to just highlight one.

[161:15.00]There are a bunch of things that are of great interest to us from our perspective.

[161:20.00]Let me just not pick one.

[161:23.00]That makes sense.

[161:25.00]And yeah, just to wrap, we always like to ask our guests, who are you looking for?

[161:29.00]You know, we got a lot of AI engineers, researchers in the audience.

[161:33.00]Like, who are the type of people that are going to have a good time working with you?

[161:36.00]And what are some of the open roles that you have?

[161:38.00]So let me start with the roles.

[161:40.00]And then, you know, we can talk about the kind of people who are going to have a great time with us, right?

[161:44.00]We have a number of open roles and they span the entire spectrum from applied researchers to data scientists, to machine learning engineers, AI engineers, right?

[161:55.00]People who are listening to you right now that we really weren't interested in what we are doing.

[162:01.00]We have roles at various levels of seniority.

[162:03.00]We have individual contributor roles.

[162:05.00]That's one of the things that we have really, really double clicked on in the last several months since I've joined.

[162:10.00]And that was a thrust also before I joined, but especially in the applied research field, right?

[162:15.00]We are looking for, you know, what would be the equivalent of, you know, principal research scientists, right?

[162:20.00]Distinguished research scientists, individual contributors.

[162:23.00]We are looking for fresh graduates, right?

[162:26.00]Masters and PhD students, you know, just out of school, all the way to people who have, you know, a decade or more of experience.

[162:33.00]So, really there is a huge spectrum of talent that we want to onboard and we are looking for.

[162:41.00]Now, what is the characteristic of someone who will really come and enjoy here, right?

[162:46.00]I think I started giving that to you earlier when I said people who are interested in actually solving the problem.

[162:52.00]That last mile is hard, so we really want people to know that they have to have the stomach for that last mile.

[162:58.00]People who are good at understanding what the product requirements are, they sometimes make the best applied researchers, right?

[163:07.00]Because they can understand what our needs are and formulate problems, change architectures.

[163:15.00]We are looking for people who are very good at dealing with ambiguity.

[163:19.00]A lot of what we are doing, a lot of the advancements we are seeing, they are empirical data driven, right?

[163:25.00]And so, we need to have that as a skill.

[163:29.00]How do you actually build algorithms, systems and solutions that you can show improvement on our data?

[163:37.00]It's great if somebody is doing some architecture or some network is doing great on some of the benchmarks, right?

[163:45.00]That get published outside, but it's equally important.

[163:48.00]It's actually more important that we are able to show to ourselves that these algorithms, architectures do well on our own internal data sets and benchmarks and evaluation, right?

[164:00.00]And so, to that point, how do we actually come up with meaningful evaluation frameworks and methodologies, right?

[164:05.00]That itself is something of great interest to us.

[164:09.00]So, people in general who like to solve problems, problem solvers, people who like to go from theoretical to practical and people who are not afraid that the empirical data may not bear out their best idea and they have to go and rethink it, right?

[164:25.00]And redo it.Those are the kind of people that will really do well here.

[164:28.00]Yeah, it was great to hear your story and not a lot of people in the world, I think, that have the same depth of experience in AI, so this was awesome.

[164:35.00]Yeah, a lot of people that you're looking for are also like the kind of AI engineers that we want to encourage.

[164:40.00]So, yeah, thanks for sharing your thoughts.

[164:42.00]Thank you, Sean.

[164:43.00](音乐)

[165:07.00](音乐)

[165:09.00]字幕by索兰娅

[165:11.00]我看你不太好看

[165:12.32]我看你不太好看

