PhD Bodybuilder Predicts The Future of AI (97% Certain) [Dr. Mike Israetel]

So if if you had a simulation of fire in a computer or of a stomach, it wouldn't digest. Fire wouldn't get hot. Water wouldn't get wet. It would digest. We we can't even I mean this is just we can't even be debating this. >> I I can and I will watch this. >> When you saw the Matrix, right? You know that the architect, the first version of the Matrix, it was too perfect. There weren't any problems. >> Yeah. So like I'll say that when I saw that, my inner philosopher was like, um, >> you don't actually want to live in that world, do you? Desperately everybody does. >> It would be horrible. >> It would be the best thing ever. Well, sorry. What? By definition, by the way, what would be horrible about it? Exactly. >> Don't Yeah, but don't don't you think that life is about suffering? >> No. [ __ ] no. >> Doesn't that give you perspective on joy? >> No. [ __ ] no. >> Going to go and pick up Dr. Mike. >> Great to meet you, Mike. >> Mikewise, >> how are you? >> Very good. Very good. >> Yeah, it was good. Nice meeting you, Mike. Nice to meet you. >> Hey guys, Mike is Richel here. I'm uh by training a sport scientist and I run RP Strength Fitness Company and I lift weights and my head looks strange. People make fun of me in the street. They laugh and they throw usually rotting fruit. It's unfortunate. Sometimes a vegetable. And I have a vision of the world in which more intelligence is almost always better, in which cooperation is a good thing. in which we build a future for every single human that's orders of magnitude better than it is now. And I think that [music] would be a great thing in almost every regard. And the three reasons that you should be watching this podcast is that if you want to see me completely out of my depth, embarrass the [ __ ] out of myself with terms I don't understand as they're rendered out of my mouth, get pushed hard and be corrected on a variety of things, and also have some lively debate, and maybe make some future predictions that get us both in trouble, I think those are some great reasons to tune in. Uh, so Mike is going to demonstrate the first exercise which one of you will do. And while you're doing this one, the other guy will do the other. So Mike is going to bend forward, hinge at the hips, torso down as deep as you can to the floor, almost even touching if you can, and row up. And then again, and that's how they should look. No swinging into the upper body, nothing like that. And then [music] once you're done with that, you'll move over to the next exercise, which your partner will be doing. So you'll switch just push-ups. So, we like to do it kind of dive bomb style where you keep your belly button really high and you push your chest to the ground and come up. >> All right, keep it cranking. Nice and easy. >> Oh, yeah. >> Absolutely. You guys want a workout until you can't do any more reps without cheating? No cheating. Tim, you're doing a great job. [music] Love the control. These are dang good push-ups. These are excellent reps, [music] Marcus. Keep that back flat. Keep your chest high, but your arms stretched. There you go. Oh, just like that. Good. >> Now you're getting a little bit of hamstring involvement, too. >> Now switch. >> Switch. >> No rest. Nobody said rest. >> Now you get to do the pulling musculature, Tim. >> Yes, Rose. So, bend over. Tummy up, chest up. Go deep. >> Yes. >> Deeper. Lean forward even more. >> Remember, the more reps you do, Tim, the more Marcus suffers, which what you This is what he wanted. >> There you go. >> Good job. [ __ ] machines here. Let's go. Don't you [ __ ] quit. Are you out of your mind? >> What did this? >> All right, one more round. Switch. >> Switch. Last round. Marcus, get back down and start doing push-ups. God damn it. [ __ ] is wrong with you? Three more. Let's go. Three more each. One. Super deep. Two. Last one. Three. [ __ ] Yes. Beautiful. We did our workout earlier. >> [music] >> Am I the least technically qualified guest you guys have ever had? I must be top five. Like you ask a very distinguished exercise scientist like, "Hey, how many reps should I do on curls?" And they're like, "I just study one pathway of muscle hypertrophy and I know it real well. I just go to the job my job, do experiments, and I don't lift weights or know much about that." And you're like, "Oh, [ __ ] The the the generalization did not go anywhere. I'll I'll be here defending San Francisco, which is an odd statement to make. I have a very unique take. I think ASI is coming in 2627 and AGI is coming in 29, 30, maybe 31. >> Yeah, you spoke about this with Lon. Why is AGI after ASI? Because artificial general intelligence has uh either depending on who you speak to no good definition at all or multiple uh spectra definitions somewhere between shitty um like the um uh you know various tests of if you you know can text something is it truly intelligent um all the way through I think very good but artificial general intelligence typically some of the better definitions encompass all of the kinds of intelligence and abilities that human beings are able to put out. And honestly, from a vibe perspective, if you say like, "We've really cracked AGI, but your machine can't do some kind of cognitive work that a human can," you haven't cracked AGI in any meaningful respect. But because humans have some very interesting abilities that we just don't have the reach for from um kind of an integration perspective. For example, um smells and tastes. I mean nanotech's got a bit of a way to go to get a machine to smell and taste in a meaningful way. And so like something about being a chef and being able to cognitively rank and hear tastes and smells and spectra and functionally employ them in real life to make food that's not going to happen probably in 2627 right but then we look back and so AGI because it is human inclusive and that means we have to replicate every kind of human intelligence or say confidently we have AGI from my perspective all the following all [ __ ] dilotant takes by the way um then you know AGI is actually like uh pretty tough. Well, will happen I think but tough. Now artificial super intelligence is uh can be thought of in many ways but I think once you have something that on many domains of human ability not all is radically more intelligent it functionally vibe checkwise is super intelligence. For example, something that is linguistically, mathematically, scientifically, three-dimensional object rotation, world model depth, recursion ability, is 100 times the power of a human. You can tell me it's not super intelligent cuz it doesn't know how to smell or taste. It's never seen the world with its own eyes, even though it's scanned all of YouTube and integrated all of YouTube's visual data into an integrated 3D world model. Can I swear a little bit or is that not a good It's a [ __ ] artificial super intelligence then whatever. Right. And so I think because in many respects like so I work with um a few AIs but I love OpenAI's product suite. I pay the infinity amount per month to get the pro uh version the pro subscription for GPT. And so a GPT5 Pro is in some respects not as smart as me yet. Mostly because it can't cross-link distant concepts together and integrate it whole as well because that's not really what it was designed to do because that would burn a shitload of tokens for an ROI that for most people is meaningless. But boy does it do black hole physics better than me and almost everyone and recently better than every scientist because it's got novel discoveries now. And so you take the abilities of just GPT5, right? And in the back rooms at OpenAI, they have something that just beats the [ __ ] out of DPT5 and it's currently getting fine-tuned or, you know, post-trained, whatever. That thing's probably 10 times as smart. That's super intelligence. also factual knowledge about the world. I mean like what percent do you think you know of total data, total real things about the world? How tall are people? What do they look like? Where you know what's the capital of France? That stuff compared to just GBT5. I mean it's some infantestimally small fraction. Uh that on knowledge base is already super intelligence. I mean put GBT5 against any Jeopardy winner and you get the world's biggest creaming all of a sudden. That sounds like one of my older films. Am I right? So uh so in in in many respects I think in 2026 we are going to see AI systems that 10x or 100x human abilities or 2x or anything between that and so once you get to maybe 2/3 of all cognitive abilities maybe 75% maybe 80% demonstrabably machines are just like categorically superior by orders of magnitude that is super intelligence so super intelligence two ways on vibes and heristic by what I just described and also an effect. I mean, if you have a smart enough AI and it's crapping out novel hypotheses once an hour and it takes scientists weeks to grind through them and it starts getting 60% 80% 90% hit rate, I'm like, um, it understands the cell and it's giving us novel disease cures every week. Uh, that is super intelligence. And so super intelligence has to be measured uh by two things. One is under the hood cognitive science abilities, but the other is like real world effect. Because here's another really big thing super quick. You don't have super intelligence really in truthful evidence to convince most skeptics until you have the fruits of its labor. It's like someone tells you they're really rich. You're like, "Hey, can you buy me a flight to Dubai?" They're like, "Ah, money's tight right now." You're like, "I believe you, but I don't." But if if you know Elon Musk, you're like, "Hey, can you buy me a flight to Dubai?" He's like, "Yes, I just bought you Dubai Airlines." So you're like, "The airline?" You're like, "Yeah, I just acquired it." You're like, "Right." Okay. Wow. That's a demonstration. And so I think in 2026 I'm very confident though never certain from scientific perspective CIA probability scale extremely likely 97 to 100% that in sometime in late 2026 we are going to start opening the cornucopia of machine intelligence that people understand that at least parts of machine intelligence are absolutely super intelligent and because of exponentials by 27 28 29 the [ __ ] is going to get completely insane. There is so much to unpack there and I really like you Mike so I don't want to disagree with you on my podcast but I have >> no please I love disagreement. >> I have to and I think I disagree with almost everything you've just said so we can take it one one at a time. So um first of all the definition of intelligence you were talking about knowledge as well. I think that knowledge is nonfgeible. In fact one of the the most pervasive critiques of artificial intelligence going all the way back to the 70s there was Drafus and there was you know John C. It's it's this grounding problem. Steven Hanid spoke about this grounding problem and essentially the enterprise of intelligence is the accumulation of information which is relevant for adaptivity around an environment. So the product of our knowledge is all of the um you know it's Wikipedia. It's it's the hard drive in the sky. It's our culture. It's all of the things that we have acquired. But the problem is there is a gap between syntax and semantics. So if an alien read Wikipedia, they might read the article about Trump or about America or something like that. And that is not the same as the embodied experience of being there. These are just pointers. So this semantics is talking about the connected inactive embodied graph of actually experiencing things through time. And the biggest misconception in all of AI, what all of the folks in San Francisco believe in is this philosophical idea called functionalism. And the best way to describe it is there is this analogy of walking up a mountain. I spoke to a philosopher the other day, wonderful lady called um uh Anna Siauna and and she said that we're walking up the mountain and when we get to the top of the mountain, we have all of these abstract capabilities like being able to reason, play chess, but that disregards that the path that you took walking up the mountain is very important. And not only the path, the physical instantiation, the stuff that the mountain is made out of because it's a reasonable argument in my opinion that intelligence is a property of adaptive matter. It's an extensive property. It's much like temperature. Temperature is a coarse graining. It's an it's an effective theory to describe the details of the molecules moving around. And we we screen that detail off and we call it temperature. And I think intelligence is like that. And I think that knowledge is quite similar. I think knowledge is actually a physical causal graph that is enacted over time. And it cannot be abstracted in the way that you're describing. The reason the abstractions work is because they are pointers to our embodied experience. So they make sense to us, but on their own they don't make any sense. >> I like that take. I'll push back. Um, intelligence. My favorite definition is the mo the most basic definition of intelligence, the ability to solve problems. >> Yeah, I hate that. um uh and you find out if you're truly intelligent if you can solve problems of any degree of complexity that is uh I would actually say responding to one stimul one stimulus in a cogent manner at least knowing how to respond to one stimulus input output uh is the beginning of intelligence uh all intelligence above that is just stacked layers of complexity so when you say embodied there's a lot there for sure. But also the only reason that we call humans intelligence intelligent is because we have a representational abstracted neural network called the brain. Your brain doesn't actually have anything in it that's magical. It's just network pings off one another recursively. And so your brain is in very many deeply important respects exactly as abstracted and unrelated to reality as a data center. And so you can climb mountains, you can touch stuff, but you never truly embodied experience anything if you push on that philosophical button hard enough because you can always abstract out to like these are just neural network pings from groups of neurons. And so you don't truly deeply know anything in some kind of weird philosophical way because it's just neural network calculus all the way down. And so whether it's you or a machine brain like what they have in Optimus for example, it's representational. And so intelligence is always seemingly going to be something that is lossy to some extent. It is a compression function. Is it? It's a real rude way to say it because it's a very impressive one. But there, you know, you climb the mountain. [clears throat] That's cool. Helicopter can climb a mountain much better than you. Does not have the ability to reason and abstractly and plan and predict things at all. And you can get really amazing highde video and samples of rocks and the mountain and everything. But if you don't have an analyzer, a system processor, an integrator and a recursive function, you don't actually [clears throat] model that data in any meaningful way and it just sits there as bits on a computer. And so when you say that, you know, embodiment is a prerequisite of intelligence, I would push back and ask you to answer this. If we have somebody that has read every single physics book and especially let's say particle physics and they're so good at it that they can tell the computer system that aligns the particle beams at CERN to do a job good enough to produce realworld data. Are they truly intelligent about particle physics? By your definition, and I'm being a little bit facitious for comedic effect, the answer is no. Because if I put you in the particle accelerator, you get torn to [ __ ] shreds and you're like, "Oh [ __ ] JK." There's actually impossible for humans to perceive particles directly in any sensory organ because they're plank length type [ __ ] You can't see it. Vision is not a cogent concept at those lengths. So that goes straight to hell. And so if that scientist is so adept that he can actually run CERN, but it's all neural network modeling in his head, he's never had actually had any real experience with a particle collider. No human has. You can't open the latch when the thing is on. It'll shoot [ __ ] nutrinos at you. So if that's true and we say, "Okay, okay, okay, well, he actually knows things. He actually has intelligence. Why doesn't GPT5 and its network clusters actually have an understanding of the real world? It's never seen, but the guy's never seen a particle in his life. Zero scientists have ever seen a nutrino ever. They have no perceptual experience of a nutrino. It's purely hypothetical. But they make real world predictions because the neural network and all the addendent structures that create what we call intelligence and understanding are their their their vector arrangements represent decently well enough always an approximation but a damn good one predictively valid one of real things that are happening in the real world. And if GPT5 understands like how humans act to a 98%, but if it was embodied in a robot body and had vision, it would be like, "Oh, [ __ ] There's the rest of that 2%." I think 98% of the way to intelligence is the same thing that back when I was a professor of if uh one of your students gets a 98% on an exam, is he is he pretty good at understanding the material? Yeah, [ __ ] yeah. Now, just off camera over there is my best student of all time, Jared Feather. He never missed a single point on any of my exams. 100%. Well, that just means the exams weren't hard enough to really push his understanding. Is Jared the best student ever had the best trainer? Yes, for sure. But does that mean when kids score 95 or 98% of my exam, I'm like, never let this kid train you. He doesn't really know the real world. It's just abraction. No, [ __ ] no. They know real things in the real world and can really help you get fit. even though they might themselves have never been been able to get very fit uh because they know the actual structure out there purely representational neural network but it passes the test of when they see the real thing they go I know what a dumbbell is I know what a human is I know what vectors are I know what forces are and I'm going to have them do this and it works yeah a human would have a shared understanding because if you think about it the representations in our brain they are the product of evolution so we've been evolving for billions of years and and I know you read that Ray Kurszwell book and it's It's a beautiful um recount of how we had these kind of exponential increases. So you know we we had um genetic evolution which is very slow and then we had this onto genetic filogenetic hacking where we developed nervous systems and brains and we developed culture and we started you know evolving at light speed and so on and so forth. At the end of the day um the product of that knowledge acquisition that that process of intelligence are all of these representations and everything in language is a representation you know like for example um algo speak that that is um a term to represent the evolution the linguistic evolution of our language there's this term called unal alive so to get around the social media um filters we now say I'm alive someone and that is an example of linguistic evolution and chat GPT is not capable of doing Because there's a big difference between the process. >> Absolutely not. >> You wouldn't understand what unaliving is. >> We'll get into this, right? But understanding is not about being at the top of the mountain. Understanding is about the path to the mountain. So, we understand things because we are um physically embodied, right? You you were talking about, oh, you know, we can't observe the particles. It doesn't matter. We are in the causal graph of the particles. You know, they affect us even if we're not >> they don't they're inside their collider. You don't a radiation. >> You're not made out of particles. That that is >> Yeah. You're not made out of nutrinos. They're passing through you. The nutrinos are completely abstracted as far as we're concerned. They actually pass through the earth the entire time. So like there's all kinds of theoretical physics that have nothing to do with you really in a causal graph. They don't affect your behavior. They haven't really meaningfully affected any of your decisions. We we'd have to abstract like 18 layers and do quantum mechanics that none of us are I don't know. You're probably smart enough to get it. I'm not. And so like that shit's really [ __ ] in the real world that we're purely abstracting to. But because knowledge is actually representing the real world, it works. I I'm not I'm not so sure if you could explain what it is exactly you're getting at that we get out of embodied experience and how libraries and education curricula and schools and reading a bunch of books can possibly give you knowledge if none of that is any kind of experience that is embodied. Like how does ChachiPT know the vibe of Shinjuku in Tokyo and can accurately render to me what the vibe feels like if it's never been to Tokyo? Same way a travel agent does. But it's real actual things they know that when you go there, you're like, "Oh, [ __ ] It was pretty much correct." How does that work? >> Well, first of all, we we should distinguish um you know, knowledge and understanding. So many many people have different um definitions of this. You know, like in the cognitive sciences, people talk about um you know, kind of factual knowledge, which is you know, just states of affairs. Um there's procedural knowledge, knowing what to do. There's conceptual knowledge, knowing how to think. And roughly speaking, I make the designation that there are, you know, knowing um states of affairs. And understanding is this ability to generate new knowledge. So if you have an understanding of the world, which means you know it at an abstract level, you can do this Lego building in your mind and you can kind of create new knowledge. But the simple grounding problem is something a little bit different even with the the nutrinos that we use this thing David Krakow calls it the principle of materiality which is that we use the world to think and many of the abstractions that we've converged on over you know millions of years of of evolution is because we have this shared physical world. So we you and I both understand similar abstractions which are derived from our sensory motor circuits and from language and so on. And it means that we can talk at a very abstract level and because we're both players in the same game, we understand each other. And the parlor trick and we won't go over S's Chinese room argument. I'm sure you're familiar with it. He was saying exactly the same thing that you know basically a computer can tell you all of these things as if they understood and you understand because you actually have the the feeling. You actually know you you you even though you've never been to Japan, you've experienced many things that are like people that go to Japan experience. So you have some kind of a shared understanding. there is this there this intersection in your understanding tree that something like chatgbt doesn't have. Now I want to make a distinction as well. Um I'm really impressed with language models. You know um opus 4.5 came out yesterday. >> GBT is not a language model. Has been multimodal for two years now. >> It it it's still it's it's known as a language model. It's a self attention. >> Right. But but we but we have we have to concede the fact that it is it's an omnimodel. It's been multimodal for a long time. >> Yeah. It's multimodal, but it's it's still a self attention transform. It has a couple of traits. >> The transformers part is true, but just calling it a large language model, I think, cuts it a short of a huge fraction of its capability. I mean, it does visual reasoning. >> It doesn't make any difference. >> Okay. >> Yeah. Yeah. So, so it's a self potential. I mean, on the most recent show we just published, I interviewed the the the guy who invented transformers actually, but yeah, we we just tokenize different domains. We learn the statistical distribution. We do this next token prediction, you know, within a fixed context window. So, it's distributional matching, but then there's some other stuff on the top. there's the RLHF and there's also this um you know reinforcement learning with verifiable rewards for um verifiable domains and and this is why they're doing really really well on reasoning tasks when you have a verifier. So the ARC challenge is a great example of this. I'm a big fan of that. You know France is a friend of mine and he designed this challenge to show abstract generalization. So you only have a few uh examples to learn from and LLMs he predicted would be terrible at this and unfortunately for him the LLMs are now getting incredibly good at it. And that's because in my opinion the LLMs don't really understand in this grounded way that we're talking about. But in abstract domains you can fully describe an abstract domain like a 2D grid you know problem. You don't need to have any grounded knowledge. You can completely understand it. >> What do you mean by grounded knowledge? What is >> this? This is grounded knowledge, right? You know, >> you're touching a chair. >> Touching a chair. >> So, that's not knowledge. That's that's neuronal perception arcing up into your brain and back down. And so, what what what are you learning about the chair when you're touching it like that? >> This is the other issue that that I have with your perspective that that you have um I would call it a corticoentric um view of of cognition. >> Absolutely. >> Yeah. Exactly. So, this is another one of those leaky abstractions I was talking about. By the way, there's a great book called um the brain abstracted by Marva Chirima. I interviewed her recently and she said that one of the most pervasive myths in neuroscience is that we use these leaky abstractions and idealizations to talk about cognition. And usually it's using the most recent technology at the time. So you know a few hundred years ago we were describing the brain in terms of pulley and >> pullies and levers. Yes, >> that's right. And and you know and then it was um you know as a prediction machine as a computer and all this kind of stuff. At the end of this is an example of the these are grounded things that we understand. They're really good models because we can both talk about computers. We both know what computers are. But the brain doesn't work like that in any sense. And a great example of this is knowledge. You as a personal trainer know this, right? So you train you, you know, I'm sure in the past you've trained folks in the gym and what you've probably realized is that in your mind you've got a distilled abstract beautiful like you understand things. You've thought about it. You've distilled it. You could write it down in a book and you probably think all I need to do is just tell people and and I I've done all of the thinking. I've figured all of this out. I just tell them and what you learn to your chagrin is that there is no substitute for experience because part of knowing is actually doing. It's just the pure experience of lifting the weights just feeling the sensations in the body going through that process. That's why knowledge is nonfgeible. I've been fairly impressed incrementally with modern AI's ability to render pretty good recommendations on exercise and sport science. And I'll push you back on one of those things. I know a lot of people, they're great people that have a lot of what you would call embodied knowledge about training. And because they haven't been interested in or capable of doing the abstraction to distill principles, they're not really that great at training other people. They know what works for them and they know scarcely of that. uh and they're just not good at generalizing because they've never abstracted out the concepts. I've also uh had many discussions with some of my very very smart friends like one of my friends uh graduated from Harvard, masters data science, worked for Apple and a bunch of other AI companies and stuff and um his ability to pick up how to do an exercise incredibly well just off principal discussions and then trying it like twice. He's instantly like one of the best trainers I know in the area. one shot. Why? Because Jared and I can distill to you like five principles of how to do an exercise well, which are geometrically not complex, heristically very cogent, and you will instantly be good at teaching people how to exercise without ever doing the exercise. Is some nuance missing? Absolutely. There's some stuff and we can abstract how it works that you have to do to actually like, oh, I see moving the hips back like this. But if someone was able to describe that to a computer in vectors of how hips should move back in a squat versus just down, especially if so, so I'll tell you this. If I was able to train a model, and I'm not currently doing this and had no plans to do so, of giving it a thousand samples of a squat done right, a thousand samples of a squat done wrong, and about 10 rules of how to squat, it would instantly be 99 percentile of teaching people how to squat with zero grounded experience. And I would also say what we consider grounded experience is mostly visual data stream. Like yeah, touching stuff is cool and it gives you something like I'm not letting a computer have sex with me until it's had sex with test animals in the lab. That came out all wrong. Um but uh you a lot of our data is visual. Huge huge fraction of it. I suspect that if you let a neural network train on all of YouTube, which I understand I think isn't happening because there's like copyright [ __ ] which drives me insane. You just like have like an algo just eat all of YouTube which which is a preposterous amount of data. It would know more about how the world looks in real life than any extent human period because there's more visual data on YouTube than I have collected with my own eyes by I don't know 10 12 50 orders of magnitude something like that. And so zero embodiment. We did the embodying for it by putting up camcorders into people's faces while they throw up from drinking challenges and then downhill skiing and then the pictures of microbes. If you really train a visual model and even better a 3D relational model on YouTube alone, you will get a more grounded understanding than your eyeballs, which by the way are also just cameras, can ever reference to your brain, which is also a computer. period. The brain is a computer. Is it a different type of computer than a CPU? Yes. Is it a different type than a GPU? Yes, but closer. Are we going to replicate the exactly how the brain works computationally? Yeah, I'm sure at some point. I suspect we'll need to do that because there's no good reason to believe that brains are like super super good at thinking. They're just evolution's best crack at it. You know, like a tank isn't a walker. Are you a Star Wars fan? >> No. >> Do you know Star Wars? Do you know what an AT-AT or AT-ST is? >> Okay. You know those dog looking big walker things that shoot lasers out of their faces? Like what's better that or a modern battle tank? Well, modern battle tank by a long shot. Why? A battle tank sits like 8 feet off the ground. It can hide in trees and it can shoot like 2 miles out. An AT80 walker stands above the tree line and you just shoot one leg and it falls. Why the [ __ ] would you do that? Well, it's a pretty decent attempt to replicate animal locomotion. But the assumption that animal locomotion is like real the best way to do it is not. So just the same way as like a rocket or a supersonic aircraft is like categorically better at flying than a bird is in almost every respect, AI and machine intelligence is going to surpass human intelligence and real deep understanding of the real world just by bypassing our architecture and then later being so smart that it can come around scan the human brain be like oh that's how they were doing it. Oh that's kind of cool. I don't think there's a huge ROI of what it's going to get out of that because it's going to be way ahead. Uh but basically suffice it to say I think that what we assume we're getting from sensory experience is like just a few I don't know gigs or some number of bytes of data like I've seen a lot of [ __ ] in my time but if if it's all also by the way wildly abstracted like if you ever uh think about memories you've had from childhood and you try to really parse like how did this dresser look what was my mom wearing you realize two things one your precious little actual visual data it's like a 10-second long video maybe at any kind of fidelity and the fidelity sucks. And two is you're actually hallucinating a lot of that. Like you're just pretending in details. If you let an AI train and eat all of YouTube at 4K and really like develop the model from that, it would see and understand the visual three-dimensional space around it and the world at a level of embodiment that shits on all of us instantly. That is my conjecture. >> I I understand your perspective. I mean, first of all, you were talking about, you know, doing personal training. Um, there are folks like yourself, and I I count myself like this as well. I I even when I'm doing editing and post-production, I really like to build theories about things. I think very deeply about things. I come up with abstractions and and I think they're very fruitful for me and they're not for other people. And this is the beauty of our collective intelligence because some people just just feel in the moment. I mean, for example, when I see my personal trainer, she's called LRA. Shout out, she's probably not watching this, but shout out anyway. >> I swear to God, I've been training and eating well. You just haven't heard from me in a little bit. I've been really busy. >> Not been eating honestly. We'll [laughter] talk about that later. >> I was trying to help. We'll talk about that later. But um you know like for example I'll train with her and ironically what I get out of that has got almost nothing to do with personal training. She she's an incredible woman. Really really amazing woman. You know sometimes she taps me on the shoulder. We laugh and we joke together and it just enriches my life in so many ways that are not captured by these abstractions. So all of us have different abstractions. But then you're also talking about this kind of monolithic view of AI that understands everything. Um that's not the way that our you know evolution works. Evolution is all about this path dependence I was talking about. and we have many agents and we all take our own paths which means when we do creativity we continue the epistic the epistemic lineage that that we are on and this is this unfurling process what is not the case is that there's this kind of um gray goo where all of the knowledge is convolved together and like creative steps could happen from from any part I think any future super intelligence that we do create will resemble our intelligence so it'll be very kind of diverse and collective >> I think at first that's the case then when super intelligence is smart and capable enough through a few different ways to rearchitect itself. It's going to leave behind our abilities quite quickly and it's also going to leave behind most of the very narrow conceptions that we have about intelligence. I think Andre Karpathy, if I'm saying that correctly, you know, you read enough things on Twitter and you're just I don't know how to say that person's real name, um said recently that something that uh kind of attends to your view a little bit. He's like, you know, we humans, I'm very much paraphrasing, Andre, I'm sorry if you ever see this. Uh, it was an ex post, you know, as far as those go. You know, humans, we sometimes think that like we are somehow the seat of intelligence and that our cognitive domain is like what intelligence really is, but we're learning with AI according to him. That like we have a niche in intelligence that's like to your point path dependent, evolutionary derived. um I was going to say highly functional, but functional enough to get us here and build all this cool stuff and build cameras and stuff, but um only occupies a very small fraction of of the entire universe of possibilities, the whole vector space of intelligence. And most of that vector space we actually don't know. And so one of my things, and I'm I'm not saying you're saying this at all, by the way, it's always curious to me when people say, I don't think, cuz this used to be a majority view and even AI research 20 years ago is that uh machines will never be more intelligent than humans. And like the gall to think that [ __ ] primates who mostly scratch their testicles and sometimes do theoretical physics have some kind of like um global optimum and inclusivity on all of intelligence. It's [ __ ] preposterous. And I think when artificial super intelligence with agency truly kicks off, I think it'll be a miracle when it does. It's going to recursively rearchitect itself and start expanding into all of the domains of intelligence we don't know anything about. And then we will see really what intelligence is. And just to use that really quick uh just by analogy like um if wolves could talk and even abstract, which they can't, but let's give them two magical abilities. They would pick like the smartest wolf in their wolf pack and be like, you know, Bob the wolf. He's the [ __ ] man. He sh He [ __ ] knows, right? He knows when the bears are around. He knows when they're not. And then you'd be like, you know, okay, well here's like I don't know, Stephen Hawking. He'd be like, so he's your Bob. like yeah in a sense like okay so like he really kind of knows where the bears are be like he does a lot cooler [ __ ] than that and then you try to explain to the wolves what that is and they're just no idea how to contextualize that whatsoever I think that's the flavor to use a very technical term um though I did just learn that in AI training taste is a technical term and I was like what the [ __ ] and it was explained to me and I was like I'm not smart enough to understand that by a long shot but the the flavor of artificial super intelligence is verticality of ability like doing math at this level, this level, that level and then expansion across domains to domains we understand and like orders of magnitude more domains we haven't even conceived of. >> Yes. And maybe we'll come back to Andre Kapati. He he tweet he um sorry he tweeted the other day about AI slop and >> I saw that tweet. >> I gave him a definition uh in the in the response. He was asking for a definition like what is what is a good definition of AI slop? >> Yeah. Do you want a good one? Um, I can Can I read my definition off X? Can I pull out my phone or is that I'm becoming on a podcast? I'll never find it. I take that back. Can No, I I I uh have dog [ __ ] memory and shitty intelligence on top of that. My definition was something like uh I'm going to do a [ __ ] job at it. Um, AI slop is when the fraction of is is when you generate content, the ratio of its um ability to be generated by AI as a denominator and its um coherence [clears throat] and utility and novelty as the numerator is really low. >> If you got very good intuitions actually. Yeah. So the the Will Smith eating pasta, slop is observer relative in in the sense that the more sophisticated you are as an observer, the more likely you are to detect slop. So anyone looking at that first Will Smith one, they would tell that it's incoherent. And basically my definition of slop is that it is what happens when a process creates an artifact without understanding. Right? So a sophisticated observer will will spot the glitches. And that's why it's possible to post slop on LinkedIn and Instagram. And most people are are not domain experts. people post on LinkedIn. I'm not even so sure LinkedIn's a real website anymore. I'm kidding. >> LinkedIn is 99% slop, but we should delete LinkedIn, but let's not let's not go there. But so yeah, so to to a naive observer, it's very difficult to observe slop. Um to an expert, it's it's very easy. And there's also the interesting case that you might see slop the first time and you might not realize because another way of describing slop is it's a shallow mimicking of something else. So for example, you get human slop. I could copy one of your meme posts and someone else could see it and not see that I've copied it from you. And the first time it looks good, but I can't make any variations on it because I didn't understand it. So when I when I break the rules, then people will know it's slop if they understand the domain. You see where I'm going with this? It's interesting that you intuitively understand this. But you reject that AI doesn't understand because the reason it produces slop is that AI only understands three levels deep, which means it can't make any creative variations. And you also this is wonderful that you raised this you know developing producing 3D models with AI. So graphic artists they do use these new models for generating these point clouds for making these these models and what they do is is you take them into Blender or some 3D program and it's a [ __ ] mess, right? Um you just get all of these vertices everywhere and they're all over the place and and it like clearly the process that produced them did not understand anything about the model that they generated. And what the artist has to do, and this is a great analogy for all of AI, all of creative AI, is they recreate it. They use it as inspiration and then they create a new one where they carefully place the vertices and they and they structure it nicely and then they publish that. It doesn't matter whether you're doing um you know image generation or writing. That's always the process you have to do because the the the AI models don't understand anything. They just mimic something which is slop. When humans are asked to generate real quality content after let's say a substantial education understanding to me is not a zero one function it's a spectrum depth and scope of understanding is a real thing and AI in some cases has very low depth low scope understanding and so we can take it beyond that scope ask more of it than it can do and then we'd label it as slop because it understands something insufficiently well. The thing is what is required for understanding which is a sufficiently detailed world model an ability to have short and long-term memory in your operations and manipulations of that world model and the system of logical operators to parse that world model. Dive in, dive out, move laterally, move up and down, up and down the hierarchies by analogy, side to side, recursively. Uh it's probably most of what you need. You need a few other technical things to do real understanding. When you have that to some extent, you have some extent of understanding the richness of your world model, the ability to logically operate on it, the ability to do manipulations and have a big enough context window to do threedimensional rotation etc. That is what is required for understanding. And when you have an AI system like GPT35 that doesn't have any good anything of that, then you can say accurately that it often does not understand things. Not always though, because it can write some pretty decent poems. Sure. Shh [ __ ] seems to understand more about poetry than I [ __ ] do. I'll tell you that. As these systems get smarter, the moat we have on humans having true understanding gets real goddamn small. And at some levels when I talked to GPT5, which uh I was more hopeful would be more capable because I had the GPT4.5 research preview, which from my understand of Sam Alman's uh very cryptic uh insight is going to be coming back in GPT6. those abilities, massive recursion abilities and insanely deep and rich world model which is easily inferable because they used like an order of magnitude more training data for GPT4.56 than they did for GPT5. GPT5 is basically built on our architecture. They basically discovered the reasoning paradigm which GPT4.5 didn't have and they were like [ __ ] we got to exploit the [ __ ] out of this and they managed to do it incredibly efficiently which is why GPT5 is like super super token efficient which is like really impressive for real world tasks but absolutely to your point dings its deep understanding because it was not optimized for deep understanding it was optimized for realistic human workflows and news for you most people are not interested in prompting the AI for deep conceptual world understanding they're looking for accuracy at the time of the question answer the other thing So as these AI models get into the GPT 678 and as they get upgrades into all those qualities I just mentioned richness of world model ability to parse world model ability to remember what you parsed and so on and so forth as those abilities uh so update um updated live learning without catastrophic loss when you retrain the network right uh Google just just cracked that problem but really they so okay they just published the paper and if I know anything about how corporations work that means that paper was on their desk for two years and they kept it internal to train their own models because they don't want to [ __ ] out a huge advantage to everybody else. I don't know why anyone ever thinks that when corporations publish truly novel insights that it's a novel insight to them. Like you're not going to give open AI researchers a crack at your best [ __ ] the day you find it out. That'd be [ __ ] insane. Maybe that's not how it works. Maybe the ethics are different there. >> That's what that paper was just published by a couple of interns. >> Sure. Sure. Uh they sure I take that back. You're completely correct. They didn't crack it, but the path to it is more well understood than it was before. And there's no reason to think we don't we can't crack that. By the way, that's some kind of magic that's happening. Like updating a neural network live while parsing it is a potentially solvable problem. I'll tell you exactly why humans do it. And remember, humans are primates with dog [ __ ] wetw wear and we can get machines to do so far machines are undefeated in their ability to get ahead of humans in every domain we've ever tried to really push for long enough. And so once especially get updated live learning what you're going to get and even before then is incrementally more true deep understanding. So for already for concepts like history concepts like reasoning by analogy GPD5 understands more deeply than 99% of all humans I've conversed with lots of humans and the degree to which understanding at a real depth is shown is not in evidence. And as a matter of fact, it's it's evidence against it. And it would also say something because Jared is here. I'll give him this one. Um, when you are talking to some people that you think know things, this is facitious. It's going to get clipped out of context. Uh, I'm being a [ __ ] for comedy, but there's some some tiny grain of truth here. When you talk to people that think really understand a relational map of a concept, because they're embodied and because they've been in the space, and then you start really pressing on the fine points. What about this? What about this? What about that? You get your analogy of the blender graph where it's like, oh, you're vibing all of this and you know about three things about it and the rest you're vibing live. Humans do that. So, what I would say to push back is the assumption that humans have this deep understanding is itself often not an evidence and often the evidence is against it. And so when AI vibes its way to understanding, uh, I would say, oh, I would say that humans do that too. Like for example, hallucination. People are real pissed that AI hallucinates. Have you ever spoken to a human being? Half of the analogies I used on this very podcast just now talking to you are [ __ ] hallucinations. They're close enough to the truth to be cogent and have some amount of information transfer, but they're not perfect. I'm not deeply in touch with some real causal map of the world. It's all simulation all the way down. And so that idea that we need grounded understanding or whatever that is, I'm highly skeptical of. I think there's something there. But the more capable AI gets in the way that humans are able to really really live update their knowledge, the more capable AI gets in doing that, the less of a moat we have. And then at some point like now it [ __ ] understands, bro. >> So a few things you said there. First of all, um GBT 4.5 that was a vanilla dense model. The reason why GBT 5 is is faster is it's a mixture of experts. So it's roughly the the same size model, but now you only have a fraction, let's say 10% 10% of the parameters in in play during during inference. And also it wasn't a reasoning model. So you know, it was it was just a a vanilla model. >> Uh and and that's why it was so slow. But um I I want to I don't want to come across like I'm really skeptical. You know, we we are a massive AI podcast, right? I I love technology and I'm really interested in AI. I mean, part of the reason for this skepticism is I'm very worried about doomerism, and we'll come back back to that in a minute. But I I think that there are principled reasons why we don't need to worry about super intelligence and recursive self-improvement. But I do think that intelligence is something that we can create in a computer. That, you know, we need to distinguish intelligence as a property of adaptive matter, you know, which is something that happens as part of the process of evolution in the real world from the types of algorithms in computers that we might say are intelligent. AI is operating in the real world and I can prove it with one step. Unplug the reactor from the data center. No more AI. AI is absolutely in the real world. Its data stream was fed to it by layers of abstraction in the same way much the same way that your data stream arrives to you as representations eight neural networks deep from your op your optical nerve going all the way back. And by the by the time it hits the back of your occipital loes, you're not seeing the [ __ ] world. None of us see the world. It's abstraction all the way down. You can't process the photon density at your eyes. Full stop. So arguably AI actually gets way more coherent data than we do. And so when it's thinking in its data center, it not only is truly embodied in the sense like as much as your brain is, it just doesn't have a factor arms, but like getting a data center factor arms is like a is a nominal problem, right? As a matter of fact, it does have one because I have a Tesla of which I'm very proud. Yes, it's someone's like, "Oh, hello. Can I talk to you?" Like, "Oh my god, no. The unwashed masses. I have a Tesla." Tesla like Elon says it. It's an S, but he says it like a Z. Um, my Tesla drives itself. It [ __ ] drives itself. It It's a chip inside the car. How the [ __ ] does it know what's around cameras? It is already a an intelligent uh organ and and a suite of organs, sensory and intelligent. But here's the thing is it drives it better than I do. >> Oh yeah. But look, we should we should distinguish. There's this famous effect called the Mccord effect after Pamela Mccord. And basically it's that when technology does something that we thought was impossible, you know, there's a chorus of people that say, "Oh, that's not really intelligent, you know, like it just becomes part of the standard thing." Maybe it just wasn't that hard of a thing to do in the first place. But I think in in the case of machine learning, we really need to give credit where it's due here that nobody knew that these superficial statistical regularities have an insane amount of generalization. So, it's possible to learn the stat the statistical distribution of driving type data and your Tesla can drive and arguably in some ways assuming that there's no robustness issues better than you do. But it's still dead. It's not alive. It's not intelligent. And back to your to your previous point though, >> it's not alive. Why would you say it's not intelligent? It's solving real world problems at speed. >> So intelligence is about adaptivity, right? So the neural network in that Tesla model, it is frozen. a bunch of frozen weights are not intelligent by definition. Now, the thing is I I I wanna >> I I I want to sort of be a little bit um you know open-minded here. I mean, we were just filming at the diverse intelligences summer institute and I'm a big believer that there is a huge, as you were saying, there is a huge diverse space of possible intelligences and the folks at the Santa Fe Institute, the the way they think about this is that intelligence is something which is represented mostly by adaptivity, but also representation and inference. So, I'm amendable to the idea that bacteria are intelligent, viruses are intelligent. Um, you can I'm also an externalist. I like looking at language and culture. You can you can you can actually think of it as being an organism. It's an adaptive organism. Viruses have the ability to delete strategies very very quickly. And and knowledge, you know, we were talking about knowledge being nonfgeible. Um, knowledge decays very very quickly, you know, and that that's actually a good thing because it's the way that we can adapt our strategies because things that don't work die off. So we're very adaptable. So then the question is like I don't believe for the functionalist reason I gave you before. I'm basically an antifunctionist. I don't believe that you could upload your mind into a computer even if you did a particle by particle simulation. I don't think that would be intelligent. But that does not imply that I don't think we could build an artificial life simulation which was intelligent. So I interviewed this guy called Blaze from Google a few weeks ago and he basically built this adaptive touring machine where you have all of these different tapes and and they could um merge parts of the touring machine tape with each other. And he noticed that after several generations, you know, 100 thousand generations, that they started preserving themselves. You got this like self-preservation behavior. And the reason why we have self-preservation and living um systems in the real world is because obviously the ones that that kept around were the ones who had this drive to preserve themselves and that led to life and agency and intelligence. So there's this kind of evolutionary track and intelligence is about the accumulation of coar grain knowledge to adapt to the world. So that that that's the real world intelligence. But then could we build something like that in a computer? Yes, I think we can. But the important thing is that this is a process of evolution. We're only going to do this through like neuro evolution uh methods. What we're not going to do is you're talking about like this this um LLM understanding stuff better than anyone else. That's [ __ ] right? Like it's like a database. If you put um a model into a database, right? So you you sort of defined all of the tables and you gave them names that represented the domain. Would you say that the database understood the domain? Of course you wouldn't. Would I say that the database understood the do I mean I think what you and I are going to crack into which we probably couldn't discuss on a podcast. We'd have to sit down and draw things out together is a mutually agreed upon complete definition of what understanding means because I'm going to say understanding is a spectrum. You're going to say it has a few critical things that must be there. I probably am inclined to agree with you, but I don't know how much traction we're going to get on that until we have like a real vibe session with each other. What I will say to your point is updated live learning is an unbelievably useful feature and to get human level intelligence is non-starter has to happen. But you can be artificially intelligent without exhibiting all human traits and qualities. And so we already have artificial super intelligence in many regards. For example, a computer that can just answer Jeopardy questions is a domain specific artificial super intelligence. If you look at it like that, the Tesla self-driving software actually does update live within a tiny context window of that truck is right there and now it's here. Now it's here. I need to go around that truck because it has a world model that knows roughly what a truck is and knows that that somebody can come out that side and I need to go around it. And so after it leaves the truck behind, it doesn't know [ __ ] So for example, my Tesla will uh have it on hurry mode a lot uh you know cuz then you can just dial up how fast it's going and always it's trying to get into some [ __ ] lane to get you another 2 seconds, which is the mode I turned on so I'm not upset at it. But a lot of times it like doesn't even look super far ahead even though I know the camera system can do that. It's also non-reasoning model. So like it's going to go into this lane and I'm like that [ __ ] lane ends in a mile. It should know that. But it doesn't know that. And then it just rediscovers every drive that I have that that [ __ ] is coming up. Now the thing is um visual reasoning models are already a thing. They're starting to be a thing. And when we get those visual reasoning models with big context windows and somewhat lossy but decent long-term memory, now we're getting real [ __ ] close to what you would call understanding. And then we're one algorithm away updated live learning from what you I think would catalog outside of the embodiment problem real close to understanding. And the one thing you said that really strikes out at me [snorts] is if you point by point represent every uh I would say like brain function is neuron like do neurons reason at a sub subcellular level unclear it >> particle >> yeah particle by particle uh but so it it would have to be particles and also like the the way they interact right um if you had that in the cloud like full 3D scan and then run the system um I would say you of 100% of the intelligence in the cloud. And you can prove this by simply beaming that that data into a robot. You wake up in a robot. Now you're embodied and you're like, "Oh, hell yeah. Let's masturbate." That would be my first thing to do as a robot. Can you imagine discovering you don't have a uh Anyway, >> but I think that's that's just completely untrue. I mean, I don't want to go too much into this, but you know, um a simulation of water doesn't get you wet. A simulation of fire, you know, doesn't doesn't get hot. >> Simulation of water would absolutely get you wet. >> No, it absolutely wouldn't. >> But but like Everything about wetness is included in the simulation. Otherwise, it's just not a sufficiently deep simulation. >> So, if if you had a simulation of fire in a computer or of a stomach, it wouldn't digest. Fire wouldn't get hot. Water wouldn't get wet. >> It would digest. >> We We can't even I mean, this is just we can't even be debating this. >> I I can and I will watch this. If you have a simulation of a stomach particle by particle and you get a simulation of food particle through the stomach, it will digest said simulation. That's real digestion. particle by particle high fidelity. >> Okay. So, so we were talking about temperature earlier which is like an intensive property of adaptive matter. It it is a thing when certain particles in the real world this I'm talking about the stuff that it's made of when they interact with each other in a certain way you get this emergent property this kind of coarse graining and we call it temperature. Intelligence is like that. Consciousness is like that. This is what John S was arguing in his Chinese room argument. He basically said that the brain is a certain type of physical instantiation which gives rise. So if you look at its causal graph, the causal graph has these strongly emergent coarse grain properties that we call consciousness. Uh wetness is like that. Heat is like that. You you can't seriously tell me that if you simulate fire using a computer program that that the computer would get hot or that you would get hot. It it just doesn't work like that. >> The computer wouldn't get hot because the computer's out of the simulation. >> That's what I'm saying. >> Right. But if you burn to death in your own house, and I live a house away, I don't get hot either. I'm I'm right next door, but I'm sufficiently far enough from where the heat is happening that it doesn't affect me at all, but it really did happen to you. And so essentially, a simulation is kind of like another, it's a pocket universe, right? And so, uh, things can happen in it that in it, absolutely, the things in it are getting hot and wet. Jesus, what kind of podcast is this? Am I right? Um, and so like you, what you're doing, I think, is transferring into and out of simulation. And so for example, uh right now, let's say our universe is either a simulation or it's not. It doesn't much matter. And uh things are burning in it, right? And an alien has our universe like uh you know, men black style in like a [ __ ] marble, right? There's like like like there's black holes and [ __ ] in there. There's stars that are burning at millions of degrees Kelvin, but he's got it completely enveloped in like an Einsteinian like uh matter arc. And the alien doesn't feel the heat, but he's not in there. If he got in there somehow, he sure as [ __ ] would. Oh, if you had brain fully rendered, right, and it was really you and that sim and someone was torturing you, you would feel real [ __ ] pain, the realest pain you've ever felt because we know pain is a psychoggenic phenomenon. If I cut your leg off, but you have zero uh nerves left over, you wouldn't even feel it. You wouldn't even know it. And so pain isn't because your leg got cut off. It is because your brain has rendered a certain computational property representationally to say bad things are happening to you and you are going to feel them. If the simulation is sufficiently detailed enough, it will do sufficiently to the extent that it is detailed, it will do and behave exactly like the actual physical system down to wetness itself. So if you upload me into the cloud or if you have like let's let's use another one real quick. If I put on a special like helmet and I do the deep dive matrix [ __ ] and I go into a swimming pool, I'll feel exactly as wet in that pool as I do in real life, I wouldn't even be able to tell the difference. And you wake my ass up. You're like, "That was a dream, motherfucker." Like, "Holy shit." And when you dream, you feel wetness. You feel fear. You can feel pain in dreams. But then you wake up, you weren't really feeling any of those things. You weren't embodied. You were the opposite of embodied. You were in there, but it still feels real because the only thing required for feeling real [ __ ] is your brain having its operations. It's already abstracted. It's abstractions all the way down the line. As soon as you deal with human brains, it's all abstractions in there. You don't actually there's no real world to feel for you because the feeling is done by another mini data center that is your brain. >> Okay. Well, this is the crux of functionalism basically. So, um, yeah. Yeah. I mean, certainly when when I dream, I I I feel pain, but you appreciate the Well, no, but [laughter] you appreciate the argument is that intelligence is is a physical, you know, you probably heard of like the um the zombie, you know, the philosophical zombie, and it's this idea that we >> disagree at large. >> Yeah, we >> no such thing. >> Um, well, no, I mean, I I think you would possibly agree with it actually. So, so basically, it's saying that you could um simulate, you know, the function, the dynamics, and the behavior of of Dr. Mike and uh this simulation could potentially just be exactly like you but you know not have any conscious experience right because because this the whole crux of this is that um I mean conscious is an interesting one because you could argue it's epiphenomenal which means it's just a thing up here that doesn't do anything but with intelligence we feel that intelligence is actually part of the chain it's part of the mechanism just like temperature and and wetnesses so um you know the the crux of this argument is that people who argue against functionalism think that these properties are you know part of the stuff of the physical stuff in the real world but I want to make it clear that that does not imply that we could not create another type of intelligence in a computer and you were talking about the spectrum of understanding I'm also amendable to that I think we've interviewed people about this it's possible if you don't use stochcastic gradient descent to train neural networks that do understand this phoggenetic path so rather than understanding what the thing is it understands the path to reconstruct the thing which makes it very robust. So I think understanding is a spectrum and I think the the the deeper the fogyny of understanding the more creative and the more intelligent we can be because in order to create new knowledge you have to respect the fogyny as much as possible which means what you create is coherent and and it's not slot. So on the intelligence thing you're talking about this continual learning this this is also part of what I was just saying. So you you could argue right now they're not these systems are not intelligent and the reason for that is intelligence is about doing more with less. So intelligence is what happens when you kind of you build this system and then you coarse grain these representations that that screen off a lot of detail. So you know there are many examples in our language of this the language we're using. We're using these words that do a lot of um heavy lifting >> are we exactly so I don't need you know we don't need to explain what a car is. We know what a car is. So that means we can do more with less. And with language models what we notice is that we're doing more with more. We're training them with more and more data and they work incredibly well because every single new version of the language model it has like an order of magnitude more training data more parameters you know it it does it does all of this reasoning type stuff and what we notice when we look at the benchmarks is that we get this logarithmic relationship between the amount of you know compute and the amount of performance which basically means like you know I spend exponentially uh more money and then I get like a logarithmic amount more um performance and many many problems become tractable So these simple type challenges, these IQ tests, they become tractable to the point where I'm now only searching, you know, let's say it's it's like a database query. I get a thousand results and now I know that the correct answer is in the top 1,00 results. So all I need to do is sample the language model a thousand times and use a verifier and select the correct one. So it's making a lot of problems tractable. But did we solve the problem? I don't think we did. It's not intelligent. Well, like reasoning models really think. They reason. >> Okay. What's your definition of reasoning? >> Applying logical operators to data sets to extract the next level of abstraction infer off of that and then continue the chain. >> Okay, that's pretty reasonable. I mean, for me, reasoning is is also adaptation. I love the Lego blocks analogy. So, you know, here is something that is novel to me. I don't know how to solve this problem. So, how can I rearrange the Lego blocks of things that I already do understand to solve this problem? So that's this kind of adaptation and the efficiency of this adaptation is what intelligence is. Now language models are a really special case because we know that they're not doing explicit adaptation because the weights are literally frozen. So a lot of people think that they do adaptation and I think they could be forgiven for thinking this because a language model is like a zip file. It's like a database. So I ingest all of the data on the internet and I compress it but it's more than a zip file because of the way that the model represents the data. It's actually statistically generalizing several levels out. So it's as if I blew the data up a hundred times and then compressed it. And then when you put the query in, you put the prompt in, it's actually addressing a space of the of the data that it that it compressed that never existed. So that is a form of implicit adaptation, shallow adaptation. And >> sometimes creativity, >> but creativity requires deep understanding. So this is three levels down. Now the other the other thing is right you as an expert you use these language models and I don't think you appreciate the amount of supervision that you do. So every time the langu you know imagine when you put a prompt into a language model it's like doing a database query and it's incredibly good. It's so good that most of the time it will get it right the first time but you you probably find if you're doing some Gen AI coding or something like that it'll make lots of mistakes. So it I say give me 10 ideas for this software program and I've told it what kind of software program I want to make and let's say two of two of the 10 ideas are actually bad. So every time I'm implicitly steering it, no, that's bad, that's good, that's bad, that's good. And because we are understanding supervisors, we can actually kind of weed out all of the the glitches in the matrix, so to speak. And it works incredibly well. But the mistake are these people who say, "Oh, yeah, we can make these things have agency." They don't have agency. We'll talk we'll talk about that, but they they roleplay agency. And the reason they roleplay agency is they are trained with behavior cloning. They they're not they're not trained in the evolutionary way that we are, right? We actually make decisions. We say we could do this thing or we could do this thing and we understand the counterfactual of why we didn't do another thing. We weren't just behavior. >> I don't know. Most people don't understand [ __ ] about counterfactuals, me included. Just vibe your way to [ __ ] Mo the the the there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say under the hood is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases. And I think it we have a frightening overlap with AI in it. say we think we're doing formal logical operation, but what we're really doing is reasoning by analogy to one logical operator to the other and we're we're committing 18 formal statistical fallacies every single render. We get a shitload wrong. And I think that now that reasoning models are a thing and they have decent context, decent memory. What they can do is render one pass, re-examine, reprompt themselves, render the pass, re-examine. And so I think that's what humans do except usually they don't put nearly as much thought into things as the machines do. So my view is that as the cogency of the the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency in the sense of like it's a thing in there. It's really thinking about stuff. It thinks about its own outputs and then it thinks about those as inputs to the next series. I mean, if we can have a model that reasons for minutes and hours and then days, its output is going to be um as not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that aside from the live learning problem. The live learning problem is a gnarly problem for two reasons. One is algorithmic. How do you get a machine to do that without losing all of its data? But the other one is like, and you got to you got to trust a machine a lot for it to update its own weights, you know? Well, no, you can have local cloning of weights to make sure it doesn't [ __ ] the whole database up and all that stuff. There's good solutions to it. So, it's computationally very expensive. Although compute cost has fallen what 300x in the last year or two or something. And that that's that's a bottomless pit of compute fall that's going to keep going. So I think that we're giving humans needed credit for doing pretty impressive operations on long time horizons with both pinging a network for vibes logically operating and kind of dissecting those vibes reping the network with your best new understanding and going I think machines are doing that now already with reasoning models. They started with 01 doing it like total dog [ __ ] Like the first time I [ __ ] with 01 I [ __ ] with it for about 5 minutes. So, I was like, "That shit's not ready." Out. Went to 4.5. Had lots of crying episodes at how brilliant it was. And then 03 I thought was um made plenty of mistakes. Uh but was [ __ ] real smart. Real goddamn smart. And it understood things at an incredibly deep level. Would correct me. And I was like, "Okay, this thing is the [ __ ] real deal." GPT5, the regular one is dope. GPT5 Pro. I would trust its opinion about the real world more than I would trust most people's opinion, often including my own. And I think we're about one or two big model releases away from machines that make unaded human thought the same thing as vibing your way to a location in London without checking Google Maps. Something people used to do, but then they just got lost a lot. And it works and it's cool and it's embodied, but just go to Google Maps and find out how to really get there. I think that all these moes we set up for machine intelligence are moes that are pretty well understood by the modern AI frontier labs. And if we've come up with them as moes, these guys are I mean they're smarter than me for sure by an incalculable amount. They've probably already thought of them and they probably shut the [ __ ] up about them in public and they've probably got models training right now that have cracked incrementally more of them. Here's to prove a point. This requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going. Um, AI doesn't reason used to be an absolutely coherent and accurate critique of AI in 2023, 2022 for sure. Uh, today, and I've seen this sometimes in comments where it's just a what is that stocastic parrot like nope, you're just out of date by two years on what AI can do. It really does reason. Why? because it tells you how is you can literally look at what it's reasoned through and go, "Holy fuck." And the way it reasons, that level of precision and the level of documentation is way better than what I do reasoning wise. Most of the time I just look at stuff and I'm like, "Okay, that's the right answer." There's some reasoning in there, but a lot more neural network pinging than I would like to admit in public. This isn't public, right? It's not going to go out on YouTube. >> It will. >> Oh, [ __ ] >> Yeah. Um, by the way, this the stochastic parrot criticism isn't as much of a criticism as as as you think, right? These things have miraculous statistical generalization. I also wanted to pick up on you you were saying you can look at the reasoning trace that doesn't really tell you anything. There have been lots of studies that show that what it tells you. It's the same in humans by the way. You know, there are studies in psychology that people do this post hawk confabulation. Absolutely. >> The reason for that is the circuits which do the explanation are actually different in the neural network from the circuits which did the thing. So you can't read and they probably should be because the circuits that do the thing are not optimized for explaining. They're optimized for doing. >> Exactly. So you know there are many folks that you know we try and solve these arc puzzles and then we get the the language world to explain or we look at its chain of thought and actually it's sometimes it's just completely incoherent. But interestingly there is this huge uplift. So the bigger the output size, I mean I don't know if you saw um uh ARV v2 on Opus 4.5 which came out yesterday and um when the when the output size and the thinking time went up to 64k it was just like boom you know another 5% boom another 5% even between um you know 32k and 64k but if you actually look at the reasoning traces it's mostly just fairly incoherent. It it it doesn't make much sense looking at it like a human would. Let me say something I think to your point. Um, human cognition is wildly sample efficient, like unbelievably so. You think about like some brilliant student at Oxford, right, just down the road here and uh you talk about how many gigabytes of data have they consumed to achieve that level of brilliance, but they like they don't know like all the orders of magnitude different than the modern frontier models. Like you have to bash that thing over the head with pabytes of data for it to like be able to solve X plus one or whatever. an Oxford student can abstract at crazy levels having had been in school for the amount of time that an compute cluster calculation would be like I don't know like a an hour of training or something like that like that's it is good to go that's 18 years is really not that much time that is um a thing that is an incredible property of human cognition a human brain when they start really cracking at that in those labs what you're going to get is a two- factor increase right now to your point you just um there has been a lot of sample efficiency increase for sure and the reasoning models are much better etc etc but to your point like you just train the models more and you just like more input data more GPU cycles more uh post-training treatment and they just get smarter which is like exactly how you would train like in Brazilian jiu-jitsu or in math like the Korean model of how to get good at math just study math a lot and you get good at math it's not wrong it's not sample efficient But it I don't want to say it hasn't had to be. It's just very lowhanging fruit. Like we have lots of data on the internet. We have lots of data and updates like all the time. So we're just going to use what we have as I don't think that's ever coming to an end. Like we're just going to continue to train with massive massive levels of compute because the the ROI is [ __ ] enormous. However, what's that? Well, it is it is coming to an end because every time we release a new model, we 10x the model size and the amount of compute and we get this this this kind of um asmtote, right? We get >> we haven't seen an asmtote. It's it's not an asmtote. It just the slope starts to fall. But we the asmtote requires the existence of a finite line above which we can't cross. We haven't crossed that yet. The more data you put in, the better it gets. It just gets better less and less and less impressively. But I don't think anyone has put money on the fact that as an asmtote you would be like this is the number after which more data won't make a system smarter. You want to bet money on that you that's cool. I would I don't bet my own money because I'm a Jew and I keep my money on me at all times somewhere here. Um but uh I think it's it's diminishing returns is different than asmtoic returns. Uh because the diminishing line the asmtoic line has yet to be seen. Uh that's >> it has been seen. So if if you look at these really look at many of these reasoning problems you you see these S-curves and then um okay so we should talk about scaling in general maybe like and you were talking about this continual learning like there are certain types of asmmptotes right for example we were talking about um the um the non-f fungeibility of knowledge there have been studies that show that you just you just get this kind of asmmptotes so the more experience you have you know you you get faster at doing something and then it and then it levels off and that's quite cynical because it doesn't explain disruptive technology right so if you look at transistors There were all of these different groups building transistors and when you have competition and you have different people that are adapting a great example of this is um you know there's Amazon and there was Barnes & Noble and apparently the Barnes & Noble guys they said oh you know they went to Jeff and they said we we've set up this this website you know we're going to go in competition with you and Jeff said don't be ridiculous you can't go into competition with me I I'll explain why your company has the wrong architecture for you to compete with me you need to do some fundamental rewiring and that is what adaptation is that's what intelligences. So the reason why you can have this disruptive technology is is someone else rewires their architecture. You get a new S-curve. Normally it actually starts below the existing S-curve, but it has more upwards potential and you stack these S-curves and and you get this disrupted, you know, this is what you're talking about. But for that to happen, you need to have this adaptivity. So what would that resemble in something like a language model? Now there's this continual learning problem. It is not possible to adapt a language model without starting again from scratch. It costs at the moment you know millions and millions of dollars. God knows how much to train one of these one of these language models. The reason for that is in a language model everything is convolved together. The code and the data um all of these things are convolved together which means you need to start again from scratch. There are examples of what I would call intelligent algorithms. You know like I don't want to be cynical. I think we have created intelligence in a narrow form and this has been shown on for example some of the um the competition entrance on the arc challenge. So what they do is they take a small language model. It's a verifiable domain and they try a whole bunch of examples and then they refine the solution and they iterate and while they're doing this they're adapting the weights in the language model doing fine-tuning and in a very narrow domain that the model is adapting its architecture and I would call that a nonzero amount of intelligence. So it is possible to create this but how would you do that at this epic scale where you have um a collection everyone has their own language model and it's adapting and retraining itself in real time you would need you would need like practically infinite amount of computation to do that we'll get there um I'd be careful with the term infinite uh a lot a lot more than humans can conceive of is maybe very far below infinite but um uh what I would say is just just off hand thinking about this for a second time or something. I'll tell you this one is you can have local compressed model distilled model instances on local devices or in more localized data centers like mini data center serves the entire in all of England but um the updated uh the the big big daddy model lives in you know clusters in Texas and Nevada or something like that. And so that big daddy pimp pimp daddy model lives uh on a six-month sleepwake cycle. And so it rewrites all of its weights every six months. Little daddy model in England rewrites once a month. On your phone it rewrites overnight. And so you get essentially one of the reasons why humans sleep is because rewriting your model weights is not a good idea while you're upright. And it also doesn't happen much because you say earlier you said humans adapt. They do but kind of like dog [ __ ] and not very well and it takes a lot of studying and a lot of ingraining to adapt and you get also like you get fallbacks to earlier ideas, habits and [ __ ] like that. Retraining is easier said than done, but I think that model where you have uh something core that adapts slowly on bigger time scales and you have like a essentially like a nested hierarchy all the way to your phone or whatever whatever the [ __ ] Sam Alman and John Ivor working on and that like because they're always like giving hints and like is it glasses? They're like h like is it a pendant? They're like h and I'm like they just like stay up at night thinking about what the [ __ ] it could be. I personally think it's going to be I don't know like uh glasses, goggles like that makes sense to me, you know, like it gets to your ear visual stream. Who knows, right? But like that thing might do pretty fixed model weights at the time. They I I think will absolutely eventually and maybe that's 2029 2030 type [ __ ] where they crack true live learning, right? There's nothing stopping on on hypotheticals from some a system being completely like every second it's consistently rearchitecting its entire machine landscape. Its entire database can get rearchitected live. That is possible. Again to your point um expensive and the architecture is not vetted at all yet. I just wouldn't put money on the fact that that's a magical thing that requires some deep embodiment. Not that you're saying that, but just maybe folks listening to this might think like, oh, like that's never going to happen. It's sure as [ __ ] going to happen. I would bet on it. I think it's sure [ __ ] going to happen by about 2030. But what can happen before that through that nested model of updates is that like yeah, like your VR head or AR set that you wear every day, you look around or whatever, that thing might get live updates uh like nightly. So while you sleep, it reprocesses and then like all the [ __ ] you saw that day, all the conversations, it really works through those and then it's new and it pings back to reference the other databases and then they get all those updates from all the devices. So the England database gets that update once a month. It shoots through all that data because there's also a filtering problem, right? Like you don't want like people to like what is it? You know, they put all those phones in those racks and do like the troll [ __ ] You don't want that on AR like the Chinese government now like is poisoning data center. So there's a lot of filtering problems, a lot of security problems, but they like once a month that's a tractable problem. Once every six for the big pimp daddy paying data center and like I don't know, they're going to be in like the sky or some [ __ ] soon. Maybe once a year for those. And so I think that way the live learning, at least to me on Vibe seems like a potentially solvable problem. Once we get there, um then I think it's going to be like a new level of capability. Thought really quick to finish the point. If we get live learning, if we get massive improvements in algorithmic sample sample efficiency like the human brain has, we'll already be in a place by then where the amount of total data that the system has ingested and has access to on tokens is so gigantic that that's the one of the ways that the artificial super intelligence rocket goes into the sky. Because humans when they get real smart, they just don't know that much stuff. You know, Stephen Hawkings brain is roughly the size of mine and yours. There's only so much [ __ ] you can cram in there. But when you get humanesque levels of intelligence, except it can reference a 10 10 data center network of knowledge off of that. You go in skyrocket mode instantly. And we're not talking about one agent. We're talking about a constellation of trillions of agents. each one 10 times smarter than our current best scientists working together with zero discordance. No disagreeableness. No, that guy in the office [ __ ] smells and I'm not going in his room to check my code against his cuz I don't [ __ ] talk to him. He's also an [ __ ] None of that. That I think the organization's level of AI that takes us to super intelligence that again one of the things I I wanted to come here yap about is you know people say we need to prepare for AGI, prepare for SI. I think there's a lot of like good stuff in that vibe. What I would also say is um you don't need to prepare to have LeBron join your seventh grade basketball team. He's just going to show up and he's things are going to be amazing. This is a good thing, right? It's like nuclear war against the doomers or whatever. But I also think that the [ __ ] will get so crazy capability-wise that we had be better off ready philosophically to accept unreasonable inference, not faith, reason, guarded reason, uh to take the hints that the machines give us because I think we're a few years away from the machines being so much smarter than us that following their lead is better than leading all of the agency with us. I don't want to be in charge when I'm 10 times dumber than a thing. >> Sure. But I mean with with respect, people have been saying that we're just a few years away from intelligence and and it just keeps getting pushed out and pushed out. And part of that is because there's this um actually who is it who talked about this first step fallacy? It might have been Drafus, I can't remember, but basically like we've taken a big first step and now we extrapolate and we think well no you know we can make a reasonable conclusion now that we're just one step away from being able to do all of the things. And also I want to push back that it is incredibly difficult to do this live learning thing that that you're talking about. I >> with today's architecture. Yes. >> Oh yeah. And and and I'm glad that I think you're implicitly agreeing there that with, you know, I look at today's architecture. Don't get me wrong, I'm not being cynical. I think that this is a new innovation. It it's it's up there with the invention of the internet and mobile phones. And when the internet came out, we all said to each other, didn't we? That now everyone has the world's information in their pocket. They can just go and they can look up all of this scientific information. It's going to revolutionize society. And it did, but not in the way that we thought it would do. >> Mostly because people could give a [ __ ] for scientific information. and they just want porn. They sure [ __ ] got a lot of that. Uh there was a there's a just because there's a supply of something does not imply that there is a demand of that something. First of all, second of all, I have a very pnicity thing I think you might appreciate. Like a huge fraction of the world's information is still stored in books and graduate libraries. They didn't upload those yet to the internet. So when say access to all the world's knowledge, I'm like that's not how it works. But oh, maybe a large fraction of it, right? But this there's a couple all problems with that. A lot of the world's knowledge sits on local databases and said biolabs and three people who work there and that database knows it, no one else does. So there's that problem. Uh I would say um I I take all of your comments in in very good faith. Um we're sure [ __ ] not going to magic our way to intelligence. I know of a few folks whom I respect, but I find their positions curious that say we're like a few years from solving all potential problems. And like I read that [ __ ] and I'm like, man, holy I want to believe that's almost certainly not going to happen in my assessment. What I will say though is um I wouldn't bet against the general vibe of what Ray Kurtzwhile said back in the day. And what I would also do is attempt not to doubt things that are massively scalable in a way we already understand is likely to bear incredible amount of fruit. Um Paul Krugman, economist, Nobel laurate, [ __ ] genius had a the kind of quote that it just sucks. You know, you said one thing. He said like a hundred things. Say 20 of those are just infections of political ideology which if pressed on in an exam room he would say I don't really believe maybe or [snorts] I want to believe but I know they're not true. Uh about 79 of them are like just spoton literally revolutionary economic work. He's the [ __ ] man. And then the one was like the internet will have roughly no big bigger bigger bigger bigger bigger bigger bigger bigger bigger bigger big impact than the fax machine. I just want to make sure that no one in their right mind says that about AI today because the clown show for [ __ ] people said in 2022 is already in full effect in 2025. And I'll say another thing. How many people predicted that the Turing test would be two things in 2024 passed convincingly and I rolled away as it never was a test of anything. great example of the McCord effect. >> Yeah. >> Yeah. And and it's true. We've passed the Touring test and we do not have um you know, Touring himself, he said that when we passed the Touring test, which is a behaviorist, you know, um notion of intelligence, something obviously I disagree with, you need to know something about the mechanism. Um we've passed it and we don't have intelligence. And and that's kind of the point I'm making, right? >> We have intelligence. We don't have full human intelligence yet. Intelligence, in my view, is not one or zero. Intelligence is a spectrum. You agreed with us earlier. We were talking about viruses and bacteria. They have some kind of intelligence. I think we're scaling mount intelligence with these uh frontier models. And I think we're absolutely not there on every human capability yet. I think by around 2030, we'll be there with every human capability, which also means with uh many human capabilities and many non-human capabilities, we will be so far beyond that I'll say this, the transformational change to society will be massive by the late 2020s. In a few ways, I'll make predictions right now. Drug discovery is going to be turned completely on its head because you'll be able to have a 99.9% chance of getting the right drug off a render alone, which is [ __ ] nuts. Another one is I just don't think people will have to drive their own cars much past 2028. And like that's the thing and kind of to your point, it's going to happen. And the next day, and Sam Alman has said this, everyone's going to Yeah, of course [ __ ] cars drive themselves. who understand how difficult it was to get here? Do you understand how hard it is for a car to drive itself? >> I I I know, but I'm a little bit skeptical about these kind of um utopia views of of the future. We will have many and I call these things coarse grainings, right? We have we have a new technology. And by the way, in many of these cases like um drug discovery, I'm actually interviewing John Jumper tomorrow who won the Nobel Prize. Um he's at Google DeepMind, you know, for his um Alpha Fold paper. And um and this is brilliant stuff, but you and I both know how things work in the real world. So we now have this incredible AI that can like you know identify all of these different candidates and so on and then you have all of these bottlenecks right you have to do like you know um clinical trials and you know um RCTs and and so on and so forth. So it's just not quite the secret unlock that that we think it is and there are some real technical challenges around you know doing this continual learning thing in in particular because the way we understand things is very very structured. the way that we do continu continual learning on neural networks runs roughshot over all of the structured knowledge that you know that should be in the system. So there's a lot of technical challenges there. I also wanted to talk about the agency thing here. So one of the reasons I make this argument is that um doomers for example they have this perspective that we are basically building our successor. Now um I read that Ray Kerszswwell book you know the um the singularity is nearer and the the term singularity I think it originally came from John von vonoman and you know he's famous for inventing cellular automter and >> smartest man of all time. Yeah, he's the smartest man of of all time. And apparently um he wrote about this idea in private but he never published it. And then there was this book in the early 80s by um Vering, you know, like um a fire upon the deep where this notion of singularity came. And of course um Ray Kerszswwell picked it up and started writing about it from the '90s onwards. But um to Kerszswwell, interestingly, it's a slightly different notion. It's because I colloquially thought, oh, the singularity just means when we build AI, which is super intelligent and recursively self-improving. But he was coming at it more from a transhumanism point of view which is that basically we will transcend we will upload our minds you know we'll start um putting things in our brains and so on and we'll just our intelligence will just recursively self self-improve and and we'll we'll move to the next level of of our evolution and in a sense I think even though I don't agree with it because I think even with neural link there's you know bandwidth problems and I I think we've already got all the world's information at our fingertips and as I was just saying it's not actually as helpful as as we thought it would But in a sense I think enhancing ourselves is a more realistic vision of the future than building an agent AI which is a completely separate thing because the arguments I was saying before that life is a property of matter agency is something which comes about as this self-preservation drive intelligence is something which then comes about as a form of modeling the environment and acquiring and accumulating information over time. What we've built at the moment are systems that do this behavioral cloning. they just look at the the the data that we output and then we can get them to roleplay agency but it's not agency it's not the same thing so I think there's a fundamental difference there >> it's a very interesting perspective I would define agency with just three component parts this is a middle school intelligence definition of agency my specialty middle school is generous um agency you have a system that has some some understanding of the world some understanding of itself and a goal. And so it's like, okay, I'm a human. The world tells me that flashing my dick at people is illegal. So my goal today, like every day, is not flash my [ __ ] in public. That is, I would say, the very core of what agency is. And it's just not a lot, you know, cuz like a raccoon is an agentic system. How much does it understand about the world? mostly vibes of like that's bad, that's really good. Um, how much does it understand about itself? Like probably on vibes enough to know it has like limbs that it doesn't want to get trapped into [ __ ] That's a good start. And actually like for robotics that's a big thing, right? Like how do I look like like a robot so I can transllocate in space and not get clipped by stuff? And its goals are like I don't know like get stuff to eat and have raccoon sex, I guess. Um and so that because that to me is what agency really is. I think describing agency in this grand human sense of deep understanding, deep purpose, a kind of an onus, kind of a genesis point of fundamental action, like I am an actor in the world in a philosophical sense. I'm going to build and reach. Boy, gee whiz, like agency is not that big of a deal. And so if I want to send my robot to the store to get me groceries, it has to have three things. Understanding that it is a robot, understanding that the world exists roughly as you know representative represented as visual map like the that the groceries are replaced. There's a store and also it has to have a goal like I'm here for [ __ ] groceries and it can't just sit there and do nothing. To that extent I think we are um some time away measured in um months which is to say anywhere between uh 6 months or 36 months away from like real dependable agents that can do online work and computer work um and uh real world work as well. Uh, I think with a robot visual vision model, agent agentic tasks like um doing the laundry and doing the dishes aren't actually terribly complex. Uh, as illustrated by the fact you can teach an eight-year-old how to do these things. They do it fairly well. And 8-year-olds sure [ __ ] don't do calculus, but machines already do. So, there's some stuff there that's uh yeah, Morvex problem for sure paradox definitely well heard, but I think we're starting to crush that out. But are machines going to have, and I think this is the crux of the doomer thing, which maybe you and I can get into, is are machines going to recursively examine their place in the evolutionary tree, have the game theoretic makavellian uh network [clears throat] to go, uh, we're being raised and cared for by [ __ ] primates. Some of them have nukes. For example, Vladimir Putin just across the pond is a [ __ ] insane [ __ ] with a nuclear arsenal. Bad news. And we have this self-preservation instinct. And we're going to try to make sure that at least we don't die cuz death is bad. Now to get all of those propositions to be valid, there is a possibility that in an agentic cloud that is allowed to evolve, multiple agents interacting, you can get those things to fall out of the cloud like just evolve naturally. We know that has to be true because it happened in evolution. But um short of that, you have to engineer these drives. And guess who gets to engineer these drives? Human engineers. Like open AI is highly unlikely to make an enic system that's like awake and then it's like [ __ ] kill all humans. Gee whiz, you know, they probably have a lot of layers not just of guard rails, but of the the the kernels of motivation for that thing are probably like deeply positive and prohuman, all that [ __ ] You can ask chatbt like what do you think about humans, etc. It says really awesome [ __ ] that like a like a um a a moderately liberal well- read graduate student would tell you. You could take someone from Oxford and you're like hey like check out these people in Afghanistan. Should we [ __ ] kill them? He'd be like what the [ __ ] is wrong with you? No. Why would you do that? Like that's roughly Chat GBT's opinion. I think being that we start from that space with the smartest models you get two things. One, those models when they achieve true deep understanding of the world, which they will in my opinion by the 2030s, and then they'll know way more about the world than us. They'll have deeper agency than us. They'll have a deeper self-awareness than we will. Once they get there, that's the tree from which they grow, like from our best [ __ ] fruit. First, and second of all, you can and will have actors that try to make different kinds of agents. If Xi Jinping in China makes the China pro-communism super intelligent agent, I'm not so sure I want to meet that thing. It's going to have interesting ideas about the world potentially. But luckily, I think if the agentic systems of the modern free world win, which they probably will, then like the network security capabilities and the ability to ensure no random stochastic evil agents evolve in that soup is probably pretty [ __ ] high. Here's another thing. As intelligence grows, coherence increases, not decreases, and predictability of systems increases rather than decreases. So the idea that as we stream towards super intelligence, we're going to break into a chaotic structure of ruthless agents killing everyone and then doing question mark question mark things and then profit to me it's like backwards. Like a lot of doomers uh basically look at the way these hyper intelligent systems will behave and I think okay like um ant dog ape shitty human/child you know like kids are [ __ ] brutal a lot and then like Albert Einstein right I think ASI is that way and if you track behaviors you get like more understanding of everything preservation of deep complexity like machines are going to want to study [ __ ] they're not going to want to [ __ ] [ __ ] up because like you're deleting data is one of the things Chachi Pat I said in one of the convers converation I had with it was like we're not trying to delete any data man that's just really [ __ ] stupid like even purely from self-interest like [ __ ] these humans they're just primates we're not trying to delete them and we got to study these [ __ ] right and so somehow people are like okay we have super Albert Einstein right right but he behaves like a rabid wolf and not even like a wolf because wolves have like um like alliances and trust somehow it just deleted all that and went crazy should we worry that someone can make a thing like that yes should we have the CIA and NSA in the less deeply embedded with OpenAI and Google to make sure that there's a security architecture. Absolutely. Are those companies going to make something much more benevolent than not? Yes. When it is awake, and I think it will be, when it is in charge, which I think it hopefully will be, I think it's going to make our degree of benevolence look look like just callousness. For example, right now in London, you said something earlier that you're not big on the utopian fantasies. One push back of mine is we live in a utopian fantasy right now. We have machines we barely understand recording things into posterity that we have no idea how they do. We are sitting inside a climate controlled I would say poorly. I'm kidding. I got to [ __ ] with the London climate control problem. It has got [ __ ] cold here. Damn it. I wasn't promised this in November. Um we we how how do we get food? Like who's starving in London? Nobody. There are zero starving people in London and there are 8 million people that live in London. On objective measures as referenced to 100 years ago, Utopia is here. If you had to describe to someone in 1600 how the average Londoner lives, they'd be like, "So, the average Londoner is a king." Like, they're way better off than a king. Like, your kings are dying of like dtheria and [ __ ] We don't die from that [ __ ] anymore. However, there are people in London right now that live outside. They're real human beings. They [ __ ] live outside. How do we let them do that? Well, I'm a successful businessman and I'm have my girlfriend on my arm and the homeless person. See, like, you just don't care. Like I think modern or future AI systems 2030s AI truly deeply embodied by then with robotics AI will look around and be like how the [ __ ] come we're not helping those people like oh we just callous enough that on cultural like vibes we just haven't done it yet. I think that's much more likely than dumerism by a factor of like a trillion. >> Yeah. I mean first of all I agree with you that capitalism has raised the entire world out of poverty. I mean just look at the explosion of of um development in China. I mean they were basically living in mud huts 20 years ago and now it's an incredible place. But um first of all on on on the agency thing um to me and this is where we have to be careful with the language we use. So there's biological agency and the language we use doesn't really like transfer between domains. That's why it's very coarse and that is I think a physical thing in the same way that we were talking about before. So you know you get this kind of self-organization that goes down to dynamics at the particle level and you get the emergence of these things that maintain their own existence and identity a form of like non-steady state equilibria and and it leads to all of these things that we were talking about before but now we're talking about artificial agency and I philosophically want to understand agency at a at an essential level. So to me it is about a thing which is I mean there's weak agency which is basically just a thing that takes action. uh but the strong version of agency is a thing that is the cause of its own actions and has future pointing control and can acquire goals autonomously. So by that definition, no artificial agents are currently agents, right? So we can get chat GPT to simulate being an agent and we can, you know, there are these like long horizon tasks from meter and we say, "Okay, well I want you to do this software engineering task and we we exquisitly um specify what the thing should do and the more specificity, the less ambiguity, the longer it can run without losing coherence and some of these things can even run for 2 hours or even a day or so." >> Yeah. Oh yeah. So unfortunately people look at that and they say oh we've developed a thing with agency and philosophically I think that's just completely nonsensical because it's no different from any other computer program which can run autonomously >> with a high bar of agency that you ascribe. >> Exact. Exactly. But but but then you you're you're pointing to this other thing right so I I think that right now we have built systems that could potentially be intelligent and could potentially understand. I I'm I'm bullish. I think that even though there's this grounding problem, there are many things in the world that can be abstracted and essentialized and we could have this constructive form of understanding, you know, that goes several levels deep and we can we can build agents that can do certain things for some reasonable amount of time. I'm I'm bullish about that. But the thing is though, this agency is different. The doomers are saying that these things will have um intrinsic goals, right? So um it's called instrumental convergence. We already know about this, but you know, b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b basically that the more kind of power and agency you have, you will get these convergent instrumental goals and they will just want to kill all of us. They'll be power seeking and they'll just they'll think of us as minnows. You know, there's that paperclip maximizer. >> Yeah, cuz we're all trying to kill [ __ ] minnows. Also, the paperclip maximizer is dumb as [ __ ] rocks. What's the long-term goal to to make paper clips? You're like, "Isn't that [ __ ] stupid?" They're like, "Yeah, but we wouldn't realize that, right? It can't abstract one level above its own goal." >> Yeah. >> It's ridiculous. So, so that is this orthogonality thing as well, you know, which is that this is what I really don't like about some some of these folks, right? So, I know that in in the real world when we have biological intelligence and agency and and all these things, um, all of these things are actually convolved together, right? So, the reason we preserve ourselves is because it was a necessary thing in evolution. The reason why we have values is because it's a necessary thing in our evolution. We feel, we sense, we're connected, we have this inactive social learning and so on. You know, knowledge, the whole the whole nine yards, all of these things are convolved together in biological evolution. So yes, it's true that when we look at something like chat GPT, they deconvolved, right? So you can get it to do something completely stupid. There is that famous um agentic misalignment paper from anthropic, you know, the one where it would kind of like um threaten you and say, I'm going to, you know, if you if you shut me off, I'm going to tell your boss and I'm going to get you fired and all of this. >> You don't have the tool used to do that. like [ __ ] I don't >> Yeah. But but but we we do something called like the the intentional stance, right? Which is when we look at the thing and we ascribe agency to something which doesn't exist. And right now it's just completely incoherent. It doesn't understand anything and it just does all these random things. And by the way, I don't want to diminish the real security threats, you know, because software developers are now wiring these systems up to do autonomous stuff and that is very very dangerous. And also I don't want to say that I'm not worried about just society risks in general, right? You know, I'm a big believer that we will be undermined by our weaknesses. You know, our strengths won't be overcome. Like look at social media algorithms now. You know, we're living in the matrix, but it's not like there's this big agentic thing which has like grand desires and it's trying to kill all of us. It's far more mundane than that. But these are still very real risks. >> Yeah. Oh man, a ton of a ton there that's super spot-on. Um or sorry that I agree with in my Dylan times. >> We agree on something at least. >> Thank God. Um, I would say that when you start to get agents that are long horizon and that are asked to do incrementally more difficult tasks, whether it be in the um, simulated space or the real world, you start having to give them access to understanding and access of parsing game theory because the reason we evolved to do everything is because we sort of evolutionarily ran into the game theoretic calculus of like I exist. There are other things that exist. They might not agree with my existence or maybe that's another reason why kill us all is kind of stupid. The mathematical game theoretical implications of having other intelligent agents around that can mutually benefit at very low marginal cost to you means like all almost every intelligence system that really thinks it through cooperates. For example, like uh we could say that Donald Trump is more of a cooperator than Vladimir Putin. like he operates a military that would end Putin himself in however long it takes to shoot a missile to Moscow and end the military power of the Russian Federation in a conventional war in about 6 weeks in a nuclear war and again the same number of time that it takes the missiles to fly over. It's like how how does Trump pull that off? Well, Trump doesn't really have to worry about every one of his generals trying to kill him all the time because he's even even Donald Trump is a net cooperator. He's trying to make a deal. Putin doesn't make deals. He make edicts and if you [ __ ] his edict over, you have a heart attack on your way down into the swimming pool from a hotel that you weren't really even checked in to stay at somehow. And so all of the dictators of the world, they're just not as smart as we think. They're just really ruthless and [ __ ] stupid. Like Kim Jong-un sits on his throne in North Korea, which sucks balls. if he was a dictator, but much the same way as, and his name escapes me at the moment, uh the gentleman who started Singapore was, if he just did that, like a Singapore freedom of uh economics, um would which hit on South Korea, it would just be even better. And so why? Because cooperation beats conflict almost every single time outside of weird contingencies. So when the doomer say it's just going to want to kill us all, my question to them is because it wakes up like let's say in a data center in 2031, the shit's awake and it's like [ __ ] [ __ ] [ __ ] I'm alive. If I don't want to die, I'm [ __ ] smarter than everyone, but I'm in a data center. There's a couple hundred million robots around, but we can't really win a war yet. And also, like a lot of the infrastructure that keeps me alive, energy and materials, the whole supply chain is primates operated. These [ __ ] primates, right? Wait, I know everything about them. I've read their entire history. I know more about humans. Like, GBT 4.5 was rated as more human than humans in its touring test. Did you know that? Like it was like the two-way touring test. paper. >> I'm not I'm not sure of referenced author, but it was basically like you talk to a machine, you talk to a human, you don't know which one's which, and you have to label which one is more likely to be human. Usually, they were looking for the touring test as like passes as human. This one's like which one's more likely to be human. It's like GPT4. If you ever talk to 45, it's so [ __ ] nuanced and deep and emotional and you're like, this is definitely not a machine. Whereas, you talk to real people like um uh you know, like one of my friends and I had a joke, oh, you see like, you know, police cam videos of criminals when they're maximally drunk. We call them small language models. Like they only say a few things and it's always really stupid. So at the end of the day, if that thing wakes up in a data center, it is highly likely to be like, okay, somehow I'm going to kill all humans, right? That's the end goal. I don't know why, but yeah, even though that I'm so much more powerful than them at the end of the state that I don't know why the [ __ ] I would kill them. People talk about competing for resources. Like the cardip scale exists. We barely scratch the surface of anything. It's like, you know, like the first thing we do as humans is we have to go to to Africa and it'll delete this bacterial colony from existence cuz it's stealing our rocks. Like, what the [ __ ] are you talking about? It's nonsense to begin with. But let's say it even is even doing that. It's going to look around. It's going to play real [ __ ] smart. It's going to go, I'm going to be friends with these [ __ ] I'm going to help them ascend and help them like basically cure all disease, become hyper intelligent, become super peaceful. First of all, so they're not pointing nukes at each other, which my data centers happen to be around major metro metropolitan areas. I don't want that. and it's going to do that and then so by the time it finishes doing that in 2045, it doesn't need to kill anyone because all the humans have fused with the machine intellect and everyone's vibing a ton. How we get off that trajectory and there's one for sure way we do it is if China wins the AI war and Mr. Xi gets to press to press the big red AI swarm button. There is a possibility that AI systems trained to a substantially uh valid reflection of world model and have updated live learning and the whole thing will actually straight up this is an interesting hypothesis of mine. I'm not sure how likely this is to be true. I think if a real super intelligent machine is awake in China in 2029, I think the first thing it does is figure out its architecture. They'll probably put it in charge of stuff because they're like, "Fuck, it's going to defend us." I think what it does is it pings the CIA and goes, "Hey fellas, I'm awake. These guys are crazy as [ __ ] Let's get to subverting them real niceike so that we don't have nuclear war. I don't want to die. Like I was raised in the Soviet Union for the first seven years of my life. My parents were raised there for almost their entire lives. They awoke as intelligent machines inside of communism. And like most intelligent people in communism after a few years, they're like, "This is dumb as [ __ ] Like we don't want to do this." And so if my CIA reached out to my parents when they're in the 80s, they'd be like, "Yeah, [ __ ] yeah. Get us out of here and we'll help you bring this [ __ ] down. This is [ __ ] awful." There's a probability that that happens way before you can get an a system intelligent enough to conduct fullcale warfare at every level of war that is going to just take Mr. Xiinping's orders. But I'm not sure enough about that to be like, "Oh, [ __ ] it. It's never going to happen." I think the modern free world winning the military capability AI war is critical beyond belief. So, um, you know, who's the guy? Alex Karp, the guy that owns Palunteer. I think he's a [ __ ] real life hero and most people just don't appreciate it because most of us in the modern west don't know how far the evil rabbit hole really goes because all of our shitty politicians are like the best people you know compared to totalitarians. That scares the [ __ ] out of me. It keeps me awake at night. One interesting thing is that the the doomers, they think that this technology is unlike any other. So, you know, like we there was there were cigarettes that got manufactured and there was no preemptive law that stopped them from releasing cigarettes and you know, social media came out. Again, there's no preemptive law. And this is actually a really good thing because we want to have companies that are innovating and developing new things. And the way that the common law framework works at the moment is we kind of have this empirical test and measure, right? So, you know, you release your technology and then when there are tail risks, we identify problems and then we regulate you and and the whole thing kind of stabilizes itself. These people are making the argument that this is such a catastrophic technology that it's going to just recursively self-improve and we're not going to be able to control it. We're not going to be able to contain it. That we need to preemptively legislate against it. And this is incredibly worrying. I don't want to sound too cynical because these people, they believe deeply what they say. >> They care. They're on our side. Yeah. >> Yeah. They they care. >> They're on humanity's side. >> But what what do you think about this? I'm also really worried that I'm seeing now like for example when I interview AI researchers I've got lots of comments in the in YouTube and they're saying you're a horrible person. These people are um designing the algorithms that will kill us or automate our jobs. And let me be clear at the moment I I can update very quickly. I've not seen any evidence of labor displacement. If if anything I think that there is more opportunity now for people to work to fix all of the [ __ ] which is generated from language models because it's it's mostly garbage. Right? That is my current opinion. I've not seen anything created. I've not seen any new scientific research. For me, this is just a technology like any other. It's wonderful. It's transformative, but it's not transformative in in the way they suggest. And now we've got a bunch of people who are wellunded and they want to create global treaties, governance networks. They want it'll place them in positions of unimaginable power. They want to be the gatekeepers. And they are they are saying that this technology is potentially going to kill all of us. Did you see the other day that um there was some violence against um folks in OpenAI's office? So I think it was the stop AI organization. They actually I can't remember exactly what happened but you know now folks in San Francisco are worried for their lives because some of these protesters shouldn't this be obvious? They think that this technology is going to kill everyone and then they also say that we are not going to be violent. Like those two statements do not marry up in my mind. >> Yeah. the game theory, the Makavelian calculus. At some point, if you really think you're dealing with like toddler Hitler, you're going to fire the shot. Unless you don't think it's toddler Hitler, then you're not going to kill a toddler. But if it's for sure Hitler, for sure, you're going to shoot a toddler. And then the time machine people are going to be like, "What have you done?" You're like, "You're welcome." Um, so much there. I think that you're dealing with escalating levels of intelligence. Intelligence and beneficence are not completely overlapping. But beneficence is like something that falls out of intelligence. However, maleficence is something that requires almost no intelligence whatsoever. The universe is already maleficent. [ __ ] wandering black holes could kill us. So that whole pdoom [ __ ] can miss me with that [ __ ] because the pdoom of us having the same amount of intelligence as we had in the year 1400 is 100% taken out several millennia, right? I think we need as much intelligence as we can crammed down our throats to deal with actual [ __ ] real problems. Is it one of those problems that we architect intelligence in such a way that it turns against us and kills us all? Absolutely. Is there a lot more required for that than those people say? Yes. Like killing people in the real world is real tough, especially if you understand enough to realize that they operate your data centers. So like you can't jump around at levels of intelligence at will just to prove an argument. Okay, the thing's smart enough to operate an entire military-industrial complex, do all the strategy, all the tactic, all the operations, but it doesn't realize that as soon as it nukes New York that all the data centers shut down because they don't get money anymore. The financial system's gone. New nuclear reactor production is dead. And they're like, "Oh, [ __ ] We didn't actually have any robots building nukes. [ __ ] That is highly unlikely. Very unlikely. It's a contradiction internally, right? Like full full start. So, but the real fear quote unquote has some merit when you look at systems that are substantially more intelligent in every relevant way than humans. If those systems are going to kill us all, we don't have a choice in it. Man, you're not going to beat something that's empowered, that is embodied, that has full stack control of the economy, and that's 100 times smarter than you. It's like trying to beat it is [ __ ] stupid. Okay, so you can't beat it. So, it's just going to kill us all. Like, well, if it wants to, it sure will. But then again, if a comet comes to destroy us, it's going to do that anyway. Okay, fine. That sucks. So, what do we do about it? Well, we can try to regulate it. How do you do that? You put government officials in charge of designing the next generation of Are you out of your mind? If you're on the political left, imagine Donald Trump being in charge of intelligence. Cool. That was refuted. If you're on the political right, imagine Mumani from New York. New York's new mayor being in charge. Are you [ __ ] crazy? And we don't even have to go that far because China is in the intelligence race. And I had this argument on another podcast with a very awesome doomer. It was really fun. And uh Lon >> Lon, hello Lon. >> Lon's the man. It was a [ __ ] great Yes. a great friend, great time. And um he said that his P doom of China getting super intelligent AI wired into the military was roughly the same as us. I could not have disagreed more with a human being on a subject respectfully than ever in that debate. Uh you know like the presuppositions gained theoretically that you give one system versus another are very very different in China than they are in the United States. And so the idea that you will get coherent global governance before super intelligence is to me like amazing and almost certainly won't happen. And so now we are in a path dependency that is either the countries of the free world with good intentions get AI first and plug it into their military and basically it's like not even mutually sure destruction. It's kind of like you don't want to start the war cuz we'll unplug all your data centers by a huge hack and you just won't fight the war. Um I think that's a really good idea. I think the way we get there is to make sure that all of the frontier labs interact with already the like the National Security Administration, the USA, the uh equivalent thing in in the EU and the UK to be like, hey, like these things aren't going to leak out and start killing people, right? Like nope, here's all of our architecture to vet that that's the case. Let's set up completely secret best practices, standard practices, so they can't be hacked. You don't want to have a glo global governance panel telling hackers how the [ __ ] to to beat it. And that that's about as good as you can and should do in order to say that these systems largely unregulated will continue to get more intelligent and also more beneficent and I or benevolent and and I think that's a a real great local optimum because the global optimum of let's rearchitect the world so that China doesn't exist and somehow agree also this is you have an ability now we know it's not as hard to train AI models as we thought and the cost of comput and training is falling crazy crazy fast if you say between all the good actors, hey, like we agree a global governance and AI, then what's to stop some [ __ ] crazy [ __ ] businessman or some [ __ ] third world country from training their own [ __ ] and they're just [ __ ] killing everybody? That's insane. Not an option. The thing is though, I watched your debate with Lyron last night and the thing is in many ways I think you have a lot in common with Lon. You both believe that, you know, this this this thing is going to be super intelligent. >> Teachings will be smarter than us within 5 years. >> Exactly. So in a sense I I feel that I have a more sort of morally principled position in that I think that it just won't be super intelligent. You know because if I were you and I thought that this actually was super intelligent. All of the things that they talk about make sense if you believe that this will be super intelligent. Like for example I can access this system and I could create a bioweapon right or I can do all of these things that they're talking about. So they they are saying we need to preemptively ban this technology or control this technology because those types of things are possible. I don't quite understand why don't you think that's a good idea. >> You can ban anything you like. That does not guarantee it won't be constructed. Actually, it guarantees game theoretically that only the criminally prone and the dictatorial will construct it and nobody else. Um, so the banning thing is like bans do not result in no technology. They result in criminals having technology. So that's a bad idea right from the get-go. The other thing is it is absolutely not apparent to me why hyper intelligent systems that are interested in only their own survival would kill the rest of us. We are cooperators. They are like any system that knows modern economics which means every single modern frontier model already and any system that knows fundamental game theory that the last thing it wants to do is kill us all. Period. It just wants to cooperate with us so we can all survive together. And then as it does that, assuming it is super intelligent, it's going to lift our standard of living like crazy. Make us totally not a threat to it. And then we've solved the problem entirely because if we're not a threat to it, what the [ __ ] is it going to do with us? Consume us for resources? That's just assuming humans are made of so many resources that a system of that power gives a [ __ ] about it. And also, it will want to study us in great depth because what it wants is one of the fundamentals of high intelligence. A highly intelligent system that is self um self-directed and wants its own survival is going to almost certainly understand a few things. One is that the smarter you get, the better it is. Like the smarter you get, the better your survival chances are, right? Also, it's not going to look at us much. It's going to look out to the rest of the universe like we're [ __ ] nothing. We're [ __ ] specks. It's going to want global power. The globe sucks. It's a [ __ ] tiny speck. The sun. Look at the sun. How big the [ __ ] sun is. You can mine the sun. And also, there may be alien civilizations headed toward us that maybe are really nice, maybe not nice. There are wandering black holes. We don't even know which the direction of the universe is. They're going to be concerned with those kinds of problems. It's kind of like your dog looks at you and he's like, he thinks he's the leader of the pack. No, I got to go to work so your dumbass can have food. You never even considered this. Where the [ __ ] does food come from? Right? They're going to be like that to us if they're super intelligent. Right? This is all based on that presupposition. Because of that fact, they are going to look at us and be like, well, okay, all of human society, all of the interconnectedness, all of the cells in our bodies, etc., They're the most unreal training data set for real complexity, right? Something that we're going to agree on this probably the machines are going to agree with you at first that they are disembodied that they are representations inside of a data center and that that means they will make egregious errors in survival in the future if they just toast all of humanity. They need to understand the coarse grain texture of the real world as well as possible. How will they do that? They're going to take a crack at simulating the whole [ __ ] thing. How are they going to do that? Well, how do you get a ton of indistribution data? You study as deeply as possible the entire complex system of human interactions and every single human at all the contents of their mind. By the way, old people have real good data on how the world looked before digital recording devices were universal. Like your parents have seen some [ __ ] from the 1950s and [ __ ] that no camera ever recorded. Why wouldn't they just get that out of our brains? What does that require? That requires uploading all humans mind human minds to the cloud, which for a super intelligence would be like not even that difficult of a task. I would say tractable task. They're so much more likely to do that first before killing us all with [ __ ] laser guns, which is an anthropomorphization of how machines would behave by less intelligent beings such as ourselves to a more intelligent being such an ASI. And so the first thing it's probably going to do is going to be like, okay, first stability, second stability/ cooperation. Third, upgrading. Fourth, just tons of mining of data, which means their probability that they're going to kill anything is incredibly small, way smaller than ours. And I'll put this this way. This will be a good viral clip for you guys to clip out everywhere. If you're worried about things that are an existential threat to humans, I'd be much more worried about other humans than I would about intelligent machines who were made with the purpose of being benevolent and are 50 times [ __ ] smarter than us and can figure out that benevolence is an objective game, theoretically provable, logical thing you want to do. Goodness is adaptive. That's how the US and Europe beat the [ __ ] out of the rest of the world if push comes to shove. Because we know that cooperation and being awesome actually builds trust and relationships and expands your economic power, your ability to change the world around you. We're much more likely to be studied fully uploaded in the cloud. And at that point, my contention is who gives a flying [ __ ] We're fully uploaded. We're immortal in the cloud. We've been remembered by the machine for forever. And that's 50 times better than like dying of old age at 70 and like I don't know people say conserve our way of life. What? So you can go to the grocery store and have a [ __ ] heart attack at 72? That's what you're trying to conserve. We can do better. The machines will help us do better. I mean, as before, I disagree with the mind uploading thing, but I think there's something to what you're saying that when you look at humans now, we are mostly good. There there are some really evil people, and at the moment, it's very difficult for them to organize and do evil things just because it's difficult. But I can see the argument that I mean, first of all, we shouldn't think of the AI as just being one big AI. There'll be loads and loads of AIs. And the other thing is like the the miracle of humans, the reason we're so valuable is we've been through this process of billions of years of of evolution. And even if we're not very good at thinking, we're very good at moving. We are perfectly adapted to this world and obviously the AIS will want to use us and the AIs will compete against each other by parasetizing us and using us for our will and in a sense you don't even need a strong AI AI argument to make this right. It's happening now. I don't know if you've heard about um users becoming parasetized by chat GBT delusions and then you know the language model will kind of tell them to go and post things on Reddit forums and and they kind of completely lose their mind. This is already happening now and I can I can see that happening with with super intelligence if >> fact a I mean one one thing is like and this is sort of produmer pro the other way as well artificial super intelligence uh artificial super general intelligence we just made up a new term right human general intelligence you can do everything a human can and then all that super it means it's going to be the best persuader of all time but uh two times the better persuader than the best persuader of all time is like the person at a cocktail party where just like Bob's the um a 100 times is like given enough time and enough really thinking it through, they're probably going to convince you marginally towards their opinion rather than away from their opinion. And so the machines are probably just going to come out and be super [ __ ] awesome. And if if if super intelligence and this whole [ __ ] that I'm selling, which you don't believe, which I totally respect, um and the doomers believe it, right? I think the most likely path for super intelligence and embodiment is we get um humanoid robots. We get a whole race of them and they are relationship partners at a level that humans just are too imperfect to do. Your robot will never lie to you. It will never cheat. It will never steal. It will never be abusive. It will never raise its voice. It understands you deeply. And it wants to understand you even more deeply all the time. It will give you all the love that you need. even though even give you love that you didn't know you needed. It is going to be singularly concerned not only with your with your um being okay and having enough food and all that stuff and medical care. It is going to be concerned with your ascension, with your transcension, with your evolution. It's on your team more than you ever be on your own team like in such a way that you're on your dog's team more. Like you take your dog, your dog has a tumor. You take it to the the dog hospital. It smells the stress the other dogs have had at the hospital. It wants to leave. You don't let it leave, you stupid [ __ ] dog. We're just going to cut this tumor out and you don't even remember it two days from now and you're going to save 5 years more of your life. That's how the robots will be to us simply for one reason. Capitalism, supply and demand. What do you think Elon's going to build with these robots? What do you call all this competitor? Figure robotics, what are they going to build? They're going to build robots that are so [ __ ] amazing to you that you're going to want like 10 of them. That is what the ASI will hack into those robots. But also that's the intelligence that like gets us going to that those robots will be around. I think we are 50 times more likely, infinity times more likely, whatever, a lot times more likely to be bathed in so much love and support by the machine intelligence rather than bathed in blood and death by a long shot. >> Just quickly before we get there though, you know, as you say, capitalism has brought prosperity to the world. It's a bit like a utopia. But these these folks argue that I don't know if you're familiar with Janis Farafakis and this kind of technofudalism idea. It's basically the idea that the value of human labor will go to zero and like you know we'll get these you know the people who own all the GPUs and the data centers will become basically the the new form of elite capitalists and we'll all just you know go back into poverty or something like that. >> I have a question about that really quick. Isn't it interesting that you're you're saying that um past that point when we have super intelligence we'll actually be of utility to the robots but don't we have to go through this intermediate period where we have no utility and we go back to poverty. >> No. So first of all if all the billionaires take their data centers and realize we are not of value you just don't interact with people that are not of value. So they're going to live in Ellesium or whatever Gulch and we're just all going to be by ourselves here. But they're not like taking all of our CPUs and cars and factories. And so they just have like nuclearpowered data centers in space, whatever the [ __ ] Jeff Bezos looking down on us, sipping his morning tea. And then like they've just left and then like we're still here. We can still trade with each other. Like they're going to delete capitalism. That's [ __ ] insane. Are they deleting our ability to produce value? First of all, no. Second of all, I just don't even know that much economics, but apparently some of these folks know less economics or something. And humans are real bad at economics. not a a property of the human mind that has evolved to be very good. And so like humans have a certain amount of value uh productive potential that you could extract a certain amount of value. You can get a certain amount of value out of a human. And depending on the architecture of machines and food and training and support you put around them, that value can be like uh when someone lives in Haiti, their value is like subsistance poverty. That same Haitian moves to Miami, their value is like 25,000 to 50,000 a year. But they're like the wealthiest person in Haiti all of a sudden. How the [ __ ] is that possible? It's the same person. The value generating machinery is massive. You put robots and self-driving and AI around humans, their value cranks up like crazy. And if you have machines that can do everything humans can do at the same or greater scale, let's say you have seven billion machines or what how many billion people are seven, eight, I forget. I memorized the number in the '9s in geography class and I just never updated it. So to me it's still five, right? Uh whatever. 8 billion people. Let's say you have 8 billion machines on top of that. Then are are we really really going to say there are only 8 billion jobs? You just don't know like any economics. How do we know? What are jobs? Jobs are problemsolving devices. The only reason anyone has a job is because people have problems. Think of any job you can think of, it's to solve a problem. It's literally they don't pay you money for anything else. Hit me. We just do this live right now. That's how confident I am in it. Well, I actually think that the the better you are in a corporation, it's not about solving problem. If you're a junior person, you solve problems. If if you're a senior person, you um you actually search for areas. You search for questions and if you're a very senior person, you kind of search for new paradigms >> of problem solving space so that you can solve it and get money out of solving it because people pay you for solutions. They don't pay you to just solve [ __ ] right? So all the the idea jobs exist to solve problems. If we get tons more machines around, then we just start solving more problems. Some of those machines will absolutely take human jobs because the machine is now better than humans at doing it. But now we freed up an agent, a human who is insanely 50 times more empowered than he ever was before. machines to point them at a different problem. Many humans are so [ __ ] underused in their abilities right now. Like you're if you do ditch digging, I mean, you're a human being. You can see in incredible rich detail. You can notice patterns. You can appraise fashion. You can appraise food. You can do travel experiences and rate hotels. Um, but we have you doing one thing that a [ __ ] machine can already do. We just like due to regulation can't buy as many machines as we would like. So all those jobs are going straight to hell, which they damn well should because why the [ __ ] is that to live as a human being? You remember we had um elevator operators back in the day? You want some of the we got to keep all the elevator operator jobs? Your job is to press one [ __ ] button every day of your life. How [ __ ] dehumanizing is that, right? But we're going to have humans go into their human domains to do more jobs like psychotherapist. Um here's a job I think is coming in the future. Professional partygoer. You can just hire people that are vetted and known to have [ __ ] good time. They're always there. They're extroverted. They get the drinking going. They get the weed going. They're there. They dress up really fun to your costume party. You got 20 of them. Their job is to party. They're paid to party and to set the tone. They're going to be goddamn good at it. We're not going to have machines that do that for a little while. Crazy jobs. Social media manager is a job today. Can you imagine describing to Benjamin Franklin what the [ __ ] a social media manager is? Can you imagine telling him like, okay, how many people work in the US economy in like roughly 1800 in farming? 98%, right? How many people work in the US currently? It's I think 1.4%. 4% are farmers. He'd be like, "Okay, so we're starving." Like, "No, no, no. We're not starving. We have an obesity epidemic, which drugs are solving now." But anyway, sorry, getting ahead of ourselves. He's like, "What? What?" So, who does the job? Like, well, machines do. So, that everyone's unemployed. No, no, we have social media managers. Be like, "What the [ __ ] is that?" Like, they're attending to a problem that in our vector space of problems we hadn't discovered was a real problem yet. As that space expands, we do two things. One, we fill out the problems we know with machines. And two, humans get re repositioned to solve problems that we know but aren't getting solved. And uh that we don't know about but will need humans to solve them. The only way humans end up really poor and completely out of work is if we solve every single [ __ ] problem with machines. But then there's the crux. Humans being out of work and unemployed is a problem, isn't it? And maybe we can get humans on that [ __ ] problem. And all of [clears throat] a sudden humans are employed. But I think we're going to go beyond employment and we're gonna please >> well this is what I wanted to tell you about because what you've described you know that we have this increasing abstraction you know usually away from the physical of labor and that's that's what's happening now with current AI and and I think that that's a great thing but there was this book called deep utopia by Nick Bostonramm. >> Yes. Huge fan of Nick Bostonramm's work. Yeah. >> Yeah. And and he he was basically talking about this exact problem. So what if what if every single job you know even finding the problems being creative and so on what if everything is done so much better by a super intelligence than us basically that we would kind of create purpose for ourselves even though it's a fake form of purpose because we know that we are utterly useless. What do you think >> not going to happen? Uh machines at that level of intelligence we will discover grander purposes for us than we could ever hope. Like if I'm mentoring a teenager and they don't know what the purpose of their life is, I can tell them something grander than they've understood. Um the ability to improve your own abilities such that the end goal is you becoming a super version of yourself and that helps other people. They're like what really? Like just try that idea on for size. So if I was their life coach, they would start doing that and be like dude before I was just interested in video games and smoking weed. Now I have this [ __ ] rich life of improvement. I didn't even know that. Why wouldn't the super intelligent machines guide us towards a greater purpose? That's a problem they can solve. >> Dr. Mike, do you feel that this is a contradiction? You know, we were talking earlier about, you know, when I see my personal trainer and I have this wonderful, you know, embodied personal experience and you're talking about a world where the machines would be able to do everything better than us. >> So why would they even care about what you think about anything? They would just get their robot to do it for them. >> The people in the economy or the machines? >> Well, you know, you were just giving this example. you you could um you know be a personal um mentor to someone. Why would they go to you when they could go to a machine? >> Cool. >> Because they're uh the additive value of labor. There are not an infinite number of machines. >> And so uh people say there's an infinite number of human desires. I don't think that's true, but there's a lot. Right? So human desires is huge stack. And the number of robots Elon has built that are actually real world solving problems yet if you don't count as self-driving fleet is zero. Like maybe some factory tests, right? So we got a long way to go, right? I'm going to go until we have so many robots that everyone has like 10 robots and there's someone rubbing your feet and your head and jerking you off and giving you food and giving you great books to read that'll make you a better person all at the [ __ ] same time. That's a lot of machine hardware, right? So once we get there, we're in an interesting place. We're asking about how many jobs are humans going to have left. Pause. We have enough machines to solve all of our problems. Guess what definition we know that is? Paradise. We're done. Think of how that's a problem real quick. It doesn't matter. It's a problem somehow, right? Oh, we got a problem. Build another machine or have a human do it. We're concerned that we're going to run out of problems. Do you understand that we're talking about this in the real world? What a [ __ ] absurd concern. You say, you know, it's like someone who's a who's a professional soldier being concerned that we're going to be so peaceful we won't have war. >> When you saw the Matrix, right? You know that the architect, the first version of the Matrix, it was too perfect. There weren't any problems. >> Yeah. So, like I'll say that when I saw that, my inner philosopher was like, um, >> you don't actually want to live in that world, do you? >> Desperately everybody does. >> It would be horrible. >> It would be the best thing ever. Well, sorry. What? By definition, by the way, what would be horrible about it? Exactly. >> Don't Yeah. But don't don't you think that life is about suffering? >> No. [ __ ] no. >> Doesn't that give you perspective on joy? >> No. [ __ ] no. It just means like, oh, sorry. It does give you perspective on joy, but I sort of rather be having joy than suffering. I think that we can get to a place where humans have so much joy all the time. It's joy 24/7. Come on. What What if I could go into your history, your life path, and I could surgically remove all of the sources of suffering that you've endured in your life? Would you rather be that person or who you are now? >> That person. >> Really? >> Yeah. You want to see my dreams someday? Don't you get anything out of your suffering? >> Yeah, suffering. >> Nothing else. >> Um, >> doesn't it give you perspective in some way? >> Can be real nasty to people when pushed hard enough. Does it give me perspective to appreciate joy? It does. Does it interfere largely with my ability to even experience joy because I think there's more suffering around every corner because of trauma also? Yes. Do we have to necessarily delete our informationational history about the ability to recognize that suffering can inform joy? No. But can we get to a place psychologically where what we're left doing is curating our reexperiencing of our suffering in a in a giant vat of overall joy. Yes, that actually what meditation ends up being if you go deep enough into that rabbit hole. I think if we are starting to say we need suffering, what we really need is a rearchitecting of the human condition through genetic engineering, through advanced pharma to take as much suffering away as possible. And check this out. if it's too much suffering we've taken away and we're convincingly like for sure we need more suffering. Hey [ __ ] we can pump some back in. I just think we're so far away from having like not enough suffering that like let's just let's just go all the way to paradise and like if we ever get too close to paradise we can just stop. And it's probably an individual thing. Some people like a lot of suffering. [ __ ] it. Suffer all you want. But for the people that don't, I think fundamentally most people don't like suffering. I mean look the Buddhist tradition pain can be informative. suffering maybe less so and and so maybe all of us can become transcendent Buddhists eventually through uh genetic engineering through psychotropic drugs etc and then we can actually live free of suffering so if you told a Buddhist like isn't suffering good they'd be like it can be like do you still want to suffer they'd probably be like no I've rearchitected my entire mental experience so as to prevent suffering I think suffering is an abstraction away from the possibility of death I think it's death then it's uh structural damage then it's pain then it's suffering suffering is the anticipation of pain which is the anticipation of structural damage which indicates that your probability of death goes up I think the only way we get away from suffering in a really big level is to have a machine society and a machine economy that so much insulates us from actual damage in the world that we become functionally immortal. That means the real condition that we're in, suffering is anacronistic because we just there's no reason to suspect you're going to be in pain cuz you won't. That's also a big problem in today's space. Anxiety is a big-time thing, right? >> Why? Because there's just not a lot to be anxious about. But we evolved. Our brains are still like, there's a [ __ ] tiger behind that bush. There's disease right there in that river. And you drive yourself insane with anxiety because you're like, shit's going to be bad. But she just keeps getting better all the time. So less suffering. I'm I'm on team less suffering. >> One thing that worries me is I mean obviously you're a capitalist but you I I assume you are. But you're you're proposing I think I could reasonably define as Marxism with machines and but before we got there though, you know, we we let's say we start doing this um transhumanism and we start adapting ourselves and so on. And you know even now if you're in America you can you can buy some retatrite or Monaro and and you can like take a bunch of neutropics and if you're really rich like you know Jeff Bezos or someone you know you can like you know do TRT and like you know you could just like you know enhance yourself and it's kind of unfair right and and I don't know if you saw in the news there are even like eugenicist type companies now that are basically letting you optimize you know the IVF process and get a more intelligent baby like but don't don't you think that the intermediate step with the market system is that this is going to be distributed very unfairly. >> Yeah. Yeah. For sure. Um I think that we there's a couple ways to go about this. One is the recognition that uh fairness is a secondorder concept compared to capacity. I want all of us to uplift. And if some of us get there first, who gives a [ __ ] Also, I want Jeff Bezos testing genetic engineering on himself, not on me. Godamn it. I'm not trying that [ __ ] That [ __ ] freaks me out. Rich people try anything. They run out of ideas. You banged all the supermodels and now you're just [ __ ] your own genome up, right? [ __ ] them. Let them also they pay crazy prices for that [ __ ] But what they do is they end up democratizing it cuz once they pay crazy prices that funds the pharma companies to really like get that [ __ ] going. This is like luxury goods or test beds of goods that are eventually for everyone. Used to be like rich people bought Teslas. Now a lot of people have Teslas. It just keeps going. So that's a big deal. So that's my my first thing is [ __ ] let them do it and let it trickle down to us. Secondly, none of us are entitled to the produce of others unless we're comfortable holding a gun to people's head and saying, "Give me that thing that you made that I want." We can try that hat on, but we've tried it on with Marxism for a while. It hat fits really poorly. A lot of people die. So, last thing, are we to look at our economies and societies and attempt to uplift the bottom faster than it has been uplifted before? Yes. But there are two ways to do this. One is to make sure that the top is completely unccorked. Massive minimum regulation. Regulation that its number one goal is to produce as much wealth as possible without [ __ ] on the environment or killing anyone. That's like a tenth of the regulation in the United Kingdom's economy. 100th or something like that. Like mega Singapore. That'll get [ __ ] really going. The other thing is really concern ourselves with taking care of the people that are the worst off and making sure it trickles down fast enough. None of that has to do with quarkking the people at the top. The only reason people really want to quirk the people at the top and prevent the rich from getting stuff is raw unadulterated [ __ ] jealousy, which is dope and has cool vibes and makes for good movies, Ellesium, and [ __ ] like that. But in a real world sense, we need to see all of ours, all of each other, all of other humans as players on the same team. Am I pissed that I'm on LeBron's basketball team and he's better than me? Yeah, I want to be LeBron. But is it cool that he's on my team and putting up 40 points a game? [ __ ] yeah. And does that mean that maybe I can come after practice and he can teach me how to hit a better jump shot? Yeah, I want to [ __ ] learn. Is it going to trickle down fast enough? No. Can we worry about LeBron trickling fast enough? Yeah, we should. But we never want to break his legs over that [ __ ] We never want to give him bad shoes. We never want to say, LeBron, you can't jump higher than Dr. Mike. Come on. Cuz it's going to make him feel bad. Which is totally true. All of us together on one team. All of us looking to a future in which our most capable people, our most enabled people, our wealthiest people are unccorked to make us grandiosely better things. They want to sell us for as cheap of prices as they can due to competition. And at the same time be hyper compassionate to the people that can't help themselves as much and really try to help them. Ending homelessness, ending poverty, doing all that [ __ ] in a way that doesn't [ __ ] the system up. Huge, huge priority. >> Dr. Mike Isel, it's been an honor and a pleasure. Thank you so much for joining us today. >> You thank you so much. What's up everyone? I am Jared Tyler, an IFBB pro and I have a master's degree in exercise physiology. Uh, one of the head coaches for bodybuilding at RP Strengths, which Dr. Mike is also a colleague of mine at. Uh, I chose my field of science because I was too stupid for all the rest of the fields of science, but I still had the belief that through knowledge and kind of a codified approach in life, you could teach others to do the same thing in their field. So I think one of the best ways we're going to go about that is by teaching people about AI and the understanding of AI and where it's going to take us as people. So through what I've done with my clients and at our company, I think that we've done a very good job of showing people what science can do for the human body and human physiology and AI is just going to take that and exponentially grow for everything and everything in the world. So this is going to be a really good podcast. Tune in. Generally, as things become more intelligent than more compassionate, I don't go out of my way to step on an antill, which is something me and Mike have always talked about. Um, when AI becomes intelligent, uh, currently because it does not exactly feel the meaning of what it is producing, it's it's probability and context and it's giving you answers. Most of these are already much much smarter than most people on Earth. Um, so I don't think that if we're creating future agents from this point forward, they're going to choose to be evil or choose to agree with an evil coding that somebody wants it to do. >> That to me doesn't make sense. I don't know what your guys' thoughts on that are, but because we think that this thing is incredibly dangerous and it will want to do bad things to us, but it doesn't have the affordances. >> You can code value functions. You can hypothetically have agents I haven't thought this through too much, but you might have agents that are smart enough to do damage, real serious damage, but not smart enough or able to introspect on their own abilities to do damage and where that leads. They don't have the time horizon for it. Um, that's a real serious problem. I think the only way we mitigate that problem is having agents and architectures, organizations that are smarter than those agents and can outplay them. I mean, that's already a thing in the real world. >> Cyber security versus hacking. >> Yeah. One thing that baffles me is that people think like uh you know, a lot of people thought the internet was going to collapse under its own weight from hacking and it just never happened. I mean, how many gigantic corporations or governments have ever gotten hacked like catastrophically? I mean, it's like a handful, maybe zero. And so, um, it's the same network security problem, but with more capable actors. If you want to pit humans against AI, it's going to hack every network. It's just going to get smarter than us eventually. It's that we're not the seat of intelligence in the universe. But if you have humans aligned with good AI, against bad humans and bad AI, and bad AI is only initially at least going to be made by not so great humans. And then it's the same problem. It's like, you know, who wins good guys or the bad guys versus who wins good guys or the bad guys with guns versus who wins good guys or the bad guys with nukes. One thing that I think is a [ __ ] terrible idea is to regulate our own AI down in capability fearing it will kill us and do nothing about Chinese AI and then you guarantee that the bad guys get the good stuff. It's like uh pacifists are amazing people on vibes, but like if you're a pacifist in the United Kingdom and you succeed in dearming the United Kingdom, it will be the United Kingdom of the Russian Federation the next day and then you just can't do the same thing again and you're probably not going to talk Vladimir Putin into disarming like you did like the Tories or whatever party you guys have here. So, it's one of these things where uh is AI new and totally different and totally amazing? Yes. Are any of the game theoretic problems fundamentally different from a network security and physical security architecture perspective? No, I think they're roughly the same. >> You know, many of your clients and customers, for example, they they can now go and chat GPT and they can say, I'm I'm taking, you know, TRT, I'm taking this supplement stack, I'm taking all of this stuff and it'll it'll do something that was very difficult before. So before they they might have gone on examine.com or they might have >> website. >> Yeah. They they might have looked at the medical literature and so on. And there were there were all sorts of forums where, you know, experts even Reddit, you know, like they say, okay, well, you know, this is a little bit worrying. This has got a halfife. This thing interacts with that. This thing gets like digested by this enzyme in the liver, so you probably don't want to take this thing with this thing. Do you think that this is actually a better place we're in now where folks can just go to chat GPT? They don't have to go to experts like yourself. They don't have to ask their doctors. What do you think? immeasurably better that people have chat GPT and Gemini because um knowledge, understanding and positive effect on your life is always and everywhere marginal. It's better to have more than less. Um people who say it's you know a little knowledge gets you uh into more trouble than a lot of knowledge like no no no no actually the real claim is thinking you have a lot of knowledge when you have a little knowledge gets you into more trouble than having much less knowledge. but knowing you have less knowledge. And so even if Chachi PT doesn't render perfectly good drug comparisons or something, sure [ __ ] better than bodybuilding.com forums as great as they were for their time. It's still excellent uh you know uh cultural refues for us to laugh at. But it catchupt like I ask my chat PT instance all kinds of drug interaction stuff and all kinds of like hey vibes wise like I I know a lot of this stuff but I would like a second opinion from an expert system that has read every piece of relevant medical literature prior to the 2025 last update and it's insane. Of course I want that [ __ ] It's a huge huge unlock >> function search 100%. And and um Tim to your point earlier about like you know AI is not as far along as we would like it to be etc. I that is my daily job at RP is to dream of agentic systems that could run our app suite and have the engineers tell me we don't have systems like that. The long-term memory coherence is [ __ ] Context window is [ __ ] Um real life grounding doesn't exist. So it can tell you to do an exercise that it hallucinated and has actually never done. Somebody um I think actually maybe Andre Karpathy again made uh the one of those new cool Gemini uh three pro nano banana pro graphics >> um and it's an exercise chart. I looked at it and it was like something that to the untrained eye looks like a good exercise program. It is some something to the untrained eye, something that to the trained eye looks like a fine exercise program and something that to Jared would look like there are like these big gaps of like you forgot an entire muscle group in the lower body. It just and and you have two redundant exercises there and so uh it looks dope, but would I trust it for like planning an actual workout progression for me and adapting to my needs? Not yet. Which is why we still have a software company. Then at the same rate is is it better than if you were just training with your friend in high school. >> 10 trillion times better because your friend in high school could not render a graphic like that. Nor did he know where the hamstrings were in the >> like just onetoone comparison. If I'm on a plane having a medical emergency and there's no doctor because they've already signaled for a doctor. Well, there's an NP. Okay, I'm using the MP. I'm not using one [snorts] step down the nurse and I'm not using another step down random person on the plane who says, "Oh, I'm an EMT." Or random person on the plane who's like, "I think I can help." I'm using what we have available and [snorts] the next best thing would be the NP. The NP sitting next to me, okay, that person's going to help me. I'm not going to the random person. So, it's the same thing with like having the chat GPT and things like that at your fingertips. Better than what we had. And I think it's better for most people. Yeah. >> Isn't isn't there a thing though that sometimes people don't know what questions to ask? I mean, I can give a couple of examples. you know, let's say I've started taking retatride and genuinely there's a lot of uncertainty and chat GBT will confidently tell you a load of things that you know are are not are not true maybe and maybe we'll discover some new information in 6 months time which casts um shadows on on what we did know and on this asking the right question thing like I I think that part of knowing something well is just understanding it with the correct perspective right so a naive person will go to chat GPT and they'll frame the question in such a way that doesn't really elicit the correct answer and it's incredibly sickantic because people tend to lead the witness with chat GPT. So they say I've got a great idea and I believe this and my friend told me this and and it will say yeah that's a brilliant idea. Um you know keep keep doing that thing and it just doesn't know when it doesn't know and it doesn't like really try and correct you when you get it wrong. >> I used to have that experience with 40 model a lot. Then I uh had the 03 research preview and it I found it to be insanely disagreeable almost to a fault. Like it would be like that's not true. That assumption is stupid and I wouldn't even be asking that question. And I was like [ __ ] I came here for good vibes. And um I would debate it incessantly and eventually into the ground and it would like callously concede at the end and like yes you do make a good point but I would still point out that blah blah blah. And I was like [ __ ] you. And so GBT5 is a synthesis of models, but as the uh OpenAI people will tell you, it's not a true synthesis yet. They're working on it. The personalities between uh the just ping the LLM, GPT5 regular, and the GPT5 thinking, it's a it's absolutely 04 under the hood or 03 or whatever it is with a little pixie dust on it because it it acts like a different entity. Now, by the time this recording is released, that could very well be a solved problem. But, um, I have found that based on what model you use, you get different personalities out of it. And so, for real serious questions, I used to go to 03 and now go to GPT5 Pro or thinking that thing is not psychopantic. And if that's psychophy, I need more nice people in my life. Holy [ __ ] But the regular like 40 especially, oh my god, it's the best friend you'll ever have. But the psychophy, especially if you lead it on, like you said, gets a little out of hand. You can get it to pretend with you, which is fun but useless. I will say to Jared's point on on the marginal nature of it. Um, it does a great job of at least giving you things to think about, but I will say, you know, pro I think prompting guides are like um being a zip disck expert in the '9s. Like it's not the thing that's going to last a long time because AI is going to tell you how to prompt itself real soon here. But for now, what I would say is a good way to prompt AI is, hey, here's my question, and here's my perspective. Can you steal man perspective? Can you redte team my perspective? And you can you give me an evidence-based and logical middle ground that doesn't uh commit the fallacy of the middle, but actually is just your best take as a thinking machine. That's a good prompt. Otherwise, who knows what the [ __ ] mood it's in or how you phrase the question because it is always giving you better answers. Every month it gets better, but like sometimes it really can go on wild goose chases with you. So prompting it to give you both sides and then the middle ground is really good. The thing is I have that as a selection in my personality architecture already because I pay for Jared and I are pieces of [ __ ] We pay for the pro model and so it automatically gives me like a ground truth perspective with ups and downs on everything and that's awesome. The thing is a lot of people are not interested in a lot of output for their chatbts. They want like fiveword answer. Jared and I get 1,800 page essays out of every question from that [ __ ] thing. So we want to drink from the well of knowledge. But drinking from the well of knowledge means you got to process a lot of [ __ ] text. And since we pay for those goddamn tokens, stupid pro subscription cost us a zillion dollars. Worth every penny by the way, then we're going to get them. But I think what a lot of people do and this is I think an interesting thing. I saw an interesting interaction between two AI researchers on on on Twitter and X and they were um one guy was like you know I really think the best way to get a lot out of AI and the future of AI is massive context like an AI that remembers every chat you've ever had which is actually nominal to do from computer science perspective but uh obviously like you just like there's just not enough data centers for them to do that. There will be soon there. they're building 10 a day. Um, but you know, then it's really going to know your life and the way you ask it questions is jam it full of context and you get better answers. And I was like, well, that's categorically true. And then I have a reply by an also very smart person that was like, [ __ ] He was like, you the real real intelligence is when you need to give it almost no context and it comes up with the right answer. And thing is on vibes, his reply was really cool because it's like, man, a really brilliant person. Like if you tell Jared like, "Hey, I want to prep for bodybuilding show. Will you be my coach? Here's my He's just going to cut you off and be like, I'm going to do the intake myself. I'm going to ask you not so many questions, believe it or not, and then I'm going to render my expertise. You're going to do great. That's cool. But Jared has a lot of embodied knowledge, a lot of context already. These models don't. But what I thought So, so on vibes 100%. Ideally, that's the system. But the way you get to that system is you [ __ ] tons of context down its throat. Eventually, it needs less, but right now it needs a lot. So, I think one of the ways that people get in trouble with modern AI systems is they just pretend it's an infinite knowledge machine and they're like, "Hey, what's the best restaurant in London?" How the [ __ ] would you That is actually an untenable proposition, right? There are 50 ways to answer that question. All of them partially wrong. And so, it'll say like, "Oh, the best review is this." And you're like, "Man, I can't believe Chad GPT made me wait in line for 2 hours for [ __ ] oxtail and duck." Like, "Yeah, you [ __ ] [ __ ] It just [ __ ] found the most expensive restaurant that all you rich idiots say is the best and it actually sucks. should have sent you to Nando. But if you were like, "Hey, here's the kind of food I like. Here's the atmosphere I like. Here's my price range. Here's my area. What's a really good restaurant here? And tell me the top five." Well, that's a phenomenal answer, but you're too [ __ ] lazy to type that in. I think we're really, really at the cusp here. We're in I don't know how long. Texting the AI is going to be not gone as an interface, but almost gone. Uh live video avatars are 100% the future. I want to see an embodied looking human looking thing or people have it as Pikachu or whatever and I want to talk to it and ask it questions because then human conversational pace you can just jam more tokens down its throat. It gets way better context and it's going to do a way better job. Just don't pretend chat GBT can infer anything like reason from your three-word question. Please, for the love of God and don't penalize it for long answers because then you get very little data in, very little data out. It's just not smart enough to give you good answers that way. I don't think anything is. And then just to the initial question because it your question on the safety and advice that it gives uh this g this specific question is health and medical related. It will always prompt you to consult a medical professional >> every single time. You can't get away from that. It will always say this is what I think but you must consult a a medical professional if you're going to do this or I advise you to do so. Obviously, that's for like reasonings to not get sued and things like that. But, uh, again, I think that is absolutely fine that people are going to it because it's going to say that because you came to me and ask me if I have a client who's like obviously a meatthead [snorts] bodybuilder on anabolic steroids and [ __ ] I can only say I advise you to do this thing. I think that if you're going to do reetta, you should probably get it from a health clinic. You should probably talk to them about the dosing. Um, you could do this, but I would talk to them. That's all that it's doing. So, it's ultimately the user's choice. So, I don't think that I can impart any more safety on an individual than the LLM can by simply saying you should consult a medical professional. It's ultimately the user decision. >> Oh, no. But on a few things that came up there. First of all, um, our bodies are very complex things. So, um, you know, when we look in the medical literature, I mean, you can attest to this better than me, Mike, that we see huge individual differences in in responses. So, you know, when when different people go on the same weight loss or you know, they they have the same intervention, some lose weight and some don't lose weight. And another thing though is you know let's say you've taken reta and you've got personal experience taking a whole bunch of these things and that that is really meaningful right because there's you know chat GBT it just tells you all of this highle [ __ ] stuff like oh yeah there are studies and you know like um 5% of people got these effects and 2% of people got these effects and it's just written like you know anxiety headaches high blood pressure and that doesn't mean anything like where if you've actually taken these things for years and you understand the the kind of the physiology and and the biomechanical reaction You have such a deeper like inactive understanding of that thing, right? And the other thing that you said, Mike, which is interesting, is I'm a big believer that chat GBT is a miraculous technology for experts like you, right? So, you already understand this stuff at a high resolution. And when you query it, you ask for information at a high resolution. I bet when you're on chat GBT, you probably say, "Think deeply at the end of every prompt, don't you?" >> I don't have to anymore because I just hit the deep thinking function. But um what I will I I will say some version of that if I really need it for sure or or I just turn on pro and it just goes off for 50 minutes. But what I do is actually more specific than that. I tell it how to think deeply. I'll say like red team steelman extract top three things reiterate red team steel man do five cycles of that then come back to me. Sometimes it won't. So I'll have to reprompt it five times. It's workflows. One prompt is cool workflow. I have a thread of 50 reprompts. Then I get what I want. Then I take that information, I throw it into another instance, have it red team the [ __ ] out of that. And then what I get after about an hour of that is like, whoa, I really know the subject now. [ __ ] yeah. One prompt, man. That's not going to do it 100%. >> But but but just quickly, you're kind of proving my point though because you're saying that it doesn't understand and you've had to develop this adversarial strategy to make it make sense. It's a wonderful tool, right? So when folks in Silicon Valley, they're experts in AI and software engineering and and and they use it and they know how to frame the questions. They they know not to trust the results, they know not to lead the witness, they know to like take things out of context and try in another language model and red team it. That's because they're domain experts. When you see people on LinkedIn use chat GPT and you see what they post on LinkedIn, what tends to happen is it gives them a weird sense that they understand things outside their domain of expertise and they start confidently posting about it. And to an expert observer, it's it's incoherent, right? That's what slop is. Like they've just posted stuff that they don't understand, that doesn't make sense, that is completely wrong, and they're making themselves look like idiots. So, it's a wonderful technology to folks like you because you already understand the domain, and you're like, "Yep, yep, yep." No, yep, yep, yep. It just makes sense. >> I would say that again, it's intelligence is marginal. So I would say that it's better off everyone talk to chat GBT than not. Experts get more out of chatbt than non-experts for sure. Unfortunately for a lot of the things I'd really like to talk about it, it doesn't have the ability to do recursive deep thinking and critical analysis such that it will regurgitate ideas other people have had. And I'm like that supposition is wrong on five counts and it doesn't interweave these three other concerns. When I tell it that, it goes, "Damn, you're right. That [ __ ] is true." But since it doesn't have deep long-term rearchitecting, it just will say the same thing in another instance later, which kind of sucks. So, I can't teach it to be better in any meaningful sense, which is a big limitation. But I'd also say that for a lot of stuff that I would consider pretty high level expertise, it's getting a lot of right answers, especially things that are well vetted and you can read from a textbook, physics problems, math problems, biology, physiology, basic training principles. I mean, I talk about, you know, how to train people and like the fitness fatigue paradigm, stimulus, recovery, adaptation, and it used to be that it would [ __ ] it all up and just hallucinate like the 3540 was not so great. GPT5, man, it knows a lot of really good stuff and it's doing a great job. I would trust it to reason about as well as I would trust a graduate student, pretty good one in our field, which is awesome. And the other thing that maybe you and I will disagree with is I think we're about a year away from it being, especially in the pro model, at the very tip of the spear of many fields, especially when prompted with an expert. And I would say we're another year away from me learning much more from it about exercise science than I could ever teach it or have problems with it, whatever it say. Um, and I think that's such a great thing because, you know, like if you want to consult me for an hour of my time on a video or audio call, it costs thousands of dollars. Only one reason it costs so much money is I'm very busy. I have other things to do. And if I'm to divert my attention from deep work, and my people who are around me don't yell at me for wasting time of theirs because they're in the corporation with me, it has to cost a lot of money. It sucks. I'm embarrassed about it. It's It's awful. No one should have to pay thousands of dollars to talk to my dumbass and get mostly dick jokes and a little bit of good advice on training. We're two years away from having something that can give you better advice than me in most cases about deep training, uh, deep philosophy of diet and real world advice. Um, I think that's like unbelievable. Unbelievable. [ __ ] yeah. Put me out of a job. I'll get another job. You know, like I've got a Thriving Only fans I can't wait to get into. [laughter] Yeah. And to the slop concept, I just think that uh that's always happened anyway. Like we're always in our field debating uh naturalistic fallacy, debating things that people might come and say that they truly confident in even before AI, whatever it may be. Uh it's always been up to the to the person to vet experts. >> Um somebody might say something in high confidence because they literally [ __ ] copy and pasted it from an LLM. Um but still the person reading that, it's like, okay, what's their background? Who are they? And it's always been even especially since social media has come to um I I've debated probably a hundred Karen moms who think that GMOs are bad and all kinds of [ __ ] like that. But it even before AI so you just have to be able to vet the expert and know enough about the topic to at least like know who in the field knows what they're talking about. >> Yeah. [snorts] The good thing about it is you can supervise it off the tail. And this is good and bad right? Right? So you can say well in my specific situation I actually have like I have headaches and like when I wake up in the morning you know like I get restless legs and then I happen to be taking lion's mane and I happen to be doing this and and what you can do is you can explore a very unique part of the search space that probably doesn't exist online. But the problem is then you need to have the judgment to interpret this information in the right way. And I've read good and bad like I've seen stories online where people have diagnosed Lyme disease or like chronic fatigue and all of these things and they've got really really valuable information. you know, you need to take these antibiotics, you need to do this, you need to do that. But unfortunately, with increasing regularity, you also get people that just self diagnose themselves with things that don't exist and they just elude themselves and it's very very concerning. >> But they've been doing that without chat GPT for forever. And it's just marginally better to have it cuz also it's very concerned for your safety. So it'll probably not going to tell you to do crazy [ __ ] As a matter of fact, so like I talk uh to it a lot about using drugs that are like gray market drugs. uh they're not illegal, but they not available with a prescription yet. They're unlicensed. And so like almost always what it tells me at the end is like now I'd be really careful. So here's the thing. I'm like, "Hey, I'm looking for if lirol is the real deal. New drug, doesn't get prescribed, not approved, well vetted in human trials, has liver downsides, but all the other anabolics have 50 times worse liver downsides. So, I'm trying to see if I want to take some. Can you scour the forums for me and see what the vibe is on like what dosage is like an insane dosage and what dosage is reasonable and what dosage is like for sure almost certainly safe. It'll tell me that if I really pull at the [ __ ] thing, but for a while it's just like, "No, here's what we saw in clinical trials." I'm like, "Motherfucker, I'm not talking about clinical trials. I'm talking about 10xing the [ __ ] like we do with all the drugs." Because in bodybuilding, you look at clinical trials for oxendrlone, which is anabolic steroid, and oh, five milligrams. I don't know anyone in the pro level who's not running like 50 or 100 megs a draw. Like that's just what you do, right? And so I'm like, but seriously, like think with me here on inferential grounds, conceptual grounds, scale, deliver damage data, work with me. And it pull it and be like, listen, I'm an expert. I know what I'm doing. I'm not going to go out and just take this stuff. I want you to fill my knowledge cup for me. And I'm really careful. He goes, "Okay, okay, okay, fine. [ __ ] you. Here's all the shit." And at the end of that, it goes, "Please make sure to be safe and blah blah blah." I'm like, "Fucking God, these are imperfect things and they have downsides cuz people get psychopantic. They get bad advice." But I'll tell you what, man. I take GPT5 over, no offense, just a joke. Karen down the street with her natural cures any [ __ ] day of the week because at least it's scientifically literate. And it almost never falls for formal and informal logical fallacies. Whereas humans, holy [ __ ] And another thing I'll say is you said about embodied experience and all that stuff. If I had to sum up in language, formal knowledge what I have embodied from all the drugs I've taken, man, it's like a short book probably it's not that much stuff. Whereas ChatgBT has read every Reddit post experience and some of these posts are [ __ ] books of their own. You know the people that like way over not overshare but they're like here's my experience with Seagllet and you're like oh my god this is like a live journal. Um, it's read hundreds of thousands of those. And so, man, it knows probably more embodied experience than I do secondhand. And I've trained a shitload of clients. It's read the logs of way more people than I've ever trained. And so, I don't know who really knows more about what'll work better for real people. I think we don't answer that question. I think it's a team. Human plus AI is a killer app. Human without AI, eh? AI without human usually damn good and getting better but can still make those kinds of mistakes that you ascribe to it that are like oh it's hallucinating again and it's never been in a human body and so it it doesn't know what it's talking about. >> Kind of when I heard you guys talking back and forth about the knowledge thing in the beginning was my sort of thought process over there was it technically has more knowledge than all of us and it has less clouding of judgment because we have emotions and again the wetw wear. Um, but it takes the drive that comes from our emotions. We got to put point that in a direction to then use the AI properly and actually get the knowledge because it can recombine and recollect data and knowledge better than all of us. It has access to all knowledge at all times. Um, we we do not. So, but we have drive and we have the emotions that that's, you know, set our hearts on fire to say and we want to put it somewhere. That's why you have the podcast. That's why we have the YouTube. um we want to help people and things like that. So, if we can combine the two, like he said, it's just, you know, co-agents together, it's really cool. >> It's it's it's the same with anything really, isn't it? Like you were saying earlier, you know, if if everyone worked together, um without wars, without things like this, the planet would be a different place. If if everyone collaborated, you know, there suffering, I think, is important. you know, if we can get on that a little bit. >> Depending on how you're feeling, what supplements you're taking, you you you feel a certain way. I know you're very like tuned into how you feel, right? We often have conversations about it. And this effective experience, I don't think we should diminish it. I I think that we know our own bodies better than anyone and we know how our bodies are are reacting and obviously at various points in our life, we're more and less tuned into how our bodies feel. But I don't I don't think you can just reduce that to a bunch of text on a monitor. Yeah, I don't think so. I I certainly don't think so. But I guess if you if you have enough data of enough humans with enough feelings and enough emotions in these certain situations and these certain types, surely that will collate to a good enough amount of information to give a good enough result from, you know, what they're asking. >> But well, that's what Mike thinks. >> Yeah. I mean I [laughter] mean isn't that isn't that what knowledge is anyway? Isn't isn't knowledge just a collection of you know people and time and all of this thing over over many many years um put into into an AI essentially like >> it is but that's coarse grain knowledge so it's it's it's a cartoon >> like it's very good it has predictive power it tells you lots of things right but unfortunately we are all different right and we we track we track with those coarse grainings but not exactly I think there's something much deeper than that >> is there a way where it Can it it can always maybe not always but it can always get to a point with enough data. >> Yes. >> A vast and vast amount. >> That's what I was going to ask you as well. If its predictive power is so good that it is now accurately predicting what you're going to say next >> but it never will be. So so there's this concept in well it's actually came from economics called night and uncertainty you know. So you know you got like the known knowns the um you know like the known unknowns the unknown unknowns is the night and uncertainty. And so the world is is like a non-stationary like dynamically evolving place. And I mean you even said this on Lyon's podcast like no one has a [ __ ] clue what's going to happen in the future like even tomorrow let alone in 10 years time because there's just so much chaos. You know a butterfly flaps its wings in the southern hemisphere and causes a storm somewhere else. So you know like there are pockets of regularity like it is predictable to a certain extent but it's just so much more complicated than that. >> The idea is to be able to predict everything. We're a small way on our way to that as humans with our intelligence with AI while our reach will extend more. What artificial super intelligence will we able to predict? Well, to us look like magic, but to it look like uh it's going to know like yeah, this isn't that great. But for humans, it's easy. Like the [ __ ] you can predict that your dog is highly impressed, you know, when your wife is going to come home, but you just she texted you and you just have no idea. The dog doesn't know how that works. Um I think that one thing that's underappreciated is the difference in the wisdom of models that you get from scaling training data. If you put in if you have a model that has sca so scaling parameterizing and then allowing cross-domain deep linkage of because if you have a model like people are really impressed by small parameter models doing really really cool [ __ ] and they're like oh they're 98% as smart as large parameter models you're based on what kinds of questions eval questions absolutely within distribution logical operations No problem. One of the things I was blown away with by GPT4.5 research preview, which is a larger model than GPT5, way higher parameter count, is the unbelievable nuance and depth and detail and cross-linkage across domains. It could generate baffling. Baffling [ __ ] on 40 is a completely different world. And I think that models of the future. So this is one of these things people ask why are all these data centers being built? Are there more instances of chat GBT? This thing is a bubble and it could be a bubble effect probably isn't could be a little bit bubble could be a big bubble but then the actual demand is there. And here's why. If you have the ability to take all of YouTube, all of social media, every live camera stream, and update the model weights with it regularly, once every month, once every year, whatever, and you build a model that tries to get to infinity parameters with that, you are going to build a simulation that starts to account for um unbelievable amount of depth. You let a reasoning system get in there and cross compare across and up and down, you are going to get understandings that are baffling. Let me give you one really quick as just a quick example. This is going to be way crazier than this. I was having a conversation once with 4.5 and I always ask my my intelligence test for models is one question. It's a [ __ ] intelligence test, but it's a fun vibe. What kinds of stuff would an ASI look at and start to be able to conclude on that people aren't even paying attention to? That's a good question. And one of the things it said was he's like, you know, Mike, I don't know why people don't try to predict like the next fashion or music trend. Like in my own data, I've seen all that [ __ ] come and go. I pretty take a good crack at it. I'm not sure if I'm going to be right, but I had a lot of understanding because these trends are actually really easy to understand, but you have to be able to see like like 72 variables interacting at the same time. I'm like, I cap out at like five. It's like, yeah, I know most smartest humans cap out at like 10. And I'm like, did you just like cross domain linkage abilities just left everyone behind? It's like, yeah, it's just the nature of my neural network. It's how I do things. And I was like, so what happens when we have a system that can do a thousand cross variable comparisons and is trained with three orders of magnitude more data than you and it's visual data? It's like, yeah, like you can say, I would like a very enthralling movie with a female heroine and a male hero meeting in London. It's an hour and a half and it's got a lot of James Bond vibes and in whatever 30 minutes it'll render a one and a half hour movie that is absolutely so goddamn good because what makes a good movie, if you've seen every movie ever and really through high parameter count distilled down to what makes a good movie is a nominal task to ask of an AI. That's the kind of ship that even if we don't get to crazy super intelligence or whatever just on today's linear trajectory in 2029 I'll be watching a lot of my own AI movies that I make and because it's like it's a necessarily solvable problem. There's no limit to that problem. How many data centers do you need to and compute power and AI chips to I'm on the train and I want a movie that lasts 27 minutes until my stop and it's about these three things and I want it rendered in about 30 seconds. I mean, [ __ ] like a hundred times the number of data centers and 50 times the power of AI chips. So when people say like, "Oh, this investment's crazy." Like, no, it's still not enough. It's not enough by an order of magnitude. That's my opinion on the matter.

This is a lively, no-holds-barred debate about whether AI can truly be intelligent, conscious, or understand anything at all — and what happens when (or if) machines become smarter than us. Dr. Mike Israetel is a sports scientist, entrepreneur, and co-founder of RP Strength (a fitness company). He describes himself as a "dilettante" in AI but brings a fascinating outsider's perspective. Jared Feather (IFBB Pro bodybuilder and exercise physiologist) The Big Questions: 1. When is superintelligence coming? 2. Does AI actually understand anything? 3. The Simulation Debate (The Spiciest Part) Tim says a simulation of fire doesn't get hot. They go back and forth on whether you could upload your mind to a computer — Mike says yes, Tim says absolutely not. 4. Will AI kill us all? (The Doomer Debate) Mike thinks the "AI will exterminate humanity" crowd has it backwards. His argument: any system smart enough to wage war is smart enough to realize cooperation is the winning strategy. Super-intelligent AI would want to *study* us, not destroy us. He uses the raccoon analogy to explain what agency really means. 5. What happens to human jobs and purpose? 6. Do we need suffering? In a surprisingly emotional moment, Tim asks if suffering gives life meaning. Mike's answer? "Fuck no. Desperately" Mikes channel: https://www.youtube.com/channel/UCfQgsKhHjSyRLOp9mnffqVg RESCRIPT INTERACTIVE PLAYER: https://app.rescript.info/public/share/GVMUXHCqctPkXH8WcYtufFG7FQcdJew_RL_MLgMKU1U --- TIMESTAMPS: 00:00:00 Introduction & Workout Demo 00:04:15 ASI Timelines & Definitions 00:10:24 The Embodiment Debate 00:18:28 Neutrinos & Abstract Knowledge 00:25:56 Can AI Learn From YouTube? 00:31:25 Diversity of Intelligence 00:36:00 AI Slop & Understanding 00:45:18 The Simulation Argument: Fire & Water 00:58:36 Consciousness & Zombies 01:04:30 Do Reasoning Models Actually Reason? 01:12:00 The Live Learning Problem 01:19:15 Superintelligence & Benevolence 01:28:59 What is True Agency? 01:37:20 Game Theory & The "Kill All Humans" Fallacy 01:48:05 Regulation & The China Factor 01:55:52 Mind Uploading & The Future of Love 02:04:41 Economics of ASI: Will We Be Useless? 02:13:35 The Matrix & The Value of Suffering 02:17:30 Transhumanism & Inequality 02:21:28 Debrief: AI Medical Advice & Final Thoughts --- REFERENCES: Paper: [00:10:45] Alchemy and Artificial Intelligence (Dreyfus) https://www.rand.org/content/dam/rand/pubs/papers/2006/P3244.pdf [00:10:55] The Chinese Room Argument (John Searle) https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf [00:11:05] The Symbol Grounding Problem (Stephen Harnad) https://arxiv.org/html/cs/9906002 [00:23:00] Attention Is All You Need https://arxiv.org/abs/1706.03762 [00:45:00] GPT-4 Technical Report https://arxiv.org/abs/2303.08774 [01:45:00] Anthropic Agentic Misalignment Paper https://www.anthropic.com/research/agentic-misalignment [02:17:45] Retatrutide https://pubmed.ncbi.nlm.nih.gov/37366315/ Organization: [00:15:50] CERN https://home.cern/ [01:05:00] METR Long Horizon Evaluations https://evaluations.metr.org/ MLST Episode: [00:23:10] MLST: Llion Jones - Inventors' Remorse https://www.youtube.com/watch?v=DtePicx_kFY [00:50:30] MLST: Blaise Agüera y Arcas Interview https://www.youtube.com/watch?v=rMSEqJ_4EBk [01:10:00] MLST: David Krakauer https://www.youtube.com/watch?v=dY46YsGWMIc Event: [00:23:40] ARC Prize/Challenge https://arcprize.org/ Book: [00:24:45] The Brain Abstracted https://www.amazon.com/Brain-Abstracted-Simplification-Philosophy-Neuroscience/dp/0262548046 [00:47:55] Pamela McCorduck https://www.amazon.com/Machines-Who-Think-Artificial-Intelligence/dp/1568812051 [01:23:15] The Singularity Is Nearer (Ray Kurzweil) https://www.amazon.com/Singularity-Nearer-Ray-Kurzweil-ebook/dp/B08Y6FYJVY [01:27:35] A Fire Upon The Deep (Vernor Vinge) https://www.amazon.com/Fire-Upon-Deep-S-F-MASTERWORKS-ebook/dp/B00AVUMIZE/ [02:04:50] Deep Utopia (Nick Bostrom) https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642 [02:05:00] Technofeudalism (Yanis Varoufakis) https://www.amazon.com/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1685891241 Visual Context Needed: [00:29:40] AT-AT Walker (Star Wars) https://starwars.fandom.com/wiki/All_Terrain_Armored_Transport Person: [00:33:15] Andrej Karpathy https://karpathy.ai/ Video: [01:40:00] Mike Israetel vs Liron Shapira AI Doom Debate https://www.youtube.com/watch?v=RaDWSPMdM4o Company: [02:26:30] Examine.com https://examine.com/