What Do LLMs Tell Us About the Nature of Language—And Ourselves? - Ep. 23 with Robin Sloan

[00:00] I just remember all these different sentences floating in this amorphous cloud that had some meaning to it. The idea that language could get mapped into math in this way was just so freaking cool. I just found it so evocative. What language models are is language itself, given its first dose of autonomy. It's like you rip language out of our head and it starts walking around like wind-up toys. They're such amazing artifacts. [00:30] models, even just as they exist today in our real world, they don't have to do anything. Just as artifacts, what's woven into them and bound up into them is just, it's so weird and so interesting. [00:53] Robyn, welcome to the show. Hey, Dan. Good to be here. Good to have you. So for people who don't know, you are the best-selling author of Mr. Penumbra's 24-hour bookstore, Sourdough. And now... [01:06] your latest book, Moonbound, which comes out next week. It will be out by the time this show comes out. I've read the entire thing. It's really good. I'm so excited to have you to talk about it. I'm excited to be here. Yeah, you should add to my bio. Robinson is also a Dan Shipper super fan and listener and reader. So it's great to be here. I love to hear that. So we did an interview like four years ago or three years ago, which is honestly one of my favorite interviews I've ever

[01:36] with Robin Sloan, which I thought was like the coolest title ever. And like one of the things that came out of that interview is like just this sort of like vast set of notes that you keep. [01:49] And basically just like anything that comes to your comes into your periphery that like has what you said, like a specific taste that like has that feel for you, you've saved. And I got to tell you, and you said the feel is ineffable. I got to tell you, like I have a Apple notes called my [02:06] That is like, I've been keeping this for four years. It's got so much stuff in it. And it's because of you. So I really, yeah, it really changed my life. [02:14] The great thing about keeping notes so assiduously and trying to cultivate a sense for that stuff that just appeals to you in that hard to describe way is you're essentially writing the perfect blog for yourself. And so you find that going back through it, you're like, wow, amazing. All of these things are incredibly interesting to me. And now I'd like to pursue them and follow up. And this is great. It's great. [02:44] incredible so what i want to talk to you about today uh is your new book moonbound um and i'll just try to like summarize it a bit for people or without spoiling it obviously just so we have a little bit of like context but basically it's like this mashup it's got notes of like sci-fi and fantasy it's got like ursula k i guess i feel some ursula k leguin in it it's got king arthur it's got like studio ghibli like it's all this it's got all this stuff it's really cool

[03:14] 11,000 years in the future, and he's run into this downed spaceship, but also there are knights and swords and there's futuristic technology. So you're like, what's going on here? And then it all unravels in this really interesting way. And there's a lot in it that reminds me of language models. [03:37] And I think was inspired by you thinking about language models and high dimensional spaces and all that kind of stuff. So that's why we're talking about it today. And the place I wanted to start is I knew that I think you got started working on this book because you wanted to write about AI. Sorry, you wanted to write with AI. So you're going to write it with AI and then you... [03:59] You didn't do that, but you ended up writing it about AI. So tell us about that process for you. [04:04] Yeah, you know, such a great example, and there's no shortage of these of like the best laid plans. They're just you never go down the path you think you're gonna go back, go down. But, um, [04:14] Sometimes that's all for the best. Yeah, I can probably frame it up best by rewinding in time quite a ways, actually. It's kind of shocking when you realize that I started tinkering with this stuff. And by this stuff, I mean language models, early forms of them way back in 2016, 2017.

[04:44] a long way off from the stuff we have access to today. I mean, this was a moment when people were feeding in, if you can imagine the entire corpus of Shakespeare and generating, you [04:55] cruddy fake Shakespeare, which is, you know, it's not, it's no longer impressive. At the time, it was impressive, and it was new, and it was exciting. And so I got plugged into this stuff way back then. And I have to say that actually, one of the really appealing things about [05:11] that era of models that generation that kind of point the technology was there um was for me as a writer was twofold one [05:20] The output actually was really weird. It wasn't as fluent as a Claude or a GPT-4. It was pretty messed up. But for aesthetic purposes, almost for poetic purposes, that was really interesting. The idea that it wrote in kind of a broken, weird, inhuman way that I am your human would never imagine to write. So that was one thing that was very interesting. [05:42] And the other thing that was interesting was the fact that back then, the scale of everything was so much smaller that one of your big considerations could be, what will I train this on? Now, today, to ask that question, you have to be a multi-jillion dollar lab or big tech company to be like, oh, well, I guess I'll download the entire internet and make five copies and get started. [06:12] classic public domain. [06:14] fantasy and science fiction and training these little models up on that.

[06:18] And the output to me was really interesting. Again, it was kind of screwed up and kind of weird, but evocative and surprising and all these other things. Okay. [06:26] That's a lot of backstory, but that's just all to say that led me down this path of experimentation. I made, I don't know if it was actually the first one in the world, but it was for sure one of a handful of the first text editors where you could write alongside an AI model. And again, these are my cruddy primitive AI models, but I could start a sentence on a dark and stormy night, dot, dot, dot. [06:48] and hit tab. [06:49] and this model would spin up and complete the sentence for me. I got really excited by this, as I think a lot of people enmeshed in this stuff did around that time, 2016, 2017, 2018. [07:01] And yeah, my goal, this is, I actually didn't even have the idea for the story that would become Moonbound at that time, in a way. [07:09] Maybe this is a bit of a warning sign. I was starting with process. I was starting with this idea that I wanted to both develop and then use these tools to write in a new way. Anyway, long story short, I worked on that for actually quite a few years. [07:39] was for me actually not very much fun. And certainly it didn't produce results at the level that I needed it to produce. It just wasn't, frankly, it wasn't up to spec. So that was one discovery. The other discovery though, is that the actual...

[07:52] machines, like the stuff, the code, not their output, but the language models themselves and the math that made them go and the code that kind of wove them all together was super duper interesting. And I actually just found myself almost compulsively tinkering rather than writing, you know, kind of procrastinating because that work was so interesting. So, [08:14] what happens? Moonbound, you know, out now, does not have a scrap of AI written text in it, but it's packed full of these ideas and actually some of these feelings that I gleaned from all that time spent with this technology. That's really interesting. I have so many different things I want to ask you about that. But the thing that's coming into my mind is like, yeah, [08:44] mean to you and how did it change how you think about [08:48] in the world. [08:49] Yeah, you know, there was one early on, and it's kind of off to one side from the main line of development of the language models. And there was this phenomenal project that some researchers at Stanford did. They've now since gone off to, you know, professorships at Columbia and other places. But they took sentences, like a huge corpus, again, huge for the time, corpus of sentences. And so it wasn't a language model. It wasn't trained on that generation task of sort of, you know, okay, I say this, you say that. [09:19] Instead, they just wanted to take those sentences and pack them into an embedding space. And I think a lot of people who use these models kind of maybe know what that is. If not, we can address that separately. But suffice it to say, they wanted to take these sentences and pack them into the space and set it up in such a way so that you could actually move between the sentences in a way that was sort of sensible. So you could start with a sentence like, you know...

[09:43] It was hot outside. And then you'd have another sentence that was like, you know, the dog barked at the moon or something like that. And you could actually crossfade between the sentences. And what the operation meant is you're literally moving through this high dimensional space, you know, with about 1000 coordinates in their case. And I just remember... [10:02] For me, again, those spaces and those long lists of coordinates, they're part of the generative language models too. But in this particular case, imagining all these different sentences kind of floating in this amorphous cloud that had some meaning to it. Like the idea that language could get mapped into math in this way was just so freaking cool. And I just found it so... [10:26] again, almost in a poetical sense, like evocative and provocative. And I just wanted to keep thinking about that. [10:34] Yeah. And so for people who aren't fully up on the embedding spaces... Yeah, I know. We can dive into that too. We can do a little tutorial maybe. Yeah. Basically, what you're talking about is we've figured out ways to, given a set of sentences or a sequence of text, to basically map that text on a map where things that are closer together are closer in meaning. [11:04] has a certain kind of meaning. So like, uh, there, you know, in your book, there's actually a, uh, there are people who are swimming through a many dimensional space, which I really love. Um, it's very cool. And, and one of the, one of the dimensions is bagel-ness. Um, so like the more you go in the bagel-ness direction, the more bagely it is. And this is like, this is, it's in the book, but it's also real. Like, um, it's real. Yeah. Like Anthropic has this whole new feature, feature paper where they, um, you know, they pulled out all the features in inside language

[11:34] of... [11:35] uh you know they have gold the golden gate bridge feature and you can like tune it to like always activate golden the golden gate bridge um which i i really love i think that's i think that's really cool yeah to give you a sense of the of the real the vintage here you know sometimes i feel like i just got into this stuff too early and i i basically did all dell of my experience and i completely burned out and then promptly you know the field the field exploded into stunning success but it might have been 2017 2018 where in this same little office where i'm talking to you [12:05] that sentence space that I created. And I plowed all these science fiction sentences into the space. And I basically... [12:12] created a whole list of these runs in which for each dimension, not all of them, not all thousand, but I think I picked about 60. All the coordinates were held constant except for one. And we just moved through that dimension, printing out sentences as we went. And so it was, I mean, it looked like experimental poetry. There are these printouts of these just absolutely wonky sentences. Some of them were gibberish, actually. Some of them didn't actually make sense grammatically. But the idea was we were going to identify what the dimensions meant. I was looking [12:42] you know, irony and or sincerity or descriptiveness or whatever. It did not work. But we did, we could, we, we could, we did identify that a couple dimensions obviously had to do with sentence length, kind of the most basic boring thing you can imagine. But the rest, we were looking at these lists of these transformations, and we kind of went, [13:03] I don't know, man. So I find it, I do actually find it a little bit, I don't know, reassuring or satisfying that it took until now, it took until 2024 for like the leading AI labs to find ways to interpret these features in these dimensions. Cause I couldn't do it, but I tried, but I tried. Yeah. No, I've seen some, I've seen people have demos, like my friend Linus, Linus Lee, who I interviewed on the show, like has one of those where you can sort of like scrub,

[13:33] really concise to being really long as one dimension. But then he has other ones that are weirder, like being more about space versus less about space. And that kind of stuff is really cool. I'm sort of interested in this. It seems like you were so excited about it. You got so into it. It seems like you burned out a little bit. But now there's this resurgence. [13:56] And what do you think was not... [13:59] like quite working for you about it because like I think um there's a lot of reasons why people end up like writing off this technology and um usually like they're just [14:10] a lot of them are not super curious about it, but you are super curious about it, but you're also like, I'm not really using it that much. So tell me about that. Yeah. Well, and specifically in the creative writing context, we can kind of restrict our focus to that because that's where I was most focused and kind of obsessed, lightly obsessed for several years. And also where I sort of, at the end, kind of had to close the book and say, this is not going to work for me. And I would say that it had to do, [14:40] to a project I worked on maybe a couple years ago now with a great Google model, not as high-end as their latest, but it was a very capable model called Lambda. And they, to their credit, had done some amazing work to wire it up into this writing interface. Super cool, fluent. I mean, it just made it so... [15:02] potentially interesting and powerful to be able to work with text and have the AI do these completions, and you could guide it in all these different ways.

[15:12] And so they had signed up several writers to test it out. And they were going to publish their short stories, whatever emerged from this engagement in a little online anthology. And this, for me, kind of was the test and the real kind of turning point. Because I was like, all right, maybe my stuff was all crap. You know, maybe my stuff, it was just this, you know, not good enough, actually. And now we've got these super capable models, this amazing interface. Let's try this for real. [15:42] trick of sort of [15:43] fitting into a style and a mode and parroting back, oh, it's a murder mystery. Oh, it's high fantasy. Oh, it's a business memo, whatever. It's really impressive. And especially when you kind of squint and say, oh, wow, I can't believe it can do that. [15:58] It's really impressive. [16:00] When you are working at the level of like, I would like to think fairly high-end fictional composition, [16:08] You see that it's always close, but never quite exactly right. And that has to do with kind of intention. Like when I'm writing something, for instance, I was writing this story, kind of a, I don't know how to, I see this is the thing, I actually don't know exactly how to say what I was doing. So [16:24] It's in my head and I know what it is when it comes out. [16:26] in the words. But the point is, if you can write, oh, it's a classic sci-fi pulp fantasy, it actually means that's not worth writing because you want to write something that only the work itself can describe. But even so, I had this text going and I would say, okay, your turn, Google AI model. And it put in something. I just was like, no, I'm not.

[16:48] you don't get what I'm doing here. You know, it was, it was obviously it was grammatically correct. It was fluent. It was, [16:54] Fine. [16:55] But it wasn't great. And, you know, it's, boy, it's hard enough to make, it's hard enough to make a piece of writing work. [17:02] work and make it worth publishing when everything is great. I mean, that's not the goal. That's the [17:10] beginning. That's the starting line to make it all great. And so in the end, I just was like, I got to do this myself. And that was interesting. And to see, my diagnosis is that there actually is a reason for that. And that has to do with the fact that [17:23] All these language models are essentially generating text from inside a distribution, a distribution of contents, this fuzzy cloud. I don't know what the most generic phrase in all of language is, or hello there, whatever it is. That's it. Obviously, that's the supernova hot center of this cloud, and then it goes out and out, and they... [17:43] they cover the statistical terrain. And I think the truth is good, really, really, really good writing is way out at the edge of that. [17:50] of that probability cloud, that distribution of content. And I mean, I think truly good writing actually pushes a bit beyond it. It's the stuff that expands the frontier of what we thought could be written. And that's precisely where language models are the weakest. So there you go. [18:05] That's really interesting. Have you tried, I don't know, either prompt tuning something like Claude, which I found to be quite good at changing its voice to your specifications or even fine tuning some of the more frontier models of today on that science fiction corpus or anything like that?

[18:35] It's just intoxical. [18:36] Let's just say that I wanted to find, let's say I had a list of my favorite [18:40] 30 authors and I don't know that I would do this. There's a lot of questions built into this, but let's just say I decided to proceed. My favorite 30 authors, I had the full text of all their stuff and I was like, I want the ultimate voice, right? I want it to reflect all those. Now, that's not enough to train a model from scratch, as we know, that's simply not enough data. It would be, it's paltry. [19:00] And so, as you say, what you have to do is you have to fine-tune one of these incredibly capable supermodels that have been trained on, like everything ever written. And for me, I actually... [19:11] I'm quite uneasy about the knowledge that even though it's been fine-tuned on this stuff that I provided, all that other content is still there lurking in its training. I mean, the wild thing to me about all these corpuses, like any corp, I should say corpora, all these corpora, any corpus in the year 2024, by definition, it's a artifact that cannot be read by a person. [19:41] I mean, it's at a scale that's only computational. And so even the [19:46] the makers, the custodians of these models, obviously, they can spot check, they can write other computer programs, they could even use other AI models to sort of filter or sort or select or evaluate these huge bodies of data. [19:59] But fundamentally, they don't know what's in there. And I don't know. Maybe that's OK for a helpful virtual assistant. Maybe it's not. For my purposes, the idea that these are going to be thoughts, and feelings, and ideas that are going to come out in fiction,

[20:14] That not knowing really, really makes me uneasy. I don't know. [20:19] One of the things that I really, I think is interesting about your work and what you touched on with me in our last interview is that like you make content, but you also think a lot about the container within which the content comes. And how much of this do you think like, [20:33] is a problem of sort of [20:36] shunting a new way to make content into an old form and how much of it is like a, if there's a, if there was a different container for this, it would be a lot more useful. [20:47] Yeah, I mean, what that makes me think of is, of course, some sort of hyper book or living book, right, where you say, yeah, instead of I'm not going to use a language model to bake out a bunch of text that I think is Robin level, whatever that is. I'm going to, yeah, I'm going to have it. There's dimension for that, I bet. Yeah, for sure. Robin-ness. Yeah, yeah, for sure. [21:08] Max it out. Full Robin. It goes up to 11. Yeah. And so I mean, I think that's a really interesting. [21:17] thing to imagine. [21:18] And... [21:20] But I am still, I have to confess, I'm sort of stuck at an issue that I had, and I felt, I worried about this five years ago, six years ago, and I would still, if I was someone who [21:32] building any kind of AI powered thing, you know, including a cool hyper book of the future, I would feel so uncomfortable. [21:42] putting people in front of a product or a artifact or whatever you call it, where on some fundamental level, I did not know what it was going to say. And like, I'm it, to me, this is going to sound very,

[21:54] silly or funny or, I don't know, naive. But I'm actually surprised that these big companies [22:03] are comfortable [22:04] have been comfortable releasing these systems to the world with that fundamental [22:10] Dan Levy: uncertainness in front of them. And now, obviously, I know they've done a ton of work to put in these guardrails and these filters and I, you know, some people would argue that they've done too much. But for me, and I guess the truth is, this just says more about me than it does about AI or tech companies or society or anything. [22:28] I would just, you know, if I got an email from someone saying, oh, hey, I spent some time with your hyper book and yeah, you know, look, it showed me the scene and. [22:37] Isn't that kind of disturbing? And if I read it and it was disturbing, I don't know, I'd shut it down. I just I would not be comfortable with that. So so that's a real that's a real question to answer or dilemma to. [22:50] to kind of worm your way through, I think. That's really interesting. I like for me, I'm like, that's actually like, I obviously don't want to like, [22:57] serve disturbing stuff to people, but it is sort of like exciting in a weird way. Like you're creating this like living thing where you can't live. Yeah, I understand that. You know, I understand that. Yeah. Yeah. And again, though, you know, that's actually where it gets extra interesting. And in a way that our last two kind of threads, I think, come back together and they tie together. [23:15] If there was a system where I was able to say, listen, this is all... [23:19] public, I know what's in it. I know what went in. And now it's going to operate in this living, organic, unpredictable way. To me, I would be more comfortable with that. I'd be a lot more comfortable with that. It is, I think, precisely the combination of these big, you know, sort of, just, there's such amazing artifacts. Again, you could so profitably spend all your time, so much time

[23:46] thinking about, dreaming about, writing science fiction stories about language models, even just as they exist today in our real world without ever actually, like, they don't have to do anything, just as artifacts, like what they are and what's woven into them and bound up into them is just, it's so weird. And so interesting. But that very thing, I, again, like, I don't actually know, and nobody knows what's in Claude or GPT-4. And so that's, I'm just like, I don't know, I wouldn't [24:16] I wouldn't put my name on the thing. [24:18] that [24:19] that that [24:20] you know, shoots that output at unsuspecting people. You want like nutrition facts, like ethically raised organic AI. I mean, I think that's coming. I think something like some version of that is coming. I think that's great. I think it's really cool. I mean, you kind of see this already with like, you know, Adobe has the like, we don't train on copyrighted material stuff for their image models. Like it's that's the first kind of like salvo in that. [24:44] Yeah. War. [24:46] I want to get into the book a little bit. [24:49] And I know like [24:50] I know for you, like the idea that books are not summarizable is like a sort of like this key thing. So I don't want to like I don't want to reduce it too much. I mean, no, they're not summarized. They're not summarizable. But but one must always try. I mean, yeah, well, yeah, I think it's like the full thing is not summarizable, but the summaries are like a pointer to things that you to unsummarizable things that you might find interesting. And so you have to have the pointers because we have limited attention.

[25:20] So one of the ways I could summarize this book, or one of the, it seems like a key idea that's like woven through it is, it's sort of like about stories and about how stories shape our reality. But also maybe how like we can make decisions that author our own stories. We don't have to necessarily follow the like, you know. [25:40] prescriptions of the story that we're already a part of, which seems really interesting and also sort of related to Next Token Prediction. And I don't know if I'm reading into it too much, but I'm just really curious. Tell me more about that. That's great. Actually, of course. So first of all, I should say, the book is newly released. And so I haven't had that many conversations about it. So this is really fun, actually, to just engage with someone else who's actually read [26:10] the idea is kind of bouncing around in other minds. [26:12] I will say that particular association, the idea of sort of... [26:16] you know, proceeding in a archetypal or maybe even stereotypical story form and, you know, [26:23] producing the next token in a sequence was not consciously in my mind, but I do think it totally checks out, totally checks out. I mean, that's what we're talking about, about literally these sort of patterns and templates that get used and reused in so many different ways that the language models are so good at. And I guess I want to try to find a way to say this without spoiling it, but in the course of this book, as we get to know some of these AI entities a little bit better,

[26:53] Thank you. [26:54] incredible, extreme importance. And in fact, and I think this is true in our real world and in our real language models, they are kind of woven into the systems at a really foundational level, only because they are [27:07] repeated so often and so powerfully throughout, you know, their, [27:11] their training corpora. And are you saying something about stories in relation to language models or do you feel like maybe not you're not necessarily saying it in the book but like do you feel something about stories in relation to language models and yeah. [27:24] Yeah, I do. And, you know, I think this is I want to not make too bold. You know, I don't want to make claims that are too bold because the truth is, I don't know. I don't know what's going on inside these labs these days. I understand the world of the kind of open research world of of the late 2010s a lot better. [27:43] And I think this probably is changing now. Years ago, [27:46] you know, when you thought about what was available in the world to scrape and collect and feed into your hungry language model, [27:55] it was things with narrative. It was stuff with story. And that doesn't have to mean like fairy tales and, you know, epic quests, by the way, and myths and legends. It can just mean news articles. You know, news articles have a real appetite for cause and effect. Even if something is fundamentally inexplicable or random or, you know, tragic and chaotic, a news treatment wants to say, and it's just because it's a human thing. They want to say, well, this probably happened because

[28:25] I don't have an answer to this. I don't have a... [28:27] really, or even a theory. But I think it's interesting to speculate about how these models and their [28:34] their kind of constitutions, their intuitions about language have changed with the introduction of so much code. You know, I've heard it speculated that it was the addition of huge amounts of code to these training sets around the time of sort of a GPT-3 that actually... [28:51] affected their language use in a lot of ways, because code, of course, is so structured and so if then and consequential. And maybe the thing that we imagine is reasoning in language models today comes in large part from what they've learned from code. If that's the case, I think [29:10] suddenly you're like, is code a story? I don't know. Maybe it's a very linear story. So my sense of what [29:16] is in the hearts of the language models might be [29:19] a little different today in the in the fullness of their of what they're learning from in 2024 but i can say with confidence um that in that era of like [29:28] Project Gutenberg and all the news stories on the web, yes, there is a bias. There's a bias in that kind of language model towards [29:38] if then cause and effect and the rhythm of a story. And that's pretty interesting to notice, because, you know, there are stories in the world, but there's a lot in the world that's not a story either. And so a system with that bias, you know, I think you have to be really careful. [29:51] Totally. I think you're 100% right. I've heard that too, that even if you're training a model that you don't need, you don't want to code or you don't need it to code, you train it on code because it just makes it smarter, which I think is like- And isn't that just so interesting that, and that had, this is a slightly further afield, but-

[30:08] And that has made me reflect on the fact that code, we think of it as a machine language. Of course, code is not. [30:14] for the benefit of machines. Machines have their own... A machine has no use for Python or Ruby. That's not the language they speak. Everybody knows this. They get compiled into the real language of machines. [30:26] Really, code is for human benefit. It's a bridge. It's a way for us to sort of think in a more machine way. And we express that in these linguistic terms. I mean, again, you could just... [30:39] You could think about this stuff so profitably and so with such just delight for [30:43] forever. It's great. Totally. There's tons of depth there. It makes me very, very excited. One of the other things that I saw in the book, this happens early on, but it also becomes more relevant later. And you say it outright, the central question of the human race is what happens next. And when I read that, I was just like, because we've been thinking about rebranding every or repositioning it. And we were like, how should we reposition it? And we came up with [31:13] what comes next as like we answer that question. And so when I saw that, I immediately texted our editor-in-chief, Kate, and I was like, oh my God, look at this, look at this. So I feel like we're kind of on the same page there. But like, tell me what that means to you and where you came to that from. Yeah. I mean, that's really close to my heart. In fact, it's maybe the most close to my heart. And in fact, of course, in the story, as you know, having read the book, the narrator, who is not human, the narrator is sort of a kind of a hybrid, organic,

[31:41] technological creation who exists to chronicle human endeavors. And, um, [31:47] And the chronicler says in the page that the question carved into my heart, the irresistible question is, what happens next? [31:55] In that case, this character is speaking for me. That is the great question for me. I don't actually know that it is intrinsically a human universal. I think there are different humans and different human cultures that [32:07] kind of care about it differently. I think there's probably some people that are very, they either don't care or they'd rather not find out. And that's fine. And that's fine. I don't personally attach a huge, like deep value judgment to it. [32:21] But I do cop to my own hunger for the question. For me, it's constitutional. [32:26] And that's one of the reasons I am such a huge reader of science fiction. You know, science fiction doesn't tell you the answer, but it, it, [32:34] So it suggests possible answers. And that's why I just find them so delicious to read. Yeah. And this is, I'm sorry to make it a little heavy, but when I personally think about death, at the end of a life, of course, I think about things like suffering and disappointment and sickness and all that kind of stuff. And I think about leaving behind things that I like and people that I love. I also think about just not, [33:00] Learning what happens next, you know, if, if death, if, if through some, you know, if, if we lived in the, some sort of magical world where, you know, an entity, an angelic entity visited you in the last moments of your life and just, just spoiled it.

[33:13] just told you like, okay, well, you did a good job. You lived a good life. [33:17] Here's how it ends. Here's how it all goes down. It would, for me, for me, that would, it would soften the blow. It really would. That's really interesting. I mean, I think it's, it's definitely central. Like, [33:28] I do agree that certain, maybe certain cultures or individuals like don't have that as their, as their sort of the center of their lives, but like, [33:36] The fact that like Buddhism exists and the entire whole, the whole thing is like, just like being the present. And like, that's so hard for people, like sort of says to me that there's something like pretty, pretty hardwired about our brains. It's like, what's going to happen now? [34:06] all about really great, super, super interesting, rich kind of next level, next generation biology. And one of the things they keep coming back to there is that even on the level of a cell, a dumb little cell, a little microbe, there is kind of that question of what happens next. And life in a way [34:23] thrives [34:25] to the degree that it has a pretty good simulation of what might happen next, because that allows it to do things like plan and react to possible dangers and all the other things you can imagine life doing. [34:38] So that's kind of like the micro scale, what happens next. But then there's also, you know, there's the Dan scale, what happens next of like 10 years. And then there's the sci-fi scale of...

[34:47] 10,000 years. And they're all interesting, super interesting. And not to make everything about AI, but like what strikes me is like that answering that question is the thing that forces these models to bootstrap all of the knowledge that they have is to be like, okay, what comes next? Like answering that over and over and over again, trillions of times, which that's really interesting. Like, I don't know if I would have thought that that would be the way that they would get smart, you know? Yeah, that's, you know, honestly, that is a great observation. [35:17] the sense that a question, you know, in the same way that like evolutionary, the dynamics of kind of natural selection evolution are so simple, and yet they make... [35:28] the rhinoceros and the anchovy and the Joshua tree and Robin and Dan, you know, it's like, okay, I guess it doesn't take much if you give it enough time. And likewise, I don't actually think that that question is exactly evolutionary. I think it's something a little different. [35:43] But I actually really, I think it's quite elegant to imagine that a challenge so simple could in its fullness require a, [35:51] really a lot of complexity and a rich picture of the world. I mean, I always take pains to say still an incredibly incomplete picture of the world. I mean, it's kind of, I sometimes think like, oh man, these language models. [36:03] they just think it's all text out here. [36:06] You poor bastards, you know? There's more than text. And, you know, slowly but surely, and I think the multimodal models are... [36:15] Obviously, it's interesting because they have such rich capabilities.

[36:18] I think they're interesting because it seems, I don't know, it seems like a better way to live. It seems like a better way to be in the world to be able to combine things. [36:26] different inputs and maybe find some common associations between them. [36:31] the idea of just living in a world of nothing but [36:34] tokens is, I don't know, seems kind of like hell to me. Are you starting to think of them as like beings that have that can live in hell? Like what's your, you know, no, not not yet. But again, I, you know, I hope I don't I don't I should maybe try to actually do some research on this. If there are not presently, like, [36:52] at minimum dozens of philosophers and cognitive scientists and ethicists kind of [36:58] thinking about this stuff around the world and maybe teaching introductory, you know, seminars about it and getting people together to start to have these conversations and ask these questions, then... [37:09] I don't know, the Academy is derelict in its duty, because these are really rich, interesting questions. For my part, no, I don't think these are... [37:18] beings yet. But, you know, if you ask me what my definition of a being is, or where I draw the dotted line around beings, I don't know. So... [37:29] So who knows? I mean, I think that the academy or philosophy question is really interesting because I studied philosophy in college. [37:35] um, [37:36] So philosophy is sort of like obsessed with definitions. So like, what is a being or whatever, and like trying to try to answer those questions and definitions obviously are useful for certain things but [37:47] It's always struck me as kind of interesting that like we've been debating definitions of things like knowledge for 2000 years and like we know a lot of stuff. We just don't have a definition for it. And and you would have thought that like.

[37:58] to build intelligent machines that had knowledge. [38:02] we would have to define it before we built them. And really, we just have to ask that one question over and over, like what comes next? And it just like bootstraps it, which I think is like a kind of interesting comment on like the project of philosophy that like, maybe these definitions are so high dimensional that like, you can't get it in a philosophy book. Absolutely. Absolutely. But I think that's great. I think that's totally great. Or, or in a sense, you know, again, you can kind of go around in circles and you could, you could have a whole successful, celebrated philosophy [38:32] just arguing with another philosopher over a definition using language the entire time, right? You're writing papers back and forth and you're denouncing each other in seminars and [38:45] And maybe it turned out that the thing all along was actually in all that language. The idea that you're going to compress it down to a word or a sentence is wrong. It's actually in that sort of that vast, that bulk of living language. And I say that because I don't know really how to assess this claim at all. But I personally, almost aesthetically, I'm quite taken with the claim that what language models are is wrong. [39:11] language itself, like literally the technology of language, [39:16] given its first dose of autonomy. So it's not just something that we deploy as we, as we want or need, but suddenly like, it's, it's like, it's like you rip language out of our heads and our society kind of set it up and like turn a crank on the side. And it starts kind of,

[39:32] walking around slowly and weirdly like one of those little wind up toys. But and that's, you know, again, we [39:39] people, you can argue about what's going on in language models. There is no disputing that when you talk with one, it sure seems like it's, [39:48] reasoning or thinking or deducing or just politely answering your question. [39:53] And so this argument goes, yes, that is just, that is the deduction and the reasoning and the politeness that was always inherent there in language. And we just never had this particular way of seeing it before. Again, I don't know, but I like it. I like the way it sounds. I think that's really cool. That's a beautiful, that's a beautiful metaphor. Is it? [40:12] is it the politeness that's always was always in language in the sense that like [40:17] politeness is like sort of built into our grammar and into our like [40:21] um syntax and all that kind of stuff or is it the politeness that's in humans that was like recorded into language i think both i think i think it's i think it's absolutely both i mean for sure it's it's the latter no question but then but then even the former i think it's just you know um [40:35] You know, again, language is not, all of human language is not, um, [40:39] some [40:41] It's not TCP/IP packets. It's people and they're writing letters or trying to entertain each other with books or trying to convince other people of things very often. And convincing is it takes some tricks and takes some patience or whatever. So yeah, I think it's just in terms of what language has always been for. Obviously, you can be very rude and you can be very impolite. But that in itself, that's only effective ever.

[41:10] because language is generally so... [41:12] The language understands that it's being used between people and you want to kind of get along. Yeah. I guess like on the subject of like language becoming an autonomous thing, one of the things that... [41:23] is sort of present throughout the book is like you're playing with this idea of like what an I is um an I as in yeah quote like the letter I like me I um you know you have the chronicler who's like I think you said it's a fungus right yeah yeah a fungus onto which much technology has been layered yeah exactly at great expense yeah yeah um you've got like Clovis as another character [41:53] around the world and they all are sort of connected, at least usually. And all have the same. And so rather than a sort of Borg hive mind or kind of like, "Oh, yes, the ants are all part of the same colony," it actually is Clovis, this wandering robot in all their instances is the same person and the same personality. Which maybe that sounds like a subtle distinction, but I think it's actually a pretty important one. [42:19] So you're playing around with all that all the time. What is it about that that is capturing you? Well, you spend any time with... [42:29] writing and I would say especially writing and then also reading across languages and I don't mean natively just in translation and reading about the translations and the process and you learn that even in the, you know, you don't even need science fiction for this, just within the scope of human language as it exists here and now today, there are so many different ways of thinking about

[42:53] I and the subject, the subject of anything. One very classic example is that in Japanese, the Japanese language, there's like a handful. There's not just I as there's in English. There's a handful of different I's, all of them with really interesting and usefully useful, narratively useful meanings. And so Japanese translators, like people who bring books from Japanese into English, they all complain about this because they're like, well, [43:20] it's impossible. There's no, there's no equivalent. Like there's, you just can't, we don't have other eyes we can slot in. And so fundamentally, [43:28] this shade of meaning, which is, again, really important, can be really, really useful and give something. It's like a little spin on the ball. [43:35] is essentially lost. And they try to find other ways to thread it in and create that feeling. [43:40] And I love that. And I just think it's like, and even now, you know, you think about humans and how we live in the world, I think already the I, the singular I is quite... [43:50] complicated. I think it has been complicated in the last 10 or 20 years, the era of the internet, because, you know, well, here I am, I, standing in a room. And at a certain time in human history, that meant I was just in this room. And I could only talk to people in this room. But here I am, [44:05] talking to you, seeing your face, you know, I am somewhere else in a very meaningful way. And we just still say I. [44:14] And I just feel like it's more complicated than that now. Our presences and our attention and our sense of kind of where and how we can act in the world is already sort of spreading out. And it's fun to think about how you could.

[44:27] reflect that, capture that and play with that in language. I think of that too in so many ways. There are so many, I don't know, internal family systems is like you have got multiple eyes within you. That's a sort of psychological model. Or just in meditation, like if you get deeper into it, you start to notice like [44:46] that you are a bit of a different self depending on the context you're in. And I think it's also why when you go through a breakup or someone dies or whatever, that's part of why you're sad. Because the I that is usually [44:59] like around that person, the eye that becomes [45:03] around that person is like not going to get activated anymore. Yeah, you're losing a part of yourself. I, you know, I can't say that I quite thought of it that way, but I think that's, I think that's absolutely true to which I would add, I mean, this is much less, um, I'm, I'm, [45:15] emotionally resonant but there's also the simple eye of your body systems you know you've got your gut and the microbes there which which we now know have desires of their own and are like blasting you know neurotransmitters up to be like do this do that um your organs i mean everything and these things all [45:32] They're not ancillary. They're not accessories to the pilot of your brain. Neuroscience and biology have definitively established that it's a big [45:43] interlocking committee. And so, yeah, what is the I? Is the I... [45:48] Does it include my gut? I think so. But yeah, it's great. It's weird. It's weird stuff to think about. Yeah, and I think it actually also relates a lot to language models in the sense that when you think about...

[45:59] embedding spaces and what they are and how language models end up predicting what comes next [46:05] Basically, in order to predict the next word in a sequence, they take the end word and then they figure out which version of that word is it. So like if we take the name Robin, like they'll just go to everything. [46:19] previous to Robin in that sequence and be like, which Robin are we talking about? And that could be like, there's hundreds of thousands or millions of different Robins, some of, but a thousand or 2000 of which are actually you, just you in different senses. [46:34] And I love that idea. I think that is so cool. It's like we created this dictionary of these very, very, very specific words that we all thought were one word, but it's really like we're using that same word in a million different ways. Yeah, it's really beautiful. [47:04] folks making and deploying these models, I think a lot of them are interested in these questions. They are not, however, the most urgent questions. Their most urgent questions have to do with the infrastructure and scaling these things up and turning them into businesses. And of course, I think the very direct safety questions about, are people going to use this to [47:24] to scam other people or to, you know, bring down governments or whatever. And, and there's just so much to dig into. And, and I really, I do hope, yeah, you've kind of brought this up in a, in a few different ways, a sense in which just seeing these little mechanisms operate,

[47:41] it it may how do I want to say this? Um, [47:45] It raises questions about longstanding projects. [47:48] You kind of go, well, ah, maybe I didn't need that after all. Or maybe we didn't need to do it that way. And those should be the kind of questions and realizations that kind of redirect things. [47:58] you know, streams of inquiry, I think. Totally. [48:01] Totally. So you've been working on this book for a while. It's coming out. [48:06] Yep. How does it feel? It feels good. I mean, it's, as always, a little dizzying to understand that it will soon be in other people's brains. You know, I said earlier, I've done a couple of these... [48:18] I've had a couple conversations ahead of time like this, and it sounds so silly, but truly, [48:23] in all cases, I have been sort of unprepared for other people to have actually thought about the book. I'm like, oh, right. Ah, you read it. And it's in your head. Oh, cool. Okay. This is weird. But, you know, that's, I mean, that's the goal. That's, that is so the, the dream and the, the magic. And I think, you know, I'm, of course, I'm such a book nerd and such a, such a, I don't know, book chauvinist, maybe that I just think, I think they, I think they do that more effectively [48:53] They literally get into people's heads because they have to. You know, you didn't just watch Moonbound on a screen. You enacted and kind of rehydrated the events and the meaning in your own language model inside your own head. And that is something really special. So I'm just excited for it to... [49:10] for that to happen a lot more in the coming weeks and months and hopefully years.

[49:13] I love it. Put me on team book chauvinism. The only kind of chauvinism I agree with. Yeah, you know, I will say very germane to our subject here. You know, I keep talking about how these scholars ought to be asking more questions about their work and their projects because of what we've learned. I apply that also to writers, fiction writers, and I, you know, I have. [49:34] been highly motivated to think about what we're doing and all this kind of stuff. I also have, I'm sort of nursing a pet theory. I don't know that it really has any evidence. So it's going to continue to be Robin's crackpot theory. But I think that anybody who's interested in language models should really pay attention to how they dream. [49:53] and what dreams are and how dreams feel. Because I think the, this is just subjective. I feel literally that the mechanism of dreaming is very similar to the mechanism of, [50:04] of a language model kind of saying, okay, well, that's weird, but I'm going to keep it going. I'm going to, I'm going to do the best I can to, to complete this, [50:12] this sequence, [50:13] And the reason I bring it up is that I think that's also very similar to the mechanism of a novel. My pitch for novels, fundamentally, is that they are packaged dreams. Usually you don't get to choose your dreams. They are weird or scary or surreal or boring or whatever, but you just get the dreams you get. I think this is one case where you get to sort of load a... [50:35] waking dream into your head. And so in that way, I think there's an interest. Some [50:40] not totally understood connection kind of trilateral connection between books and dreams and uh

[50:47] language models. I love that. Do you think that that implies that the true AGI is going to have a language model as its dream generator, and we just haven't built the AGI yet? Yeah, maybe, maybe so. It is, you know, I don't want to say too much because we're very close to spoiler territory here, but I will just say, suffice it to say that anyone who reads the book [51:17] during sleep is important to me. I actually do wonder, I don't know if you know this, but there's no living things that don't sleep. I mean, literally, it is- I didn't know that. Yeah, it's a fundamental part of every biological process. Do bacteria sleep? They do. They have a phase that is sort of, it's a clear sleep thing. Now, are they dreaming? [51:37] Well, maybe, but it's different dreams. Certainly, you go one notch up the scale of recognizable to us complexity, and you do see things that begin to look like dreams. And so that's, I think, really fun and interesting stuff to read about and learn about. And it's clear that the long-term flourishing through evolution of life on Earth has discovered... [52:03] This phase is really important. It's like, it's, it's not negotiable. And it makes me wonder, it makes me wonder if there is some, if we will determine that there's some analogical thing that, you know, that AI is really ought to do, either for their own long term success or health or who knows what.

[52:20] That's fascinating. Well, this was an incredible conversation. Very sci-fi. This is great. I'm delighted to be able to, you know, it's like as if the novel itself wasn't sci-fi enough. I feel like we just took it to a few further levels, which is very fun. We totally did, which I suspected was going to happen and I was very excited for. The book is Moonbound. It is out. When is it out? June 11th. [52:50] Thank you so much, Robin, for doing this. This is really an incredible conversation. Dan, this is great. Just a real treat. So let's do it again before too long. Sounds good. [53:20] at GPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat. [53:28] craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the spaceship. [53:36] So do yourself a favor, hit like, smash subscribe, and strap in for the ride of your life. [53:41] And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.