Looking to make your search, SAM Harris just a note to say that if your hearing this you are not currently honour subscriber feet, We here in the first part of this conversation in order to access full episodes of making sense. Podcast you'll need to subscribe, SAM Aristotle, work there, you'll find or private our says. We need to add to your favorite pot catcher, along with other subscriber, only content. We don't run ads on the past and therefore its may possible entirely through the support of our subscribers. So if you Don'T- we're doing here, please consider becoming one.

Okay. Before I jump in today, I want to take a moment to address some confusion that keeps coming up. I was on another podcast yesterday and spoke about this briefly, but I thought I might be a little more systematic here it relates to the paradoxical way that we value expertise in really all fields and scientific authority in particular. Saves me, there's just a lot of confusion about how this girl. His expertise and authority are unstable intrinsically so because the truth of any claim doesn't depend on the credentials of the past. in making that claim. So a nobel laureate can be wrong. Total ignoramus can be right, even if only by accident, so the truth

is orthogonal thoughts, the reputational differences among people and yet, generally speaking, we are right to be guided by experts and were right to be very sceptical. Of novices who claim to have overturned expert opinion, of course, for us right to be alert to the possibility of fraud, among so called experts. There are touted experts who are not who they seem to be. And were right to notice that bad incentives can corrupt the thinking of even the best experts. So these can seem like contradictions by their simply different moments in time I'm right with a career of reason has to pass through all these points, again and again and again we respect authority, and we also disavow its relevance by turns were guided by it until the moment we cease to be guided by it or until the moment when one authority supplants another or even a high

paradigm gets overturned, but all this gets very confusing when experts begin to fail us and when the institutions in which they function, like universities and scientific journals and public health organisations get content in about political ideologies that don't track the truth now, then many I, where I ve, talked about this problem from various angles, and I'm do many more because it's not going away, but much of our society has a very childish view of how to respond. To this problem. Many many people apparently believe that shot having more unfettered dialogue on social media and on palm casts. A newsletters is the answer, but it's not I'm not taken a position against free speech here. I am all for free speech, I'm taking a position

against weaponized misinformation and a contrarian attitude that nullifies the distinct. in between real knowledge, which can be hard won and ignorance or mere speculation, and am advocating a personal. thick of not pretending to know things. One doesn't know. My team risen I posted a few memes on instagram. These were things I said, I think on other people's podcasts, and these posts got a fair amount of crazed. Pushback. Apparently many people thought I was posting these memes myself as one I just left twitter only to become addicted to another social media platform, but any case my team posted these quotes and my corner of instagram promptly became as much of a cesspool, less twitter, and then people even took these instagram names and posted them back on twitter. So they could vilify me in that context

Needless to say, all of this convinces may again that my life is much better off of social media, but there is some real confusion at the bottom of the response, which I want to clarify So one of the offending instagram quotes read during the and dimmick. We witness the birth of a new religion of contrary aneurysm and conspiracy, thinking the first sacrament of which is quote. Do your own research. The problem is that very few people are qualified to do this. Research and self. Is society driven by strongly held unfounded opinions on everything from vaccine safety to the war in ukraine, and many people took offence to that as there was a statement of mere elite? but anyone who has followed this podcast, no Those that I include myself in that specific criticism. I

also unqualified to do the quote: research, the so many millions of people? Imagine there doing. I wasn't saying that I know everything about vaccine safety, or the worn ukraine. I'm saying that we need experts in those areas to tell us what is real or likely to be real and what's misinformation and this is why have declined to have certain debates on this podcast that many people have been urging me to have and even Leginn that it's a sign of hypocrisy or cowardice on my part that I won't have these debates. There are public health emergencies. And geo political emergencies that simply require trust in institutions. They require that we acknowledge the difference between informed expertise. And near speculation or amateurish sleuthing when our institutions and experts fail us that's not a moment to tear everything down

at the moment where we need to do the necessary work of making them trustworthy again- and I admit, many cases, is not clear how to do that at least not quickly detecting and nullifying. Bad incentives is a major part of the solution. But what isn't a part of the solution? At all is asking someone like canvas, owens or tucker karlsson. or even elon, musk or Joe rogan or brett weinstein's or me what we think about the safety of emigrants if vaccines or what we think about the nuclear risk posed by the war in ukraine, where our information ecosystem is so polluted and our trust in institutions so degraded again in many cases, for good reason that we have people who are obviously unqualified, to have strong opinions about ongoing emergencies dictate

what millions of people believe about those emergencies therefore dictated, whether we as a society can cooperate to solve them? Most people show and be doing their own research, and I say should blindly trust the first experts we meet if your face in a difficult medical decision get a second opinion, get a third opinion, but me people shouldn't be jumping on pub med and reading abstracts from medical journals depending on the topic this applies to me to so. The truth is, if I get camp I might do a little research, but I'm not going to pretend be an uncommon, the rash thing for me to do even more My background in science is to find the best on colleges. I can find and ask them what they think, of course, is true, any specific expert can be wrong or biased, and that's where you get second and third opinions has also.

I wish you'd be generally guided by scientific consensus, wherever a consensus exists- and this remains the practice, even when we know that there is an infinite number of things we don't know so well, because the last few years has created a lot of uncertainty and anxiety and given a lot of motivation to contrariness em in the world, a pocket. And news letters and twitter threads has exploded as an alternative to institutional sources of information. The truth is we can't do without a culture of real expertise, and we absolutely need the institutions that produce it and communicated and I say that as someone who lives and works and thrives entirely outside of these institutions, some, that defending my own nest, I'm simply noticing that sub stag and spotify and youtube and twitter are not substitutes for universities, and so

scientific journals and governmental organizations that we can trust and we have to stop acting like they might be. Now I got that off my chest now for today's podcast. Dance begin with Stuart Russell and gary marcus steward is a professor of computer science and a chair of engineering at the university of california, berkeley his affair of the american association for artificial intelligence, the association for computing machinery and the america. Association for the advancement of science and here author, with peter nor vague of the definitive textbook on a I artificial intelligence, modern approach and is also the author of the very accessible book on this topic? Human compatible, artificial intelligence and the problem of control Gary Marcus is also a leading voice on the topic of artificial intelligence. He is a scientist. that's an author and entrepreneur, he was found.

During ceo of geometric intelligence, a machine learning company there was acquired by and twenty sixteen and he's. Also the author of the recent book roy booting I, along with his co author earnest davis, and he also has a forthcoming podcast title humans- verses machines and there we talk about recent developments in a I chant jpg in particular, as well as the long term risks of producing artificial general intelligence. We discussed the limitations of deep learning the surprising power or of narrowly. I d ongoing indiscretions of chad, jpg a possible misinformation apocalypse, the problem of inside and creating human values, and I the business model of the internet, the matter digital provenance. Using ai to control ai the control problem, emergent goals.

locking down core values programme. Uncertainty about human values into asia, the prospects of slowing nor stopping a I progress and other topics found. It very interesting and useful conversation, a topic whose importance is growing by the hour, and I bring you Stuart Russell and gary mark as the. I am here, was Stuart Russell and gary marcus Stuart. Gary thanks for jointly thanks for having us. So I won't have properly introduced both of you in the the intro, but perhaps you can just briefly introduce yourselves as well gary. Let's start with you, you and your new to the park asked. How do you describe what it is you do and what you have the kind of problem.

you focused on I'm getting markets and I've been trying to figure out how we can get to a safe way. I future, I maybe not looking is far out is stewart is, but I am very interested in the immediate future whether we can trust the idea that we have how we might make it better so that we can trust it. I've been an entrepreneur, I've been an academic. I didn't coding since eight years old to throughout my life had been interested. Nay, I and also human cognition, and what human cognition might tell us about hey and how we might make better yet allowed you, you did your phd under our mutual friend Steven pinker and do you have a wonderful book, rebooted, nay I building artificial intelligence. We can trust and I'm told you have a coming podcast later this bring titled humans versus machines, which I am eagerly awaiting? So I'm I'm pretty excited about. That's gonna be fun nice and you have a voice for radio so

You're here take it, and I know that joan about I'll I'll, take it in good time. I started yoga is a face for radio is the joke? A voice or a voice for radio is a is high praise. That's right! Thank you stewart Who are you? What are you doing out there? so I I teach at berkeley. I've been doing. Ai for about forty seven years and I spent most of my career just trying to make a systems better and better working in pretty much. Every branch of field, and in the last ten years oh, I have been asking myself what happens if or if we, as a field succeed in what we ve been trying to do, which is to create a systems that are at least as general, in their intelligence

as human beings, and I came to the conclusion that if we did succeed, it might not be the best thing in the history of the human race. In fact it might be the worst, and so I'm trying to fix. That, if I can also add, you have also written renee, wonderful book, human compatible, artificial intelligence and the problem of control which is quite accessible than you have written, and accessible burger co written one literally, the textbook on I You ve been podcast a few times before so yet you you each occupied different points on a continue. of concern. About. Generally, I and Andy perhaps distant problem of super intelligence and stuart have always seen you on the these sober side, the worried end spoken to many other worried people on the pakistan and various events: people, ethnic boss from max tag, mark eleazer, caskey, toby, ord, spoken

may I people one private. I've always count myself among the worried and have been quite it only by you and your book, Gary I've I've seen you on me. They saw recital d they not worried and and have also spoken, people who are not worried like steve, pinker, David, deutsche roddy brooks and others I'm not sure if either of you have moved in, meaning years at all, Maybe we can, wouldn't you start there, but we're all gonna? we'll start with narrow way. I and Chad jpg ended the x closure of interest on that topic, but I do want is to get to concerns about where all this might be headed, but before we jump into the the narrow

the problem. Have you moved at all in your sense of of the risks here? There are a lot of things to worry about. I think I actually have moved just within the last month, a little bit so we'll probably disagree about the estimates of the long term risk. But something that really struck me in the last month is there's a reminder of how much we are at the mercy of the big tech companies. So my personal opinion is that were not very close to artificial general intelligence. Not just do it would really disagree, but he can jump in later on that and I continue to think we're not very close artificial general intelligence but with whatever it is that we have now this kind of approximating the challenges that we have now. This mimicry that we have now the lesson of the last month or tour We don't really know how to control, even that it's not full asia, I they can self improve itself. You know it's not sense

de I or anything like that, but we saw that microsoft clues internally, that the system was problematic, that a gas laser gas related it's customers and things like that. And then they rolled it out anyway, and then initially the press hyped. It me sound amazing, and then it came out that wasn't really so amazing, but also came out that, if microsoft wants to test something on three million people. They can go ahead and do that even without a clear understanding of the consequences. So my opinion Don't really have artificial general intelligence now, but this was kind of a dress rehearsal and it was really shaky dress rehearsal and that in itself, me a little bit worried and suppose we really did have hiv and we had no real regulation in place about how to test it you're my view as we should treat it as something like drug trials. You want to know about customers. Fats and have a slow release, but we don't have anything like regulation around that, and so that actually pushed me a little bit closer to. Maybe the worry side of the spectrum

not as worried may be stored is about the long term. Complete annihilation of the human race living in Stuart has recently get em it concerns about. Unless worried about that, because I dont see asia is having the motivation to do that. But I am here: about whether we you have any control over the things that were doing, whether the economic incentives are gonna to push us in the right place, so I think there's lots of things to be worried about. Maybe we're having this discussion about which those should be and how you prioritize them, but there definitely things to worry about Yeah. I want to return to that question of of motivation, which has always struck me as a red herring. So in will talk about that when we get to asia, I butter Stuart. Have you been pushed around at all by recent events or anything else, so what she dared dearer to recent events, one of them is, is checked, BP, tee, but another one which is much less widely design

needed. But there was an article in the financial times last week was finding out that the the superhuman go programmes that I think pretty much. Everyone had abdicated any notion of of Schuman superiority go completely. You know that was twenty seventeen in the five years since then, the machines have gone off into the stratosphere. Their ratings are fourteen hundred points higher than the human will champion, and fourteen hundred points in go or in chess is like the difference between professional and you do a five year old- is played for a few months so what's amazing is that we found out that actually an average it all good average human player can actually beat these superhuman go programmes because with them, be all of them, giving them a nine stone handicap which is the kind of handicapped, but you give to a small child, is learning the guy

Isn't the caviar there, though, that that we needed a computer to show us that exploit well. Let's see hear the story is a little bit more complicated. We we had an intuition that the go programs, because their circuits right, the the circuit, is a very bad representation for a recursively defined function. So what is it I been so in go the the main thing that matters is groups of stone, so are a group of stones or stones that are connected to each other by vertical and horizontal connections on the grid, and so that, by definition, is a is a recursive con said because I'm connected to another stone. If there's an adjacent stone to be in that stone is connected to the other stuff, and we I right that I can say in English. I just didn't is there a one all sentence? I can write it

programme. In a couple of lines of python, I could write it in informal logic in a couple of life but to try to write it as a circuit is in some real sense impossible. I can only do a finite approximation. We had this idea that actually the programmes didn't really understand. What would a group of stones is, and they didn't understand in particular whether a group or stones is going to live, we're going to die, and we concocted by hand some positions in which we thought that the just deciding whether the programme, either to rescue its group or whether it could capture the opponents group, that it would make a mistake Is it didn't understand, group, and that turned out to be right so that we can just

been for one second shore. It actually relates to the thing: that's on the cover of perceptions, which was one of the most famous books in the history of artificial general intelligence. There was a argument by minsk game paper that two layer perceptions which are the historical ancestors of the deep learning systems. We have now couldn't understand something three basic concepts and in a way which steward in his lab did, is a riff on that old idea. People hate that book in the machine learning feel they say that a prematurely dismissed, multi, They are networks and there is an argument there, but it's more complicated than people usually tell, but in any case I see there result as a descendant of that, showing that, even if you get These pattern recognition systems to work that they don't necessarily have a deep conceptual understanding of something as simple as a group in go in, and I think it's a profound connection to the history

hey. I in kind of disturbing that here we are in a fifty. Some years later we still struggling with the same problems yeah. I think it's it's the same point that minsk was making, which is expressive, power matters and simple, beset troms. Heavy incur the blue, limited expressive power, but even larger the deep networks and so on. in their native mood, they have very limited expressive power. Are you could actually take a recurrent? neural net and use that to implement a turing machine and then use that to implement a pilot, interpreter and then the system could learn all of its knowledge python, but There's, no evidence that anything like that is going on in the go programme so- The evidence seems to suggest that, actually there not very good

recognising what agrippias in life innocent and death, except in the cases in so they ve learned sort of multiple, fragmentary, partial finite approximation to the notion of a group and the notion of liveliness, and we just found that we could fool it where in aware, constructing groups that are somewhat more complex, a look and kindly typically show up, and then Sir is jerry said: sorry. Is that same? As you said there, there is a programme that we used to explore whether we could actually fine this occurring in a real game, because these were contrived positions, the whig by hand we couldn't we couldn't Fourth, the game to go in that direction and Indeed, when we started running this programme with cards, sort of an adversarial programme is just both to find ways of beating one particular go program called category

indeed it found ways of generating groups kind of like a circular sandwich. So you start with a little group of your pieces in the middle and then the programme, the computer program surrounds your pieces to prevent them from spreading and then use around that surrounding. So you make a kind of circular sandwich and it said He doesn't realise that its pieces are going to die because it does not. and what do you know? What is the structure of the groups and it has many opportunities to rescue the money paid, no attention, and then you capture, sixty pieces and has lost the game, and this was something that we saw that in our everyday, your programme doing, but then a human could look at, say. Ok, I can make that happen in a game, and so one of us, he members is a good go player and he played this against category, which is the best go programme in the world. beat it easily and beat it with a nine stone handicap. But also,

now that all the other go programmes which were trained by completely different teams using different methods and different networks structures and will arrest. They all have the same problem. They all failed to recognise the circular sandwich and lose all their pieces, so it seems to be not just an accident is not sort of a peculiar hack that we found for one particular programme. It seems to be a qualitative Failure of these networks to generalise properly and in that sense it somewhat similar to two adversarial images where we found that the systems that are supposedly superhuman recognising object are extremely vulnerable to tiny, making tiny tweaks in images that are those tweaks, a totally invisible to a human, but the sis changed his mind and says: oh, that's, not a school bus is an ostrich right and it's again a weakness in the way the circuits have learned to represent the concepts they haven't really

learn the visual concept of a school bus or an ostrich, because there are obviously for human, not confusing and this notion of expressing power is- is absolutely central to computer science. We use it all over the place when we talk about compilers who talk about the design of hardware if you use and inexpressive representation then, and you tried represent a given concept. You end up with an enormous and ridiculous. He over complicated representation of that concept, and that representation, the analyses, the rules of go in an expressive language python. That's a page in an annex as language like circuits. the million pages so too low, that million page representation of the rules of go requires. Billions of experience and the idea that oh well we'll just go

more data and we'll just build a bigger circuit and they will be able to learn the rules properly, that justice, not scale. The universe. Doesnt have enough data in it, and we cannot really does not material in the universe, to build a computer big enough to achieve general intelligence using these inexpressive representation? it's, so we wright right. I don't! I don't think without close to a I and I've never said eighty I was imminent. Generally, I don't. Ah So the question when do I think it's coming, but I am on the record because someone violated the off the record. was it? Let me someone, someone sliding scotch when they hit literally just broke it. I was at a chatham house rules meeting an ice literally reference. My sentence with off the record Twenty minutes later, it appears on the daily telegraph website. So

So I was at the daily telegraph. You could look it up. What I actually said was, I think, it's quite likely to happen in the lifetime of my children, right which you you could think of as as another way of like some time in this century, but before we get into that, can I jump into a sort of wrap up stewart's point, because I We with him. It was a profound result from his lap there's some people arguing about particular go programmes and so forth. I wrote an article about stuarts result. Called david beats goliath. There was on my sub stack and out of just read a paragraph. Maybe we can get back why it's worry so callin pell ryan, I guess, is the name of the player who actually beat them go programme. I said his victory The profound reminder that, no matter how good deep learning data really I looks when it is trained on an immense amount of data. We can never. Be sure that systems of this sort really can extend with a note, a novel circumstances. We see the same problem, of course,

the many challenges that have stymied the driver. This car industry, in the bat shit crazier as we ve, been seeing with the chat, but in the last week, and so that in peace also increased. My worry level, it's a reminder that these things are almost like aliens with we understand, like all of that thing knows how to play go, but through these little we, mrs. They are some of which you know turn into adversarial attacks and some of turn into bad driving in some of which turn into mistakes on chat? But I think we actually separate out genuine artificial general intelligence which may becomes in our lifetimes and make it doesn't from what we have now, which is this data driven thing that is stuart, would put his like a big circuit. dont, really understand what those circuits do and they can have these weaknesses. And so are you about alignment or something like that. If you don't really understand what the system does and what weird circumstance it might break down, and you can't really be that count but in around alignment for that system

may. I totally. This is an area that my research centre is now a lot of effort in is. We had we're going to control these systems at all, we ve got to understand how they work. We ve got to build them according to much more. I guess traditional engineering principles were system is made up of pieces. We know how the pieces work, we know how they fit together and we can prove that the whole thing does what is supposed to do and there's plenty of tackled Google elements available from the history of a high that I think can move us forward in ways where we understand what the system is doing

But I think the same thing is happening in G p, t in terms of failure to generalize right. So it's got billions of examples of arithmetic. You know twenty eight plus forty two is what seventy right and yet, despite having millions of examples, it's completely failed to generalize. So if it, if you give it into a three or four digit,

Addition problem that it hasn't seen before and particularly ones that involve carrying it fails. I think it can actually just to be accurate. I think he can do three and four edition addition to some extent a completely fails on multiplication at three or four digits. If we're talking about Minerva, which is, I think, the state of the art to some ix to some extent yet, but it I think it it works when you don't need to carry because it's the tickets has it has figured out that you know eight plus one is nine, because it's got into a few million examples of that, but it had. You know when, when balls carrying or you get two more digits outside the training said it hasn't extrapolated correctly, it hasn't learned the same with chess. It's got. Lots

lots of grandma suggest games in it's database and it thinks that thinks of the game as as, as a sequence of you have notation like and in a four d six night take c three b, three b: five right: that's what a chess game looks like when you the doubters notation. It has no idea that that's referring to a chess board with pieces. It has no idea that they're trying to checkmate each other, and you stopped just with it just make an illegal move, the cause it doesn't, We understand what what is going on at all and that the weird thing is that almost certainly the same thing is going on with all the other language generation that is doing. It has not figured out that the language It is about a world and the world has things in it and there are things the true about the world. There are things that are false about the world and you know if I, if I give

my wallet to gary than Gary has my wallet and if I he gives it back to me than I have it and he doesn't have it as we figured out any of that stuff. I completely agree. I think that people tend to anthropomorphic and actually needle steward a little bit and say he used words like think and figured out. These systems never think and figure out there, just finding closer approximation to the tax it seen and its very hard for someone who's tutor to nay I to really get that is to look at I see this very well formed output and realise that is actually more like an illusion. Then something that really understands things. Is absolutely right. You can talk about having a wallet or whatever. But it doesnt know that. There's a me out there that there's a wallet out there I've it's hard for people to grasp that, but that's the reality, and so when it gets the math problem right people like it's got a math and then it gets what's wrong. We're like oh, I guess it made a mistake, but really it never

at the math. It's just it's fine, some bit of tax. That's close enough! Some of the time that it happens. They have the right answer and sometimes not I want to return to that point. I think I need to back up for a second and define a couple, a terms just so that we don't lose people I realize I'm assuming a fair amount of familiarity with this topic from people of her previous podcast on it might not be fair, so quickly. We we have introduced a few terms here. We talk about narrow way. I generally I or ire artificial general intelligence and super intelligence, and those are those are interrelated concepts. Stewart. You just one hour, break those a pardon and suggests what we mean by them so narrow eyes, the the easiest to understand, because that typically refers to high systems that are developed for one specific task, for example, playing go translating french into english

or whatever it might be, and asia I or artificial general intelligence was sometimes call human level, artificial intelligence or general purpose. Artificial intelligence would mean I systems that can quickly learn to be competent in pretty much any kind of task to which maybe to which the human intellect is relevant. I'm probably a lot more besides and then artificials super intelligence or as s eye means systems that are far superior to humans and in all these aspects- and I think this is something worth mentioning briefly about euro. A a lot of commentators talk as if king on narrow a dozen present any kind of risk or problem. Because all you get out of narrow. I is a system for that particular task, and you know you could make hundred narrow eyes systems and they would

be little apps on your laptop and that none of them would prevent any risk, because all they do is that particular task facing us. A complete misunderstanding of how progress happen the day. I so let me give you an example. Deep learning, which is the basis for the over the last decade of of exploding, Aye aye capabilities, merged from a very, very narrow ay. I application, which is recognising hand written digits on checks by bill. jobs in the nineteen nineties and you are you caught me if find a more narrow application than that, but whenever a good day? I reset your works on a narrow task and it turns out that the task is not solvable by existing methods. They are likely to push on methods right to come up with more general, more capable methods and those methods will turn out to apply

to lots of other tasks as well. So it was the anglican who who is working in the group that worked on these hundred digits and he didn't write a little program. That sort of follows the s around and says. Ok, found one band. Ok, let me see if I can find another band. Ok, good! I've got a left band and are right then, so it must be an s right. That would be a very happy, Hurry non general. Very brutal way of of doing had ridden recognition. What he did was he peaches developed a technique for training, deep networks that had various kinds of invariably is about images, for example it and ass, his and ass, no matter where it appears in the image. You can feel that in to the structure of the networks and then that produces a very powerful image recognition capability. That applies to lots of other

things turned out to apply to speech and die in a slightly different form, is underlying. What's going on in chechnya, bt, so don't be a fool. Into thinking that as long as people are working on narrow way, I everything's gonna be fine here. If I could just jump in So on the point of general intelligence and what that might look like in Chad is in, sitting, because it's not as narrow in some ways is most traditional, narrow way I and yet it not really generally, I either. Doesn't it doesnt perfectly fit into the cattle Isn't let me explain what I mean by that so become narrower. I is, I will fold proteins or I will play chess or something like that. Really does only one thing Well in anybody whose play with tat she BT realises that those many things maybe not super. Well, it's almost like a jack of all trades and master of none. So you can talk to it about china.

send it will play ok, just for a little while and then, as stuart points out, probably eventually break the rules, because it doesn't really understand them or you can talk, what about word problems in math and it will do some of them correctly and get some of them almost anything you want to do not just one thing like sage ass: it can do to summit dead, but it never really has a good representation of any of those in. So it's never really reliable at any of them. As far as I know, there's nothing that chatty pity fully reliable that, even though it has something looks like I like generality, and obviously when we talk about artificial general intelligence, we're expecting something that is trustworthy and reliable. They could actually play chess. No, let's say as well as humans are better than them, or something like that. They could actually you know, do word problems as well as humans are better than that and so forth, and so it gives in illusion of generality, but it's so superficial, because the way it works in terms of approximating bits of text

doesn't really deliver on the promise of being what we really think of as an artificial general intelligence. Yes, ok, let's talk more about the problem. with narrowly I hear an incomplete. We should also add that most narrower I, although Chad cheap, it is perhaps an exception here is already insofar as we dignified say I it and implemented its already superhuman I am your your ear, calculators, superhuman for arithmetic and their many other, forms of narrowly, I that are just that they perform better than people do and one thing that's been surprising latest store just pointed out, is that superhuman ay. I of certain sorts like are our best go playin programmes have been revealed to be a highly imperfect

said there less than human in specific instances in these instances are surprising and can necessarily be foreseen in advance, and therefore it raises question of as we implement narrow way, I because it is superhuman it seems that we might always be surprised by its failure modes because it lacks a common sense. It lacks a more general view of what the problem is that it solve in the first place, and so that then I suppose some risk for us. If I could champion for once again, I think it cut right, air actually has to do with the mechanism. So a calculator really is superhuman. We're not gonna find in achilles. He'll where there is some regime of numbers that it can't do right. You know with within what it can represent, and so is the same thing with deep blue I'd be curious if stewart disagrees, but I think the blue is going to be able to beat any human in chess in it's not clear that we're actually going to find an achilles heel, but when we

when we talk about deep learning, driven systems are very heavy on the big data for using these particular techniques, they often have a pretty superficial representation stewards analogy. There was a python programme, its concise. We know that it captures something correctly verses. This very complicated circuit, that's really built by data, and when we have these there complicated circuits bill by data. Sometimes they do have achilles heel, so some narrow way. I I think we can be confident of so gps systems that navigate turn by turn. There's some problems like the the map could be out of date. There could be a broken bridge, but basically we can trust the algorithm there, As these go things, we don't really know how they work, we kind of do and it turns out. Sometimes they do have these achilles heel there in there, and those achilles collegial, can mean different things in different contexts in one on. It means where we we can beat it go what it's a little bit surprising

another contacts. It means that were using it to drive a car and there's a jet there and that's not in the training set, and it doesnt really understand that you don't into large objects and doesn't know what to do with the jet, nor actually runs into the jet, so that the weaknesses can manifest themselves in a lot of different ways. and some of what I think, stuart- and I are both worried about- is that the dominant paradigm of deep learning often has these kind of gaps in it. Sometimes I use the term point realistic that they have they're like collections of many points in some cloud, and if you come close enough to the points in the cloud, they usually do what you expect, but if you move outside that sometimes people call it. Distribution shift to a different point, then they're kind of unpredictable. So in the example of math at stewart- and I both like no It'Ll- get a bunch of math problems that are kind of near the points in the cloud where it's got experience in and then you move to four digit multiplication in the cloud is sparser, and now you ask a point: that's not next,

the point that it knows about it. Doesnt really work anymore. So this this illusion. Oh it learned multiplication will know it didn't just learn to jump around these points in this cloud and that had so normal level of unpredictability that makes it hard for humans to reason about what they systems gonna do and surely there are safety consequences that that arise from that and something else Stuart said that I really pretty It is in the old days and classically I we had engineering techniques around these. You you built modules. You knew what the modules did. There were problems, then, to I'm not saying, was all perfect, The dominant engineering paradigm right now is just get more data if it doesnt work and that still not giving you transparency into what's going on and it can be hard to debunk and so, like the ok. Now you built this go system in you discover a cat can't beat humans doing this thing. What do you do when they collect some data pertaining to that? But can it be general? You gonna have no way to know, maybe we'll be another attack tomorrow. That's what

seems driverless car industry is like through. Research may be of a different sort. They're not deliberately find some error and then people try to collect more data is no systematic science. There, like you, can't how much are we a year away or ten years away or a hundred years away from driverless cars by kind applauding out what happens because Most of what matters are these outlier cases? We don't have metrics around them. We don't have techniques for solving them, This is very empirical, we'll try, stuff out and hope for the best methodology anything stewart was reacting to that before and I certainly worry about that, a lot that we don't have a sound if the where we know hey we're getting closer here- and we know that How can I ask them to before we get to where we want to be ok Are you both have doubts as to whether or not the current path of of real? I and some deep learning and similar techniques to scale is not going to deliver us too

the promised land of asia I whether aligned with our interest or not, is just too We need more to actually be able to converge on something like general intelligence, because these networks as powerful as they seem to be in certain cases their exhibiting obvious failure of abstraction and they're, not learning the way humans, learn and or word discovering these failures, perhaps to the comfort people who are terrified of the gee. I singular we been reached. Like again, I wanna keep focusing on the problem- and potential problems with narrow eyes, two issues here this narrowly, I that fails- that doesn't do what it purports to do and is narrowing. I that is applied in ways that prove pernicious. intentionally or not a bad actors or good actors are reaching unintended consequences. Let us focus on Chad jpg for another month,

We are so and were all things like chat, you betty Many people have pointed out that this seems to be a potentially eight thermal. nuclear bomb of misinformation. Right and we are, we already have such an enormous misinformation problem. Just well in the apes concocted now have we have created a technology that that makes the cost of produce nonsense nonsense that passes for knowledge almost go to zero. What are your concerns about where, da headed where, where narrowly I've disorders headed in both of its failure modes failure to do what is attempting to do that is its making inadvertent errors or is just his failure to be applied ethically and wisely, and now. We are, however, effective it. Is we plunge into the part the map that has just done in a bursting without with unintended consequences here, I find all this.

Terrifying. It may be worth speaking for a second is to separate out two different problems. You kind of hinted at it. So one problem is that these systems hallucinate, even if you give them clean data, they don't keep track of things like the relations between subjects and products or entities in their properties, and so they can just make stuff up. So an example of this is a system can save it. Elon musk died in a car crash in two thousand and ten, that's a real error from a system called Galactica and that's contradicted by the data in the training said it's contradicted by things you could look up in the world and so it's a problem where these systems hallucinate, then there's a sec in problem, which is the bad actors can induce them to make as many copies or variants really of any specific misinformation that they might want. So, if you want to tune on perspective on the january sixth events will you can just have the system make that and you can have that, make one hundred versions of it? Or if you want to make up propaganda about coven and vaccines, you can make up one hundred version

each mentioning studies and land said in jail a may with data, and so, with all of the data made up. The study is not real and so for a bad actor. It's kind of a dream come true, so there are two different problems there on the first problem. I think the worst consequence is that these chat style search engines are gonna, make up medical advice. People are going to that medical advice and they're going to get hurt on the second one I think. What's going to get hurt is democracy because the result is going to be there's so much misinformation nobody's gonna, trust anything, and if people don't trust that there's some common ground, I don't think democracy. It works, and so I think there is a real danger to our social fabric. They are so both of these issues. really matter, and it comes down to in the end that if you have systems that approximate the world but have no real representation of the world at all. They can't validate with their saying so they can be abused. They can make mistakes. Is it's not a great basis? I think, for I certainly not what I had

for a start? So I think I have on a number of points by just wanted to go back to something you were saying earlier about the fact that the car Paradigm may not lead to the promised land. When I, I think that's true, I think some of the properties of chad, gb t have made me less confident about that claim, It is an empirical aim. As I said this, in a sufficiently large circuits with sufficient recurrent connections can implemented. Machines and can learn these higher level, more expressive representations and build interpreters for them. making a lamb elate them. They dont really learned what, though they can. Actually they could do that right men think about your laptop, you lot of is a circuit, but it's a circuit that supports these high level. Abstractions. Your brain is a circuit bonuses. They tried it. It is a question of representation for his work,

Right so get that supports that. So we can learn those internal structures which support representations that are more. Expressive and condemn, learn in those more recent representation. So, theoretically it is possible that happen well, but what we always see in reality is your example. Before about the four digit arithmetic, like the system's: don't in fact converge on the sufficiently expressive representations, they just always converge on these things that are more like masses of conjunctions of of different case and they leave stuff out. So I think I'm not saying you know learning set him to get up at these learning systems done well. We don't know that right. We see some failures, but we also see some remarkably capable behaviors that are quite hard to explain as just sort of stitching together bits, attacks from the training said, I think we're gonna disagree. They're up the same, how far he wants us to go down that route

Well, actually, lustrous spell out the point has been made and also to want to lose stuarts reaction to get his gently rose in his errand to bed at narrowly I, but I think this is an interesting point, intellectually. So yes there's some failure to use symbols or to recognise symbols or to generalise and is easy to say things like here's, a system? That is plain- better than then any person, but it doesn't know what go is or doesn't know. There's any beyond this grid, it doesnt recognised the groups of pieces etc, but can some level the same can be said: at about the subsystems of the human mind. Right me like that either. Yes, we use symbols, but the elite if one which symbol uses an instantiated in us in our brains is

its self symbolic raymond there is. There is a reduction to some piecemeal architecture, women and that there's just atoms in here right at the bottom, is just atoms again ass. True right up, there is nothing magical about that. Have an amendment mean navy sooner in the case of your laptop. If you want to talk about something like I don't know the folder structure in which you store your files, it actually grounds out and computer scientists can walk you through the steps. We could do it here if you really want to of how you get from a set of bits to a hierarchical directory structure and that hierarchically, just rectory structure can then be computed over. So you can vote. Apple movie, sub folder to the inside of another sub folder, and we all know the algorithms for how to do that. But the point is that the computer This is essentially a model of something and it manipulates that model there is a model of where these files are or representation might be better word. In that case, human these- have models of the world. So I have a model of the two people

but I'm talking to and their their backgrounds in the their beliefs and and desires. To some extent it's gonna be imperfect, but I have such a model and what I would argue is that a system like catchy pitied doesn't really have that in any case. If even if you could convince me that it does would the along appeal battle. We certainly don't have access to it and so that we can use it in reliable ways and downstream computation. The output of it is a string where In the case of my laptop, we have a very rich representations, I'll ignore some stuff about virtual memory they make, of a complicated and we can go dig in- and we know like which part of the representation the stands for a file and would say, for a folder and an outer manipulate those and so forth, we don't have that in these systems. What we have is a whole bunch of parameters, a whole bunch of text and we kind of hope for the best

asked yes or no disagreeing that we don't understand how it works, but by the same token, given that we don't understand how it works, it's hard to rule out the possibility that it is developing internal representational structures, which may be of a type that we wouldn't even recognize if we saw them, did their very different and we see I have a lot of evidence that bears on this. For example, all of the studies of arithmetic or guy vander brooks work on reasoning where, if you control things, the reasoning doesn't work the properly in any domain where we can look, are math problems or anything like that. We always see spotty performance. We always see hallucinations, they always point to there not being a deep, rich, underlying representation of any phenomena that we're talking about. So, for my mind, yes You can say the representations there, but they're not like world models, you're, not world models that can be real, ably interrogated and acted on, and we

see that over and over again, ok, I think we're gonna just agree to disagree on then. But the point I wanted to make was that you If Gary and I are right- and we're really concerned about the existential risk from asia. I wish you just keep our mouths shut right. We should let the will continue along this line of the bigger and bigger deep circuits will ya I think, that's the really interesting. That's a really interesting crash. I wanted your take on steward and it goes back to the word. Sam used about promised land and the question is, de I actually the promised land we want to get too so I've kind made the argument that will living in a land of very unreliable, a and said. There's a bunch of consequences. For that, like we have chat, search, is bad medical advice. Somebody dies, and so I have generally made the argument, but I'm really interested in sorts take on this- that we should get

more reliable ay. I, where its transparent its interpreted, what kind of does the things that we expect of it does forded? We ask it to do for digital arithmetic. It's gonna do that which is kind, The classical computer programming paradigm, where you knew you have supper, teens their functions and they do what you want to do, and so I'm kind of push towards let's make the ai more reliable and there is some sense in which that is more trustworthy right. You know that's going to do this computation, but there's also a sense in which maybe things go off the rail. At that point, that I think stewart is his interest and so Stuart might make the argument, let's not even get to asia, I'm like haywire this lousy point with this unreliable? I surely it must be better forget reliable ally but steward, I think

he's somewhere along the way where we get to a transition where, yes, he reliably does it's computations, but also he poses a new set of risks. Is that right? Stewart and you want to spell that out- I mean the if, if we believe that building bigger and bigger circuits isn't going to work and instead we push resources into, let's say methods don't publicity programming, which is a symbolic kind of representation language that includes possibility theory, so it can handle uncertainty, can do learning. It can do all these things, but There are a number of restrictions. On, our ability to use are realistic. Programming to achieve I, but- suppose we say: ok, fine! Well, we can put a ton of resources into this. Much more engineering based semantically, rigorous, come component composition, kind of technological approach, and if we succeed,

right. We still face the problem that now you build a system that actually more powerful than the human race how'd you. How do you have power over it? And so I think the reason to just keep, I would be give us more time to solve the control problem before we make the final push towards I against and if I'm being intellectually honest here, I don't know right answer there. So I think we can, for the rest of our conversation. Take probabilistic programming is kind of standing for the kinds of things that might produce more reliable systems like I'm talking about. There are other possibilities there, but it's fine for present purposes. If we could get you a land of probabilistic programming, that at least is transparent generally does the things we expected to do Is that better or worse than the current regime and stewardess make the argument that we don't know how to control them? either I mean. I'm not sure we know how to control we ve got now, but that's interesting question yeah. So is there

We give you an example. Let me let me give you a simple example of systems that are doing exactly what they were designed to do and having disastrous consequences the recommended system, algorithms though, in social media, let's take youtube, for example, When you watch a video on youtube, it it loads up another video for you to watch next. How does it choose that? Well, that's, learning algorithm minutes watched the behavior millions and millions of youtube users and which videos day they watch when their suggested in which videos they ignore or a different video or even check out of youtube altogether. And those learning algorithms design and to optimize engagement right. How much time you spend it platform. How many videos you watch, how many ads you click on and so on? and they very good at that. It was not that they have unpredictable failures like these, get it wrong all the time

and they don't really have to be perfect anyway right, they just have to be weighed up considerably better than than just loading up a random video and the problem is that there very good at doing that but that goal of engagement is not aligned with the interests of the users the way the algorithms have found too, to maximize Engagement is not just to pick the right next video, but actually to pick a whole sequence of videos that will turn you into a more predictable vic. and so there are literally brainwashing people so that once their brainwashed, the system is going to be more successful at keeping them on the platform. That's a completely drug deal there like drug dealers, and so so. This is the problem right that if we made that system much better

maybe using power programming if that system understood that people exist and they have political opinions, it's a system under stood the content to the video. Then there will be much much more effective at this brainwashing task that they ve been set by the social media companies, and that will be designed dress right. It wouldn t have promised land. It will be a disaster stuart. I agree with that example: its entirety, and I think the question is what lessons we draw from it. So I think that has happened in the real world. It doesn't matter that they're not up Well, they're, pretty good. They ve done a lot of harm in those algorithm. We do actually largely understand. So I accept that example. It seems to me like, if you have a g, I can certainly be used to good purposes or bad purposes. That's a great exam but where it's too near the good of the owner of some technology in the bad of society, I could invasion and approach to that in a curious what you think about it and it

really matter whether they are worthy of decent day. I grady. I, in the sense of being able to do what its told the deal. I'm is already a problem now could imagine systems. They could compute the consequences for society. A sort of asthma law approach may be taken to an extreme. There would compute the consequences of society and say: hey, I'm just not recommending that you do this from the strong words just wouldn't do it and weak version would say, hey here's. Why shouldn't. Do it. This is gonna, be long term consequences for democracy. That suck would be good for your society. We have an axiom here that democracy is good. So the one possibility is to say: if we're going to build a g, I it must be equipped with the ability to compute consequences and represent certain values and reason over them. What's your take on let's do it well, that assumes that is possible for us to to write down in some sense the utility function of the human race. We can see. The initial efforts there in how they have tried to put

rails on chad, BT, where you ask it utter a racial slur. Any won't do, even if further fate of humanity hangs in the balance right. So that, like a if into I mean that that's not really true. That's a particular example in particular context you can still it What are you going to read and write? We, we have not been very successful. You know we ve been trying to write. Tax law of six thousand years still haven't succeeded writing taxpayer that doesn't have loopholes right You ve been so I've been. I always worry about a slippery slope argument at this point, so it is true, for example, that we're not gonna get uniform consensus on values that we ve never made it up good work, but I don't think we want anarchy either and I think the stable we have now is either your systems with no values at all that are really reckless or you have kind of guard rails based on reinforcement, learning that are very sloppy and dont really do. What you want,

the do, or in my view we looked behind your number three, which is uncomfortable in itself, but which would do the best we can to have some kind of consensus values and try to work according to those consensus values, while I think there's a donor for I don't think turn on three works because really there are sort of in currently many ways to to write, thought the right. The wrong objective an early and say that in our society- and you know that but though not without doing gray, but were you know better than anarchy? I mean it's it's it's! The churchill line about democracy is in the best of some lousy up. That's that's because individual humans tend to be of a pro. Similarly equal capability and if one individual human starts doing really?

the bad things that the other friends thought of you know tend to react and squish them out. It doesn't always work. We've certainly had your near near total disasters, even with humans of average ability, but if once we're talking about hey, I systems that are far more powerful than the entire human race combined, then the human, This is in the position that term as a samuel, butler in eighteen. Seventy two that the beasts of the field with respect to humans? we would be entirely at their mercy. Can we separate two things stewart for we go to your number four, yes with which our intelligence and power, so I think, for example, our last president was not particularly intelligent. He wasn't stupid, they wasn't the brightest in the world, but he had a lot of power and that's what made him dangerous was the power not the sheer intellect, and sometimes I feel like in these conversations. People confound super intelligence with what a system is. Actually I'm enabled to do with what it has asked.

To do and so forth. At least I think it's important to two separate and those two out, so I worry about even dummy I like we have right now having allow the power like there's a start and wants to attach all the world software large language and there's new robot companies, I'm guessing his power in the eu, lloyd robots with large language models that terrified maybe not on the existential threat, humanity level, but the latter full of there's gonna be a lot of accidents because the system's don't have good manners. I saw your so completely agree right. I mean I wrote an ipad called. We need an the for algorithms about six years ago. I think no any driedup so sorry story. I just I think we should should hold the conversation. diy for a moment yet. But I would point out that that separation of concepts intelligent and power might only run in one direction, which is to say that for narrow way. I you can have it become powerful and-

depending on how it hooked up to the rest of the world, but for true, gee I that is superhuman one could wonder whether or not the intelligence of that sort can be constrained. They then you're in relationship to this thing that you are you. Are you sufficiently smart to keep this thing? That is much smarter than you from doing whatever it intends to do. One could wonder? I've never seen an argument that compels me to think that that's not possible and we like go programmes, have gotten much smarter, but they haven't taken more power moreover, the world. Nobody ever do not generally with debts unjustly gary. That's it ridiculous, exonerate right, since they want you're, let way way way they started story. Reviewer, look before we plunged, and I want to get there. I wanna get there. I just wanna want to extract whenever Lessons we can over this recent development and narrowly eye, and then promise you won't be able to fight about hiv in a mere matter of minutes. I doubt steward. So do you have got a few?

files over here I just want to acknowledge them. One is you. You suggested that if, in fact, you think that this path of throwing more more resources into deep learning is gonna be a dead with respect Agee I and your work about asia. I maybe its ethical to simply keep your mouth shut or or even cheerleader, for the promise of deep learning so as to stymie. all field for another generation. While we figure out the control problem is due to read you cry: lay there. When you, I guess, that's a possible argument that it occurred to me and and and people that put it to me as well. Ok, it's a difficult question. I think it's hard to rule out the possibility that the the present direction will eventually pan out, but it would pan out in a much worse way because it would lead to systems that were

extremely powerful, but whose operation we completely didn't understand right where we have no way of of specifying objectives to them or or finding out what objectives there actually trying to pursue because we can't look inside, we don't even understand their principles of operation, Soak is worthless. Table table that for a moment, what we're gonna wanna talk about asia, but on this issue of narrow way. I getting more and more powerful, especially especially concerned especially concerned, is about the information space and again I I just view: what ordinary engagement with social media is doing to us as more or less. entirely malignant and the algorithms a simple as they are endless, diabolically Fact of as they are. I have our proven sufficient to test the very fast.

rick of society and in the long term, prospects of democracy. But there are other their other elements here. So, for instance, The algorithm is effective and employed in the context of what I would consider the the perverse incentives of the beast This model of the internet, the fact that everything based on ads, which gives the logical ro endlessly gaming people's attention. If we solve that problem right, if we decided okay, this is Is the ad based model this pernicious here with the problem of the very new No problem, you pointed to worry nobody with youtube algorithm say with that go, Hey are you? Are you still just as worried in some new buy some new rationale about that problem, my view and then store can jump in with his. My view is that the problems around in from

asian space and social media are not going away anytime soon that we need to build new technologies to detect misinformation needed built new regulations to make a cost for producing it wholesale way and then there's this whole other question, which is like right now, maybe stew and I could actually agree that we have this sort of video career, I can't fully be counted on, has a whole set of problems. That goes with it and then really. The question is: there's a different set of problems. If you get too and I that could, for example, say for itself You know I don't really want to be part of your algorithm, because your algorithm, gonna have these problems like that open? a whole new can of worms. I think steward is terrified them, and I'm not so sure, is to not be worried about them. You have a little bit less concern that then store, but I can't and all intellectual honesty say it. No problems lie there. Maybe there are problems, lie there so funny

long before we get and an algorithm that can reflect in that way. Just imagine effusion of what we all we have with chad jpg with deep, fake video technology right. So you can just get in less content. That is a persuasive simulation from of a real failure, laying a crazy things, yeah in its away not years away, I haven't editor, the major paper is an ordinary at various special role that at paper everybody knows. He read a debate that I and on twitter two days ago where I said, I'm worried about these things and somebody said eyes, not a problem and

he showed me how, in like four minutes, he could make a fake story about, like Antifa protesters cause the january six thing using his company's templates and a image from mid journey, and it looked completely authentic, like this can be done at scale right now. There are dissemination questions, but we know that, for example, you know russia has used armies of of troll farms. out of you know iphones and fake accounts and stuff like that too. This is like an imminent, probably will. Certainly right, I no longer life. It's it's a past problems right. It- is an ongoing problem. It is here or it's already happened many times fact. If you, if you go google news and look at the fact tat section Since the last day, the urban faked videos of president Biden saying that all twenty to twenty two year old in the united states will be drafted to fight in the war

This is just a year now and it's a question in scope and spread and regulation. This doesn't require really further, advances in AI. What we have now is already sufficient to cause this problem, and is it just going? I think I think sam's point. Is it it's going to explode as the capabilities of these tools and their they'll ability increase our. I would completely agree. I think I don't really get the impression that I only care about extinction risk and none of the other stuff matters. I spend a ton of time truly working on lethal autonomous weapons which again already exist, despite the russian ambassador, claim that this is all science fiction and won't even be an issue for another twenty five years, gets this nonsense, as he was saying that There was a turkish company that was getting wet to announce a drone.

Capable of fully autonomous hits on human targets. So so I think solution here, and I have a sub group within my centre at berkeley that is specifically working on this headed by Jonathan stray the solution is very complicated. It's an institutional! solution that it probably involves the setting up some sort of third party info structure much, as you know, in real estate, there's a whole bunch of third parties like title insurance, land registry notaries, who this to make sure that there is enough truth in the real estate world that it functions as a market the reason we have accountants and auditing in the stock market. So there's enough truth that it functions as market. We just haven't figured out how to deal with this avalanche of disinformation and defects, but it's going to require similar kinds of institutional solutions and our politicians

have to get their hands around this and make progress because otherwise eyes there is the worry about democracies all over the world, the only I can add to what stuart said is all of that with the word yesterday like we don't have a lot of time to sort this out if, if we wait till after the two thousand twenty four election it might be too late, or do we really need to move on this? I have to thank them business model of the internet has something to do with us, because if there was no money to be made by gaming people's attention with misinformation this attitude, it would never happened, but the incentive would evaporate radically and I mean as a reason why this doesn't happen on netflix right, there's, a reason why we're not having a conversation about how netflix is destroying democracy, in the way it serves up each new video, too and it's because there's no incentive, may I guess, there's they ve been threatening to move ads and in certain markets, or maybe they have done so. This could go away eventually. But

or you know, heretofore. There's been no incentive for netflix to try to figure out if they're trying to keep you on the platform, because they want you not to churn. They want you to and every day feeling like netflix is an integral part of your life, but it's in there. There No one said they want. They want you to dinner, watch for thirty, eight hours exactly Entirely innocent didn't know he s thoughts, but about it, it is not having the effect of giving them a rationale to serve you up insane confections of pseudoscience, and overt lies, so that, someone else, can drag your attention for a moment. or hours because it's there, business model, because if they ve sold the right to do that if it's not an tiredly, an internet phenomena in the sense that, like fox news, also as a kind of engagement model did it does

centre around. In my view, maybe I get sued for this but sent around misinformation. So, for example, we know we know that the executive there, we're not all on board for the big lie about the election, but they thought that you know. Maybe it was good for ratings for something like that. did you look at the weekly world news right that thought: rosa back to get the yellow journalism in the morning. Every print outlet which every week would tell you that the creatures of hell have been funded oft emerging from cracks in the streets of LOS angeles reason, you name right wait to apple historically, the last time we were this bad off was the eighteen nineties with yellow journalism, hearst and all of that and that's when people started doing fact checking more and we might need to revert to that to to solve this. We might need to have a lot more fact, checking a lot more curation, rather than just random stuff that

it's up on your feet, and this is not in any way fact track. That might be the only answer here. Some are probably taking it. Not so not saying. Okay, facebook has to factcheck all the staff or Google has to fact check all the stuff, but facebook asked him available filters. Where I can say, okay, I dont want stuff in my newsfeeds that hasn't passed, some some basic standard of of account. ability and accuracy and equipment that he has raised. A business models are come coming back to this business model question. I think that the tech. Companies are understanding that the deal It'll banner had has become pretty ineffective. And advertisers are also starting to understand this, and I think when you Could the mischievous and say well? What on earth is the business model here right? Why they spending billions and billions of dollars? and I went to a conference in south korea where the business model basically revealed by the previous speaker, who was name

researcher, who is very proud of being able to use chat jpg technology along with the fact that you're in the better versa, you have these avatars to create fake friends. So these are people who their avatars who avatars to appear to be avatars of real humans, who spent Weeks and weeks becoming your friend learning about your family, telling you about their family But the boy in a casually would drop into the conversation that they just got a new bmw where they really love slow lex watch, blah blah blah right so the digital banner add, is replaced by the czech jpg, driven fake human and the metaverse and goes from thirty million seconds are trying to convince you to buy something to six weeks. That's it is this model of the metaverse and this You know this will be far more effective, far more insidious and destructive although we do when you

think about it is what happened? What is what people do to one another anyway? I'm is this like product placement in relationships there's a little that, but they really expensive right now in an influence I on on youtube, you know you have to pay them. Tens of thousands of dollars to get ten twenty seconds of product placement out of them. But these us these are crazy humans. You know because pennies run and they can take up hours and hours and hours somebody's time. Trusting lady the european union, the a I act, has a strict ban on the persecution of you being so you. you always have a right to know if you interacting with a real person or with a machine I think this is something that will be extremely important. It sounds like yeah. Ok, it's not a big risk right now, but I think it's gonna become an absolute linchpin of human free

in the coming decades, I don't things- are gonna, be a story of a lie to the rescue. Here were the only way we can detect deep, fax and and other sources of of information and in the future will be to have sufficiently robust ay, I that can go to war against the other s eyes their create the emotion. I think it's a useful tool, but I think what we need actually is provenance I think that we need to do, for example, that's generated by your camera is water marked and time stamped and location coded, and so if a video is produced that doesn't have that and it doesn't match up cryptographic

with you know, with real camera and so on. Then it's just filtered out, so it's much more than it doesn't even appear. Unless it's verifiably real, it's not that you get you let everything appear, and then you try to sort of take down the stuff. It's fake! It's much more of a sort of positive permission to appear based on authenticated provenance. I think that's right way to go. I I think we should definitely do that for video. I think that for text we're not going to be able to do it, people cut and paste things from all over the place. We're not really going to be able to track them. It's going to be too easy to beat the watermark schemes. We should still try, but I think we're also going to need to look at content and do the equivalent of fact checking, and I think that ai is important because the scale is going to go up and we're not going to have enough humans to do it. We're probably going to need humans in the loop,

I think we can do it fully by machine, but I think that it's going to be important to develop new technologies, to try to evaluate the content and try to validate it and something like the way that a traditional fact checker might do hm yeah. I think also, I think the full text is Thirdly, more validation of sources so at least until recently there are. There are trusted sources of news and we trust them because in our effort, journalists was to to generate a bunch of fake news, they would be found out and they will be fired and I think we could probably yet agreement on certain standards of operation of that type and then, if, if the platforms provide the right filters, then I can simply say, I'm not interested in new sources that don't subscribe to these standardize principles of operation. I am less optimistic about that particular approach, because

I had it for several years, and most people just don't seem to care anymore in the same way that most people don't care about privacy anymore. Most people just don't care that much about sources, and I would like to see educate national campaigns to teach people, ai literacy and web literacy and so forth, and hopefully we make some progress on that. I think labeling particular things as being fought, were, I think the most interesting ones are misleading has, value in it. So a typical example of something is misleading is if Robert Kennedy says that somebody took her billy vaccine, and then they got a seizure. The facts might be true. It is an invited inference that taking covert vaccines is bad for you and those lots of data that show on average it's good for you, and so I think we also need to go to the specific cases in part, because lots of people say some things that are true and some that are false. I think we're going to need to do some addressing of specific content

and educating people through through labels around them about how to reason about these things. Ok general. asia. I alignment and the control problem was jump in Gary early on, you said something sceptical about this, be a real problem, because you didn't necessarily see that asia I could ever form the motivation to be hostile to humanity, and this echoes something that many of you Well said I certainly Steve pinker has said similar things and I think that is put words into storage mouth and then let him complete sense, I think that really is a a red herring at this point or a straw. man version of the concern is not a matter of a rope overlords spontaneously become in evil. It's just a story of what mismatches in competence and and power can produce. In the absence of pearl

alignment in it. In the absence of that, the ever increasing com but as we are talking about a situation where do the machine? are built in the next generation of it even better machines? The question is: if they're, not perfectly aligned with our interests issue, which is save human well, being isn't there paramount concern, even as they they outstrip us in every conceivable or everywhere. woven cognitive domain. They can begin to treat us spontaneously based on You have goals that town, we can no longer can even contemplate the way we treat every other animal on earth. It can't contemplate the goals we have formed. Rice was not that we have to be hostile to the the creatures field or the ants. You know that a walking across driveways, but is at the moment. We do we get into our has to do something that anson and farm animals can do even dimly glimpse

We suddenly start behaving in ways that are total. Scruple to them, but also totally destructive of their lives, and just by analogy, it seems like we may create the very entities that would be couple of doing that us maybe give you much of the sends to finish stuart, but for a further away in and then not less give it a gary yeah. I mean there are a number of variations on this argument so since keeping it has, as you know, there's no reason: to create the alpha mail. I if we just build a along, Feminine lines. It'll have no incentive to take over the well yeah le says what, there's nothing to worry about. We just don't have to build it. Instincts like self preservation, and I made a little by little grid world in an empty p, which is a mark of decision processes just a little grid where the ice system has to go and fetch the milk from you know, a few,

away and one corner of the great desert, you know, there's a bad person who wants to steal the milk and so what is the aid system learn to do learns to avoid the bad person and go goody goody at the corner of the grid, to go fetch the milk so that there is no chance of being intercepted and we didn't put self preservation in. At the only goal system has its to fetch. The milk and self preservation follows as a sub go right, because if, your intercepted and killed on the way than you confess the bill. So so this is it this. An argument that a five year old could understand right. The real question, in my mind, is: why are extremely brilliant people like young girl and stephen? thinker, nor able pretending, not understand. I owe pretending either I think there's some motivated colony,

going on, I think, there's a self defence mechanism that kicks in when your your whole being is, is a few feels like it's under attack, because you so in the case of Yan, he devoted his life to ai. I in the case of steven is his whole thesis. These days is that progress and technology has been good for us, and so he doesn't like any talk of that progress and technology could buy. apps end up being bad for us, and so they go into this defence mode. Where you come up with any sort of argument and I've seen it with AI researchers. They immediately go to all. Well, listen! There's no need to worry. We can always just switch off right as if super intelligent? I would never have thought of that possibility. like saying all we are weak in a weaken easily be deep, blue and always other chess programmes. We just play the right move. What's the problem

is a form of thinking that that makes people makes me worry even more about a long term process. It's, because it's one. It's one thing to have the solutions. If you'd like to continue listening to this conversation, you'll need to subscribe at sam harris dot org once you do, you'll get access to all full length, episode of the making sense podcast, along with other subscriber only content, including bonus. Episodes and animals. and the conversations I've been having on the waking up at the makings has podcast his ad free and relies entirely on listener supported, and you can subscribe now at sam harris dot, org