In this episode of the Making Sense podcast, Sam Harris speaks with computer scientist Stuart Russell about the challenge of building artificial intelligence that is compatible with human well-being.
SUBSCRIBE to listen to the rest of this episode and gain access to all full-length episodes of the podcast at samharris.org/subscribe.
This is an unofficial transcript meant for reference. Accuracy is not guaranteed.
Today, always speaking with Stuart Russell. He is a professor of computer science and engineering, and you see Berkeley he's also AY adjunct, professor of neurological surgery, and you see San Francisco. He is the author of the most widely read textbook on the subject. Today, I artificial
elegance, a modern approach and over the course of these ninety minutes, or so we explore the topics that you may have heard me raise in my TED talk anyway. Stuart is an expert in this field and a wealth of information, and I hope you find this conversation as useful. As I did. I increasingly think that this is a a topic that will become more and more pressing every day and if it doesn't for some reason, it will only be because
scarier, things, have distracted us from it. So things are going well, yes
we worry more and more about the consequences of ay or so it seems to me, and now I give you
store Russell. I'm here was Stuart Russell store thanks for coming in the park ass, you welcome,
Our lesser should know you ve been up nearly all night working on a paper relevant to our topic at hand. So double. Thank you for doing this, no problem. I hope it will be coherent. While you you, ve, got.
Now nearly infinite latitude not to be so perhaps you can tell us a little bit about what you do. Have you describe Europe job at this point, so I met Professor Berkeley, a computer,
scientists, and I worked in the area of artificial intelligence.
For about thirty five years now, starting with my phd it's for most of the time I ve been.
What you might call a mainstream. A researcher
work on Machina paying and Power ballistic reasoning planning
game playing all the things that that day, I people work on
and then the last few years, although this has been something is that's concern me for for a long time,
I wrote a textbook in ninety ninety four, where I had a section of a chapter talking about what happens if we
Accede in I mean what happens. If we actually build
machines at a more intelligent than us. What does that mean
so that was forever in into intellectual question in its become.
A little bit more urgent in the last few years as progress is accelerating and
The resources going into a I have grown enormously, so I'm really asking people to take the question series
What happens if we succeed? As you know, I've join the chorus of people who really the last two years,
Have begun worrying out loud about the consequences of a I or the consequences of all of us, not building it with more or less perfect conformity to our interests and one of the things about this choruses that is mostly made up of non computer scientists and therefore people like myself for Elon, musk or even physicists like MAX tag. Marken Stephen hawking are seemingly dismissed with alacrity by computer,
scientists who are deeply sceptical of these worried noises, we're making end. No, you are not so easily dismissed because you are, you have the the really the perfect Bonatti Data of a computer scientist, so I want to get us into this territory and are actually want in. I don't
actually know that you are quite as worried is, as I have sounded publicly serve as any difference between your taken,
mine. That would be interesting to explore, but I also want us to to at some point I'd like you to express the the sound is basis for this kind of skepticism, that you know that we are crying Wolf in a way that is unwarranted, but before we get there right at one, ask you a few questions to to get our bearings. The main purpose here is also just to educate our listeners about me to what artificial intelligence is and what its implications are, whether if everything goes well or everything, goes less than well. So very a disarmingly, simple question here at first: what is a computer? Well,
so pre much everyone. These days has a computer, but that doesn't necessarily understand what it is. The way is presented to the public, whether it's your your smartphone or your laptop, is something that runs a bunch of applications and the applications. Do things like edit word documents of a face to face video
Jack, and things like that, and what people may not understand is is that a computer is is a universal machine that any process that can be described precisely can be carried out by a computer, and an every computer can stimulate every other computer, and this. This property of universality means there at tat. Intelligence itself is something that
a computer can in principle, emulate- and this was realized.
Among other people, by eight lovelace in the eighteen fiftys when she was working with Charles Babbage
they had this idea that the machine they were designing might be a universal machine, whether they couldn't defined that very precisely,
and so the immediate thought is well, if its universal than it can it can carry out the processes of intelligence,
well as ordinary mechanical calculations. So a computer computer is really end
thing anything you want that you could describe precisely enough to turn into a programme to relate the concept of information to that they sound like very simple questions, but these are very good at this
Certainly deep questions. I'm aware, I think everyone understands that out there is, is a world the real world,
and we all know everything about the real well, so it could be one way or could be another. I think it could be theirs gazillion different ways.
The world could be in all all the cars are out, there part could be parked in different places, and I wouldn't even know it
So there are many many ways that we could be an information is just something that tells you
a little bit more about what
you know what the world is wit, which way is that is the real world out of all the possibilities that it could be
And as you get more more information about the world through typically, we get it through are:
eyes and he is an increasingly we're getting it through the internet, then that that information helps to helps a narrow down the ways that the real world could be,
and Shannon, who is a logical engineer at MIT, ah figured out a way to actually quantify the amount of information. So if you think about a coin flip, if I can tell you which way that coin is going to come out,
heads or tails than that. That's one bit of information and of that lets you give you gives you the answer for a binary choice between two things:
And so from info
in theory we have. We have wireless communication, we have the internet, we have all
all the things that allow computers to talk to each other through physical
medium so information theory, has been in some sense. The
complementary or the hand maiden of of computation- and
Allowing allowing the whole infiltration revolution to happen is an important difference between what you just described, computers and in the information they process and mines fell asleep
justness aside for that for the moment? But if I asked you what is a mind, would you have answered that question differently? So I
I think I would because the mind the word mine carries with it this this motion, as you, as you say, of consciousness, it's not
I the way you can't really put aside the notion of consciousness, except if you're taught me talking about, are the unconscious mind,
you know it likely all the unconscious cognitive processing. We do his mind
a misnomer there without consciousness it might yeah unconscious.
Kind of like saying, artificial grass. It isn't grass, but he kind of like grass, so just to give you a quote
John Hoagland has written a lot about
he's a philosopher and he describes the notion of strong, I as it as it used to be called.
As building machines with mines in the fallen literal sense so
So the word mind there is carrying the idea that there is true, conscious awareness, true semantic understanding and perception. No perception is sexual experience and I think this is an incredibly important.
Thing, because without that nothing has moral value. There are lots of complicated physical processes in the universe,
you know stars exploding in rivers. In you, know
issues, melting and all kinds of things I've got, but none of that has any
moral value associated with the things it
generate moral value? Are things that have conscious experience? So it's that's a very it's a really important topic, but I have nothing to say about it. What
However, will not yet, I guess says it: we're we're gonna get there in terms of if consciousness is at some level, just an emerging property of information processing. If, in fact, that is the punch line at the back of the book of nature will then we need to think about the implications of building conscious machines, not just the intelligent machines, but you introduced ay, ay,
term here which we should define. You talked about strong verses, weak ay. I guess the the more modern terms are narrow versus general artificial intelligence. Can you define those horse right so that the wood
the words strong and weak about change their meaning over time so strongly. I was, I believe, a phrase introduced by John, so in his chinese
paper romanian slightly than that, but what he meant was
the version they. I that says that if I build something weird
human level intelligence, then, in all probability, it's gonna be a conscious device that that the the functional properties of intelligence and consciousness are inseparable.
And so strongly eyes the sort of the super ambitious form with a sigh and
we K. I was about building I systems that have capabilities that you one that you want them to have, but they don't necessarily have the the consciousness or indeed the first person experience.
So that and then, and then I think, there's been a number of people.
Both inside and outside it feels it of using strong, and we are in
various different ways and now largely you.
See stronger. I and some time
generally I or artificial general.
Talents to mean building. I systems that have the capabilities.
Comparable to or greater than those of humans without any
opinion being given on whether this consciousness or not,
and then narrowly, I meaning they. I systems, I don't have the general
multi, they might be very capable, like alpha, goes very quickly
we'll go player, but it's too narrow and sensitive cant do anything else. So we don't think of those general purpose in telling
its role and given that consciousness is something that we just don't have
a philosophical handle, let alone a scientific handle on. I think, for the time being, we'll just have to put it to one side, and the discussion is gonna have to focus on capabilities on on the functional properties of intelligent systems. For this, as other term, one here is in this area, which strikes me as a an actual Roy, a term that their names almost nothing possible. But it's.
Human level ally, and that is how it often put forward, as currently they nearer landmark too soon.
Or intelligent I or something it's beyond human, but
Seems to me that even our narrow at this point, you know the the calculator in your phone or anything else that gets good enough for us to dignify it with the name intelligence very quickly,
comes superhuman even in his narrowness, so is the phone is a better calculate
than than I am or will ever be, and if you imagine building
system that is a true general intelligence, its learning,
not confined to one domain as opposed to another, but much more like a human being and that it can learn across a wide range of domains without having
learning and one domain degrades learning.
Other you very quickly, if not immediately will be talking about superhuman ere. I, because presumably this system will
it's not going to be worth calculator than my phone right? It's not going to be a worse chess player than deep blue. It's like I did at a certain point. It's going to very quickly be better than humans at everything it can do
so it is, is human level. I am mirages it is some service of awaited to think about that counts
So. I think human level I is, is just
notional goal, and I I basically agree with you that if, if we can achieve the generality,
of human intelligence, then we will probably ex seed on many dimensions the actual capabilities of humans. So there are, there are things that humans do
do we really have no idea how to do yet, for example, what what
humans have done collectively in terms of creating science
we don't know how to get machines to do something that we can. We can imagine that, theoretically possible. We know it's some somewhere in the space of programmes. There exists a program that that could be
high quality scientists, but we don't. We don't know how to make anything like that. So it is possible that we could have.
Human level, capabilities on sort of money on all the mundane intellectual tasks that don't require these really creative re formulations of a whole conceptual structure that happen from time to time in science and in this is sort of what
it's happening already right, I in in all in, as you say, in areas where computers become confidently, they quickly become super competent, and so we could have
super competence across all the mundane areas, like the ability to to read an answer suited the kinds of questions that
You know an undergraduate could answer by reading it. Reading a book we might
see those kinds of capabilities, but it might be then quite a bit more work which which may we
learn how to do to get it to come up with the kinds of answers that the truly creative and deep thinking human could do from from looking at the same material
But this is. This is something that at the moment is very speculative mean what we.
What we do see as the beginning of generality, so you often see
people in the media claiming Orwellian oak computers can only do what they programme to do there only good at narrow tasks. But when you look at, for example, Deak, U N, which was
Google deep mines first system that they demonstrated to this learn to play video games and learn com.
Lee from Scratch, so it was like a new born baby opening its eyes for the first time it
no idea what
kind of a world ASEAN. It doesn't know that there are objects or that
move or there such a thing as time or good guy
is in bad guys or cause, or roads or bullets or spaceships, or anything. Just like a new born baby.
And then, within a few hours of messing around with with a video game, essentially throw cameras reefs looking at the screen, it doesnt have direct access to the internal structures. The game at all is looking at the screen very much the way
Human beings interface with the game. Yeah exactly are, the only thing it knows is. Is that once it more points
and so within a few hours, its able to learn a wide range so so most of the games at Atari produced. It reaches a super human level of performance in a few hours entirely, starting from nothing. That is important to say
It's the same algorithm playing all the games, not it's not like deep blue, that is the best chess player, but he can't play TIC tac toe and will never play TIC. Tac toe is so completely different. Correct approach,
this is this is one are really. We know it could be a driving game, it could be stays invaders, it could be packed and ban it. Could the undersea yo see quest with submarines? So in that sense, and when you look at that, you know if your baby did, that woke up,
you know the first day in the hospital and by the end of the day was, was beating everyone beating, while the doktor.
That Atari Video games, you be pretty terrified, so you and it is
demonstrating generality up to a point. I there there is certain characteristics of video games
that don't hold for the real world in general main one of the main things
seeing that in a video game, the idea
Is it you can see everything on the screen
in the course of the real world in any given point
this tonnes in the real world that you can't see, but it all it still matters, and then
So with video games they they tend to have very short horizons because you're supposed to play them in the pub when you're, drunk or whatever, so they typically on. Unlike chess, they don't require deep thought
about the consequent long term consequences of your choices so but Ill Adam LOS do things which is certainly important
They like Dick, you and and various other reinforcing learning systems are beginning to show generality, and we seeing with the with the working computer vision that the same basic technology, these the convolutions deep,
networks and with their and their recurrent cousins, that the technologies with fairly small modifications not really can
Joe changes just sort of minor changes in the details of the architecture can
learn a wide range of tasks to do an extremely high level,
kooding recognising thousands of different categories of objects in photographs doing speech recognition?
learning to even right captions for photographs, learning to predict, what's gonna
but next video and so on. So for so so I think we're.
Arguably you, if there is
to be an explosion of capabilities that feeds on itself. I think we may be in the beginning of it now. What are the implications,
with respect to how people are designing these systems. I e, if I'm not mistaken me,
If not all of these deep learning approaches
generally machine learning approaches are essentially black boxes, in which you, you can't really inspect how the the algorithm is accomplishing, what it is,
accomplishing is that the case, and if so in or wherever it is the case, are their implications there. That we need to be worried about her. Is that just a novel way of doing business, which doesn't raise any special concern
What I think it raises two kinds of concerns, one we three, so one is a very practical problem there
Why? It's not working you re dont know why it's not working
and there is a certain amount of blundering read about in the dark. Some people call us graduate student descent.
Which is that's a very nerdy joke, so great gradient descended,
you know walking down down the hill. Is is a way to find the lowest point
so graduate students send meaning that you're you're trying out
from system designs and in the process you using up graduate students at a rapid rate, an end that is it that's clearly, a drawback,
We know- and I in my research I generally favoured techniques where
the design of a system is derived from the characteristics of the problem that you're trying to solve
and so the the function of each of the components is clearly understood, and you can.
You can show that the system is going to do what is supposed to do for the right reasons and the Black box opera. There are people who just seem to have great intuition
about how to design the architecture of these deep learning network so that they know they produce good performance? I think they're also practical questions from the list
the point of view that there are a lot of areas, for example medical diagnosis or
treatment, recommendations, recommend
being for or against parole for a prisoner,
is approving credit or
cleaning credit applications
where you really want a explanation of why the record
nation is being made
without that people simply won't accept that the system is used.
Some one at one of the reasons for that is that a black box could be
making decisions that are biased, racially buys, for example, and without the ability to explain itself, then you, you can't trust that
the system is unbiased and then there's a third reasons which I think is what, behind your question
about why we might be concerned with will systems that are entirely black box that
we said we cannot understand how the system has been is reaching its decisions or or what it's doing that gives as much
less control. So as we move towards more more capable and perhaps general intelligent systems. The fact that we
really might have no idea how they working or what they thinking about so to speak. That would give you some concern, because,
then one one of the reasons that the ay I community often gives for. Why, then did not worried rice are the people who are sceptical about there being
risk is that while we design these systems, you know, obviously we
We were designed them, so they did what we want, but if they are completely open
black boxes that you don't know what they're doing them, that that sense of control and safety disappears. Let's talk about that issue of what passed from corporate control problem. I guess we called the safety problem as well, and this is many people were soon will have watch my TED talk where I spent fourteen minutes Warren.
About this, but just perhaps you can briefly sketch they concern here. What is what is the concern about
Generally I getting away from us. How do you articulate that? So you mentioned earlier that this
tick can be to scientists and busters book. Super intelligence was certainly instrumental in was
instrumental in bringing it to the attention of all of a wider audience and young people like Bill gates
on Elon, musk and so on. But the fact is that these concerns have been articulated by the central figures in computer science and ay. I so much
going back to age, a good environment
well I'm in
during himself right so people a lot of people may not know about this. I'm just canoe read a little quote two hours
During gave a talk on BBC radio Radio, three in nineteen fifty one, so he said if a machine can think
might think more intelligently than we do and then wish we be, even if we could keep the machines in a subservient position, for instance, by turning off the power at strategic moments, we should, as a species feel,
greatly humbled? This new danger is certainly something which can give us a anxiety. So that's a pretty clear period.
Eve super intelligent ay, I we could have a serious problem. Another person who talked about this issue was no, but we
and about window was day.
One of the leading applied mathematicians of the twentieth century use the phone.
Under of of a good deal of modern control theory.
And automation site is often called the father cybernetic. So he was he was concerned. Is he saw Arthur Samuel,
complain programme, ah in nineteen, fifty nine learning to play checkers by itself a little bit like the dick. You end that I described learning to play video games, but this is nineteen. Fifty
nine and more than fifty years ago, learning to play check is better than its creator.
And he saw clearly in this the seeds of the possibility of systems that good our distant.
Human beings in general, so
He was more specific about what the problem is. That of cheerings warning is, in some sense the same concern that guerrillas might have had about humans if they had
would you know she's a few million years ago, when the human species branched off from from the evolutionary line of the guerrillas at the guerrillas, had said to themselves. You should we create these?
when being's right, gonna be much smarter than us yeah. I kind of makes me worried
right and in the EU
they would have been right to worry because as a species there they sort of completely
lost control over their own future n and humans control, everything that that they care about. So so ensuring is really talking about
This general sense of unease about making something smarter than you is not a good idea, and what we said was was this: if we used to achieve our purposes a mechanical agency with whose operation we cannot interfere effectively,
We had better be quite sure that the Pope was put into the machine is the purpose which we
really desire so this taking sixty nowadays, we call this the value alignment problem. How do we make sure that sir,
The values that the machine is trying to optimize are, in fact the values of the human who is,
trying to get the machine to do something for the values of the human race and in general, and so now we know actually points to the sorcerer's apprentice story as a typical example of wind.
You give a got off to a machine, in this case fetch water, if you dont, specify correctly, if you don't cross,
every tee and dot every eye and make sure you ve, covered everything, then
genes being optimizing. They will find ways to do things that you don't expect
Those ways may make you very unhappy, and the story goes back to the king MIDAS,
five hundred in whatever B c, where he got exactly what he said, which is the thing turns to gold,
It is definitely not what he wanted. He didn't want his food and water.
Turn the goal of his relatives to turn the gold, but he got what he said he wanted
the stories with Jeannie's the same thing right, you you gave her wish to a genie. The genie carries out
wish very literally in the near the third way. She's always can you under the first two does I got them wrong and the problem with
Cooper, intelligent, I is that you might not be able to have that third wish, or even a even a second, where
yeah, so she so if you get it wrong, you and you might wish for something very banal
fine sounding like you know, could you cure cancer? But if, if you haven't told the machine that you want cancer cured, but you also want human beings to be Eliza was simple way to keep
cancer in humans is not to have any humans, a quick way to come up with a cure for cancer,
is to use the entire human race as guinea pigs or for millions of different national drugs. That might you a cancer,
So that's all kinds of ways. Things can go wrong and you know what we have governments.
All over the world. Try to write tax laws that don't have these kinds of loopholes
they fail over and over and over again,
and there are only competing against
ordinary humans in tax lawyers and rich people, and yet they still fail
despite there being billions of dollars at stake, so track record of being able to specify objectives and constraints completely.
We are sure to be happy with the results are track. Record is, is abysmal and
Fortunately, we don't really have a scientific discipline for how to do this
so generally, we have only scientific disciplines, a I control theory, economics operations, research that are about how do you optimize an object
but none of them are about. Well. What should the directive be so that we are happy with the results? So that's really. I think.
The modern understanding, as described in Boston, was broken and other papers of why a super intelligent machine could be problematic is because, if we give it an objective, which is differ
from what we really want, then we will. We basically looked like creating a chess match,
with a machine right now. There's us with our objective in it with the directive we gave it which different from what we really want, so it's kind of like having a chess match for the whole world. Ah, were not too good a beating machines, a chess, so it s a great image, a chess match for the whole world. I, when I drove out on a couple of things you just said there, because I'm hearing that these skeptical voice, even in my own head, even though
I think I have smothered it over the last year of focusing on this, but in amazingly easy, even for some.
Unlike me, this was really committed at the framing of my TED talk, where it is it's just. I was talking about these concern
and there any value alignment problems actually. But the real message at my talk was that it's a very hard to take this seriously emotionally, even when you are taking it seriously, intellectually, there's something so diaphanous about these concerns and they seem so far fetched, even though you can't give an account where I have certainly haven't heard. Anyone give an account of why, in fact they are far fetched when you look closely at them. Feel like you did the idea that you could build a machine that is super intelligent and give it the
to cure cancer. Fetch water and not have anticipated that one possible solution to that problem was to kill all of humanity or to fetch the water from your own body, and it just seems it. We have an assumption that things.
Inconceivably go wrong in that way, and I think the most compel
inversion of of push back on. That front has come to me from people like David Deutsche, who, you probably know, he's one of them, the Father of of quantum computing or worthy the concept there. A physicist at Oxford has been on the pike asked he argues, and this is something that I don't find compelling, but I just want to put it forward and I am told him as much. He argues that Super intelligence entails an ethics. If we built a super intelligence system, we will have given it our ethics inside
to some approximation, but it will have a better ethics than ourselves, almost by definition and to worry about
the values of any intelligent systems. We build
is analogous to worry about the values of our descendants or a future teenagers where they might have different values, but they are an extension of ourselves and now we're talking about eight, an extension of ourselves that is more intelligent than we are across the board, and that could be slightly misrepresenting him here. But this close to what he advocates that there is there is,
something about that. They should give us comfort it almost in principle that there's just no. Obviously we could stupidly build a system that is gonna play chess for, though the whole world against us, that is malicious, but
wouldn't do that and what we will build is, by definition, going to be a more intelligent extension of the best of our ethics.
I mean that that's a nice dream, but as far as I can see its
is nothing more than that
There is no reason why the capability to make decisions successfully is associated with any particular goals,
your family of gold structures. So let me give you a much simpler example. Take a chessboard, you can look at a chess board and you could say well the notion of czech major just cause
and of inherent in the chessboard in any intelligent system would cease from
chess board and and and the shapes of the pieces that that check made his is the inherent objective of this game,
and when would therefore
learn to play chess in
and be an enjoyable chest partner for us, but this is completely false because you can play reg
which ass or you can play. Suicide chess is neither to completely in some sense opposite objectives, the opposite, definitions of what winning means
in the game, and neither of them is inherent to the chess board and I can build a programme to do either of these two things. So there's no there's no sense in which objectives that would tend to produce behaviour them that we are happy with. Ah I, just intrinsic to the world
We know that their activity is certainly don't. Don't think that viruses, and so on that day, just do whatever they do to achieve whatever they want, which is you know which evolution has has created them to do so adjust. It just seems like wishful thinking that has absolutely no causal basis. Other than that, you wanted to be true. The fact that
Humans have been talking about analogy to our own descendants. So humans,
We evolved over well,
globally hundreds of millions of years to function in a certain way
and one of the things that we have is we have built in biological wiring. That gives information about what is desirable,
we don't like hunger. We don't like pain. We are drawn to pleasure,
and so on, in the desire to reproduce these things are built in biologically their approximations too.
What we might think of as an evolutionary fitness function, the and sometimes are not very good approximations. For example, many of them can be satisfied by by ingesting certain chemicals
and that usually does not lead to evolutionary fitness even
It saddens fight all seems to satisfy all the boy.
Biological bearing so in some sense we can. We can short circuit and fool the drives that biology
build into us in a way that
would be evolution narrowly unfit
we also have, I think,
a way of learning about. Are you know what our objectives are and should be free
observing the behaviour of other people around us, and this is a key idea that
we ve been using in our work on value alignment, which is the idea that, when any intelligent end,
DE behaves in the world is revealing information about its internal preference structure, about what it once you know, and as this is a very common sense idea. If you see me struggling
out of bed, bleary I'd going down to the kitchen and doing stuff with a machine
that makes grinding and steamy noises and so on. Eventually, I get myself a cup of coffee and a drink it you can turn off.
While this behaviour that that that's that's what I'm trying do I,
they get. I wanna have a cup of coffee in the morning
it is perfectly normal. As humans perceive each other, we are constantly trying to explain. I mean this is sort of how
how the human mind works. We were always trying to explain
we perceive and when we perceiving
television behaviour by others we're internally explaining it in terms of objectives, and I think is likely that the human species actually has a built in.
Process not just for explaining the behaviour of others in terms of objectives, but actually then
assuming those objectives for ourselves as as we grow up,
So we internalize and may become our own objectives because they are explaining
behaviour of appears and parents, and so on.
So I don't mean to say that this is the end of the story, but these two things the
the biological wiring and this more social process which which people sometimes called.
Cultural transmission of value system, but is not necessarily cultural transmission by some weeks
aiming to you. Ok, this. These are the values of our culture, rule, one rule two or three there's much more pervasive process of of.
I'll use absorbing values from from the of
if you're of everybody else in your society, so I think these these processes are very complicated and they still don't work that well I mean, from time to time, groups of humans sort of go off the deepened and end up it values systems
are highly inimical to the continued existence of the groups. Are they gonna? They can come cultures that are very vital.
Onto a nihilistic,
and that doesnt end up well, nothing things! Don't go well for groups that develop those kinds about these systems, but what about them?
later claim. Where that we will we'll be safe, because we will together these super intelligent machines, quite literally to our brains. We will essentially become the Limburg system of these new machines
and therefore, by definition, their goals, both long term
Instrumental will be anchored to our own values system. You see
That is a durable basis for hope. Not really, I do think, that's a possible direction that that we, we may see
electronic enhancement of human cognition and
people hold that out, as I hope in the sense that
will grow our own intelligence by by electronic prostheses.
That will enable us to compete in this chest match that people think of is inevitable and unfairness. I should say I think, that's David Deutsche his position as well they'll be extensions of ourselves in in every conceivable
hence, but we don't use machines. Currently, that way mean there
there are lots of- I mean there- are just lots and lots of machines here there's
machines that kind of are extensions of ourselves. You maybe it maybe my laptop is he's an extension.
It serves as an external memory and and remote communication device and so on, but when I
about Hydro Electric Power Station, the gets it's not an extension and myself it's it's a whole other thing in a bicycle, ass, sort of an extension myself, but is a is a jedi craft and just an extension to me. You know I mean it's. It's is quite happy to fly around without me or I could be a passenger in it and I think we will. We will have.
Intelligent systems that a separate from our brains physically separate. We know we want that. We want robots that can
where we can say. Ok, you know go into
is coal mine and rescue the minors who are buried in some Rockville
You know, there's all kinds of reasons why why you want autonomy and not have only devices that are physically connect?
to our own brains. What I want to do what she is
away from his idea that that we forecasting were either forecasting do more with forecasting you'd appear, it seems that there are developed and possibly could follow, that that don't seem to have very
good long term prospects for human rights, and there are other parts that do, and I think what we have to do is find the path that do have good long term prospects. For this
when race, and that means understanding how
design a I systems so that they are highly capable that they do that.
Find things that we want them to be able to do, but at the same time that day
We remain fully under control and so visitors
the girls I have I have a new centre was funded by the open philanthropy project and so that the goal of the centre is it. We calling it proved to be beneficial
hey. I systems we would like it
What sort of pious hopes that, if we design thing this design things this way, it should be good we're. She won't mathematical, theorem, saying that
certain kinds of designs for I systems are intrinsically safe and, oddly enough is quite easy to explain in a sense one key idea. So if you go back to what we know said right, we better be quick.
Sure that the purpose put into the machine is the purpose which we really desire so that that's operating on this
function that the only way to design and intelligent machine
that we formulate the objective, and we put it into the machine- am that's precisely the source of the problem right. What have we done
put exactly the right objective and we will don't even know how to write down properly
we're gonna have all these problems book and you can you
have useful machines where you don't put in a precise objective and the answer seems to be yes, we want the machine
in particular because we don't even know ourselves really. What would our true objectives are? What will truly make us happy or not? Happy things? We don't know it yet then
We want the machine to be explicitly unburden, about the objective that the human has
The machines goal is still to try to optimize it to help the humanist.
Whatever it? Is? They really want
the machine needs to be clear that he doesn't know what that is released in the night. Precisely and oddly enough, this is this sounds like a very simple idea, but it hasn't really been explored
in the field, because, as I as I mentioned earlier, you know all these fields ay. I statistics operations, research control,
theory. They all assume that the objective is just something this exhort specified Minnesota by definition, correct
And is put into the machine, and why
what I would argue is the best that assumptions false and if we have machines that are uncertain about
Jackie, then they will behave in very different ways. For example, they will, though, ask questions like what would it be a good idea? If I did this or
did you do with? Did you really mean that you
everything everything to turn to gold or ok, how bout doesn't sound rights, not consistent with other things? You said in the past.
Last year. Maybe what you meant was just the things I point two and say: Abracadabra will then I'll make those things turn into gold, but
you. Don't want your food and drink to him to go because then you'd be dead, and I know I'm pretty sure you don't want to be dead right. So that's the kind of conversation you want to have with AI system and
It can only happen with an eye citizen. That is explicitly the uncertain about your objective and you can tell us
All kinds of things about what you want- and this is just its evident about what you really want, but it should never be taken as literal truth about what you want
And by definition it would remain perpetually open to your saying: that's not what I wanted or that's not what I want and that's not what I meant so that the EU that you can get it would be a break to pull.
It is somewhat counter intuitive. If you imagine, building a truly super intelligent
general eyes so that its this is a mine that exceeds us in every capacity that we have, and the only difference is that we built it to serve us in some sense again was leave consciousness out of this injustice.
That you can get human level intelligence and beyond, without consciousness necessarily coming along for the ride. As you said at the outset, the existence of consciousness create another problem which we might want to talk about, but now we have an ethical obligation towards this super intelligence system that can be made happy or suffer or be deprived of happiness in
in ways that perhaps we can't even imagine- and I would argue that if we build a superhuman, I that was superhumanly conscious. Well, we ve just built something that is actually ethically of greater concern than than we are just as where do of greater concern within squirrels and crickets up the Leslie. That aside, we have a super intelligent artifact that we have no reason to think is conscious, but
if it is in fact, super intelligent across the board and the what would that dialogue be like with our own goal?
our own preferences, because at some level this thing-
one at one of the reasons why we would want this thing is to tell us what we should want and how to deepen our wants and goals so as to maximize human flourishing to a degree that may be currently unimaginable in how do you keep a a slave that is in every respect?
more competent to judge the wisdom of what you're doing than you are. I mean the diva great question. I mean that the
The notion of human flourishing is itself problematic.
You know who used to define what constitutes human flourishing.
Some people might have visions of
you know, conquering the universe and others might have vision of you know a stable, peaceful,
dutiful agrarian existence and who knows- and I think one point to Make- is that
It would be nice to have a choice as opposed to being
given by conflict by limited resources by the sort of inexplicable tides of history that that sometimes just leaders to utter catastrophe you'd be nice to have a choice. How we make that choice this day? I can't tell you how we make that choice, but one thing is clear to me at least: is that one of the things humans, value
is autonomy and a future where every decision is made by a machine everything
difficult decision is made by machine and and are you
a material needs are met. This doesn't feel to me like a desirable future and it it might be the case that, but what a icy systems do for us in the long run he's give us this freedom of choice to have to have some ability to shape our future and of
these catastrophic problems that we frequently get ourselves into, but to some extent, to stay largely in the background so too, to give us freedom and help us understand.
What we do really want, but not to be constantly apparent for us, and
So this is a story by E M Forster called the machine stops, which I
I recommend to all your listeners and
written in nineteen o nine, I think nine. I it's an amazing story because it has the internet. It has basically ipads. Video chat has moves
massively online open courses, in fact, is what most people do is is give milk lakes,
fears and listen to move lectures
and and the machine is its golden?
The book, the machines that looks off to everyone material needs to the point where humans become in full,
people day they don't go,
outside anymore. They don't even bother visiting each other because they can do video chat. They become very, very a feat,
and in fact it even the sort of couch potatoes obesity is described in this nineteen. Oh no,
story which we think I was a very modern thing had heard
I've never read it. I dont associate Forrester with and hd well
the ability to ya. You truly amazing. In fact I would say it's much more pressing than most of what HD wells route.
So what happens to human society is. Is that as a society,
he becomes gradually more more dependent.
On the machine until it cannot function independently at all,
and gradually loses its like the disk process that we talk about now with with pilots, for example,
I regret the human body gradually loses its understanding of even how the machine works, and so when it when it stops working
he came in races in real trouble, so we we clearly would want to avoid that kind of future and and if you,
If you ask me what what is this scenario is a bust him has is
paper clips, an area where we accidently tell us, you can us machine to make paperclip certain ended, we know turns the whole planet into paperclip.
All the Kansas scenario that I mentioned. So though, it a very simple just thought, experiments just to illustrate the point
that seemingly innocuous or even
honourable goals by themselves can be arbitrarily bad, but he
you, if you ask me what is a likely.
Scenario that we may
follow. That will lead us
you know into a dead end. It seems to me that this in Feeblemind, Andy, Skilling and gradual loss,
Autonomy is is a much more likely scenario
do occur Eve, and we have to be vigilant against this
Have you ever seen, Wally, which is a very
first it movie. That's precisely the the scenario that its describing
for the future of the human race that we become in feed,
obese and completely dependent on machines?
serve all of our material needs.
And we lose autonomy. Almost completely yellows are actually had another podcast gas David crack. Our who runs the Santa Fe Institute has its concept of two different types of cognitive
artifacts cooperative ones and competitive one. The cooperative ones actually make us better, even in their absence his example is it with. You, learn to use an abacus
You actually become better at calculating, even without the abacus you can. You can do that you can. You can internalize the advocates and you can do calculations that wouldn't otherwise be able to do
and language is a cooperative artifact and at things things at the concept of numbers for things that we have invented that we we can internalize whether or not they there actually devices in the world, and yet there are other things that are cognitive, competitive artifacts that we actually we get worse because we have delegated all of this operation.
To a machine or to some other artifact which, when it breaks down, we no longer know how to do the thing we used to know how to do perfectly well, weathers long, division or or anything else flying plans. Yap driving cars is in the very near term, something we are going to, probably to the to the benefit of
tens of thousands of lives every year. We will blissfully forget how to drive cars at some point, because it will be irresponsible to drive your own car once robot cars are truly safe
you can argue that even without a I, the internet is already almost there in terms of if this machine stops it's hard to picture what would happen to a society like the United States with it with an internet
that lasted six months. Many think we are already fatally dependent on our technique.
Long before we ve birth, superintends and I did about face.
We recently very, very dependent. I mean our economy would would almost collapse,
I think we would muddle through their way
squarely in some sort of disappear for a good long. While, yes, I think we would, we would see possibly starvation in cities.
You know the complete termination of a large fraction of industrial production and air transport?
asian systems and so on
Do you see this need only when an airlines,
Online reservation system goes down on. You know they end up having to cancel tens of thousands of flight. So if it goes down for several days so yeah, I think this. This is just a small example.
Of something that we need to learn. How do we avoid this? Because, in other is always easier to take the institute's convenient without thinking about the loss of autonomy that it entails and before you lose out before you
suffer any consequences of lost autonomy, you would have the immense power of having successfully delegated you're a fish operations to a super intelligent, a eyes so the community that decides well. I just want intelligence,
my Beck and call that community will be far more power?
Then, by comparison kind,
luddites? You know the kind of the amish of the future where there
trying to maintain in their autonomy in the face of all of these, these immense computational resources that another disturbing wrinkle here, which I talked about and in my TED talk, which is even of
everything goes exactly as we would want it to go. So it is, though, we're just handed a perfectly obedient. Benevolent super intelligent ai. You can still see things going spectacularly wrong for us because we are not economically or politically poised to absorb this perfect labour saving device, because it is just they say, wealth creation
vice of the sort that has never existed it can design the machine they can build a machine that can do any thing it can replace any whom
labour, not just drudgery, but the sort of labour that that you- and I do so- you know in a perfect world. We would then all be free to just enjoy ourselves in every way that we want and to have those pleasures be supported, perfected, protected by increasingly powerful labour saving technology and so would be plain frisbee in the park, and no one would have to justify their existence by working because they would be more wealth and then
could ever consume just pulled out of the ether by these superhuman machines. But of course we are nowhere near ready, apparently to share this wealth in this way. What I pictured in a somewhat in somewhat of a cartoon, but a picture
we can, it would have trillion errors on the cover of our business magazines and need a fifty percent unemployment in the United States, so that there's a bottleneck layer that we have to figure out how to negotiate. Even if we were given a perfect version of this technology which, as you have already said, is by no means guaranteed, we need the political and economic mechanisms to use this gift for the benefit of all humanity and- and I dont really see many people.
Thinking about how to do that. Yes, oh remedies! That's a beginning of another hours, conversation at least two to talk about the effect on the economy and employment
and I'm not I'm
one of those who is as well. We know we ve had these technological revolution,
Four will have amended again and, and we always figure out new jobs, people to do because, as you say,
If, if the machine can now replace not just the physical,
Member of the human and and very few humans are, relatively speaking.
All physical labour compared to what was the situation in the nineteenth century and if you
to replace the mental labour than we know. What is what is left in a new era. There isn't just an infinite supply,
new capabilities, that humans have that they can that they can use for employment. So I think you know the briefly one thing that humans have is the fact that their human and the fact that other humans value the attention, the company, the services of of of human beings and as long as that's true there,
and they can still be used for roles, but the the kind of society, the economy that that entails. If it's not to be, as you described, a very thin elite supply
what it by a somewhat thicker layer of personal servants,
and then every one else. You know ninety five percent of the population being
essentially unemployed, just fed and housing entertained and left to play play that that to me it is a terrible future for the human race, so so on both of these fronts, what one
being in a? How do we think about? Are the value
autonomy in the future.
And the other being. How do we construct a functioning society and in the presence of of these kinds of capabilities,
I think that there is a limited amount that at the technocrats, the a I researchers and were bought, assists and so on
or even the economies can do. I think it has to be a cultural process, though your huge amount of cultural,
work has to be done to work through all these questions.
And and first of all, to give people understanding of what the questions are an end. I think either, for example, movies are doing a good job of this. Are there any movies that you think get this get these concerns right and then what? What's the most compelling vision of whether its utopian to stop in that you ve seen so eager
there there are bits and pieces, the story that have done well, indifferent movie, so I think transcendence gets across the idea of of intelligence explosion in Canada. Does the same
of humanity? Losing control in the face of a much more
elegant machine. You know some other parts, the plot a bit less it less excellent, but it has excellent aspects. I think Ex Mckenna gives you a very
the essence of the as we talk about earlier than the Black Box of it, of a system that
he's very intelligent, but you really have no idea how its working or what its objectives are
or what's going on and in the EU
There is a one on one since you it's it's like the chest match idea, except in micro, bosom, less linger on that for a second
is it that displayed a property of of an intelligent system, which I think, may people find totally counter intuitive. The idea that intelligent machine we would build might lie to us or manipulate us to meet its own goals or even meet goals. We ve given it, but it sees another way of meeting them as a reason to think that manipulation on the part of superhuman. I is implausible and in principle
No, not at all. We have poker playing programmes that that bluff, so in its arms and slight, they lied to us to be. Does it Boca? It seems to fall right out of doors having goals in the first place. You I mean, did deception, I mean you, know kid, kids, let me sit a fairly
early age. They have goals and the they lie in order to achieve them.
They say. Oh no, I haven't seen the candy daddy. I have no idea where it is now. I think of
the stories that
people, whether it lay people or the argument, Anti tell themselves in order, not
have to worry about it. I just stories, and I I wrote recently
He wrote a paper where I listed about fifteen of the arguments that have come out of the air I community as to why there is no reason to be concerned and when you, when you look at these, they they all have the waiver of a kind of a fright response.
A joke, kind of denial and the arguments are just not worthy of the people who are making them.
And I think there
our arguments, which say that
supervision? I is a long way off, and as yet we don't have much sense of of what it will be like what shape it will have at all, and so it's hard tee to know what to do. Even if you were a knowledge that potentially in one kind of superintendent tough Ocean, a system could present a risk to the human race, but I think you know that's a legitimate line of
argument, I think our research suggests that, in fact, even in the absence of any details about what assistance will be like we can, we can start to do work that will allow us to construct templates that, unlike more classical
fines of optimizing of objectives. Do deuce to be potentially safe. They clearly have better properties from
You have you? U remaining in control.
Avoiding these miss alignment issues. The reason why the time horizon
in for me is a deeply uninteresting, and an unconcern thing to reference is twofold. One is that there is a quote I used of yours in my TED talk, which suggested that we have under different descriptions, a very different response to certain blocks of times over you, you said,
I believe this quote originates with you, but it's I think you first appeared in the article you rode jointly with MAX End and Stephen hawking in it in a british paper, but I think I've heard you use it at the keys to the conference. We were at an import Rico couple years ago. The idea that if we received an email, I think you said from AIM Alien civilization,
and it said, people of earth we will arrive on your planet in fifty years, get ready, and we would
far more motivated
then we are when we hear a technologists denying that there's a problem here, because it can take these fifty years to develop so fifty years is just not that much time once you imagine this thing actually arriving on a certain day, but the at the other problem for me is that lets say we didn't we
knew it was, can be fifty years or a hundred years. We still don't know how long it will take us to do it safely. Me that's only consoling on the proposition that that
you know it you eat. It only takes you forty nine years to do this safely. So you ve got enough time to you.
The right to an end, the idealistic, the other thing is that you know with with, for example, the aliens who take they're gonna, be fifty years or an asteroid where you can calculate the collision,
time achieving super intelligently. I requires breakthroughs that they can happen overnight
and you know the example I often gave in talks is is what happened? Nuclear physics, where the physics establishment
as embodied, for example, by rather food
who is partly the leading nuclear physicist of his time was convinced that
even though they knew there was massive, a massive amount of energy locked up in the autumn. They were convinced that there was no way to release
Ah, you know in relevant, gave a famous speech.
On September, eleventh, nineteen, thirty three, where he said that anyone who talks about this possibility is talking moonshine
The next morning, LEO Szilard figured out the
key idea, which is a chain reaction, that's mediated by neutrons, so the nuclear chain reaction, so it went from never too
do you no less than twenty four hours early the next morning, that say
sobering. Analogy though I like, I would say it was hard to quantify these things, but I think we need about half a dozen breakthroughs of that magnitude.
To get something that we would call General Ere, I
it is very hard to predict how. But given the rate of investment that the massive concentration of intellectual resources into the feel right now and in the fact that progress seems to be accelerating prompt one problem after another is being knocked down that that you know has been problems of decades of work.
Yeah. I just think it be much optimism to say: well, you know it was still a long way off
I am seeing you know what one of the arguments that I'm seeing in a community of fifteen.
I mentioned one that seems to be cropping up quite often, is well. No, don't worry, because in fact
human level, s eyes impossible, so we can get there. It is like what
my committee, has been saying exactly the opposite. For the last sixty years, the philosophers have been saying that is impossible and a community
has been saying: no, no, it's not impossible and you just confused and an old fashioned and an absolute
Lee human level, I as possible, while the argument that attempt
possible presupposes that there is something man,
all about a a computer made of meat, or that is something that our brains are doing. That is
something. I cannot believe mediated in any other physical system were or any physical raised in that way, you're going to dance and its services
is not is not as is not as complicated as that they present no justification whatsoever or being impossible other than well. If was impossible, I don't have to worry about it,
so I'm just going to assume it's impossible, rightly made in its so inconsistent with our own history as a field, and it comes with absolutely no technical justification. What's
However, even even of the magical kind that it's not only be a form of denial and in that disappointing
Is this paper of yours available online now. Can I can I link to it on my blog from more where I am bed. This part has not yet
Oh buddy, it's impressed. I think it will come out of its it's a chapter in a book that will come out from the early next year,
I wanted to say one one more thing that people may want to watch, which is the the tv series called humans which come
comes from rising channels,
for in the UK and to me that gives us a very clear accounting of the kinds of socio ecological issues relating to employment, sense of purpose, autonomy and sullen that would arise in in the series. The assumption is that we have created effectively human level.
Not particularly superhuman but human level intelligence embodied in very physically realistic, Android stay there
look very much like human beings and does so it's it's very, very interesting. It very much done not in a sort of a giant syphon style, but in looking at a family that the people in the family that particular
Android, who works with the family and only when there is a logic plot which will have to watch the series of doubt about, but I think it really in its white excellent in in describing the very practical day to day adjustments of human society to the presence of the is intelligent, intelligent robots. We ve been great. They ve been very generous where their time,
and sorry about the law sleep at land briefly tell us: are you on social media surname web address your twitter account? You want people to know about so much
my web pages. The easiest thing is just a type my name into Google that'll. Take you straight to my web. Page will be on my blog as well thanks against Europe, and a real pleasure
like you them before. I close. I would like to thank those of you who have begun supporting the podcast,
This has allowed me to spend more time doing it, and it also has inspired me to begin thinking creatively about how I could make us a more substantial project, and this could entail many things that could entail renting a studio where I can do more of these interviews face to face rather than over Skype, which would improve them significantly. It might allow me to hire producer who could bring a level of professionalism,
this that I can't seem to muster on my own when I listen to podcasts like serial or any of these other highly produced, shows their aspects. To that that I would like to be able to emulate here. I can hire a research
that could help me move into new content areas and I could even produce live events associated with a podcast. This sort of thing becomes possible once this achieves a stable basis of support. So if you have begun helping in that way, please know how much I appreciate that
and if you like to help, you can do that at SAM Harris DOT, Orgy Ford, slash support or you can support the pod cast on a per episode basis through patron Dotcom, forts, Lash, SAM Harris
and again, many thanks if you find this podcast
I believe there are many ways you can support it. You can review it
Itunes or or wherever you happen to listen to it. You can share
social media with your friends. You can blog about it or discuss it on your own podcast, or you can support it directly and you can do this by subscribing through my website has samharris dot org and there you'll find subscriber only content, which includes my ask me anything episodes. He also get access to advance tickets to my live events as well as streaming video of some of these events
and of also get to hear the bonus questions.
Transcript generated on 2020-03-24.