So, I’ll start with asking you to share your name and your position at Carnegie Mellon?

Richard Scheines, and I’m the Dean of the Dietrich College of Humanities and Social Sciences.
Professor in Philosophy. And I also have appointments in Machine Learning and in Human
Computer Interaction.

So, Richard, can you tell me a little bit about how your work in AI began?

Sure. When I was a graduate student at the University of Pittsburgh in the 1980s in the
Department of History and Philosophy of Science, my advisor Clark Glymour and I were working
on a project on causation in the social sciences, and he sort of gave me a stark choice. On one
hand I could go read all the dead white men philosophers, right, and what they said about
causation which nobody in social science or statistics cared about or read, or I could go read the
social science and statistics literature and really confront what people were doing and what they
cared about, in social science. So, I sort of took that route and then I quickly discovered that and
to make any dent in that I had to learn a bunch of statistics and a bunch of computer science.
The basic thing that happened was once we sort of figured out a minimal way to represent
causal structures which was these directed graphs, you have an X that’s some variable that has
an arrow to Y and has an arrow to Z, right, and we understood a little bit about what predictions
those causal structures could make in data, we quickly figured out we couldn’t compute
anything more complicated than a couple of variables, and so we had to use the computer to
help us compute it because we were too stupid to do it reliably. So, I learned how to program
the computer a bunch, that helped enormously. And then, we added many other people to the
project, so it really wasn’t all me, it was a bunch of people. And then what happened was we
figured out that if I had data to start with, I could learn from the data something about what’s
causing what. But I couldn’t learn exactly what was causing what. I could learn it’s one of a
series of theories. One of five theories or ten theories or a hundred million theories, right? So
sometimes you were lucky, and the data could tell you a lot and sometimes you were unlucky,
and the data could tell you just a little. But what we really set out to do was understand what the
data constrained and what it didn’t, so you could tell exactly what you could learn and what you
couldn’t learn from the data. Which was extremely exciting because, you know, you don’t want
to say more than the data tells you, but you don’t want to say less either. And to do that you had
to start with data and then compute all the possible causal explanations that would explain the
constraints or the things that you could see in the data. So it’s like reverse engineering, once
you say here’s a theory, I can see what it predicted, that took a lot of computing, but then once
you had the data and then you wanted to go back and say what are all the theories, that was
where sort of the AI part of it came in because you had to be extremely efficient about how you
did that or you couldn’t possibly do it because as you increase the number of variables you’re
measuring, the number of theories goes up super-exponentially. So, by the time we have five
variables, which is kind of small, you can’t write them out cause there’s twenty-seven or thirty
thousand different theories. By the time you have ten variables, there’s like ten to the
eighteenth. So, by the time we have twenty, you know, you’re in space that nobody can possibly
articulate or search without being super intelligent about how you do the search. And so, a lot of
the early work we did in the late eighties and early nineties tried to pay attention to how we
could efficiently search that space, right, in order to find out anything from a series of variables
people measured. And so that’s where I sort of confronted, you know, a lot of the literature or
research on foundations of AI. And the second thing to say to that is a lot of what’s done in AI
isn’t looking explicitly for, you know, a causal theory or a full theory of the world, it’s looking for
something a little bit more simple, right, like if I had this x-ray, you know, does this person, right,
have pneumonia? Right, and that’s sort of what machine learning is focused a lot on.
A diagnostic.

A diagnostic that’s a very clear, right, predictive or categorization or classification task. But I
think AI’s a much bigger task, right, you have to sense what’s going on in the world intelligently,
you have to actually make decisions about what’s happening. And build a theory about what’s
going on and then decide what to do, how to act. And so, when you really think about, you
know, what do you have to know in order to decide what to do or how to act, some part of that is
causal, right? Because it turns out that I can predict, you know, very closely, right, what the
weather is outside if I just look at what people are wearing. But obviously I can’t change the
weather by making people change what they’re wearing. Alright and so, to teach a computer or
robot that, that means you have to impart or represent at least some causal knowledge
otherwise it’s easy for the robot to say, yeah, I think I can predict it’s somewhere in the mid-
sixties or somewhere in the mid-seventies, right, and if I want to make it ten degrees warmer,
I’m going to have everybody take off a pound of clothing. That doesn’t work. Same thing with all
sorts of other examples, but that’s the correlation is not causation. But foundation of almost all
machine learning is in some sense being extremely sophisticated about what will loosely
determine this correlative relationships. So, the whole task for us was, you know, can we find
out more about what’s driving, right, the underlying world and the data that we’re producing. And
so that’s actually sort of my take on it. It’s all been about representing in a sophisticated way
what’s causing what and then being able to learn about it from data you observe, either by just
passively looking at the world or doing experiments that say okay, I’m going to actually do
something and see what happens and then make inferences about that. So, the first is sort of
called observational studies or non-experimental studies or just passive observation. And
there’s a whole big field that deals with that because most of social science, they can’t do
experiments, right? And then the other is in situations when we can do experiments, we can
intervene, right? A lot of psychology puts people in the lab and they intervene on one condition
or another and then they see what happens. So, they can actually do something a little more
manipulative. So, drug testing, all that sort of stuff is more experimental design. And so, learning
isolated causal relationships in that context is pretty well studied and pretty well understood, but
inferring what’s going on causally from a whole lot of observations just passively, that’s a big
challenge.

Yeah, so can you tell me a little bit more about how this foundational work was applied in
different ways through the course of your career leading up to now? It doesn’t need to be
comprehensive, but maybe one or two examples.

Sure. There are lots of different areas of sort of science, right, which really need or use this
work. Some of the early work we did was in psychology. So, when psychologists ask
questionnaires of people and they use what’s called the Likert Scale, right. So, they give you a
statement like you can’t sit through a movie because you get too restless and then you have to
say I strongly disagree, I disagree, I’m neutral, I agree or I’m strongly agreeing, that’s called the
Likert Scale. And they ask many, many of these items to measure one particular right thing that
they think might apply to your brain, like whether you’re impulsive or not, right. You might not sit
through movies, you might be unwilling to take a certain amount of time to make decisions
about travel, lots of different questions they could ask. And so, inferring something about what
the underlying structure, right, that’s going on psychologically became a way we applied this
technique in helping psychometricians, psychologists and the like do that sort of thing. So, to
make a long story short, one of the ways I applied that was to research on whether or not lead
in exposure causes deficits in children’s intelligence, right. So, there was a lot of research on
lead poisoning, right, it’s been known about since really Ancient Roman times in industrial
applications. But low-levels of lead didn’t make people go to the hospital and get clinically bad,
there was a huge debate about that for many, many, many decades. And it wasn’t really until
the eighties that people really had some solid bit of evidence that kids who were exposed to
lead, right, actually showed deficits in their cognitive function. So, combining the work about
what sort of stuff formed the underlying constructs that we could use and the techniques for
discovering like what’s causing what, we ended up reanalyzing a lot of data on lead and IQ that
was actually taken by Herb Needleman. So, he’s a hero. Let me digress for a second to talk
about that story cause he’s a great hero of Pittsburgh. So, in the lead industry which was used
for gasoline, right, this totally revolutionized the way the auto industry functioned in the 1920s,
engines knocked, they didn’t get good gas mileage and when the U.S. did an estimate of what
the fuel supply would be like going to the future, a study in 1919 came back and said in 1933
we’ll be out of fuel. Done. Over. And so there was this enormous pressure put on the industry to
find better ways to, you know, be fuel efficient and when lead was added it was like a miracle,
nothing stopped fuel efficiency, this huge amount of work happened in adding lead to gasoline
and in the factories that produced the lead, people died, all the time, and there was a huge
outcry and in New York City, lead was banned in gasoline temporarily in 1925 while they figured
out whether it was safe. How did they figure out whether it was safe? Well one of the ways was
a surgeon general’s committee, blue ribbon committee was put together. And right here in
Pittsburgh, right here at CMU, in the building that used to be the Bureau of Minds that is now
Hamburg Hall with the Heinz School of Public Policy, they literally hooked up a truck and had a
palette in the back that had roosters, right, other small animals and rode around the courtyard
for almost 24 hours, right, with leaded gasoline, so the fumes would go over the animals, and
then they took the palette off and they said the animals are fine. We must be okay. And that was
the last time we really worried about lead in gasoline until about the 1970s. So, that’s a fun
story. So, Herb Needleman was a hero because he came in and he did some actual studies in
Boston where he took children’s baby teeth, and as they fell out, he could measure how much
lead had been deposited in their teeth over the course of their entire childhood because lead
poisoning is like temporary in blood, it eventually cleans it out, but lead in the teeth that shows
how much you’ve been exposed to. So, he had a much better measure of lead. And then he
correlated that with all sorts of deficits in student performance on IQ tests, deficits in behavior,
impulsiveness, etc. etc. etc. and it was a very controversial study as you can imagine and
everybody was worried because the basic problem there is that it might be that kids who
actually consume or are exposed to more lead are also exposed to less stimulation in the home,
right, and have other sorts of challenges they have to overcome. So, to make that long story
short, we came in and we reanalyzed the data and applied our techniques and managed to
actually make a fairly compelling case that even after one controlled for all the possible
confounding there was still a very clear and detectable signal that low levels of lead exposure
chronically really cause deficits in children. And now since then that literature has exploded. And
now there’s literally thousands and thousands of studies to demonstrate that all the people in
my generation who were exposed to lead and gasoline in the air, we probably could’ve been a
lot smarter. I really could’ve been somebody. So, that, that was one of I think the first
applications for us became sort of an important benchmark. And then, since then there’s been
applications in economics and psychology and educational research and biomedicine, etc. etc.
And very happily for me and for I think the group I work with, there’s a group over at the
University of Pittsburgh who are in biomedical research. And we combined forces with them
about eight years ago. And we applied for a National Institutes of Health Center grant and we
got it, and it’s now a center for causal discovery which is really applying these techniques and
developing them further, right, but applying them to cancer, lung disease, and brain imaging
studies. So, another, another fun story I can tell is when you look at the brain and you say what
can I measure about activity in various regions which you’ve probably seen in the pictures of
fMRI studies where they have light blue or dark blue or red to light up where you’re active. It’s
very difficult to distinguish people who are on the autistic spectrum disorder from neurotypicals,
right. That’s very difficult to do. So, this group at Rutgers, they actually put neurotypical and
spectrum people through fMRI and then tried to use our techniques to infer what’s the causal
structure among regions in the brain. So, right, you take in visual input and they did face
recognition task, they did theory of mind, so if I’m trying to understand from your facial
expression what you’re feeling, right, if you’re smiling and I’m a neurotypical I can attribute some
state of mind to you. And if I’m on the autistic spectrum, it’s more difficult for me to do that. So,
they looked at the structure of the brain and they discovered that in the theory of mind tasks
vast difference between the way your brain was causally arranged if you’re neurotypical from
the way your brain is arranged if you’re on the spectrum. So that was an exciting...

Yeah.

And then now, there’s work in cancer that’s looking at inferring whether or not certain genes are
real drivers, or certain combinations of genes are drivers in cancer. So now we have a bunch of
hypotheses that have come up and said that these are genes you should check out, and now
they’re in the lab testing out with rat or mice models to see if those are actually drivers. And then
there’s work in lung disease that was similar sort of sort, in particular we have a team at Pitt
doing work on cystic fibrosis and they actually think that they’ve discovered a new mechanism
through iron that might actually be involved and lead to eventually a therapy.

Yeah. It’s a considerable range. It sounds like your work has really been driving by an academic
pragmatism, but were you influenced ever by reading science fiction?

Oh yeah.

Really?

I mean I love science fiction, I always have. I’m not sure the work was influenced by it.

Yeah.

So, you know, no, I can’t say the work was influenced by it. I think that.

No causal relationship there.

Yeah, I mean the kind of work we’ve done you know in terms of the utopian or dystopian
versions of AI fairly low level and much more, you know, in the trenches from a work like what’s
the mathematical and statistical relationships that have, you know, we have to really take
advantage of to learn stuff.

It sounds like this baseline development of a methodology has just given rise to just an
extraordinary diversity in the work.

Yes.

In terms of its application.

Yes.

With various teams, various experts. And I also know that you’re an educator, as a sitting dean
but also as a professor in philosophy. I’m wondering if you can speak a little bit to what you see
as your responsibility in communicating with the public in regards to AI? It sounds like so much
of your work is deep down in the methodology of the work and yet that communication with a
broad public seems to be quite relevant and, and important.

Right, that’s a great point. I think the general public is so focused on the way out in the future
dystopian or utopian view of AI that all the stuff that’s happening sort of in the trenches, you
know, in my area or in other areas, you know, is sort of generally kind of lost to them, you know,
the things that come out to the public are the big hits, right. Or the big failures. And not what
was really, you know, was really going on in the bottom levels. So, I think that we’ve done a
very, very poor job of really communicating to the public. And, I think the public has done a poor
job in sort of what they’ve asked for of us. Because I mean, you know, when you go to see a
Steven Spielberg movie AI, right, and you know, that’s going to be big box office hit or The
Terminator movies, right which are going to be another dystopian version of AI, you know, all
those questions come up and everybody can ponder like well what are we going to do when you
really can’t distinguish robots from people in a certain set of ways. And what are we going to
actually have to do about that. But the state of the art, right, as anybody will tell you who’s
working in it, is so far away from that, that its sort of, you know, it’s still science fiction and still
more literary and more dramatic and fascinating to consider, right. And maybe we’re getting
closer now to the point where we’re actually going to have to make those dimensions of the
problem salient and we’re going to have to think about them. But I still think we’re quite a long
way away from anything that is being considered in the media. So, you know, when the public
goes to the movies, when the public watches something on television, right, they’re not going to
want to hear about the details of, you know, causal inference or the details about deep learning
in the thing, there’s some people who do, you know, they’re going to want to consider, you
know, what’s life going to be like, right, when we’re really in the Jetsons kind of world and you
can just say something and this robot will take over and do everything you want for you.
Including, you know, make decisions about whether your elder parent, right, should be
supported, right, in a particular way or not. So, I think those sort of questions are still going to
dominate. And I think that the reality about what’s possible and what, where the work really is
still quite a ways away from publicly consumable things. So, I’ve tried on various occasions to
make the work a little bit more consumable and exciting and I think that the general consensus
is that we failed completely. Now there’s a book that Judea Pearl, so he’s really a hero of mine
and a hero of everybody in my space. He’s done a lot to push the whole theory of causal
representation, causal inference, etc. and so a lot of the work we’ve done is very in sync with
what he’s done. And he is far more famous than we’ll ever be. He’s won the Turing Award and
he’s just this towering figure. But he just wrote a book with a science journalist that’s called “The
Book of Why” and this is an explicit attempt to really bring this work to the public so that people
can understand it, you know, who are very, very curious but really minimally educated in the
space. I read the first two chapters of it, my reaction of it was it’s pretty good but we’ll see. You
know what I mean.

Yes.

There’s still sort of a bunch of moments when you’re saying this is still going to be a little bit
difficult for people to really get their head around because it goes into a level of detail that
they’re not going to like. But.

But that’s part of the rational of asking about the influence of something like science fiction. The
literary and the cultural circulation of those stories and the ways in which we can signal a
specific genre sometimes to a public, but the ways in which that’s not always signaled in
mainstream journalism pertaining to AI.
Right. I mean I think that the thing that is getting out to the public which should get out to the
public partly is that the whole paradigm of AI and machine learning has changed from, you
know, let’s build a system we can preprogram with lots of rules that will mimic human
intelligence to a system that’s let’s set it up so it can learn. Might take a long time, might take a
million examples, but it will learn. And so that I think has really taken over as the paradigm for
where we’re going in AI. And I think that’s right. You know, I think that human evolution, right,
chose that we’re all born, right, as incredible learning machines, but we don’t know a lot. Right,
and so, the more adaptable and intelligent beings, right, tend to be, the more they have to learn
before they get to that point. And I think we’re probably going to hit the same, the same thing in
AI and computers. That’s my sense of it.

Yeah, so as we’ve been speaking with people, we’ve really been asking questions that pertain
to the, the boundary time, not far, far in the future, but really asking individuals who’ve been
practicing in this area for a long time to offer their predictive or projections into the coming ten to
twenty years. So, one of the things I’m curious about is, how you think AI systems have
changed the way that people worked up until now? And also, what you think will be some of the
most dramatic changes we might see in, in labor, in the coming ten to twenty years?

I think there’s two areas that are going to be really dramatic changes in the next ten to twenty
years. One I think we’ve all talked about a lot which is driverless cars, autonomous systems that
can do, you know, lots of tasks that we sort of take for granted now. So, I’m not going to really
comment on that because I think there’s a bunch of people talking about that and they’re really
a little bit unknown, but I think we’re getting a better sense of economically labor force like who’s
going to be put out of work and how. What I want to focus on and what I think has been ignored
in this discussion too much is the potential for AI in education. So, we have a group here at
Carnegie Mellon that was really started by Herb Simon way back in the 1970s and the 1980s
which was carried on by John Anderson, Ken Koedinger now and other people like it who have
applied artificial intelligence to teaching people, right, with a computer in an intelligent way. So
when we tutor, right, and I’m teaching you whatever it is, right, I’ll explain something to you, I’ll
give you a problem, I’ll watch you do it and if you are faltering if you are, if you’re stumbling or if
you’re confused or something I’ll step in and say no, no you should be doing this or you look like
you’re thinking this thing but it’s really that thing. So, I’ll intervene and guide you and for me to
do that I have to have a fairly sophisticated understanding, right, of what you’re trying to do and
then an interpretation of where you are when you’re trying to solve the problem. So, you start as
a complete novice, presumably I’m an expert teaching you and then I have a model in my head
of some sort that says okay she’s a novice, she knows this stuff, but she doesn’t know that. And
so, I have to teach her that in order to make her able to do this problem. And so, they took this
approach and showed that in domains like algebra, teaching students algebra or teaching
students programming tasks or teaching students geometry or other things that are fairly well
defined we could make enormous gains in how quickly people learned and how well they
retained the information later and how well they could apply it to new contexts. And so, this work
is now twenty to twenty-five years old and it’s led to curriculum and computer tutors that are
used by over 600,000 high school students every year in the U.S. and in lots of outside
evaluations, you know, it’s been shown to be dramatically effective. So, right, when we actually
teach anything, right, and I think that there’s many, many places in which we don’t have the
labor, right, so if I want to teach writing or if I want to teach, whatever I want to teach, I don’t
have the number of professors I need, right, to go out and give the students the kind of one-on-
one help that they would really require to actually learn in the way we’d like them to. And we’re
never going to overcome that because there’s just not enough money in the world or at least in
the way I look at things to overcome that labor shortage. So, to give students of any strata, right,
the kind of one-on-one intelligent help they need we need AI systems that can do that. And
we’re making huge progress in getting that to happen. And so, I think that that’s a very, very
underappreciated way when we talk about work force retraining, right, if you want to actually
gain new skills so you can be employable, what do you do? Well there are lots of online courses
now, most of the online courses are sort of designed for people who are already curious and
fairly well-educated, right. And they’re not really effective for people who are in a position where
they actually have to reskill, and they don’t have a lot of education or they don’t have a lot of
education in that space. So, what the AI is going to have a big influence on is really getting
those systems to be much, much more effective. So, I’m very optimistic about that and I’m just,
my only pessimism about it is it requires a lot of capital to build these systems and a lot of
iteration to actually make the systems so they’re actually happy to use so that you’re using them
and say this actually helps, I’m getting what I need out of it. But what they do is they provide us
this amazing potential to customize the learning to each particular person. I mean now the mode
of education that’s typical is you know I’ll get in front of a class and I’ll talk for twenty minutes
and if I’m enlightened, right, I’ll stop. I then get the students to do something like in small
groups, but I don’t have enough of me to walk around and correct every person it’s twenty-five,
thirty, fifty people. In the situation where we have the computer that can actually be truly
personalized and helpful every single person that can do some exercises can learn some things
and have particularly customized advice. That takes a very intelligent computer.

So, the tailoring sounds fantastic in terms of the possibility of that level of sophistication.

Yes.

Of responding to individual users and it sounds to me within the domains of the algebra,
geometry and specific subject matter.

That we got it.

That has very specific parameters, rules, method, perhaps grammatical structure, certain writing
conventions, I can imagine an instruction in teaching as well. So, there’s a lot of possibility
there. But back to that notion of communicating with a public or communicating with a user, how
do we work with teachers to not over reach the efficacy of these tools that continues to combine
the sophistication of the tailoring but also doesn’t translate to substitute? I know this has come
up.

No, no it’s a great question.

A lot in Baltimore County in, in Maryland because the number of devices has skyrocketed in the
public schools in Baltimore. And the number of students in a classroom has also skyrocketed to
close to forty kids in a classroom and there’s this notion of substitution for instruction rather than
supplement to human instruction. I’m wondering if you can speak to that a little bit.

Absolutely, it’s a passion of mine. It’s a compete red herring. So, start from the beginning, right.
I think it was completely evident that sometime in the 1500s, right, when printing was involved,
was really going, that many of the people who were teaching and concerned about education
got very alarmed because now the possibility arose where you didn’t have to have a teacher.
The book would replace the teacher.

It leads to the reformation.

Right, so bad things maybe by some people’s eyes happened, but I mean, the same fear arose.
Now we had this device which back then was quite novel, right, you could get books into
everybody’s hands, why are we going to need a teacher anymore? Everybody could just read
their way into education. Well, obviously, it didn’t happen. Obviously, what did happen was the
teacher was still essential but used the book as an auxiliary device, right, to get more
information to be available to the student and available to the class, right. The same thing is
going to happen with the computer. You’re never going to replace the teacher because part of
learning is social, and part of understanding what the students, right, need is going to be social
and even if we get a computer system that’s much, much better at understanding the social
millilux or the social situation or reading the face of the student so they can understand whether
they’re confused or not and we’re working on that. You’re still going to have an incredibly
important function that involves people discussing things in groups of every size, right, and then
lending the expertise that people have in a way that’s nuanced, that’s very, very sophisticated in
a particular case it’s very complicated and nuanced. So that the stuff that we have, that the
computer is good at, is exactly what you said, right? Very well-defined domains that are very
rule-based. Solving algebra problems is extremely simple at some understanding, right? Solving
complicated policy problems, right, about, you know, immigration policy, that’s not nearly so cut
and dry and it’s never going to be sort of a simple rule-based thing it’s going to have lots of
different dimensions that are ethical, that are historical, that are political, that are economic etc.
etc. So, I think that you’re never going to come to a situation at least in our lifetimes where that
level of complexity and nuance is better handled by the computer. It’s always I think it’s going to
be much, much better handled by people who are experts and by groups of people who want to
explore and talk about and understand a domain. But, just like the book, right, and the article,
right, in college teaching for generations now, right, has supplied students with something
common they brought to the class which they then discussed. The same thing can be done with
a computer but in a much, much more powerful and sophisticated way. So instead of me
coming into a class and having to show people how to solve a certain kind of mathematical
problem or a certain kind of legal problem, right, if they could go back and not just read about it,
but actually have very, very sophisticated problem-solving environments, so they can do those
problems and get feedback right away and learn how to do those problems, and then come in
and do much, much more nuanced case studies. I think that’s going to transform education for
the better, I mean I really do. I think that what we now assume is that when we actually want
students to develop some skills that are precursor to more sophisticated discussions we have to
give them assignments, those assignments have to be done, turned in, graded by a TA or the
professor, and then the student gets them back sometimes a week later, right, sometimes two
weeks later, depending on the work load of the TA or the professor and then the student
somehow is supposed to learn. Now we all know very, very clearly now from learning science
research, it’s much more effective to get feedback right away. So, and waiting a week is almost
a joke it’s so bad, right. So, if you’re trying something out and someone can sit down and say
here’s what I think right now, you’re going to learn a lot more. And so, the computer can do that.
That will be where the computer comes in and provides the labor that we will never be able to
provide. And I’ll tell you another, a little bit of my own history that I think deals with this that I
didn’t mention, when I first worked at Carnegie Mellon in 1987, 88, I worked for Wilfried Sieg
and he was a logician and we were teaching this logic course at CMU which was built by one of
the pioneers of philosophy and education Pat Suppes that was a whole course on the computer
teaching formal logic. So I was the TA for the course and what I discovered was everybody kept
getting stuck on the same problems at the same point in the course and the way that course is
structured they couldn’t move on until they had solved those problems and of course they didn’t
want to wait for me at two in the afternoon to have office hours, they wanted to get that answer
at two in the morning which I wasn’t willing to be available at. So, Wilfried had this idea that let’s
build a computer tutor that was intelligent enough to sort of solve these problems for the
students, and so that’s what I did, so we actually did AI for a computer tutor in logic, right, in late
80s, early 90s and then demonstrated empirically that this had a huge effect on student
learning, right. So, it was the same kind of thing. If you could give the students very, very
immediate feedback that was intelligent to sort of say what you are experiencing as a problem,
not what everybody else which we suspect is that it was much more immunable to the students
and much, much better for the students in terms of learning. So, you know, that was I think
where I first said this is really going to be a big help. And then there’s been a whole bunch of
work, and let me just comment on your very astute observation that in geometry and algebra it’s
one thing but in more nuanced less-defined domains is another, so there is a subfield of AI in
education that’s looking at what do we do when we have more ill-defined domains, legal
reasoning. And legal reasoning is much more rule-based than something like just, you know,
vague political reasoning, but it’s much less well-defined than something...

It’s logic, but it’s interpretation.

Right.

Applied to logic.

Right. And so, getting the interpretative side into the computer is totally challenging and totally
interesting. So, I think there’s a lot of research and acknowledgement in this field that to
conquer ill-defined domains, right, with this sort of approach, right, is a big challenge. Very, very
exciting.

Yeah, and within the education example or perhaps there’s another example that you might be
able to draw on, we’ve been really concerned with notions of power negotiation. So, moments
where humans may defer to an AI system by virtue of assuming its correctness so to speak by
virtue of its capability to process data that might be beyond what, what we as individuals or even
collectively might be able to work through. Or, the, the circumstances where individuals or
groups are actually pressured due to institutions or politics to defer to that system. I’m
wondering if you can speak to that just briefly.

Well, I view it as a problem that’s bigger than AI or the computer. I view it as a problem of, you
know, being in a society of any level, right, a family, where the kids are supposed to defer to the
parents but don’t want to. You have kids you know.

Yes.

There’s a negotiation about that power dynamic, right? Parents tend to think they know better
than the kids and generally I think they do, but not all the time, right. And, so there’s a
negotiation there and good parents do not by rule autocratically. They rule, right, somewhere
between autocratically and democratically. I mean my parents, my father always said to me
“This is not a democracy.” So, there’s power dynamics everywhere and I think that’s, you know,
the family, right, the community written on your block, the village, the city, the country, you
know, the company you work for, the institution you work for, etc. etc. etc. So, you know, I view
sort of these power relationships as something we have to negotiate every day and we make
decisions, right, as people who are put in positions of power over others and are under others
that have power over us, right. We’re constantly negotiating, right, these questions of, you know,
do we trust the people who are telling us this is the way it has to be done, right, or do we think
there’s a little bit of room in that policy to do what we think is more commonsensical, right. Do
we communicate power over others in a way that allows them to feel empowered within a
certain boundary and constrained within another, right? So, I just think this is one of these things
that you can never get away from, it’s just part of everyday life, like fifty-thousand times a day.
Alright, so, now you enter, and you put in the computer, right, and then somebody gives your
evidence, or somebody gives you a story that says in this particular domain, right, this computer
is more reliable than a person would be, right. So, if I go to a doctor and the doctor says “I think
you have, you know, disease X” right, obviously I don’t know as much as the doctor. I might
question the doctor because I think, you know they have a particular pet diagnosis they want to
do, and they’ve always done this diagnosis, etc. Like a surgeon you have to be suspicious of
because they do surgery and so everybody who’s a patient in the world today is instructed, if
someone goes, if you go to the doctor and he’s a surgeon or she’s a surgeon and she says
“Yeah, you’re shoulder joint looks like it needs surgery, let’s schedule it.” You’re like hold on a
second. Let me go get a second opinion from a physical therapist who’s stick is more like “Let’s
do physical therapy to fix you.” So that’s a power relationship and I think we have to be active in
figuring out whether we trust, whether they have biases, or interests, etc. And so, the computer
has the potential in my view at least to be a much more dispassionate and less agenda-toting
agent, right, to say here’s what I think I know from the evidence that I’ve gotten in, right. And
then if we have some data about whether or not that’s reliable, like you know if I’m taking an x-
ray of my lungs and the computer says this is what I think is going on in your lungs or now we
have an alum from CMU who owns the biggest database of dental x-rays on the planet and is
now interested in applying sophisticated machine learning and vision to seeing diagnosing,
right, what’s going on in your teeth and your mouth as a function of these x-rays because when
people are told by dentists that they have a certain problem and then the dentist wants to do two
thousand dollars of work, people tend not to trust the dentist because it’s not the doctor who’s
going to do it for free. It’s now the dentist who you’re going to have to pay. The evidence now
looks like people might be more willing to trust a machine which says no, no, you really need a
root canal or you need, right, something that’s going to be painful and expensive.

Back to your point on some level that also requires some level of transparency on the data
that’s fed into the system.

Absolutely.

To make those choices, right, and the sophistication in the user, the individual to be able to be
able to question that data source.

Right.

And receive a transparent answer also, right?

Right, so I find it totally fascinating what happens to all of us as people when we go into a
doctor’s office or we go into a lawyer’s office or any expert, right, and we’re put in a position
where they’re giving us an opinion and we have to make a decision whether we trust that and
act on that opinion, right. And of course, the data we have is so thin, right, you go into a doctor,
they have, you know, a diploma on the wall, now maybe you can get Yelp recommendations,
and so you can have some evidence, right. We read ridiculous amounts of things into their
demeanor. Oh, they seem to be confident, they seem to be capable, they seem to articulate
what they’re saying in a good way, they must be, right. And so, the ways we now judge
expertise are so flawed, are so painfully obviously biased in the systematic way. We can be
conned so easily, right? That I think that the potential for having the computer be experts, right,
is both very large in a good way, very large in a bad way. Because I think ways in which we’re
going to judge whether or not the computer really knows something are going to, by necessity,
be much, much more connected to trials of whether or not they’re reliable and less to “Oh they
have a nice facial expression. Or they seem very confident. They seem very relaxed.” You
know, that kind of stuff which we’re evolved to use in ways of judging expertise that we know
are incredibly flawed. So, yeah, I think that my sense of it is that we’re very generationally split
on this. Like I talk to my mother-in-law all the time. She’s eighty-seven now, but we’ve been
having this argument for fifty-seven years, you know. Her distrust to technology is so visceral
and so intense that she misses out on all the benefits of technology in a very, very clear and
systematic way. She’s utterly convinced right away that it’s just the worst thing we can possibly
do, right. And, I’m sort of, you know, in my mind at least, in the middle, so I’m much more eager
to take advantage of the benefits but suspicious as well. My kids and their generation, right, are
much more eager to just blindly adopt what the technology is doing for them because they’re so
bathed in from really early on. So, it’s partly a generational, you know, transition and partly, you
know, a way we had to navigate into what we’re, you know, going to do in the world. I think that,
you know, we’re exposed to be a little suspicious as we should be, but probably too suspicious.

And it’s not just the generations in terms of the individuals, but the different generations and
sophistication of the technological tools in those moments...

Yes.

To each generation too, right? We’ll close now with maybe just some thoughts on what you see
as valuable in the prospect of machine autonomy and also what you see as some of the
potential pitfalls with any example that you think is illustrative of your views.

Right, so I think the big advantage I think I mentioned before, I think in education we’re going to
have an enormous, enormous benefit and this is going to be something that really might, you
know, transform the world in a positive way. All over the world. I mean I didn’t mention this
project but in the Robotics Institute at CMU Jack Mostow worked for many years on a reading
tutor that would use speech recognition in a fairly artificial intelligence way to see if the student
was reading the right word and then give them feedback. And then he’s now turned this into
what’s called RoboTutor which we submitted as an entrant into the Global XPRIZE for Learning
and it’s now a finalist. It got to the final five, so now it’s being field-tested now in Africa to teach
in Swahili and English basic writing, basic reading, basic math, to six-year-olds in rural villages
all across Africa. The idea of the competition being the most effective tablet application for
teaching these things is going to get a ten-million-dollar prize and hopefully be deployed, right,
in thousands and thousands of schools all across the continent and everywhere else where kids
don’t have access to the kind of schooling that they would need to become literate in the way
we think. So, I mean if this works, right, and it should, this is going to really transform the world
in a hugely positive way. So, I think that that’s just an enormous upside. I think the other upside
is kind of obvious. You know, now when we get off the, the turnpike, you know, you have
EZPass, right. There’s maybe one or two toll collectors still left. But that used to be, right, filled
with people sitting, right, in this horrible booth, breathing in fumes all night long, and, you know,
mind-numbingly horrible, boring, menial jobs that it’s much better to have labor from the
computer to replace. So, I’m very optimistic and I think there’s a huge benefit in replacing jobs
that you really would wish on no one, right, with the computer. And then I think that there’s this,
you know, huge complicated difficulty that I don’t see so much as in a dystopian end game, but I
see it as a huge problem in getting there. I think that between now and some future in which we
really do have a more sophisticated and enlightened, you know, set of tools that would help
everybody, you know, there’s going to be an awful lot of displacement and we don’t have any of
the political apparatus we need, right, to actually manage that and navigate it with foresight,
right. Cause we’re reactive, I mean, the whole politics we’ve set up is we react to crises. We
don’t really have any way to intelligently look ahead and say, you know, if we actually invested a
lot of money here, we’d really prevent ourselves from a lot of pain on the way. So, my worry is
that we’re spending much too little time and money on understanding what’s the challenges
we’re going to face as we go through this transition and then taking the political steps necessary
to do it in enlightened and constructive ways so everybody benefits. I think it’s a political
problem more than anything else. And I think that any, I didn’t express this to you, but my
fundamental belief about all technology is by itself it’s neither good nor bad. It’s just like, you
know, atomic energy, it’s been incredibly important in a positive way for the environment in a
certain sense and then terribly harmful in nuclear weapons in another. So, you know, every
technology is both good and bad.

Any last closing thoughts?

No, I think this is great. I’m very appreciative that you’re doing all this stuff. And well let me
make a pitch for the two of you because you guys are teaching this course on AI and society,
and I’ve heard nothing but great things about it. And the basic message I get from the, the
audiences everywhere is can you clone yourself and do this a whole bunch more times. So, I’m
very appreciative you’re taking this on, and I think this is very exciting work, so thank you for
doing this.

Thank you.