In this episode of the podcast, Sam Harris speaks with Renee Diresta about Russia's "Internet Research Agency" and its efforts to amplify conspiracy thinking and partisan conflict in the United States.
SUBSCRIBE to listen to the rest of this episode and gain access to all full-length episodes of the podcast at samharris.org/subscribe.
This is an unofficial transcript meant for reference. Accuracy is not guaranteed.
Welcome to the maids Asthma against SAM hers,
I am speaking with renamed arrested for nay, as the directive.
Search at new knowledge and ahead of policy at the nonprofit data for democracy and investigate the spread of hyper, partisan and destructive narratives across social.
Edward she's co, authored a recent report on the the russian disinformation campaign, both before and since the twenty sixteen presidential election.
And I we talk about all that she's advised politicians and policymakers members of Congress, the State Department
Her work has been feature in the New York Times and the Washington Post and CNN and
and many other outlets he's a member of the council on foreign relations and a Truman National Security Project Security fellow she also holds degrees in computer science and political science. From
soon. He stony Brook as you'll hear Renee, was recommended me by my friend and former podcast guest Tristan Harris.
recommended her as an authority on just what happened with the russian influence campaign in recent
yours and Renee did not disappoint. So without further ado, I bring you Renee Tourist
I'm here with Renee Director Renee thanks for coming on the packet thanks, you have me SAM, I was insured
few through our mutual friend Tristan Harris had you know if just done
in mid twenty seventeen. I had written
say about bots.
and I'd. He read it and he showed at Facebook one enough
now. We discover them about sixty mutual friends, even though it never met, and I we met for breakfast couple days later and he wanted to talk about
What I was saying things I was writing about and how they intersected with his vision of of social platforms as having profound impact on individual.
All my research into how social platforms are having profound impacts on policy in society, and we had breakfast headed often think had breakfast again a couple days later, Sonia fast friends or Tristan has grown to so many people will recall, he's been in the past and I think he's actually been described as the conscience of
CAN Valley has just in terms of how he has been sounding the alarm on the toxic business model of social media in particular. So you touched on it there for a second, but give us
a snapshot of your background and how you come to be thinking about the problem of thoughts and also just a specific problem, or get me talking about Afy russian disinformation campaign and packing of democracy.
Yes, a sort of a convoluted way that I got two to investigating russian disinformation had actually started back in twenty fourteen
I became a mom and I was just move to San Francisco low, but prior and I had to get my kid onto a preschool waiting list which is not all
raising yeah, not like a nice preschool. Just like opera
and I and I and I I knew Calvin
hats, manufacture problems and I started good looking for the data sets the California to pump part department. Public health has public data sets where they tell you vaccination recent schools anyway, luck tonight
goddesses is a disaster waiting to Happen-
and am alone to hold a couple months later,
The Disneyland missiles outbreak, in fact it happened, and I reached out to my congressmen- is the first time ever done that I said hey, you know we should have a law
for this now we should we should eliminate the vaccine opt out and
they told me they were introducing something. So I said, graded I'd love to helping I've data signs background. I can mainly be useful as an analysed and what wound?
happening was that there was an extraordinary thing. Is that as the bill took shape, which was that the legislator,
were finding that pulling in their districts was about eighty five percent positively people really liked the
idea of eliminating what we're called personal belief, exemptions, the writer just kind of voluntarily after kids out, but these social media conversation was
like ninety nine percent negative. It was very hard to even find a single, positive, tweet or positive facebook post expressing support for this bell, and so I started looking at
why that was and discovered this entire kind of ecosystem of what was this hybrid between almost activism and me,
population. So there were very real activists who had very real points of view, and then they were
things like using automation, so the reason that they were dominating the twitter ecosystems that there actually turning on automated accounts, MR there's gonna spanning hashtags that any time use
fur anything religion, the bill in the hashtag, you would find their content. So this is kind of you know. The sort of
guerrilla marketing tactic and I thought how interesting that they were using it and then really
There were like fake persona in their there. There were people pretending to be from California who work
from California Power
figuring out. How are you assess, in effect, persona
are they were. They were created within days of the veil being introduced and they were. They existed solely to talk about this bill, and then I discover these communities on Facebook. Things with me,
like tweet for vaccine freedom, where there were actually moderators in the group who were posting instructions on for people from out of state how they could
involved and the answer was created- persona change
your location idea somewhere in California and then start tweeting, so they sort of me no kind of em
at the time it seemed brazen. Now it seems so quaint, but these tactics to shape consensus to really cool,
eight, the illusion that there was a mass consensus and opposition to this bell, and so a varies
small group of people using social media amplifier or able to achieve dominance to do just really all the conversation, and it led me to think this deserves
fascinating, because what we have here is this one.
activism work. There is kind of like a real
core and then there's some manipulative tactics layer on top of the real core. But if you're not look,
king for the Manipulation- and you don't see it and most people aren't going looking digging into this stuff. So it was a kind of a first indication that are policy. Conversations are social, come
stations were not necessarily reflective of,
me no kind of the reality on the ground of the step that we are still seeing in the polls.
It was. It was an ocean experience and then a couple months after that law is all in all done. I got a call from LA from these some folks, they Obama administration
the digital service, saying hey, we believe we have read your research published about this unwired hey. We read or research we'd like you to come,
down and and and look at some of the stuff. That's going on with ISIS
So you know I don't know anything about about ISIS or raw terrorism candidly.
is it not all you have to understand. The tactics are identical. The same kind of
no kind of owning the narrative owning the hashtags reaching out to people pulling them into secret facebook groups. The idea that the terrorist directly following some of these kind of radicalization pathways, these efforts to come
dominate the conversation any time there was a real world events related to ISIS. They would get things trending on twitter and so people need a minister.
In to understand how this was happening and what they could do about it so
that was how I wound up
getting more involved in this sort of a more official capacity was a first kind of conspiracy. There is than terrorists, and then- and then Russia was some. Russia was following the twenty sixteen election. There is a sense that again there had been
these bizarre bought operations and they were far more nefarious unsophisticated.
Anyone had realized and we had to do a formal investigation,
before we get into the Russia case, specifically howdy,
view the the role of
social media. In this you distinguish between the culpability or the negligence of twitter versus Facebook. Verses Youtube
are there right lines between how they have Miss played this war
or are they very similar in the roll, their plan at the end of kind of they really evolved, a lot of sense
fifteen in the early conversations about isis- there was
you're. Just a take you back to twenty fifteen. The attitude wasn't, oh god, we ve got
Harris on our platform. Let's get ahead of this right. It was you know, Facebook, to its credit today
attitude from day. One and justice is a violation of our terms of service. We take down their content, we find them. We shut them down. Youtube would kind of take
the beheading videos, as they popped up Twitter,
go back and you read. Articles from twenty fifteen is
I've been doing a lot of them going back in looking at the conversations from that time. You see a lot of sympathy
the four twitter and this idea that if you take down ISIS, what comes next since a slippery slope
It really is entering call on to ponder
How about Satan?
so you know I mean if we take down ISIS, I mean who knows what we have to take down next outta one man's terrorists is another man's freedom fighter and I would be sitting there in these rooms hearing this conversation saying like these are behalf
thing videos, you guys are terrorist recruiters. These are people
were killing people. What the hell is this
station. I a guy, can command around it, but that's where we weren't
fifteen and you know, go back and read things that people like to eat at United, he's like that. Yes, after putting out and you'll, see that that this was a topic of of deep concern. What would you know what would happen if we were to silence?
ISIS would we inadvertently silence things that were tangentially related ISIS and then from there? Would we silence you know certain types of of the victim
of his and so on and so forth. An end it
was a very different kind of mindset back then. I think that the context has changed so much over the last year in part because of stuff, like what trust not is doing on the Tec hearings, and I think the twenties
team was almost like the sort of bein a pearl harbor that made people realise that in a holy shit, this actually does have an image
action- and maybe we do have to do-
than to get ahead of this cause everybody's doing it now written recent article specifically about Facebook makes me think that it is that there is just an insuperable problem here. You can't put enough people on it too,
appropriately that the content and the algorithms don't seem to be up to it entered the mistakes that people plus algorithms are making are so
flagrant eminent, preserving. You know that the accounts of known
terrorist organisations there deleting
the accounts of.
muslim reformers or ex Muslims who simply say something critical about the faith and industrious people can't figure out which ends up apparently end. Once you view these platforms as publishing platforms are responsible for their content is, understandably, would want to given the kind of things were going to talk about, but I dont know how they solve this there's a lot of thy in interest on and others have done, a lot of work on.
Changing the conversation around culpability and accountability, and I think that again in twenty fifteen two thousand and sixteen you know there would be references to things like to CDA. Two hundred and thirty, the communication Decency ACT section two hundred and thirty, that gives them the right to moderate, which they chose to use us the right to not moderate and
norms. I would say that that involved in the industry around not wanting to be seen as being censored
in any way at the time, which meant that they left a whole lot of stuff up and didn't really do very much digging and then
the shift kind of the pendulum swinging hard in the other direction, which is leading to allegations that the conservatives are being censored and allegations. That per your point,
unsophisticated moderation. I don't think there is an article about this in the New York Times over the weekend has led to some disasters. Were they take down people
aiding extremists in Myanmar only be early extremists. Upso gathers. I think that the recognition that they are culpable, that fundamental change in the attitudes of the public has led them to to start
try to take more responsibility and right now, it's being done in something of a kind of a heavy handed way. You were with their certainly culpable for,
business model that I have kind of a less of a view of twitter here, because Twitter doesn't seem to have its business model together in the way the Facebook does, but clearly facebook per Tristan point that their business model promotes outrage and sensationalism preferentially, and the fact that they continue to do that. As I just selecting for these crazy conspirator
Oriole, divisive voices and then they're trying to ignore tight curate against those but they're still amplifying those, because it's their business model and at least that's the way it seems as of whom, on my route,
reading the New York Times. Is that still your understanding of
bad geometry over there. I would say that accurate. So I see
a lot of midnight. I try to focus on the disinformation peace. There are some people who work on privacy, some who think about monopoly in a lot of different grievances with textile firms these days. But I see a lot of the the manipulation specifically, I would say, comes from a combination of three things: there's this mass consolidation of audiences on a handful of very few platforms,
just because as the web move from these kind of inner decentralization, whether there has always been manipulation and disinformation and lies on the internet right, but the mask.
The addition of audiences onto a very small handful of platforms meant that, if
we're going to run a manipulative campaign, much like if you're gonna run it
pain? For you know, Pepsi, you only had to really blanket five different sites and then the second piece was the precision targeting right, so they
as business model. The thing that Europe offering to these are attention brokers, which means they make money. If you spend time on the platform, so they gather information about.
The user in order to show the user things that they want to see is that they stay on the platform and then also
as their gathering, then information it does double duty and that they can use it to to help advertisers target them, and I would say the last piece of this is
the algorithms at you're describing and the fact that four very very long time now they ve been very easy to game.
And when we think about what you're describing the idea that net outrage gets clicks, that's true and,
them, particularly things like the recommendation engines- they're not sophisticated
I have to know what their showing, so there is no sense of downstream harm
Psychological harm or any other type of harm
they know, is this content gets clicks and this content drives engagement in. If I show this content to this person, they're gonna stay on the platform law.
I can see no mind them for more data can show the more adds. So it's its beneficial to them to do this, and I think one of
Interesting challenges. Here is, as we think about recommendation engines that swear there.
In my opinion, a greater sense of cod,
ability and in a greater requirement for responsibility on the part of the platforms and that's because they ve moved into acting as a as well as a curator right there, they're saying you should,
see this and the recommendation engines in particular often surface things that are not necessarily
You know where we would necessarily want them to be showing. This is how you get it
like you know my Auntie vaccines right Thea. I had an anti facts account in account that that was active in active x groups,
and it didn't engagement, eighty people at a sort of sat in the groups and knew no kind of observed, and it was being referred into pizza gate groups,
so long before pizza gate was a matter of national conversation long before that, guy showed up with a gun and shot a pizza place. Thinking that Hillary Clinton was running a sex dungeon of the basement. These person, as that were prone to conspiratorial thinking
the recommendation. Engine recognise that there is a correlation and people who are prone to conspiracy in a conspiracy type. A would be interested in pizza gait, which would call conspiracy, type b and then soon
I've Q Anon started to show up in the in the recommendation engine and so that the question becomes. You know
where, where is the line you do what the platform is actively making recommendation here? These accounts have never gone and proactively searched for pizza gate and killing on it. There it there being sued
just to them. So where is the responsibility? Should we have
the recommendation engine not surface that type of content or is even making that suggestion a form of censorship? This? These are the kinds of conversations I think we'll start to see more of an two thousand and nineteen focus on the topic at hand, which is russian interference in, I guess democracies everywhere, but specifically they, the U Dot S presidential election in twenty. Sixteen and the recent report that you helped produce on this, which is runs to a hundred pages and I'll, put a link to that where I post this. On my blog first, I got a big picture orchestra political parties in question. It seems to me that many people- certainly most trump supporters, continue to doubt whether Russia interfered in anything in twenty sixteen and
It's just me. This is fake news. Is there any basis for doubt about that? At this point nope? This is just a crystal clear as a matter of what our intelligence services tell us and as a matter of what people like, you can ascertain budgets so
in online behaviour. It happened at his. There is really no
I'll say about the intelligence agencies. Now it happened, foreign governments now
happened. Researchers no
happened. The platforms acknowledge it happened, I mean beard there sure there can be some small group of people who continues to ignore live like ostriches, but that doesn't mean that it didn't happen and what are you?
with the charge that we do the same
thing all the time everywhere ourselves. It is not this really nothing to complain about her. Well, we planning we gotta, do it to each other.
At this point. There is evidence of that as far back as twenty sixteen, you know some things that insinuations
not all about matters. A whole lot of love me no evidence that domestic groups can and do this as well on and that's why we're right, where I keep going to when I talk about this topic publicly, is that this is
not a partisan issue. This is not a one in a one state
What one foreign actor interfering in one moment issue is a sort of
just an ongoing global challenge. At this point, if we're speaking specifically about
Russia, and when that happened, I
Think that it's? It's incontrovertible truth at this point and the other thing that seems incontrovertible, that it happened to favour the election of trust,
in many obvious ways and in many surprising way as that will go into, but it they were not playing ball. Sides of this was not a pro Clinton campaign and in Europe
or you breakdown three ways which their meddling influence things and or attempted tenfold
things were going to talk about one of them, but a bit run through those three quickly in and will focus on one. The first is their attempts at
hack, online voting systems and that that's been reported on elsewhere. Secondly, there were those just this very well known and consequential cyber attack on the Democratic National Committee and the leaking of that material through Wikileaks, and that was obviously to the brain disadvantage of the Clinton campaign. Then, finally- and this is what we
going to focus on. There was just that this social influence, based on the disinflation campaign of the sort that year we ve just described using Botz end fake persona, is and target in various groups. Have. This was surprising that when you get into the details of who was targeted and the kinds of
messages that were spread its fairly sophisticated and amazingly cynical, there's a kind of morbid
one you can imagine these people are having at our expense in how they played one community
yes and other an american society, so us focus on this third method, and this was coming from something called the internet reach.
Urge agency, what what will call them the IRA. Is you doing your report? What
is the IRA, and what were they doing to us
so the Irish as I can think of them a little
as I am social media-
marketing agency meets in Mantelish Ants Agency, so what they did to a large extent was they kind of built these pages. They built these communities, they built these persona us and they pretended to be Americans, Americans of all stripes. So some were southern conferred.
At summer, Texas, secessionist some were black liberation nests, really that all these prisoners they really round the gamut. What they were
Ling was they were creating pages to appeal to tribalism, so a lot of them
a lot of the conversation about the IRA or the last two years has referred to this idea, that they were exploiting divisions in Society- and that's true, but
notice that I had access to, which was provided by the Tec platforms to the Senate. Intelligence committee was the first time net,
Anybody saw the full scope, a korean or through the full two and a half years, and what we
saw there was not a meal,
marketing. You know me shit, poster type agency that was just throwing out
names have has really in and trying to exploit divisions. What they were trying to do was grow tribes, so little
little bit different the IRA
A entity that was designed to propaganda eyes to russian citizens, to ukrainian citizens to people who were in Russia's
of influence and the early stuff in the dataset twitter page
at the earliest possible LE information and of the of the material accompanies gave us was actually russian language tweets
talking about the invasion of Crimea. It was talking about. You know. His creator
conspiracy theories about the downing of the Militia Airlines flight. I made seventeen so the earlier
it is of the Irish were very much focused inward focused domestically and then around twenty fifteen. They turn their
due to the United States in what they mean,
and some of the Eastern District Court indictments have been referring to as project locked her so
Jack locked. I was when me the effort to grow these american tribes really started. This perceives the election rights of this procedure, trumps plausible candidacy- and there
Still this goal of amplifying tribalism in the? U S! Yes. Indeed, goal was to to create these, so
This is a long game. This was not a short term, so
media operation to to screw around in an election. This wasn't long game to develop, extended relationships, trusted relations,
with Americans and what they did was they created these pages so
an example, would be hard of Texas was a page that really amplified notions of Texas Pride, almost all of their pages, that algae BT pay
age pages targeting the black community pages targeting confederate aficionados all of these
ages were designed around the idea of pride, and
pride in whatever particular tribe they retargeting. So the vast majority of the content, particularly in twenty four
in the early days was
Oh, we are algae, BT and proud. We are Texas and proudly Texans and proud. We are proud descendants of confederates answered this idea that you should
have pride in your tribe was what they reinforced.
Four and over and over and over again and then you would see them,
periodically slide and content. That was
either that was either political or divisive most sometimes that would be
out of the ring. Another group, so
we are in some of the M so that the content targeting the black community in particular, did this. This country is not for
we're not really part of America? We are. We exist outside of Amerika,
and so a lot of our lot of exploiting
of real grievances tied to real news events, so constant drumbeat of pride plus leveraging real harm
comes to exploit feelings of alienation. Sometimes you would see them do this with political content such as these
primaries heated up. That was where you sorry to see them weaving in their support for candidate Trump weaving in their opposition to candidate Clim. Looking at your report now- and I am saying this list of themes took off some of these, because
again rather diabolical and clever. How they were plain. Both sides of the board here today would focus on
the black community and black lives matter in and issues of police brutality, but also they would amplify pro police.
Blue lives matter pages. You had anti refugee messages and knew no immigration, border issues, Texas cultures. You said southern culture, confederate history, very separatist movements, muslim issues-
eligibility issues, meme culture, red pill, culture, Conrad,
in the second amendment pro trumpet Anti Clinton and more
aunt. I went in the form of pro Bernie Sanders and Jill Stein Tea Party, stuff, religious rites, native american issues, and all of this is just sewing, divisiveness and conflict. Although aid, it really does seem that those two a surprising degree of focus.
On the black community. Due to have more information about or or just in a pin,
in about why that was such an emphasis for them. Yes,
so there were about. There were eighty one, Facebook pages hundred thirty three instagram accounts.
I'm the eighty one, Facebook pages, thirty focused on the black community. Now there were
there were other pages it focused on other kind of traditionally left leaning groups is
as you mentioned- Muslims native Americans Latinos, so there was another
they're kind of non black lefty pages before we go on and that those numbers dome
sound, very large. Eighty one facebook pages sounds like not even
drop in the ocean. I think we should give some sense of the scale of what happened here.
So. There were eighty one facebook pages thinker about sixty two thousand posts across them
there were hundred thirty three instagram counts hundred and sixteen thousand posts across them. There were about
Britain, eighty seven million engagements on the Instagram content and another seventy five million engagements on Facebook content at an engagement,
like a like or a share or a comment, the pages to to be totally totally clear
They had what I would call it a long tail like twenty of them were successful. Enough
but they had. You know in the hundreds of thousands of followers.
and then the lot of the remainder of the long tail was just crap. They were just failed pages, and so on
the things I was actually interesting was you could see them in the dataset pivoting those pages so pivoting her failures going in there and actually an insane
I go cable, one exam.
was. The army of Jesus page of a lot of people have seen some of the means of like Hilary fighting Satan, thereabout now
an hundred posts by that account before it found Jesus. It started as a Kermit. The frog mean page, you know me,
if they Kermit sipping, tee and stuff- and they didn't seem to again of trash and there they pivoted it to a Simpsons mean page
and it was you know, a sharing. These kind of ridiculous homer Simpson means again dislike
passing around with american culture. Seeing what stock? When that didn't stick all of a sudden, it became a religious page devoted to Jesus that seem.
Seem to have that kind of like now. Did you start to see the names doing things like like for Jesus, when you do something like say like like for Jesus share for Jesus there,
getting people to share their content organically, so you actually see them kind of hitting their stride with awe standard kind of tat
acts of social media audience growth with examples like this, this army of Jesus account, so there
It is absolutely true that many of their pages, where complete failures that had no left but then some of their pages, were
So if you go and you look at the audience reach using things, I crowd tangle and you look at their engagements versus the engagements for other conservative pages or other black media. You do see them kind of popping up in there you know top twin
top fifty in terms of engagement overall, so when
nope. Am I saying this? These were like the best possible pages for this cause
tat for these audiences know, but what they did do
Was they achieved substantial success
some of them and they use their successful pages to direct people to their other pages
so the black community was particularly they did this particular
really- this was a- I can't say, effectively, necessarily because
I can't see the conversion data. I know that they showed people. These other means I dont know of people converted to the page for these other means, but
They were doing was they were saying if you like this content from our page, blacks diagram that you're following here? Some other, be no hey. Look at this other group called
Williams and Calvin. Now, there's no disclose
or that the internet research agency is also running Williamson Calvin and their thing look at this other content from this page, Coalblack Divest. Look at this other content from this page called never tv scheme,
so a lot of this kind of cross pollination of audiences in an attempt to push people so that if they're following one of their accounts, one of their pages there
inundated with posts from the others and also
I would like to continue this needed content. You'll need to subscribe at San Aristotle. You get access to all full length, episodes of making as buncoed into other subscriber only content, including bonus episode,
can Amas. In the conversations I can have him in the waking up, app and making sense. Podcast add free and relies entirely on listener support, and you can subscribe now at SAM Harris,
Transcript generated on 2020-10-07.