Should We Be Wary of Algorithms?

Transcript

GUY RAZ, HOST:

It's the TED Radio Hour from NPR. I'm Guy Raz. And on the show today, we're solving for X, the hidden numbers that shape everything around us like, say, for example, when you flip through the radio dial.

(SOUNDBITE OF RADIO)

UNIDENTIFIED WOMAN #1: (Singing) Taking over from one to 10.

KEVIN SLAVIN: When I hear music these days, I wonder...

(SOUNDBITE OF RADIO)

UNIDENTIFIED WOMAN #2: (Singing) Number one.

SLAVIN: ...Why I'm hearing this particular song...

(SOUNDBITE OF RADIO)

UNIDENTIFIED MAN: (Singing) One, two, three, four - get it.

RAZ: This is Kevin Slavin.

SLAVIN: ...What it is in my behavior or the behavior of others or whatever produced this particular track at this particular time.

RAZ: Kevin runs a research group at MIT that's obsessed with algorithms, and he thinks that we're now living in a sort of age of algorithms.

SLAVIN: Our lives are made up of a series of decisions. What time does the train arrive? And what's on the radio? And what's happening to my retirement stock? And more and more, those decisions are made automatically by machines, increasingly without human supervision.

RAZ: So here's an example from Kevin's life.

SLAVIN: So yesterday, I had to make my way from New York to Boston, so I needed to take a flight. And so I went online to one of the sites that just sort of lists, you know, what all the prices are and what's the price of that ticket.

RAZ: A chain of complex calculations figured out...

SLAVIN: Routing and where the fuel depots are, projected seat fill, weather patterns, et cetera, et cetera, et cetera, et cetera.

RAZ: And once Kevin arrived in Boston, he decided...

SLAVIN: I'm going to call for a car, and so I use Uber.

RAZ: But then, an algorithm decided...

SLAVIN: What's that going to cost? Is it going to cost whatever it costs? Or is it going to cost 1.2 times that? Is it going to cost 3.9 times that, et cetera, et cetera, et cetera, et cetera.

RAZ: Then, once he got home...

SLAVIN: My wife and I are expecting a child...

RAZ: Oh, wow.

SLAVIN: ...So we were ordering...

RAZ: Is this your first child?

SLAVIN: It is, yeah.

RAZ: Congratulations.

SLAVIN: Thank you. So we're sort of looking to see, OK, which crib?

RAZ: And if you shop on Amazon like Kevin does...

SLAVIN: A lot of sellers on Amazon aren't actually setting the prices themselves, they're actually just using small algorithms that are looking at the prices of other cribs on there.

RAZ: And that changes - and that changes all the time.

SLAVIN: Yeah, it changes many times in a second. So the price that we paid may never have been actually approved or even considered by human at any point.

RAZ: In fact, Kevin says, you can find these kinds of things wherever you look. Here he is on the TED stage back in 2011.

(SOUNDBITE OF TED TALK)

SLAVIN: And so Netflix has gone through several different algorithms over the years. They started with Cinematch, and they've tried a bunch of others. There's Dinosaur Planet. There's Gravity. They're using Pragmatic Chaos now. Pragmatic Chaos is, like all of Netflix's algorithms, trying to do the same thing. It's trying to get a grasp on you, on the firmware inside the human skull, so that it can recommend what movie you might want to watch next, which is a very, very difficult problem. But the difficulty of the problem and the fact that we don't really quite have it down, it doesn't take away from the effects that Pragmatic Chaos has. Pragmatic Chaos, like all Netflix algorithms, determines, in the end, 60 percent of what movies end up being rented, right? So one piece of code with one idea about you is responsible for 60 percent of those movies. So if you need to have some image of what's happening in the stock market right now, what you can picture is a bunch of algorithms, and that's 70 percent of the United States stock market, 70 percent of the operating system formerly known as your pension...

(LAUGHTER)

SLAVIN: ...Your mortgage. And what could go wrong?

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #1: A stunning and dramatic crash on Wall Street yesterday appears to be the result...

(SOUNDBITE OF TED TALK)

SLAVIN: What could go wrong is, is that a year ago, 9 percent of the entire market just disappears.

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #1: In a matter of minutes...

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #2: Nearly 1,000-point drop...

(SOUNDBITE OF TED TALK)

SLAVIN: And they called it The Flash Crash.

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #3: Flash Crash, which many people...

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #4: Flash Crash, as it was described.

(SOUNDBITE OF TED TALK)

SLAVIN: Flash Crash of 2:45.

(SOUNDBITE OF ARCHIVED RECORDING)

UNIDENTIFIED REPORTER #5: Stocks go do-oop - boink (ph).

(SOUNDBITE OF TED TALK)

SLAVIN: All of a sudden, 9 percent just goes away.

(CROSSTALK)

UNIDENTIFIED REPORTER #6: It's a fast market.

UNIDENTIFIED REPORTER #7: The market was down 900 points, we're now down 688.

(SOUNDBITE OF TED TALK)

SLAVIN: And nobody to this day can even agree on what happened. And that's the thing - right? - is that we're writing things, we're writing these things that we can no longer read. And it's - we've rendered something kind of illegible, and we've lost the sense of what's actually happening in this world that we've made.

RAZ: That's - but - that's crazy.

SLAVIN: It is crazy, right? And so one of the questions is, when we create mathematical models of such complexity - and let's remember that any given algorithm within the stock market is impossibly complex and then it is the interaction of all of these complex algorithms with one another. This is far beyond what any human could ever, ever hope to understand. And it doesn't leave a forensic trail that provides an explanation; it just provides a bunch of data.

RAZ: But you're saying that, actually, increasingly we're solving problems in a way that we don't really even understand?

SLAVIN: That's exactly the point. I think that's one of the most important aspects of what's happening, and I think that characterizes our time.

(SOUNDBITE OF TED TALK)

SLAVIN: So let me take it back to Wall Street, OK, because the algorithms of Wall Street are dependent on one quality above all else, which is speed. And they operate on milliseconds and microseconds. And just to give you a sense of what microseconds are, it takes you 500,000 microseconds just to click a mouse. But if you're a Wall Street algorithm and you're 5 microseconds behind, you're a loser.

(LAUGHTER)

SLAVIN: So if you were an algorithm, you'd look for an architect like the one that I met in Frankfurt who was hollowing out a skyscraper - throwing out all the furniture, all the infrastructure for human use and just running steel on the floors to get ready for the stacks of servers to go in - all so that an algorithm could get close to the Internet. And you think of the Internet as this kind of distributed system - and of course it is - but it's distributed from places, right? In New York, this is where it's distributed from - the Carrier Hotel, located on Hudson Street. And this is really where the wires come right up into the city. And the reality is that the further away you are from that, you're a few microseconds behind. But if you zoom out, you would see an 825-mile trench between New York City and Chicago. It's been built over the last few years by a company called Spread Networks. This is a fiber-optic cable that was laid between those two cities to just be able to traffic one signal 37 times faster than you can click a mouse - just for these algorithms. And when you think about this, that we're running through the United States with dynamite and rock saws so that an algorithm can close the deal 3 microseconds faster, all for communications framework that no human will ever know, that's a kind of manifest destiny. And we'll always look for a new frontier.

RAZ: So - I mean, so basically, you could argue that our lives are controlled by algorithms.

SLAVIN: I think it's more - the word that I use is shaped. The rough edges of what determines our sense of the day are, in fact, sort of hinted by these mathematical models that are all around us. You know, there's still an awful lot of things that happen in the world that can't nobody predict and probably won't ever be able to. And I think that's great and that's fine. I just think that those unpredictable elements are - have this weird kind of frisson with an increasingly predictable set of models that are sort of at every level of our lives.

RAZ: So there's a limit to all this?

SLAVIN: Yeah, and it may be - so there is a limit, but I think it's also - it's not just that there's a humility in being reminded that there's a limit, there's also a value in realizing that there's a limit. I think that it's a reasonable dream to have that we can take a fundamentally mathematic model to everything in the world and then just solve all the problems in it, but it's also fundamentally impossible. And I think that there's a great value in recognizing the idea of striving toward something impossible but also the impossibility of the task.

RAZ: Kevin Slavin runs the Playful Systems group at the MIT Media Lab. Check out his talk at ted.npr.org. Transcript provided by NPR, Copyright NPR.