Hands-Free, Mind-Free: What We Lose Through Automation
Nicholas Carr's books are the nagging, tech-wary conscience of the digital age. In The Shallows, he warned that surfing the Internet is destroying our attention span.
Now in his new book, The Glass Cage, Carr warns us that computers are making more and more decisions for us, and we risk forgetting how to make those decisions ourselves.
He writes a lot about cars. Cars that do many things for us automatically, things we used to do and had to think about. And cars of the future that may take over the driving from us altogether.
NPR's Robert Siegel picked him up in a state-of-the-art driving machine, a 2014 Mercedes-Benz S550 4Matic.
The car is Mercedes' top of the line, highest-tech model you can drive. It parks itself. It controls the windshield wipers. And it automatically dims the high beams when oncoming cars approach.
It's even equipped with a special camera and radar that allows for semi-autonomous driving, says Michael Minielly of Mercedes-Benz USA, who was also along for the ride.
As Siegel was driving in rush-hour traffic to Carr's hotel in Washington, D.C., this system kept him in his lane — had he strayed, it would have taken over the steering — and it maintained his distance from a taxi that cut in front.
"The car ahead of me is moving, so the car is following it. I'm not accelerating right now," Siegel says.
"That's correct. And you're not braking," Minielly adds.
"I'm not braking. And now the car ahead of me is slowing down so this car is slowing down," Siegel says.
No hands, no feet. The Mercedes was driving itself.
For Carr, features like automatic navigation demonstrate how technology gives to human beings, while also taking away.
"At least you used to have to figure out where you were," Carr says. "And even with a paper map, you'd have to locate yourself somewhere and figure out what the landmarks around you are and kind of get a sense of place. And that's no longer necessary when you have the voice come on and say, 'In 500 yards turn left, 200 yards turn right.' I do think there's something lost there."
And it's not just behind the wheel of a vehicle that can drive itself where Carr sees a worrying loss of autonomy.
"Well, you see it in a lot of professions," including doctors and pilots, he says.
When you go into your doctor's office today, Carr says, the doctor spends a lot of time entering data into a computer — information he used to dictate or write down — and going through different templates to help give him hints on diagnosis.
"Can be good, can be not so good, but [it] changes the doctor-patient relationship in very interesting ways," Carr says.
The same goes for flying. Flight has become much safer since the inception of autopilot and more automated systems, Carr says.
"The name of the book, The Glass Cage, refers to what pilots call the glass cockpit — that more and more they're flying by looking at banks of computer monitors," he says.
As a result of autopilot, though, pilots aren't getting enough practice in manual flying. So when something bad happens, pilots are rusty and often make mistakes.
"I think the lesson isn't that automation is bad, but we have to be very wise in knowing how to automate and when to say no; let's not take more control away from the human being," Carr says.
But a lot of automated driving features work to avoid accidents.
Virginia Tech and the Virginia Department of Transportation have placed short-range radio transmitters on selected roads and highways in the state, says Ray Resendes, executive director of the Virginia Tech Transportation Institute's National Capital Region.
Resendes' Cadillac receives their transmissions and displays them on the dashboard navigation screen. Recommended speeds are shown on exit ramps, and as the car passes a school, an alert warns that school children are nearby.
Carr says he worries about people being bombarded with such automatic alerts — in our future cars and everywhere else.
"This is something called 'alert fatigue,' " he says. When you receive too many alerts, "... you actually become less alert yourself because you're just dismissing the alerts."
And here's another question that Carr writes about: If the car of the future will make decisions for us, how will it decide what to do when a collision is unavoidable and a computer is in charge of the steering?
"You have to start programming difficult moral, ethical decisions into the car," Carr says. "If you are gonna crash into something, what do you crash into? Do you go off the road and crash into a telephone pole rather than hitting a pedestrian?"
Resendes says addressing these issues will be a very difficult task. "A lot of times people will steer to avoid a rear end collision and then you can run into a head on collision, which is the worst crash, or you can hit a pedestrian," he says. "Having to codify these issues into the algorithm on a vehicle is a very serious issue."
Carr's complaint against intrusive automation isn't just about how well or how poorly computers might make moral decisions for us. It's about the very erosion of human autonomy.
"Once we start taking our moral thinking and moral decision-making away from us and putting it into the hands not of a machine really, but of the programmers of that machine, then I think we're starting to give up something essential to what it means to be a human being," he says.
9(MDE1MTIxMDg0MDE0MDQ3NTY3MzkzMzY1NA001))
Transcript
ROBERT SIEGEL, HOST:
And I'm Robert Siegel, on the road for a talk with author Nicholas Carr. Carr's books are the nagging, tech-wary conscience of the digital age. In "The Shallows," he warned that surfing the Internet as destroying our attention span. And now in his new book, "The Glass Cage," Nicholas Carr warns us that computers are starting more and more to make decisions for us and that we risk forgetting how to make those decisions ourselves. He writes a lot about cars. So we are going to pick him up in this state-of-the-art, high-tech driving machine - a 2014 Mercedes-Benz S550 4Matic sedan. It's a loaner, and Michael Minielly of Mercedes-Benz USA is our chaperone. Thank you very much for lending us this car this morning.
MICHAEL MINIELLY: Absolutely. Happy to be here.
SIEGEL: And since this car so far out of my league - talking about a hundred twenty thousand dollar car here?
MINIELLY: Just about that. Yeah.
SIEGEL: You're going to have to explain to me all the amazing things this car can do for me without me having to do them.
MINIELLY: Absolutely.
(SOUNDBITE OF CAR DOOR CLOSING)
SIEGEL: This is Mercedes' top-of-the-line, highest tech model car you can drive. It parks itself. It controls the windshield wipers. Driving in the dark?
MINIELLY: So we have a system called active high-beam assist, which automatically illuminate the road in front of you.
SIEGEL: And if a car's approaching in the opposite direction?
MINIELLY: And it will automatically lower than them and to continuously lower them to the amount where it's - it will never dazzle the driver.
SIEGEL: Talk about dazzling the driver - this car has something called Distronic Plus.
MINIELLY: There's the stereo camera, and then there's also short-, medium- and long-range radars that are continuously monitoring the traffic situation, both in front and behind the vehicle.
SIEGEL: At one point, as I was driving in rush hour traffic to Nicholas Carr's hotel in Washington D.C., I tried this Distronic Plus system out. The system kept me in my lane. Had a strayed, it would've taken over the steering. And it maintained my distance from the taxi that had just cut in front.
The car ahead of me in moving, so...
MINIELLY: So you don't have to use your...
SIEGEL: The car is following it.
MINIELLY: Yeah.
SIEGEL: I'm not accelerating right now.
MINIELLY: That's correct. And you're not breaking.
SIEGEL: I'm not breaking. And now the car ahead of me is slowing down, so this car's slowing down.
No hands, no feet - the Mercedes was driving itself.
Nick Carr, welcome.
NICHOLAS CARR: Thank you.
SIEGEL: I decided to pick you up in the ALL THINGS CONSIDERED mobile studio.
CARR: (Laughter) It's very impressive.
SIEGEL: (Laughter).
I have arrived driving Nick Carr's nightmare. We set out for a glimpse of what's to come in nearby Arlington, Virginia. And as I drove, we talked about automation and what it has done for us and to us.
(SOUNDBITE OF GLOBAL POSITIONING SYSTEM)
UNIDENTIFIED WOMAN: In 700 feet, turn right onto New Hampshire Avenue Northwest.
SIEGEL: For Nick Carr, automatic navigation demonstrates how technology gives to human beings while also taking away.
CARR: At least you used to have to figure out where you were. And even with a paper map, you'd have to locate yourself somewhere and figure out what the landmarks around you are and kind of get a sense of place. And that's no longer necessary, when you have, you know, the voice come on and say, in 500 yards, turn left - 200 yards, turn right. I do think there's something lost there.
SIEGEL: And it's not just behind the wheel of a vehicle that can drive itself where Nicholas Carr sees a worrying loss of autonomy.
CARR: Well, you see it in a lot of professions. So if you go in these days to have your annual physical, usually now the doctor's spending a good deal of time looking into a computer to enter information that they used to dictate - notes about the visit - or write down themselves - to kind of go through processes, templates that give them hints on diagnosis and stuff. Can be good - can be not so good, but changes the doctor-patient relationship in very interesting ways. Pilots - the name of the book "The Glass Cage" refers to what pilots call the glass cockpit - that more and more, they're flying by looking at banks of computer monitors.
SIEGEL: But you acknowledge that, overall, the numbers of air crashes we've had over the past couple of decades since we've automated is way, way down.
CARR: It is. I mean, the whole history of aviation is a history of flight becoming much, much safer. And some of that is definitely due to autopilots and automation systems. But I think what we're seeing - and, you know, the Federal Aviation Administration is concerned about this and has sent out alerts - is that pilots just aren't getting enough practice in manual flying. Almost the entire flight now is done on autopilot.
And so on those rare - thankfully, rare occasions when something unexpected and bad happens, pilots often make mistakes because their skills have gotten rusty. So I think the lesson isn't that automation is bad, but we have to be very wise in knowing how to automate and when to say, no, let's not take more control away from the human being.
SIEGEL: I've been talking with Nicholas Carr, whose new book, "The Glass Cage," is a - is a warning against the potential dangers of some kinds of automation. And so far, we've been in the present - at least, a very expensive version of the present - driving this amazing Mercedes-Benz. But now we're at the Virginia Tech Transportation Institute, at their research center in Arlington, Virginia. And now we're going to get a glimpse of the future with Ray Resendes. Tell us about the Cadillac we're about to take a ride in.
RAY RESENDES: So we have a Cadillac SRX. We have added a technology called dedicated short-range communications. Coupled with GPS, the vehicle puts out a basic safety message, which, basically put, is saying, here I am. Here's the location of a vehicle. Other cars equipped with the same technology can hear that and just like an air traffic control system, can start figuring out who's in my path. Am I in imminent crash danger?
SIEGEL: Virginia Tech and the Virginia Department of Transportation have also placed short-range radio transmitters on selected roads and highways. Ray Resendes' Cadillac receives their transmissions and displays them on the dashboard navigation screen. We see a recommended speed on an exit ramp. As we pass a school, we get a warning that schoolchildren are nearby. When a second Virginia Tech car outfitted like the Cadillac passes us, we get an alert. Nicholas Carr says he worries about people being bombarded with such automatic alerts in our future cars and everywhere else.
CARR: This is something called alert fatigue. Doctors now - when they use computers, they'll often be alerted to possible drug interaction problems. These alerts come up so often now that the doctors are just dismissing them automatically because a lot of them are completely useless, it turns out. There is a big problem that the systems become so sensitive and give you so many alerts that you actually become less alert yourself because you're just dismissing the alerts.
SIEGEL: And here's another question that Nicholas Carr writes about. If the car of the future will make decisions for us, how will it decide what to do when a collision is unavoidable, and a computer is in charge of the steering?
CARR: You have to start programming difficult moral ethical decisions into the car. You know, if you are going to crash into something, what do you crash into? You know, do you go off the road and crash into a telephone pole rather than hitting a pedestrian? Or - you know, things that we do - and we might not do them that well all the time, but...
RESENDES: Very interesting answer 'cause if you look at crashes, the best crash to get into is to rear-end somebody else 'cause cars are well-designed to deal with that. A lot of times, people will steer to avoid a rear end collision. And then you can run into head-on collision, which is the worst crash, or you can hit a pedestrian. Having to codify these issues into the algorithm on a vehicle is a - is a very serious issue.
SIEGEL: That's Ray Resendes of Virginia Tech. Nicholas Carr's complaint against intrusive automation isn't just about how well or how poorly computers might make moral decisions for us. It's about the very erosion of human autonomy.
CARR: Once we start taking our moral thinking and moral decision-making away from us and putting it into the hands not of a machine, really, but of the programmers of that machine, then I think we're starting to give up something essential to what it means to be a human being.
SIEGEL: That's Nicholas Carr, author of "The Glass Cage: Automation And Us." By the way, he told me that he drives an Audi A4 with no navigation system. Transcript provided by NPR, Copyright NPR.
300x250 Ad
300x250 Ad