The Singularity Summit
October 29, 2008
On October 24th and 25th I attended the Emerging Technology Workshop and Singularity Summit, both hosted by the Singularity Institute.
“The Singularity” is basically a point in the near-future where technology has expanded and evolved to such an astounding degree that, from where we stand today, we cannot even conceive of it. Part of the reason for this, Singularitarians believe, comes from the fact that eventually AI will surpass human intelligence and then continue to improve itself. So we standing here can’t understand what that further-future technology will be like because AI more intelligent than us will be creating it. It’d be like a small child trying to predict what a genius engineer is going to make next. The kid just doesn’t have the cognitive tools necessary to do that.
Now I’ve been interested in the intersection of technology and ethics for quite some time – I got funding from UCLA to go to Stanford’s Human Enhancement Technologies and Human Rights conference back in 2006 and started a project during my senior year at UCLA on constructing an ethical framework from which to view enhancement technologies (or basically anything that has the potential to alter our relationships), oh, and I took a fantastic Science Fiction/ Speculative Fiction course at UCLA during my senior year that Katherine Hayles taught. So I’m into this. That’s not to say I’m “pro” the Singularity – I just find thinking about these topics engrossing.
But here’s the problem: I fear that the vast majority of the people who go to these sorts of conferences (both the one at Stanford a while back and this one last weekend) are so infatuated with the prospect of these technologies that they fail to take seriously the sorts of moral and ethical questions that ought to be raised in light of them. There is such a blind naive conviction that technology is both inevitable and good that their moments of reflection come off as little more than a mere gesture to the worries people are raising. A gesture and nothing more.
There are exceptions, though. I spoke a bit with Eliezer Yudkowsky (mostly I listened… since another guy ended up hijacking the conversation) and the sort of stuff he’s working on is an example of someone actually thinking about how to answer people’s Asimov-type worries that AI that is self-improving could end up harming us. So the question he thinks about is sort of a meta one (if I understand it correctly) – what sort of governing principles would you give to AI such that it would be "friendly” in all its future incarnations. We can’t foresee all the situations this AI will be in, we can’t even foresee what it’ll be like, but suppose you can create fundamental principles that would govern it’s future self-improvements.
But of course, if you’ve read “I, Robot” (NOT the movie people – the book is TOTALLY different…and by “different” I mean infinitely superior), you’ll remember that even with the three laws of robotics, a lot happened that, while aligned with the rules, was counter to what people may have wanted. [see, for instance, “The Evitable Conflict” for an enjoyable and scary example]
So there’s no doubt Eliezer has his work cut out for him 
I’m actually not really that interested in the question of whether some AI is going to enslave me. Instead, I focus my attention on a question that I think is a bit closer to home, namely, what do we need from each other in order for us to have eudaimon (or flourishing) lives as individuals. This question, when asked in the context of technological innovation, leads me to examine a technology to see if it somehow fundamentally alters our ability to get those needs met from each other. Because if that’s the case (that the foundations from which we create a base that allows us to live flourishing lives is jeopardized or corrupted), then that technology is in fact not an advancement of human good. Why? Because it’s undermining my very ability to be the sort of person who could even have a flourishing life. This is exactly the framework that I’m trying to flesh out – and I think it’s exactly the framework we should be using to examine these technologies.
So back to the conference…
While I enjoyed hearing about a lot of up-and-coming technologies (of particular interest to me was the semantic web stuff and a talk given by Esther Dyson about 23andme), I was more than a little concerned that instead of taking the time to have a discussion about the moral and ethical questions that should be addressed before we decided to move in a certain technological direction, the two days were basically spent hearing about the latest and greatest tech candy.
Here are a few things that really did stand out:
1) During the first day a guy from the Department of Defense got up and basically made a plea to the people in the room to take seriously the exploration of the ethical issues surrounding the creation of technology that can create autonomous vehicles. He said that it’s not like he’s in a position to really have those discussions but they are important. First, it was a bit disturbing to think the DoD think itself exempted from ethics discussions but that withstanding, this was quite exciting. The excitement, however, was quickly followed by disappointment. The response from those with the power to continue creating and advancing those technologies was along the lines of, “look this technology is a-moral – it just is the ability to, say, have a vehicle roam around all on it’s own. If someone ends up hooking it up to a bunch of machine guns so it can go out and kill people, that’s an ethical question about the application of the technology – which is beyond my scope.” And THAT captures the problem. The people inventing feel that the application of their technology is beyond the scope of their concern. They just want to keep working in their bubbles, oblivious to the rest of the world. And I want to contend that it’s not until the people making technologies acknowledge their moral culpability (at least to some degree) for the use of their technology that we will see compatibility between our moral and ethical convictions and the use of technologies. We are always trying to push the ethical deliberation off to someone else. That’s got to stop.
2) Like I mentioned, Esther Dyson from 23andme’s Board spoke. I about died from excitement. During the break I summoned all my courage, telling myself that “even if the rest of the conference totally sucks, it will have all been worth it if you say one sentence to Esther Dyson”. And so I did. At first I’m sure I was a bit incoherent but finally I think I asked her a mildly interesting question regarding my concerns about 23andme. She seemed interested and told me to email her so we could talk further. At that point I about squealed with delight. So that was neat.
3) People kept saying the singularity was “inevitable”. This drove me insane. Like eye twitching/ head jerking to the side insane. You are depriving yourself and all of us of our agency when you suggest that a certain course is inevitable. We are making decisions (with more or less care) and those decisions are what result in a certain future. Jamais Cascio gave the closing keynote after the first day and in the course of it said both that the Singularity was inevitable and that we were making incremental decisions every day that were leading us in a certain direction. You can’t have both dude! I’d like to stick to the latter please!
And as a side note, it’s very difficult to have a discussion about if we should proceed in a certain way if you’re talking to someone who thinks we have no choice but to!
So that was the conference. I’m happy I went – it’s given me more to think about… and a lot of things to look up!
Comments
5 Responses to “The Singularity Summit”
Got something to say?
GREAT post, Kiddo. From my view, I’d say Dyson was lucky to receive a reasonable question from an enthusiastic person like yourself.
For whatever it’s worth, I don’t recall saying anything about the Singularity being “inevitable” in my talk. (I didn’t compose the talk, I just spoke extemporaneously, but I’m fairly certain I wouldn’t have said that.) I *did* say that some of the outcomes I described would be the inevitable result of individually rational choices, but I wasn’t including the Singularity in that set.
For my actual thoughts on the Singularity, see my talk at the summit last year. Text here: http://www.openthefuture.com/2007/09/singularity_summit_talk_openne.html
Great post, Heather. Seeing you at the conference was especially fun.
I agree that ethical discussions are especially important here, but I disagree with your point in (1) that it’s the job of inventors to worry about applications. As long as they’re not negligent, they’ve done their job in my book. Any technology (and I do literally mean *any* technology) is going to have good and bad potential uses. It’s the job of the inventor to do due diligence and not be negligent, but as long as he’s done that, then his job is done, and it’s up to people/markets/government/society to ensure that the technology is used properly.
Since I am sans-blog, I thought I’d share some outstanding Conference Quotage:
During the panel on robotics, about how better sensor technology will make cars safer:
“You’ll never have to die in a car crash again.”
The lovely Heather Whitney, after a long bioethics discussion (about what, exactly? I don’t remember):
“I don’t know anything about that — I should blog about it!”
Some guy Heather and I ran into while taking in San Jose night-life, talking to Heather:
“Aren’t you my friend on Facebook?”
During the opening reception mixer:
Me: “Hi, Tyler. I want you to meet my friend Heather.”
Tyler: “Hi, I’m Tyler Emerson.”
Heather: “Oh, are you helping out Ken with the Summit?”
Me: “Actually, Tyler is putting on the Summit. I’m helping him out.”
Heather: “Oh. My bad.”
To Jamais…
I’ve only had a brief chance to look over the text from your previous speech so I only can say a few quick things
1) I enjoyed reading it and like much of what you say
2) Perhaps it’s just me but I still get the sense that you’re presenting four possible routes to the Singularity — and we have to decide which to take. Granted, the list may be non-exhaustive, but there’s still a base assumption that that’s (the Singularity) is what’s going to happen — but we can speculate about how it’ll come about. And on one hand I understand this — look, you’re giving a speech at a Singularity conference! But, on the other hand, I still get the sense that some questions haven’t been fully, or at least sufficiently, answered. For instance, at the Singularity conference, when I would ask someone why they thought the Singularity was a good thing, I’d hear stuff like “because our lives are going to be way better” — and then I ask “well…why?” to which I end up getting “because this technology is amazing. We’re going to be able to do the most unimaginable stuff!” And at that point I am left thinking a few things.
1) It’s unclear to me why just having a bunch of super advanced technology is grounds for thinking my life will be so much better. It’s true that it could, given the existence of certain applications of technology, but it’s certainly not the case that tech is a good in itself.
2) what exactly is the Singularitarian or Transhumanist picture of the good life? What do these groups think constitutive of a flourishing life?
3) And once 2) has been answered, exactly how will these Singularity-like technologies aid in the realization of that picture?
So basically I think the thinking-structure most Singularitarians have is misguided. Do they really have such a clear understanding of what the good life looks like that they think all these technologies (autonomous vehicles, nanotech, AI smarter than us, etc.) are a part of it? And if they really are, there should be a lot more explanation for why that’s the case. Because in the meantime
1) there are very real reasons why reporters who show up at these events say there’s a “cult-like” feeling.
2) it looks like a bunch of tech guys (and maybe a half dozen women) sitting around at the conference equivalent of a day long Fry’s fawning session
Wow Heather! It sounds like you were in your element learning and discussing some neat tech stuff and meeting interesting people. I LOVE that you are always challenging others and yourself. After all, someone needs to be asking questions!