@starbreaker It really highlights the difference between his settings and our world. Thanks for posting; I am always pleased for any opportunity to rethink how I first read/considered #SFF

There’s nobody anywhere to hold anyone accountable for making a really unFriendly AI. #Asimov assumed widespread compliance so his stories could work but like... how and why do we get there? Drone warfare alone would seem to show we dgaf abt implications as a species.

@Cobalt I honestly think we need to do a better job of making sure techies have the ethical grounding to handle "should we" questions along with "can we" questions.

We've got developers, mostly white dudes, making decisions for the rest of us that quite frankly they're not at all qualified to make on their own.

And if it's their bosses making the decisions, the Nuremberg Defense ain't gonna save techies who just do what they're told and ignore the implications.

@starbreaker Nooooo kidding.

IDK how so many think they can build intelligent creatures w/o any of that humanities stuff about how people think and bond and come to decisions about right and wrong (and most importantly, how we check our work). Like. They don’t think they need to learn how humans work singly or in groups, how their own minds work, but they think they’re ready to build a whole non-anthropomorphic person (w/ mounted weapons, probably). Wut.

@Cobalt I know, and I've got a story that might illustrate just how bad it is.

A bunch of people at my day job were talking about the possibility of the firm getting a contract to design and implement a system for Federal law enforcement. One of the managers handling the proposal got a bunch of us developers in the room and asked us if we could figure out how to ensure the system would never be misused.

(more...)

@Cobalt

When I suggested not building it at all, and exposing the Feds requesting the system to the press, everybody looked at me like I had gone insane.

It didn't occur to anybody else that when you make a weapon, it will eventually be used. Somebody will find a way to justify doing so.

And none of them wanted to be told that they were being asked to build weapons as surely as if we were sitting in an Armalite factory.

@starbreaker 😱😱😱😱

SERIOUSLY

Sounds like a disguised ask for a fucking ethical fig leaf, ugh, and while I was reading this my brain was yelling, “no NONONONONO don’t do it at all! AHHHH!” Because 😱 maybe play a nice game of chess thanks

@Cobalt This was a few years back. I'd love to say that I was in on the ground floor at Palantir, realized what they were doing, and decided to get out with a semblance of a clean conscience, but the outfit I worked for wasn't nearly as famous or as successful.

@starbreaker Still. Fucking creepy, and you never know where your coworkers and boss may end up someday. Setting the precedent of thinking about this stuff and an expectation that acting on it is what normal, well-grounded people do, is still important. 👍

@starbreaker @Cobalt Nah. Judging from what I have read of Palantir’s culture of irresponsibility, I’d not wager you for a Palantir worker...

There is something sweetly menacing in how empire’s belief in its innocence neatly dovetails with the puerile Neverneverland of tech nerd culture. It’s all a jolly boyish romp to make something big, to believe that “disruption” is always good. It must be fun to have such power and so little care for where it lands.

@jordyd Which is why Dr. Wily is the end boss...

@starbreaker I'm just waiting for Elon Musk to turn supervillian

@jordyd Peter Thiel already has. Haven't you heard of Palantir?

@starbreaker oh wow I had not that's incredible

@starbreaker and not in a good way

@starbreaker

Going off a tangent here, but have you considered that the only thing robots need to do to destroy us, is just stop working? We've grown so dependant on tech and so few understand it that a massive failure could bring us to the #collapse. We depend on black boxes few can repair or even understand.

To quote #Clarke: "Any sufficiently advanced technology is indistinguishable from magic."

We need more wizards, gentlemen. Many more.

@jordyd

@starbreaker

anyway, while in theory this is great, I currently fear humanity far more than any AI

so really, I'd feel awful putting some baby AI into the world only for humans to rip it to chunks, cackling like fucking howler monkeys, claiming they "beat a robot" and "humans rule" and whatever else

the omnics are pretty much how I think humanity will handle AI

so I'd rather my bots can defend themselves from people when they need to, people are

well

awful

@starbreaker

(I mean, if AI starts terminatoring the earth, I will call you personally and apologize, but like, I think Facebook II is way more likely for our apocalypse -- whatever thing(s) end up replacing that basic goddamn parasite-monolith's place in our lives will try to eat us like Facebook I did. but if Skynet happens, I will seriously apologize. especially if it's because of my own babies. total mea culpa situation.)

@sydneyfalk Facebook is something Mark Zuckerberg should have thought twice about building. He never considered the implications, and acted as if the negative externalities were somebody else's problem.

@starbreaker

> Facebook is something Mark Zuckerberg should have thought twice about building.

Agreed. But let's imagine that alternate reality for a moment, because in it, whoever *did* just plow in and create the Facebook we know bought whatever *that* Zuckerberg created and Zuckerberg's nobody in that universe and that person who owns that thing was the one who should have been more careful.

The system encourages reckless, exploitative behavior.

So humans do them.

@sydneyfalk In that case, Zuckerberg should have refused to work on the system.

You can't just make something and hope nobody will misuse it.

@starbreaker

> You can't just make something and hope nobody will misuse it.

The corollary to this is that you have to be willing to look at misuse vectors for a limited amount of time that is still greater than zero, because "make something and hope nobody will misuse it" is precisely what humans do all day long with everything.

We can't just say "be more careful", we need specifics. What determines 'preventing unsafe use', etc. etc., so these things can be detected.

@sydneyfalk All of that's true, but that's not the point. The point is that any machine that doesn't follow these four rules shouldn't be built.

@starbreaker

But *no* machine can follow those four rules. Any given *wheel* can't follow those rules.

People get beaten to death with crowbars, but they're 'machines' and they exist and nothing about their weaponization potential was going to stop that from happening.

@starbreaker

I mean, I can beat people to death with mechanical keyboards. How would that have been prevented?

You can't make safe hammers. If hammers can think, then you still have to decide what constitutes 'safety', and that's hazy.

I'm trying to say: There need to be explicit lines about what is ethically expected of the machine to 'prevent', because *we* can't 'prevent' our own weaponization. We cannot expect our creations to somehow be better at it than us.

@sydneyfalk I've been trying to say that it isn't the machine's job to prevent itself from being misused. It's our job to refuse when asked or commanded to make machines that we know or suspect can be easily weaponized.

We're the human beings. We should damned well know better.

@starbreaker

Well, this:

> Rule 0. Do not design robots capable of harming humanity, or, by inaction, allowing humanity to come to harm.

The last part specifically implies a robot has to prevent its own misuse -- it cannot allow, via inaction, humanity to come to harm.

So it has to *detect* misuse, so it can shut down, retract blades, etc.

> We're the human beings. We should damned well know better.

Well, based on available evidence, we don't, and never did.

@starbreaker

As for the ethics of whether *we* should do things: Yeah, we should be more careful about that. But we "should" do a lot of things and we don't. We, humanity, don't bother. And even the subset of us that *does* bother cannot actually predict and avoid, successfully, all possible work that is detrimental to humanity.

Maybe robots could be designed to not be manipulatable into murder, but humans are basically born for the job, as far as I can tell. :\

@starbreaker

Case in point: Zuckerberg. He didn't design this thinking "what if this becomes a global power tool that can be manipulated into targeted political hacking". He didn't start with an end in mind, and could not have predicted that *his* product, code, etc., would somehow be the thing that *did* do this.

He couldn't know he was gonna Skynet. He was a less dumb fuck who correctly identified a bunch of dumb fucks giving him information.

If not him, someone.

@starbreaker

I agree we *should*.

But nothing about humanity has suggested, from what I can see, that we *will*.

So we're gonna have to build ethical considerations into something that *can* potentially think that *isn't* afflicted with meatbag instincts.

At least, that's what I think *might* have a shot at working.

Humanity has no shame. We have no ability, as a whole, to look at our behavior and reject it as immoral.

We just do what we do and things happen. :\