@Cobalt I honestly think we need to do a better job of making sure techies have the ethical grounding to handle "should we" questions along with "can we" questions.

We've got developers, mostly white dudes, making decisions for the rest of us that quite frankly they're not at all qualified to make on their own.

And if it's their bosses making the decisions, the Nuremberg Defense ain't gonna save techies who just do what they're told and ignore the implications.

@starbreaker Nooooo kidding.

IDK how so many think they can build intelligent creatures w/o any of that humanities stuff about how people think and bond and come to decisions about right and wrong (and most importantly, how we check our work). Like. They don’t think they need to learn how humans work singly or in groups, how their own minds work, but they think they’re ready to build a whole non-anthropomorphic person (w/ mounted weapons, probably). Wut.

@Cobalt I know, and I've got a story that might illustrate just how bad it is.

A bunch of people at my day job were talking about the possibility of the firm getting a contract to design and implement a system for Federal law enforcement. One of the managers handling the proposal got a bunch of us developers in the room and asked us if we could figure out how to ensure the system would never be misused.

(more...)

@Cobalt

When I suggested not building it at all, and exposing the Feds requesting the system to the press, everybody looked at me like I had gone insane.

It didn't occur to anybody else that when you make a weapon, it will eventually be used. Somebody will find a way to justify doing so.

And none of them wanted to be told that they were being asked to build weapons as surely as if we were sitting in an Armalite factory.

@starbreaker 😱😱😱😱

SERIOUSLY

Sounds like a disguised ask for a fucking ethical fig leaf, ugh, and while I was reading this my brain was yelling, “no NONONONONO don’t do it at all! AHHHH!” Because 😱 maybe play a nice game of chess thanks

@Cobalt This was a few years back. I'd love to say that I was in on the ground floor at Palantir, realized what they were doing, and decided to get out with a semblance of a clean conscience, but the outfit I worked for wasn't nearly as famous or as successful.

@starbreaker Still. Fucking creepy, and you never know where your coworkers and boss may end up someday. Setting the precedent of thinking about this stuff and an expectation that acting on it is what normal, well-grounded people do, is still important. 👍

@starbreaker @Cobalt Nah. Judging from what I have read of Palantir’s culture of irresponsibility, I’d not wager you for a Palantir worker...

There is something sweetly menacing in how empire’s belief in its innocence neatly dovetails with the puerile Neverneverland of tech nerd culture. It’s all a jolly boyish romp to make something big, to believe that “disruption” is always good. It must be fun to have such power and so little care for where it lands.