You may remember – it happened only last week – that Microsoft’s Tay got shut down for being a jerk.

The commentators already have speculated – and in the broad strokes properly, in my opinion – that this is because people were jerks. Tay- which, from what I saw, was pretty much a simple ELIZA brought forward a few degrees¹. It’s a bit more complicated than that, but in the really broad strokes, one has to wonder what Microsoft was actually thinking by setting Tay up on Twitter and not having some safeties in place.

Please tell me someone on that team said, “You know, I think that this is a bad idea. We should add some safeties.” It’s like sending a kid on her own to a prison to hang out with the inmates and not expecting something to go wrong. “Pick you up at 5 p.m., have a great time sweetie!”

Maybe you’re thinking, “Twitter isn’t that bad.” Maybe for you it isn’t – for me it isn’t – because I attenuate who and what I ‘listen’ to. Clearly Tay didn’t. All things considered, that’s a pretty important thing for people to do – and frankly, it does seem like we humans get that wrong more often than not.

As Software Engineers, we tend to forget sometimes that while we are building interactions within complex systems that our participation is a functional part of the design². We shouldn’t just be tossing things out into a Production environment without understanding the environment.

Clearly, the decision to put Tay out there did not factor that in – because it’s blatantly obvious from the days of Usenet forward, people can be jerks. Trolls abound. It takes something a bit more to deal with that, and they should have known that.

I’d wager that at least one engineer said, “This is a bad idea”. Listen to your team.

¹Sorry, Microsoft, but that’s kinda what it looked like to me.
²Recommended reading: Design as Participation.