Better Mousetraps.

Head in HandsIn ‘Solving All The Wrong Problems‘, Allison Arief tackles something that has been bothering me for some time:

…We are overloaded daily with new discoveries, patents and inventions all promising a better life, but that better life has not been forthcoming for most. In fact, the bulk of the above list targets a very specific (and tiny!) slice of the population. As one colleague in tech explained it to me recently, for most people working on such projects, the goal is basically to provide for themselves everything that their mothers no longer do…

I’ve always wanted to work on things that matter, that actually have a positive impact on the world or society. Over the last 2 decades, I’ve had the opportunity to work on a few things that did.

It seems more rare to find work like that. And it seems like it’s not a pressing issue when it comes to business, either. The great revelation for me was the lawsuit between iFart and PullMyFinger back in 2009, where millions of dollars are spent on making sounds on demand that most mammals can make for free, but it’s more endemic and less obvious.

All those features that you don’t use are bought and paid for by someone – yes, even Free Software and Open Source.

Allison Arief is correct, and it’s something that is irksome. We’ve been solving all the wrong problems. It would be good to work on some of the right ones – but the risk of working on one of the right ones is high because people don’t necessarily want to pay for ‘useful’.

The market drives.

1 TANSTAAFL: There ain’t no such thing as a free lunch. Even if you pay nothing for software, it doesn’t write itself and basic amenities are required by humans to write it. Remember that the next time your company leverages open source and doesn’t give back to the project in some way.

Evolution: Process Trumps All

3D view of Mar03wjc1bOnce upon a time, software was sold on disks that were actually floppy. They were encased in boxes, with user manuals.

Then the disks were no longer floppy, but they were called floppy as a sacrifice to the Gods of Inaccuracy.

Then the Internet came along, and you could buy the boxes over the Internet.

Then you could download the applications over the internet when you purchased them.

And finally, Software as a Service (SaaS) came into being, and all was basically the same only faster (and with significantly worse documentation).
And the people were generally as miserable as they were, and could be disappointed at a faster rate when they found bugs, or when systems went down, or when their internet access was sloppy.

The real difference between competing companies now is how they produce and maintain the software. That’s the process.

Bugs happen. How fast can the bug be fixed, without introducing new ones? That’s the competition. That’s where competing companies actually should be competing, because the faster a company can fix bugs, the faster it can implement things as well.

That boils down to Software Process. And that’s why Agile, Lean, DevOps and so on are so important.

Because nobody is ordering boxes from the back of magazines anymore and getting it in the mail a few weeks later. They want it now.

If you don’t even have a software development plan (SDP) for a project, you’re already behind.




Getting Past the Mystery of MVC

One way for Software Engineers/Developers/Programmers to impress the hell out of Managers, Directors and Executives is to confidently shoot acronyms at them in rapid fire. Dilbert is rich with this; since the late 90s I’ve sometimes tossed ‘EIEIO’ into the mix when I heard it being abused.

MVCOne of these acronyms is MVC. It’s really simple. See the image on the left? It’s that simple. And it’s not new by any stretch. It’s great for architecture of small systems, and done properly can do well for much larger systems – if you understand how to define the Model, the View and the Controller.

I say it’s not new. In fact, it was introduced in 1978-1979 by Trygve Reenskaug and others for the Smalltalk-76 programming language at the Xerox Palo Alto Research Center (Xerox PARC). You can figure that it has been around almost 40 years, and that it’s from the same era as the mouse and the graphic user interface (GUI).

As a concept, it’s fairly solid – and used properly, it can be powerful. But it doesn’t mean that every system should be MVC – in fact, with application development the way it is these days, a lot of systems probably shouldn’t. Why? If you’ve got multiple application programming interfaces (APIs), you may be going down the rabbit hole into complexity and making software entropy that much more accelerated within your Software Development Life Cycle (SDLC). This article makes that point quite well and offers alternatives, though it stops before getting to some of the conclusions I draw.

So let’s get to the crux of MVC. It’s one of many architecture patterns out there, and the main thing that really needs to be talked about in the context of MVC it’s about implementation – languages and technologies that allow for proper use for MVC. Some don’t give as much flexibility as others.

And MVC isn’t always the answer – more often than not, in the last 2 decades, I’ve seen it used as the straws a drowning application clings to. It’s not bad – it’s not good. It depends on how it’s implemented, and proper implementation in a complex and open system – which describes most – can be outright scary.

A good Software Architect knows these things. A good software architect puts the complete system requirements ahead of the architectural skeleton that is to be and decides on the architecture – while planning for future growth.

Is MVC right for you? I’d say mostly yes. But don’t think it’s new. It’s the architecture that has lasted 40 years and has brought us the applications we have today. Is it the right architecture for the future of your systems? That is a question that should be posed to architects to challenge them.

Your mileage should vary if you’re doing it right.


Avoiding Complexity In The Software Process

ComplexityOne of the greatest enemies of software is complexity, since it invariably leads to Software Entropy. It increases the cost of producing software. The good news is that we’ve gotten better over the decades at assuring best practices steer away from complexity – but sometimes people who implement the software processes themselves do not understand the why of things and create complexity in the software process itself – and even create new and interesting problems.

A simple thing like code reviews can be made complicated by different disciplines sharing the same space. Let’s say that you have code that uses stored procedures. Clearly, you’ll want a DBA doing the review of the stored procedure(s) and table changes – you might not want to call it a code review, but it is a review of a sort. Would you create a new column on your Agile board to track DBA reviews? I would hope not, if you’re using a decent tracking system. You can expect a DBA to make loud noises about them being so different, but if you take a step back… it’s just another type of review that can stay on the board without adding complexity to the board.

Further, differentiating that from the other reviews almost always assures that the code review and the review of the SQL related code is done separately – and since they will go into Production together, it would make sense that the reviews be synergistic. The Software Engineer and the DBA should review together, not separately, so that there are no assumptions made. The short term ‘gain’ of doing them separately results in a potential loss of quality.

How could it be done? Sure, the DBA needs to know to do the review, but that can be done by assigning two people to the review (a DBA and an Engineer) or creating two linked tickets for the same issue and making sure that the review is done at the same time. Simplify, simplify, simplify.

Simplify. It will take a little more time, but it will avoid quality issues – and keep your Agile process more simple.



Know Your Environment: Oh Tay

Artificial IntelligenceYou may remember – it happened only last week – that Microsoft’s Tay got shut down for being a jerk.

The commentators already have speculated – and in the broad strokes properly, in my opinion – that this is because people were jerks. Tay- which, from what I saw, was pretty much a simple ELIZA brought forward a few degrees1. It’s a bit more complicated than that, but in the really broad strokes, one has to wonder what Microsoft was actually thinking by setting Tay up on Twitter and not having some safeties in place.

Please tell me someone on that team said, “You know, I think that this is a bad idea. We should add some safeties.” It’s like sending a kid on her own to a prison to hang out with the inmates and not expecting something to go wrong. “Pick you up at 5 p.m., have a great time sweetie!”

Maybe you’re thinking, “Twitter isn’t that bad.” Maybe for you it isn’t – for me it isn’t – because I attenuate who and what I ‘listen’ to. Clearly Tay didn’t. All things considered, that’s a pretty important thing for people to do – and frankly, it does seem like we humans get that wrong more often than not.

As Software Engineers, we tend to forget sometimes that while we are building interactions within complex systems that our participation is a functional part of the design2. We shouldn’t just be tossing things out into a Production environment without understanding the environment.

Clearly, the decision to put Tay out there did not factor that in – because it’s blatantly obvious from the days of Usenet forward, people can be jerks. Trolls abound. It takes something a bit more to deal with that, and they should have known that.

I’d wager that at least one engineer said, “This is a bad idea”. Listen to your team.

1Sorry, Microsoft, but that’s kinda what it looked like to me.
2Recommended reading: Design as Participation.

So You Think Documentation Sucks.

GNOME doc sprint 2013I had a few conversations today involving documentation in email and otherwise. One was griping about someone else not writing documentation. I had someone saying developers don’t like to document, which is a fair statement, but it’s like saying that they don’t like holding heavy bricks over their naked feet. That’s the equivalent, and if you’ve been around a while, you know that. Even the best coders with perfect memory don’t like explaining their code to others over and over, and it’s a blessing for them in that way. 

Later that day, that person griped about someone writing documentation because they needed something done fast.

Another person and I were going back and forth by email about documentation (oddly related to the same thing), and the perspective presented was that automated documentation was the way to go. I’ve seen automated documentation. Developers work around it. “Oh, I have to put something above the header for my class. I’ll write something witty”, instead of accurate, or both, and 3 years later you get hired in to stare at functions named after this person’s cat, or SuperDuperIntegrator, or – my personal favorites – the Monty Python characters (seen it). That’s just one of the many problems.

The idea behind documentation is that things should be easy to find, easy to read and comprehend if your IQ is above 90 and below 300. It should be accurate, clear and concise – or as close as you can come to it.

Over the years, people taking large amounts of your time to explain the same documentation over and over again causes you to write better documentation. You may not be the best at writing it, but when you’re pestered by people over the years, you end up spending more time about the questions that will be asked and answering them in advance. I’m not kidding. What’s more, it actually helps you with your coding because of it because you’re thinking as you do it, “How do I explain this?” It’s a code review in your head by people’s past implicit criticisms of your documentation by making you repeat yourself.

Documentation becomes a habit.

Or you can keep griping about the lack of documentation by not writing any. See where that gets you. Frustrated teams, tired of having to research everything goddamned thing as if it were the first time, every time. That makes it a bit harder to hire people, because the people who leave aren’t going to be happy. Maybe you’ll be one of those people to go work in a cushy job where there is great documentation (the El Dorado of career software engineers).

Good documentation assists software processes. Requirements traceability alone can become easier. Distinguishing scope at a code level becomes easier. Dependencies are easier to police. Design decisions can be made understandable with things like, “We did it this way because we were rushed for a prototype.” It necessarily slows development because development sometimes needs to be slowed – QA assures quality, software engineers put the quality in and quality, even with a peer review process, is adversely affected by lack of time and in a culture like that the software complexity increases until you’re swimming in a sea of software entropy. Code so fragile that a glance from a user can level it.

Yeah, documentation sucks. I actually hate it though some people seem to think otherwise. Documentation is anticipating needs in the future while describing the present and, if done well, the past as well.

If it was hard for you to figure out, it will be hard for someone else. Document it. If it was easy for you but after a moment’s pause while writing the documentation and looking at the code again, you’ll think about the how, why, what, when, where and who of what you’ve done and may even see things you have missed or see you don’t even understand.

Yeah, documentation sucks. But a lack of it is much, much worse, and bad documentation is almost as bad.



Buying the Future – What are we buying?

Skynet for DummiesA while back, I wrote about the Tyranny of an Inefficient Skynet. I found the thought of a Skynet that is buggy and makes a lot of mistakes a bit scarily amusing. We project our logic onto what we build and we almost always imprint at least some of our irrational behavior on it. Software developers of all ilks have their own styles; the conformists are usually the ones who made the wrong choice in major and feel like they have to suffer for it for the rest of their lives. Either way, all these people hammering out Code. Remember Lessig’s Code: And Other Laws of Cyberspace, Version 2.0?

Now we have the Internet of Things. IoT. A dressed up and marketable version of Web 3.0. People attaching all manner of things to the Internet, collecting data, acting on data when the people themselves may not even know what the data is. Privacy is traded for convenience and the ability to post cat videos on Facebook. The data is collected, decisions made – enough so where people will quite literally have worse lives if they don’t meet the criteria the algorithms (written by those software developers, remember them?)  fit, even if the data is misrepresentative or outright wrong.

That’s where we are now.

And with 3 decades of using and adapting technology behind me, I can’t help but wonder where exactly we’re headed as a society. Feynman spoke about it in his lectures (The Meaning of It All: Thoughts of a Citizen-Scientist) – from his perspective, it’s society that drives how we use science and technology. From the Atomic Bomb to your smartphone and it’s applications, small decisions add up to societal decisions… and ultimately, this Skynet we’re building. The Cold War gone, we live in an era where governments war over the Internet with propaganda machines powered by technology and hackers who have an allegiance of some sort, be it to a government or to a corporation or to themselves – rarely for society itself.

Children that used to go outside to play stay indoors, using software (games), living in worlds created by the imagination(s) of teams and implemented by programmers, increasingly educated by the same software with data that is selectively converted into information. And it happens faster and faster. Remember Gleick’s Faster: The Acceleration of Just About Everything?

So where am I going with this? I’m just wondering more and more often what sort of society we’re buying with our finances that are upended by algorithms on Wall Street. I’m wondering about all the shoddy software pushed out to meet some business need before it’s ready, fragile enough in certain spots that it allows breaking and bending at the weak points. We’re changing the world and we hardly know it as we drive into work, sipping coffee as we dodge traffic – but soon, the cars will be transporting us around.

The Internet has allowed people with common bonds to work together, play together – but in doing so, inadvertently, it has also allowed us to war against those we dislike – from CAPS-LOCK stuck on to outright attacks on someone else’s systems.

Transporting us around so that we can write code to buy things and influence our own future without a thought as to the long term consequences of our actions in a period of time where medicine and associated technologies will have us living longer to see the consequences of our collective decisions.

The small unconscious decisions making the big unconscious decisions for us, mindfulness out the window.

I suppose I may be in a dark mood this morning. I suppose that this may seem pessimistic or cynical. I suppose it’s disturbing if one were to think about it all.

We should go buy something to feel better about all of this…