Why Your Software Sucks: Complexity

Wrangling complexityThe more complex a system is, the harder it is to maintain. However, the computer you're using right now probably is much more powerful than a computer from the 1960s. It's easier to use a keyboard rather than punchcards or tickertape. The system you're using is a complex system, and much of that complexity is hidden from you.

That's the way it usually works with software. The personal computing device you're using, though, is an evolution of what works and what is sustainable. It has a shelf life too, according to Moore's Law. The cost has decreased over the decades. It's really not a bad model for sustainability in technology.

Software Entropy is almost the reverse of that when it comes to technology. It refers to a tendency of software to become more costly and difficult to maintain (technical debt being a part of that) over time. If you've ever worked or owned a long term software project, you know this is true.

There are two perspectives on the complexity. I'll start with the one that engineers and developers need to communicate to management the most.

Business Complexity

Your software is probably trying to do too much for too many. If everyone wanted swiss army knives, there would be no market for corkscrews. Software as a service (SaaS) and Software as a product constantly have this difficulty. The key issue is trying to please everyone. You can't. Stop it.

When customers are using your product in ways that it is not designed for, you don't have to modify that product. You can... create a new product for them (if it's worth it). Sure, you can use existing software to build upon - ask the (hopefully) smart people you hired - or maybe it's an opportunity to get rid of some of the technical debt and do something new that will help existing projects. Investing a lot or a little is not the key. The key is to invest wisely.

Spinning off new products sounds like an administrative nightmare, but it isn't necessarily so - particulary if the changes are based on the same code. Good object oriented practices, coupled with good software engineering and good architecture, will allow for reduction of complexity and technical debt over time.

An anecdote. Almost 2 decades ago, we got a new Director at the Sensor and Guidance Products at Honeywell, Inc. - Ben Simmons. At the time, we had an issue with engineering departments overflowing with work because our salespeople were very effective. Ben, in a meeting with the sales team, said: "We're getting 100% of the business we bid on." Cheers. He continued, "...and that means we're not bidding high enough."

Development and Maintenance Complexity

For software engineers and software developers, this is the obvious complexity. The more complex a project is, the more complex it is to maintain or add new features to. It can be argued that proper practices can avoid such complexity, but it's a poor argument. All best practices and good development do is mitigate complexity.

How many times have we heard, "We need this new feature!", or, "We have to fix this bug!"? All too often it's thought to be a 'simple thing to add' or 'a simple fix' when it's complicated by a culture of technical debt that hasn't permitted the refactoring of code and architecture to make the fixes simpler (or not necessary at all). In a world where businesses expect continuous change, they need to understand that there is a need for continuous delivery approaches that require investment in both time and finance to get started: the foundation for the continuous delivery must be set.

And the foundation moves.

Why Your Software Sucks: Technical Debt (and Hidden Opportunity)

Technical Debt, Business Value and Efficiency

This is part 2 of a 4 part series, starting with Why Your Software Sucks: Intro

The phrase, 'technical debt', is used to explain all manner of issues that organizations encounter when dealing with software. A lot of things contribute to it, as the Wikipedia article will show. What the Wikipedia article doesn't cover is probably the most important thing: the Culture of Technical Debt.

As an example, as a young software engineer, I worked on a project that was 25 years old. Suddenly, after decades of alterations where the only requirements were someone walking into the lab and saying, "Wouldn't it be nice if..." would have to conform to a process to meet a business need. The design was ad hoc. The software development plan didn't exist. This same software was being sold to other companies in the 1990s at $25,000 U.S. per license - and we didn't even have license tracking aside from an Excel spreadsheet. Suddenly, we were required to conform to a software process and the hard work began even as people continued walking downstairs and continued saying, "Wouldn't it be nice if..."

Fortunately, the Culture of Technical Debt was put in it's place by an organizational decision to minimize creation of technical debt while paying me to remove it. Documentation. Coding standards. Configuration Management. Process. Relaxing of business pressures for a period. Testing. It was a painful process from my end of things, but it was worthwhile. Since then I've been working technical debt issues just about everywhere I've been. Modernizing legacy code has been my bread and butter for most of my career. 

So, your software sucks. Technical debt sucks. If you're reading this, you likely already know that. But there is opportunity with technical debt. Consider how much it costs to remove the technical debt of a project against creating a modern project that does what is needed, or revitalizing a project. A developer or engineer might be able to do create something with significantly less technical debt (by doing things properly with management support) than supporting the technical debt that is holding the present project back.

There is opportunity when it comes to technical debt. It can prove an incentive to leapfrog and get ahead. 

And sometimes, because of business needs, the technical debt grows. 

Why Your Software Sucks: Intro

Frustration (was: threesixtyfive | day 244)Your software does suck. You may not think it does which is exactly why it sucks. Of course, no one means for it to suck - I have yet to hear someone, either in business or software engineering/development, say, "We require this software to suck". Invariably, it does, and sucky software has kept me employed for decades. Some of it I got to fix. Some of it I got to redesign. Some of it continued to suck, some of it sucked more and some sucked less.

But yes, software sucks.

Two articles worth reading on this are 'Why the Great Glitch of July 8th Should Scare You' and 'Everything is Broken'.

Why does software suck? We have terms for why software sucks - that's how much we know it sucks. It falls down to:

  • Technical Debt: This is the result of business and developer decisions to do things quickly instead of properly in the hope that they will fix them someday (if they're even aware). This is a big one for startups as they enter their sustainable phases. In my experience, this issue tends to be enforced by management, and later on the software engineers who did not create the debt are forced to pay for it. The original people who worked on the code have probably moved on to that greener pasture across the fence.
  • Complexity: Aside from NASA and whatever commercial ventures are going into space, software does not exist in a vacuum. Further, users almost always use the software in ways in which it was not designed for. In an increasingly software 'rich' world, this is almost always a problem.
  • Not Fixing Real Problems: I've heard phrases along the lines of, 'Chasing the Shiney' so many times and yet it always surprises me how the Shiney is never the underlying issue that needs to be addressed. At one particular company I worked for, we hated when our management flew anywhere because they'd read the magazines on the planes and think that they had a grasp on what was wrong. They trusted the magazine article more than they trusted their own people. 

Almost all of this could be fixed by better software engineering practices combined with management actually trusting the people they hired to think about these things and acting on what is being said, but that requires a culture shift. The bad news is that the software will still suck and that it will be more expensive. The good news is that the software will suck less and be less expensive to maintain in the long run.