software engineering

Identifying The End of the SDLC (2015)

~ Taran ~ Leave a comment

Killer QueenDeath is not the opposite of life, but a part of it. – Haruki Murakami

Software projects die. Call it retirement, ‘end’, whatever you wish – they die. In a perfect world, this is planned for – expected – understood. In the real world, many software projects were never established formally and have to be brought, kicking and screaming, to the table for hard decisions to be made.

Some software projects continue living out of some perverse masochistic need to keep them living – but they’re dead but for the efforts of those keeping them alive. Sometimes it’s necessary to admit that the software is beyond repair if only for a business to stop losing money and by that time a company is already deep in the quagmire in the project.

This, of course, is written mainly for businesses with software projects. Some are defined as software projects and some are not – I’ve encountered quite a few software projects over the decades that started off with the question, “Wouldn’t it be nice, if?”, and evolved based on that – a spiral model of requirements that doesn’t necessarily translate into a spiral model of development. While I have worked on successful projects and the world celebrates such successes, it’s the failures that create best practices and other rules of things-not-to-do.

A best practice should be planning for the end of the software life cycle throughout the life of a project – including planning the project. That’s part of any solid software development plan – a living document of a project. that exists in one form of another, be it in one person’s head or many people’s head. The latter creates the conflicts in a project that putting the document in written form so necessary. If this development plan doesn’t exist, identifying the end of the software project – it’s death – is almost impossible.

There are a lot of my fellow professionals out there who know about the software development life cycle to varying degrees of intimacy. In theory, we all want to begin things properly but we never discuss the end too much – and, oddly, the end is typically what employs the most software developers outside of relatively few software companies – the fact that when the software project was started no one planned for planning for it’s death. “Just one more change”, someone will say in consternation, “and we’ll move on to the next evolution because…. <insert your phrase of choice here>“.

It’s rarely blatant to management, and depending on what motivates the software developers, they may be immune to the pungent smell of necrosis if only for fear of not finding employment in a job market that is less than friendly.

For any technology to be sustainable, the economy surrounding it must be sustainable. Identifying the end of a software project requires attention to new technology adaptation both inside and outside of the user demographic as well as an actual interest in what the user demographic wants (will pay for) and needs (what it cannot do without).

The key to knowing when a project is dead is assessing whether the project will make more money than is being spent on it. I’ve worked on projects decades old that were still quite profitable and I’ve worked on projects a year old that were money pits – and a business that keeps throwing money into the money pit is unlikely to remain in business longer than their capital permits.

So how can you tell whether a software project is necrotic? There are a lot of very subjective things that have to be considered, such as the business direction and vision of the future as glimpsed by management – and they color the items listed below as well. In the end, it’s up to management.

There are also very, very simple things that will tell you whether it’s time to break out the candles for the project and will help assess whether any software project is in it’s death throes – or toxicly necrotic for the business.

  • Spending More On The Project Than It Brings In: While this is true of most startup software, the subjective aspects of vision and business direction overshadow it – and sometimes they continue to do so longer than they should. A software project should have a budget. Your company can fight over what is considered an expense and what is considered revenue (it’s not always cut and dry), but if you’re paying software developers more than the project is bringing in, someone should begin asking questions. Startups might be considered exempt but they shouldn’t be – at every iteration, this has to be looked at and with startups even more so than mature projects. Why? A misstep could be the end of it all.
  • Underlying Technologies Are No Longer Competitive/Supported: If you’re required to run a project under DOS or CPM, it may be time to pull the plug. If there’s a niche market, that’s nice. Just make sure that the niche market is paying your overhead and profit. Another example might be a lot of code that is not compliant with the latest and greatest strides in the language. I’ve seen PHP 4 code stuck on LAMP servers that belong in museums – running live on the Internet, where script kiddies are more than happy to show the willing that it’s time to move on with their probing.
  • The User Demographic Says So: Users are a peculiar group of people who don’t necessarily write long and detailed emails about what you’re doing wrong for them – in a world full of software options, it’s hubris to think that your company is so special that your users should invest themselves in your product. Social media makes getting feedback easier but tweets like, “your product sucks” just don’t give the granularity necessary to steer the project.

There are no silver bullets in here. Everything has to be considered within the context of the business and the key problem is that there’s a level of introspection required that wouldn’t have allowed the software project to get where it is now. It can be an uphill battle with those who make decisions since they may have seen how much they invested in the technology in the past as some form of wedding ring.

Image at top courtesy Louise Docker, who made the image available under this Creative Commons License.

The Human Factor: Tangible Results

~ Taran ~ Leave a comment

Mas allá de lo tangibleOne of the issues I faced across the decades of my professional and private technology endeavors has been, simply put, the amount of intangible there was.

A visit from my father in the late 1990s saw him proud of what I was accomplishing, but he had really little idea of what I was doing. He was of the electro-mechanical engineering sphere, a meshing of the arcane art of visualizing the magnetic fields of motors and the results that they churned out, physically. He enjoyed the recliner, the space in my apartment, and the ability to watch a flat screen in any room he was in – he appreciated the rewards of my work but not the work itself. Later, when he saw me strip myself of those ‘rewards’, he had no idea what I was doing even when I was getting media attention.

The human factor of sharing any achievements is difficult enough given the shifting sands of technology and the ability to comprehend them to understand the achievements. Dealing with things covered with Non-disclosure agreements, non-compete agreements, trade secrets and so forth creates a divide between people one doesn’t work with – sometimes unbridgeable. The idea of being the keeper of secrets is a romantic notion when it can be pretty tragic. I know I lost a few girlfriends to my being lost in thought about something that we could not communicate about. Call it a personality flaw. Mine. I live what I’m doing and employers and businesses loved me for it.

And then there’s the disconnect within a business, where the tangible results are misted by horrid implementations of the Agile processes. It’s why I prefer a more DevOps methodology. In the latter, there are tangible results with Operations, who are part of the process.

There is no complaint in any of that, simply a statement of fact. I bring this up because in comparing my battling the Trinidad Roseau versus Software Cost Estimation.  On one hand, I have tangible results that I can write about and share with others, and on the other hand I am writing about how developers can’t properly estimate given the effective silos in a company that keep them from being a true part of the larger project. This is where startups are awesome to work with.

Everyone has a balance. Some people can balance these things better than I can, some people cannot balance them at all – at points, I balanced them well – but life isn’t a sprint, it’s a marathon.

But I do think that Software Engineers and others in Information Technology deserve more in the way of expressing tangible results not just to others, but for themselves.

On Software Cost Estimation

~ Taran ~ 1 Comment

University of Maryland and Sourcefire Announce New Cybersecurity PartnershipA recent video had me considering the problems of software engineering cost estimation – something that has plagued software engineering. It has also plagued people who think software engineering is just coding because, frankly, they’re idiots.

Since I’m out of the industry – by my choice and on my terms – I can now tackle some topics and speak my mind more freely without worry of repercussions when it comes to the next contract, or the next job.

The video is, “How To Price Design Services“, and I’ll embed it at the end of this post.

Now, when it comes to software cost estimation, we start off by gathering requirements. We come up with a design, or alternative designs, built on architectures and technologies that may be new or not, that a company might have the resources to do or not, etc. Some people call this ‘discovery’. Based on what is found in discovery, an estimate is done by reading tea leaves, a magic 8 ball, estimates of coders, and perhaps killing a gluten-free chicken and reading it’s entrails. That’s about as scientific as most people do it.

And how does one get the estimates? As a software engineer over the decades, I know an estimate given by someone writing code (not necessarily a software engineer) is:

  • based on assumptions based on the documented or communicated (mistake 1) information that leads to assumptions (mistake 2).
  • based on their skill level and experience, as well as innate ability.
  • dependent on how much pressure is applied to them, with different thresholds for different individuals.
  • usually wrong.
  • never tied to the value for the company.

Now, I’m not saying that the value of a project for the company should be the estimate – far from it, that’s just not how business works. But let’s talk about profitability – immediate and recurring.

The immediate profit usually doesn’t work out unless a marketing department is brilliant, or there is a monopoly, or both. So it’s about recurring profit. How much would be expected as a return withing – oh, let’s say – 6 months?

The point is that there is a value associated with the project that is rarely communicated to the development team, which is usually – hopefully – smart enough to pick this apart. And the people asking for the project almost never want to let the development team know the value that they’re contributing because those salaried employees will want more money.

Meanwhile, every software project encounters problems because of technology changes, changes on the development team (a problem of poor hiring or poor retention policy), requirements creep (‘someone in the sales department just had a great idea!’), design flaws (they happen), architecture problems and…. well, just about everything.

The main problem with all of this is that estimates simply aren’t accurate – which, actually, is exactly what they are supposed to be. An estimate is never accurate. It approximates, and people don’t get fired too often off of teams because of their coding abilities, but because of their or someone else’s estimation abilities.

There is a point here. That point is that the development team is only as invested in the project as the business team and management permits them to be – and if they’re better invested… productivity will increase and there will also be more careful and thoughtful estimates.

Now, does ‘invested’ mean ‘more money’? Sometimes. Free gummi bears and a gym might work in Silicon Valley, but they also make a metric buttload more money than a development team in – oh, let’s say Orlando, Florida. No, that ‘incentive’ is to get people to stay physically in place in a brick and mortar establishment which, sadly, is still the norm. But they don’t actually make people tweak that bit of code just a little more efficiently. You don’t toss gummi worms at developers and they prance around.

It’s about being vested. And that’s the core problem of cost estimation – the developers are given a black box to fill with little to no actual information from the business.

Oh. That video:

[youtube https://www.youtube.com/watch?v=RKXZ7t_RiOE&w=560&h=315]

The Coding Precarity

~ Taran ~ 5 Comments

Decoding Alan TuringThere’s a brave new world coming, and it’s something I’ve been considering for the past few years. As artificial intelligence begins writing more and more code, and troubleshooting it, the women and men who have made their money speaking to machines are slowly joining the precariat.

It’s not a bad thing or a good thing. It’s not a matter of the sky falling. It’s a simple reality that the precariat needs to consider and find a way past.

Coding is a part of the Software Engineering discipline – wherever it actually exists as a discipline. Coding, on the other hand, is not the entirety of Software Engineering despite what the market appears to think.

HR departments all over the world are trying to fill what are really glorified code monkey positions. And that’s not the future of Software Engineering as I see it, or as circumspection will show. We’re seeing more and more code generated by AIs, and really, if you actually

It’s going to be about ‘soft skills‘. It’s going to be about getting the right requirements to hand off to the coders, because the coders in time are less and less likely  to be human. It’s going to be about interacting with the people in dealing with what they want and making sure that they get what they want.

Of course, wanting what they actually need is something that they still need to get better at. 

 

On Quality In Production Code (2012)

~ Taran ~ Leave a comment

So I’ve been playing StarCraft II: Heart of the Swarm

Once upon a time in a universe resembling this one at a younger stage, I dreamt of game programming. I don’t think that there’s any coder of my generation that didn’t have similar aspirations – perhaps getting published in Compute! magazine, perhaps getting one’s name on the box, or maybe even just writing a really cool game. I stand with the majority of developers who have never done that sort of thing. Times changed.

Consider what the person who wrote Galactic Adventures and Galactic Gladiators had written on the Well (apparently before they thought of putting a name to it):

To me, the most amazing thing about doing these games (and I’m about to date myself) is that I did virtually everything myself. I designed the games, programmed them, drew the graphics (quite primitive), wrote the rule book I did get help from friends and people at SSI in testing and making suggestions. Compare that with today where new games are developed with a million dollar budget and use teams of 30 or more people, programmers, artists, tests, story consultants, etc, etc. And you know what — the games of today are better (Gee, why is that a surprise?)

With that in mind and considering that I knew about Blizzard when the original Warcraft was a freeware game where you paid a bit more to get a full version – no, not the MMORPG you kids play now, but the Warcraft that started a new genre that Starcraft continues – Blizzard has come a long way. Since then, Battle.Net has come online. Since then, numerous titles have come through Blizzard and, while I haven’t played Diablo or messed with World of Warcraft, not one title hasn’t been popular. Blizzard has a kind of magic that works for them. I just like a particular genre and have found MMORPGs to be disappointments when it comes to content; I dislike leveling a character by what is properly termed grinding.

All of that said, I’m sure the budget for the Heart of the Swarm expansion was over a million dollars. I’d wager that it was at least 10 times that amount. The detail is great, the storyline is good (though how a prisoner in a high security prison has a pistol escapes me) and it’s playworthy – though I keep mixing up Q and A on the keyboard. But here’s the thing: I played through the game that took so long to develop in about 8 hours of game time. Hard difficulty, much the same. Brutal – well, that’s still in progress. There’s always the PvP maps and the challenge maps. I figure that, all things considered, the game will last me well over 1,000 hours of playtime.

It sold 1.1 million copies in 48 hours. At $40 per expansion, that’s $40.4 million in 48 hours.

Yeah, we’ve come a long way from typing in games from magazines. We’ve come a long way from a single developer writing a game. There was a magic to that era, though – we didn’t yet expect a great game. We expected a game and hoped it would be good. Now people get their sodden underwear twisted when they start talking about game balance and other things on the forums. As a fuddy-duddy, I see that as an odd form of entitlement – people expect a lot more for $40 and have the privilege of time to gripe about it in poor writing.

Game development has come a long way. Humanity? Not so much.

Thank you, Blizzard, for what I consider to be another great game. As a developer, I wish I had a hand in it. I’ll wait for the Protoss section. Take your time.

Code Responsibly (2015)

~ Taran ~ Leave a comment

Control YourselfWe are stuck with technology when what we really want is just stuff that works.

– Douglas Adams, The Salmon of Doubt

More often than not I’ve had to deal with projects where I sincerely wondered whether salespeople and upper management – those that charge admission for the voyeuristic tendencies of the public in relation to technology – understood the implications of decisions. Examples abound. Planned obsolescence of a business infrastructure based on a technology tree. Coding one’s business into a corner. Complaining about the spaghetti code but forcing the conditions where all programmers can do to keep their jobs is make meatballs (features) and slathering the thing in sauce so as to keep those that sign paychecks appeased with something… resembling… something palatable. The list goes on.

Somewhere in the 1990s, Free Software and Open Source came along and put programmers in charge of much of the business process. Because of experiences I’ve had over the years, it was easy to say, “hey, this might fix everything!” Years of experience, though, made me bite my tongue because… well, because programmers are mostly interested in writing code. It’s rare to find a programmer that is interested in actually creating a product. Just as there are no unfinished poems for poets, there are no unfinished programs for programmers.

This has haunted Linux. Granted, Linux at its core has a benevolent dictator that keeps feature creep and feature tsunamis under control – but the desktop Linux never took root as much as it could have because software developers aren’t as good at keeping things simple. Frankly, in general, we suck at it. It’s not a bad thing. It’s not a good thing. It’s the way it is. There are outliers, of course, but those are rare. I’m one of them and it’s because of something quite simple I learned as a Navy Corpsman:

First, do no harm.

Harm is a subjective term. There are even people who appreciate the common understanding of the word as pleasure though this is a family friendly blog and we won’t discuss that much. Let us say that, generally speaking, there are almost no people on the planet who would wish to have harm propagated against themselves. Yet software developers have a tendency to do it every day. I’ve done it and, on much rarer occasions, I can catch myself doing such things because I’ve learned to identify them.

That’s why I really enjoyed Paula Rosenblum’s article, ‘Is Software Quality Going To Hell In A Handbasket?‘:

…Can anyone yet understand why the MS Office interface has changed completely twice in the last three releases? How many hours of productivity have you lost hunting for menu items that you knew by heart last time around? And do we even want to start talking about Windows?  Not today. These posts are only supposed to be so long.  Let’s just say when your Executive VP banker brother-in-law calls to ask you how to print a document in Windows 8 you know things have just gone silly…

Paula completely nails it, and that’s just a piece of the article.

Simplicity is really the key. Yet simplicity isn’t that easy when expectations are constantly evolving. The only way to actually assure a good balance of features and meeting user expectations is something that most software developers suck at: Communicating with users.

It’s almost akin to having Marines who were naughty getting stuck with KP duty – where they cook for other Marines. The Marines don’t see a flaw in this logic.

To make matters worse, businesses are also not that great with the balance – and in being bad about the balance, they often spend more money than they would have to in chasing their tails – or as they see it, catching that thing that’s oh-so-close-all-the-time.

Programmers do it to themselves. Businesses do it to themselves. Worse, both can propagate it on others – and often do.

First: Do no harm. Code responsibly. And use the right technology – just because you’ve invested heavily in it doesn’t make it right. The world has been littered with the bodies of those that meant well while reinvesting in bad debts.

After all, the income you save may be your own.

Image at top left courtesy Flickr User JOSE VICENTE JIMENEZ RIBAS through this Creative Commons License. Enhanced by Zemanta

Getting Out of Firefighting Mode (2015)

~ Taran ~ Leave a comment

WildfireIf Prometheus was worthy of the wrath of Heaven for kindling the first fire upon earth, how ought all the Gods honor the men who make it their professional business to put it out? – John Godrey Saxe

Sooner or later any seasoned software developer, programmer or project manager finds themselves in firefighting mode – where it’s a matter of putting out a fire so that it doesn’t spread to other things, taking a project down. Some people think that the ‘fingers in the dyke’ are a more appropriate metaphore – and at times that is true – but I’m writing about firefighting today.

Getting out of firefighting mode is relatively easy. Like fire, there are certain things that are needed to create the conditions for the fire in the first place. A fire needs fuel, oxygen and a source of ignition.

Oxygen is easy in a software project – it’s a requirement for the software project to start and survive in the first place: users.

The fuel, all too often, results from poor software and system administration practices and poor management decisions.

Like fire, the ignition sources can be many. It can be the fuel rubbing together to form the friction (bad decisions culminating into a burst of flame), or a visiting pyromaniac (someone taking advantage of an exploit), or enough oxygen (users) blowing over a hot spot (friction) that causes a burst of flame.

The only way to get out of firefighting mode with a software project is to deal with the cause of the fires. Sure, putting out the fires is important, but sometimes you can’t put them out fast enough – and unless you treat the underlying cause, the fire will gut the project.

Almost all the time, the people who have been fighting the fires have opinions on what is causing all the fires – and they typically have a plan to tackle them, if only they had time or management approval. Despite our best efforts in Hollywood, time travel is not yet viable – and that leaves us with management decisions. Management decisions over a period of time become company culture. A culture that is derisive of good software project management is almost impossible to change unless something drastic happens.

About 15 years ago, I came up with a phrase that seems to be pretty accurate since: Management doesn’t smell the fire until the flames are licking their posteriors. Until they understand that there is a problem that needs to be solved, they will not try to solve it. The time can vary depending on how much attention they are giving to the issues and, where the real problem usually is, how much they trust the people that they hired to talk to warn them about these things.

If you can’t solve the management/leadership problem, you’re sunk. You’re done. Cash in your paychecks and update your resume.

If you can get management to see the need for change – probably with the help of someone in finance who will point out how much money is being spent fighting fires – the only tried and true way to get out of firefighting mode is:

(1) Take a hard look at whether the project is dead or not. If you’re so bogged down in maintenance mode, it’s time to figure out what Plan B is. If not…

(2) Start implementing best practices, from source control to documentation of the project itself. When there are too many things going on, find a finite area to begin controlling and start controlling it. It’s an uphill battle but it can be done with management making the path clear.

(3) Draw lines in the sand. If fires don’t meet certain criteria you set – don’t fix them. This is triage and has to be decided with the overall project in mind.

Image at top courtesy U.S. Fish and Wildlife Service, Southeast Region, and made available under this Creative Commons License.

 

Better Mousetraps.

~ Taran ~ Leave a comment

Head in HandsIn ‘Solving All The Wrong Problems‘, Allison Arief tackles something that has been bothering me for some time:

…We are overloaded daily with new discoveries, patents and inventions all promising a better life, but that better life has not been forthcoming for most. In fact, the bulk of the above list targets a very specific (and tiny!) slice of the population. As one colleague in tech explained it to me recently, for most people working on such projects, the goal is basically to provide for themselves everything that their mothers no longer do…

I’ve always wanted to work on things that matter, that actually have a positive impact on the world or society. Over the last 2 decades, I’ve had the opportunity to work on a few things that did.

It seems more rare to find work like that. And it seems like it’s not a pressing issue when it comes to business, either. The great revelation for me was the lawsuit between iFart and PullMyFinger back in 2009, where millions of dollars are spent on making sounds on demand that most mammals can make for free, but it’s more endemic and less obvious.

All those features that you don’t use are bought and paid for by someone – yes, even Free Software and Open Source.

Allison Arief is correct, and it’s something that is irksome. We’ve been solving all the wrong problems. It would be good to work on some of the right ones – but the risk of working on one of the right ones is high because people don’t necessarily want to pay for ‘useful’.

The market drives.

1 TANSTAAFL: There ain’t no such thing as a free lunch. Even if you pay nothing for software, it doesn’t write itself and basic amenities are required by humans to write it. Remember that the next time your company leverages open source and doesn’t give back to the project in some way.

Evolution: Process Trumps All

~ Taran ~ Leave a comment

3D view of Mar03wjc1bOnce upon a time, software was sold on disks that were actually floppy. They were encased in boxes, with user manuals.

Then the disks were no longer floppy, but they were called floppy as a sacrifice to the Gods of Inaccuracy.

Then the Internet came along, and you could buy the boxes over the Internet.

Then you could download the applications over the internet when you purchased them.

And finally, Software as a Service (SaaS) came into being, and all was basically the same only faster (and with significantly worse documentation).
And the people were generally as miserable as they were, and could be disappointed at a faster rate when they found bugs, or when systems went down, or when their internet access was sloppy.

The real difference between competing companies now is how they produce and maintain the software. That’s the process.

Bugs happen. How fast can the bug be fixed, without introducing new ones? That’s the competition. That’s where competing companies actually should be competing, because the faster a company can fix bugs, the faster it can implement things as well.

That boils down to Software Process. And that’s why Agile, Lean, DevOps and so on are so important.

Because nobody is ordering boxes from the back of magazines anymore and getting it in the mail a few weeks later. They want it now.

If you don’t even have a software development plan (SDP) for a project, you’re already behind.