When I was just a software engineer in the eyes of managers, I usually got tossed into undocumented complex systems that nobody could figure out easily, largely because they were… undocumented. What overwhelmed and/or intimidated others was exciting for me – exploring something new, understanding how it worked. I’d developed a toolset for exactly that, finding and fixing memory leaks in things I’d never touched before, finding the circumstances that created specific bugs – but the real joy for me was in understanding systems. This transcended computing systems, but here I’ll write about computing systems.
My motto, adapted from something my paternal grandfather would mutter now and then, was, ‘Man made you, man will solve you.‘ I also tried to teach this to others. I recall telling one young intern, just spreading her wings on a GIS system, that I knew that she was smarter than the code she was looking at. I was right, she was – brilliant young woman handpicked from UCF – but she was one of the few; too often I said it and was wrong, probably inadvertently destroying someone’s confidence in what they did for a living. That was probably for the better, I hope – if you can’t hack it, you can’t hack it and you need to find something else to do that better suits you.
Not everyone can think on a systems level. There’s no shame in that. Not everyone can put the puzzle pieces together, even with well written documentation. Not everyone can eat elephants, particularly when they are afraid to get close to one. There’s a few of us that can. And it gets down to details, the moving parts of systems, understanding the principles of operation, understanding what and why things were done that way. It’s forensics, it’s imagination, and it’s also being able to understand the developers who wrote the code and their different styles or lack thereof.
When it comes to multiple software packages working together, the intrinsics of each interacting system are more important than people who create silos think; the more complex the system, the more personality it has for lack of a better word. Some software is plain grumpy, some is pretty and shallow, a reflection of the development cycle. If you’ve been around long enough, you can see something that was pushed out into production too early and abused over the years – scars of undocumented patches cover the code, each done differently as maintainers came and went, their lack of time afforded shown in how deep the scars run in the code. Frankensystems, held together by scar tissue. Let the healing begin.
Almost every time, this requires talking to people who have been around long enough and listening not just about the requirements of the system, and how things evolved from their perspective, but also understanding the developers involved. The brilliant developer who wrote undocumented code that never made sense to anyone else, who snuck things in when they could without others knowing. The plodder, who took their time and was always behind schedule. So many personalities, and all of that feeds into the Frankensystem in ways that defy management silos. How many times did I get in trouble with managers for talking to people outside of the development area to understand the system? Too many. I have references. But I always delivered, which probably boggled them.
There’s more to these systems than code, and the more complex the systems involved, the greater the interactions, the more one needs to see it from different angles. The metaphor of eating an elephant is scalar, from only one perspective, but to truly get into a system you need to eat that elephant from as many directions as you can.
Knowing only the code of a Frankensystem is failure. Sure, there will be landmarks, sprints, whatever – but those ‘successes’ are just small things created by people who just want their pain to go away and have been sold on the idea that these systems are solved that way. If you truly want to understand the systems, you have to mine all the data – from social engineering to code, from network diagnostics to documentation, and what is lacking you have to create.
The power of understanding is in the interactions, the intersections, and cannot be taught.
