2019: New Year, Same Problems.

Experimenting with proximity and remote control with @anki Vector.I’ve managed to avoid the deluge of end of year posts by people, as well as their bright and shiny posts of what they expect in 2019. After you’ve seen enough of them, you know the recipe and you can make your own – even if it’s not a very good recipe, even if it typically doesn’t stand the test of time.

A ‘New Year’ is just another date on the calendar for me these days – and truth be told, it has been for some time. So I spent this ‘holiday’ running some experimental code associated with the Anki Vector I picked up.

As a way of tracking what changes and what doesn’t, years are fickle. As an example, when it comes to code, the thing we sent that is furthest is still running 8-bit code, and it still seems to be working well. Looks like hunspell (that’s what you call it for pip) is the droid I was looking for, though the documentation on that… well…

Things that haven’t changed that much is the acceleration of technology – because it continues to accelerate, and documentation on it is simply horrible in some areas. I spent roughly an hour delving into replacements for PyEnchant, as an example, reading all sorts of the same thing that Google thought would be useful – and which wasn’t.

And this is, sadly, the sort of detritus that software projects leave behind. As a friend mentioned today, a lack of documentation is better than bad/misleading documentation – and when it comes to documentation, a lack of date tagging condemns people to whatever algorithm the search engine uses when college students are trying to find hardly known authors to plagiarize from.

It goes beyond that. There’s a trend where technology gets disposed of so fast that there is almost no documentation on any of it, or if there is, it’s dated and/or misleading.

This is why we’re not fixing things as much, those of us that have that mindset – because there are always a few people, statistically, that can fix things – remember repair shops? And then there are the people who pay to fix things. The way intellectual property – really, copyright – has gone in a legal sense keeps a space between people who would repair and the owners of copyright. And the contracts, threats about warranty… even more space, starving the ability for products to be supported by third parties.

Heaven forbid you reverse engineer something to fix it. That can get you in trouble with people have chain-linked bracelets and lawyers who love killing trees.

That’s where Open Source and Free Software were supposed to step in, at least in the context of software – but after a few decades, it’s all relatively young and the documentation is done largely in crayon hieroglyphics. The successful projects are documented, at least to some degree.

If there’s one thing that I’d like to see change this year, it’s people getting better at documentation. It’s as if they think what they do isn’t worth that investment.

And when they don’t, it isn’t.

Bureaucratese

bureaucracy :-) #jboye14Bureaucracy has it’s own language; Bureaucratese. Over the course of my lifetime, I have learned to translate it on the fly – but apparently this skill is one very few have. In this entry, I will cover a few different words and what they actually mean.

This post will be updated as I things jar my memory.

Acronyms: The more acronyms associated with a bureaucracy, the more inefficient it is. This complexity also leads to potential corruption.

Analysis: A means of finding meaning from metrics or data which may or may not be appropriate depending on the context of the metrics and/or data, as well as what the actual point is.

Bureaucrat: Someone who takes exception to these definitions. Typically someone who has drank the Kool-Aid and is trying to be a distributor.

Capacity: The potential to do something. Unless this word is associated with a concrete way to measure things (see ‘Metric’), it’s a useless word that communicates the hopes and dreams of the Stakeholder writing it.  They might want Funding.

Committee: A group of legal entities, perhaps even Stakeholders, that attempts to do something of worth. It involves meetings, agendas, and all manner of documentation.

Data: A generic term that may or may not involve metrics; metric that comes from forms, as an example, may or may not be useful based on the questions asked and the audience.

Diversity: Diverse, but having little or no conflict that needs to be resolved by people who are actual stakeholders but have no actual say in a committee or group.

Documentation: Something that rarely reflects the reality of a situation in a bureaucracy.

e'<insert word here>’: A 1990s methodology of trying to bring the word ‘electronic’ to bear on matters that are about technology implementation of the word used. For example, ‘eBureaucracy’ would be about using technology to implement Bureaucracy.

eBusiness: What the rest of the world calls business.

eDemocracy: There’s really no such thing. It’s either a democratic process or not.

eGovernance: A reference to policies and processes that the bureaucracies wants to create to make their jobs easier. They usually create committees and have multi-stakeholder approaches to doing this (see associated definitions).

Funding: Money necessary to create more bureaucracy while making sure people who are creating the bureaucracy continue to get paid.

Human Capacity: A reference to people who do concrete things.

ICT: What the rest of the world calls ‘IT’. It stands for Information Communication Technology and assures correspondence has a character that makes people believe that it is somehow different to the popular use of “Information Technology” -‘IT’.

Leverage: A poor reference to physics; the idea is that a small amount of energy can be used to do great things with a lever. When the word ‘leverage’ is used, it’s usually associated with a notion that something will be done. This word, especially when used with ‘Synergy’, typically means you can ignore the sentence since they’re discussing hopes and dreams. Hopes and dreams have a purpose, but in bureaucratese, they are tongue in cheek since bureaucracy chokes hopes and dreams with red tape.

Metric: A way of measuring things; this can be useful if the appropriate things are measured and are simply fluff if the inappropriate things are measured. This word is typically used with the word ‘Analysis’ wandering around somewhere nearby, and thus bears scrutiny to assure that the appropriate things are being measured in the first place.

Multi-stakeholder: See ‘Stakeholder‘. When this word is used, it refers to a mix of stakeholders. Stakeholder bias, the bias implicit in the types of stakeholders involved (such as companies alone) is masked by the use of this word.

Synergy: This used to be a really impressive word that implied collaboration or integration of some sort. Now it’s sort of an empty word, where you can ignore the sentence it is in – especially if the sentence includes ‘Leverage’.

Stakeholder: A legal entity that is involved. The casual reader might think this refers to people – and sometimes it might include people. It is used in various ways in various dialects of Bureaucratese, but typically it means any legal entities involved – from a government organization to a non-government organization, to a business, to a actual human being.

When the word is used, make sure that who the stakeholder is is communicated, and why they are a stakeholder. The reason of why they are a stakeholder is typically associated with what sort of legal entity they are. A company will try to be more profitable, a government organization might wish to have more control over citizens, a non-profit might give you a nice warm and fuzzy intention underneath there is the need to get the next round of funding. Actual humans will typically be interested in opposing at least some of all of that. If there are no humans as stakeholders, members of society, then that tells you a lot.

Strategy: Typically a methodology of using bureaucracy to create more bureaucracy, typically an unconscious bait-and-switch while saying that the bureaucracy will be accomplishing other tasks. This almost always entails the need for Funding, as well as the creation of new Acronyms.

Did I miss something? Is there a word confusing you? Drop a comment and I’ll help you clear it up. 

Natural Language Processing, Health Records and the Developing World.

Case Investigation Team

The Veterans Administration will be using Natural Language Processing (NLP) for their medical records. It can be a powerful tool for searching for trends and getting the right people to the right treatments in a timely manner. That’s a gross oversimplification.

I know a bit about medical records1. I also happen to know quite a bit about Natural Language Processing, since I’ve worked with it in the context of documentation management.

And, as it happens, I know a bit about the developing world – the Caribbean and Latin America. And I know a bit about the hospitals in the region, where hand written records are kept, but lack the rigor and discipline necessary for them to truly be useful. I recently looked at the medical record of someone in Trinidad and Tobago, if you could call it that, since I found it odd that the Doctors and Nurses didn’t seem to communicate not only with each other but their own subgroups. I saw why.

I know of one doctor who keeps patient records in Microsoft Word documents – a step in the right direction.

There is an opportunity here for the developing world in general, but it’s a technology leap that must be undertaken with the discipline of good medical records in the first place. These delapidated medical systems, despite new buildings, need to have medical records that enable good care in the first place.

There’s no reason that medical care in the developing world should suffer; it can be done much more cheaply than in the developed world and with the advancements such as NLP already being implemented, it’s vacuous to build shiny buildings when the discipline of the medical records themselves should be paramount.

But then, maybe implementing electronic medical records properly would be a good start to building that discipline. 

1Medical Records have interested me from my days as a U.S. Navy Corpsman, where we were assiduous about medical records – Doctor’s orders, nursing SOAP notes, lab results – all had their place within a folder. It was just on the very edge of the medical databases that the U.S. Navy rolled out. When I was at my first USMC command, myself and other corpsmen’s first job was  to get the medical records ready enough to allow us to deploy – and it was an onerous task, with those who had gone before not having taken the records as seriously as they should. Later, I would work with a Reserve USMC unit at Floyd Bennet Field where I would be commended for my database work as related to their medical records.

The Why and Why Not Of Documentation

FAIL stampMy core pet peeve of the last 10 years is the lack of documentation I’ve encountered with Software Projects. Sometimes it’s as the consultant that’s called in after someone already charged a lot of money for a substandard job. Sometimes it’s that legacy project that sticks around through generations of developers who leave a company. Sometimes it’s the “we don’t have time” factor.

Documentation is a value-added part of any software project for a variety of reasons:

  • Reference: The ability of software engineers involved with a project to look back and see why things were done the way they were, and to allow for knowing the limitations and potential of a project.
  • On-boarding: Bringing new software engineers up to speed quickly without having to bounce around the company finding answers.
  • New Projects: Proper documentation of old projects, including estimates and other data, is of great use for estimating similar projects. (Project Management and Business Analysis)
  • Complicated code reviews are simplified.
  • If someone gets hit by a bus, someone else can take over. Quickly.
  • Legal: If a project is supposed to meet certain requirements, those requirements should be traced to the actual implementation.
  • Teams: Proper documentation in an Agile or DevOps process allows team members to see what others are doing in similar areas of code, and allows easy identification of problem areas quickly.

These are just some of the reasons documentation is important. More often than not, I’ve ended up writing documentation at different companies because it simply did not exist before.

Why didn’t it exist? Here are the standard excuses:

  • No Time: We didn’t have any time to write the documentation because we’re busy working on the next thing!
  • “Me no write so good”: Fair enough, but there’s only one way to get better at writing.
  • “It’s not my job, man”: Erm – no. Documentation is a part of software engineering. A big part.
  • We don’t know how to organize it: Fair enough. Spend some time and figure it out and implement it.

At the base of all the excuses are 2 things that are really simple:

  1. Software Developers generally don’t like writing documentation.
  2. Management fails to have developers write documentation.

These 2 things are understandable, wrong, and amazingly easy to deal with if a company wants to.

Exploring Lunar Code

#wcw Margaret HamiltonLast week I’d found this article about the Apollo 11 code being on Github, and I took a look beyond what the article mentions. The code on GitHub is written for the Apollo Guidance Computer (AGC) – and Margaret Hamilton, at right, was the Director for the development of that code. In fact, that’s a stack of the code next to her back in the days when paper was used for development.

It was all the buzz for a week or so, and while it was pretty cool to look over the project, particularly the code comments, it interested me more to consider how much interest this project has had – and what projects from my lifetime will be looked back on with the same level of interest.

I can’t think of a single one. This is from an era where we aspired to put our feet on the moon, not look around for Pokemon. Can you think of one project that will generate this much interest in 3-4 decades?

 

Writing and Tech Documentation.

cubiclecreativity

The tough part about good documentation is that everything has to make sense. All the stuff has to ‘line up’ along the user stories for the use of the documentation.

There’s the high level story and the lower level stories that make the high level story work – and those same low level stories can have multiple dimensions of their own if written conscientiously with modern tools.

Documentation is usually dry and boring. Dry and boring is great reading for those who read the DOS manuals and Unix manuals end to end (I did), and you can amaze other people with how much you can’t be understood when you talk. That’s where the social engineering aspects of documentation come into play. Or, as writers would call it, writing. The documentation has to be sticky in the marketing sense, such that when someone reads it, they remember it. For the software engineering side – the technical side – less so. On the user side, it has to be… usable.

We’ve come a long way when it comes to our capacity to organize documentation online1. The actual writing, though, has to lean toward a SEO type of writing – repeating key phrases and using possible words and phrases that a user might search for. This requires understanding how a product is expected to be used as well as how it might be used. The latter is not as important for the planned use, but is important for disruptive use cases that might pop up on the radar and allow a business to leverage quickly.

Simply put, good technical writing allows for what is planned, and enables potential uses1.

1 Something I’ll be writing about some more this week.

So You Think Documentation Sucks.

GNOME doc sprint 2013I had a few conversations today involving documentation in email and otherwise. One was griping about someone else not writing documentation. I had someone saying developers don’t like to document, which is a fair statement, but it’s like saying that they don’t like holding heavy bricks over their naked feet. That’s the equivalent, and if you’ve been around a while, you know that. Even the best coders with perfect memory don’t like explaining their code to others over and over, and it’s a blessing for them in that way. 

Later that day, that person griped about someone writing documentation because they needed something done fast.

Another person and I were going back and forth by email about documentation (oddly related to the same thing), and the perspective presented was that automated documentation was the way to go. I’ve seen automated documentation. Developers work around it. “Oh, I have to put something above the header for my class. I’ll write something witty”, instead of accurate, or both, and 3 years later you get hired in to stare at functions named after this person’s cat, or SuperDuperIntegrator, or – my personal favorites – the Monty Python characters (seen it). That’s just one of the many problems.

The idea behind documentation is that things should be easy to find, easy to read and comprehend if your IQ is above 90 and below 300. It should be accurate, clear and concise – or as close as you can come to it.

Over the years, people taking large amounts of your time to explain the same documentation over and over again causes you to write better documentation. You may not be the best at writing it, but when you’re pestered by people over the years, you end up spending more time about the questions that will be asked and answering them in advance. I’m not kidding. What’s more, it actually helps you with your coding because of it because you’re thinking as you do it, “How do I explain this?” It’s a code review in your head by people’s past implicit criticisms of your documentation by making you repeat yourself.

Documentation becomes a habit.

Or you can keep griping about the lack of documentation by not writing any. See where that gets you. Frustrated teams, tired of having to research everything goddamned thing as if it were the first time, every time. That makes it a bit harder to hire people, because the people who leave aren’t going to be happy. Maybe you’ll be one of those people to go work in a cushy job where there is great documentation (the El Dorado of career software engineers).

Good documentation assists software processes. Requirements traceability alone can become easier. Distinguishing scope at a code level becomes easier. Dependencies are easier to police. Design decisions can be made understandable with things like, “We did it this way because we were rushed for a prototype.” It necessarily slows development because development sometimes needs to be slowed – QA assures quality, software engineers put the quality in and quality, even with a peer review process, is adversely affected by lack of time and in a culture like that the software complexity increases until you’re swimming in a sea of software entropy. Code so fragile that a glance from a user can level it.

Yeah, documentation sucks. I actually hate it though some people seem to think otherwise. Documentation is anticipating needs in the future while describing the present and, if done well, the past as well.

If it was hard for you to figure out, it will be hard for someone else. Document it. If it was easy for you but after a moment’s pause while writing the documentation and looking at the code again, you’ll think about the how, why, what, when, where and who of what you’ve done and may even see things you have missed or see you don’t even understand.

Yeah, documentation sucks. But a lack of it is much, much worse, and bad documentation is almost as bad.