Exploring Lunar Code

#wcw Margaret HamiltonLast week I’d found this article about the Apollo 11 code being on Github, and I took a look beyond what the article mentions. The code on GitHub is written for the Apollo Guidance Computer (AGC) – and Margaret Hamilton, at right, was the Director for the development of that code. In fact, that’s a stack of the code next to her back in the days when paper was used for development.

It was all the buzz for a week or so, and while it was pretty cool to look over the project, particularly the code comments, it interested me more to consider how much interest this project has had – and what projects from my lifetime will be looked back on with the same level of interest.

I can’t think of a single one. This is from an era where we aspired to put our feet on the moon, not look around for Pokemon. Can you think of one project that will generate this much interest in 3-4 decades?


The Limits of Open Data and Big Data

Open Data Awards 2015A couple of days ago, one of the many political memes rolling around was how many police shootings there were under different presidencies. People were, of course, spewing rhetoric on the number of lethal shootings there were between one administration in the 1980s and one in the present. I’m being obtuse because this is not about politics.

The data presented showed that there were less shootings under one administration than another, but it was just a raw number. It had no basis in the number of police at the time, the number of arrests, or the number of police per capita.

I decided to dig into that.

The U.S. population has gone from roughly 227 million people (circa 1980) in that time to 318.9 million as of 2014. That’s fairly substantial. But how many police were there in the 1980s? A search on how many police officers there were in the 1980s was simply useless.

I went to the Bureau of Justice Statistics page on local police. It ends up that they only did any form of police officer census from 1992 to the present day in 4 year increments, which means that they didn’t have the data from the 1980s. If that’s true – if there was no data collected prior – it would mean that decisions were being made without basic data analysis back then, but it also means that we hit a limit of open data.

And I’d expended too much time on it (shouldn’t that be easy to find?), so I left it at that.

Assuming that the data simply does not exist, it means that the limit of the data is by simply not collecting it. I find it difficult to believe that this is the case, but I’ll assume good. So the open data is limited.

Assuming that the data exists but is simply not available, it means that the open data is limited.

The point here is that open data has limits, either defined by a simple lack of data or a lack of access to the data. It has limits by collection method (where bias can be inserted), by the level of participation, and so forth.

And as far as that meme, I have no opinion. “Insufficient data” – a phrase I’ve used more often than I should be comfortable with in this day and age.

Disruptive vs. Sustainable

Anachronistic TechnologyIt has been driving me a little nuts over the last few years with all the drivel posts on ‘disruptive’ this and ‘disruptive’ that, particularly when ‘sustainable’ was the catch-phrase from a few years ago that still lingers doubtfully in the verbage of non-profits. In fact, I tend to gloss over ‘disruptive’ these days when it shows up because so many people don’t balance it with sustainability.

You see, I was fortunate enough to read The Innovator’s Dilemma: When New Technologies Cause Great Firms To Fail¬†back when it first came out in 1997 – I still have a copy of the first revision. So for this post, and some thoughts on a potential startup or two, I referred back to what I consider the best work out there on disruption and sustainability.

Here are the high points from the Introduction of Christensen’s book. ¬†I use ‘product’ as an interchangeable word for ‘service’ in this context since a service is a product of sorts.

Sustaining Technology

  • Can be discontinuous or radical (so many internet posts seem to confuse this with disruptive when it can be either),
  • Can be of an incremental nature, or as I like to think of it,¬†iterative.
  • Improves performance of established products along the dimensions of performance that mainstream customers in majority markets historically value.
  • Largely the most advancements in an industry.

Disruptive Technology:

  • Results in worse product performance, at least in the near term (in the majority market context).
  • Brings to market a very different value proposition.
  • Under-performs in established markets.
  • Has new fringe features/functionality.
  • Is typically cheaper, smaller, simpler and has more frequent use.
  • Lower margins, not greater profits.
  • Typically is embraced by the least profitable customers of the majority market.

These are very, very simple ways of looking at the differences between the two. A startup can utilize disruptive technologies and enter the market, but there has to be a plan for sustainability (other than being bought by another company) to present itself as a value proposition to anyone involved.

And that’s the key issue that most of the posts I’ve read on disruptive¬†anything fail to mention. Sure, there is risk, but where there is risk, there should be risk mitigation. Don’t get me wrong, I understand solving problems as they come, but only presenting one half of disruptive technology – or disruptive¬†anything, for that matter, is disingenuous.

The disruption of today, to be successful, should be successful tomorrow. Sustainability. Sustainability is why alternating current is used to transmit power over long distances, marketing is why people still think that Edison was more inventor than he was and that Marconi invented the radio.

Writing and Tech Documentation.


The tough part about good documentation is that everything has to make sense. All the stuff has to ‘line up’ along the user stories for the use of the documentation.

There’s the high level story and the lower level stories that make the high level story work – and those same low level stories can have multiple dimensions of their own if written conscientiously with modern tools.

Documentation is usually dry and boring. Dry and boring is great reading for those who read the DOS manuals and Unix manuals end to end (I did), and you can amaze other people with how much you can’t be understood when you talk. That’s where the social engineering aspects of documentation come into play. Or, as writers would call it, writing. The documentation has to be sticky in the marketing sense, such that when someone reads it, they remember it. For the software engineering side – the technical side – less so. On the user side, it has to be… usable.

We’ve come a long way when it comes to our capacity to organize documentation online1. The actual writing, though, has to lean toward a SEO type of writing – repeating key phrases and using possible words and phrases that a user might search for. This requires understanding how a product is expected to be used as well as how it might be used. The latter is not as important for the planned use, but is important for disruptive use cases that might pop up on the radar and allow a business to leverage quickly.

Simply put, good technical writing allows for what is planned, and enables potential uses1.

1 Something I’ll be writing about some more this week.

Crisis Informatics

DisasterPeople who have known me over the years know I’ve always had a passion for responding to disasters. I can’t tell you why it is that when most people are running away, I have a tendency to run in – something I did before I became a Navy Corpsman (and learned how to do better because of). Later became a stab on what this is about by first enabling the capture of the data itself by enabling the communication. I even worked a year at a company that does weather warnings and other emergency communication, and was disappointed at how little analysis was being done on the data.

Years later, I now read ‘The Data of Disasters‘. Some folks have been working on some of the things that I had been thinking about and working on as I had time, and they seem to have gotten further. I’m excited about since the Alert Retrieval Cache was necessarily closed and didn’t gain the traction I would have liked – and open systems present issues with:

  • Context: A post may be¬†about something mentioned prior (a.k.a.¬†ibid) but not tagged as such because of size limitations.
  • Legitimacy: Whether a source is trustworthy or not, and how many independent sources are reporting on something.
  • Timeliness: Rebuilding a timeline in a network full of shares/retweets can pose a problem because not everyone credits a source. If you go by brute force to find source date and times, you can pull on threads – but you’re not guaranteed of their legitimacy in unit time.
  • Perspectives: GIS allows for multiple perspectives on the same event in unit time.
  • Reactions: When possible, seeing when something at a site changes when all of the above can change in unit time.

It gets a bit more complicated from there – for example, languages can be difficult particularly with dialects and various mixes of languages (such as¬†patois in the Caribbean, where I got into all of this). There’s also a LOT of data involved (big, quick and dirty data) that needs to be cleaned before any analysis can happen.

This is all why I envisioned it all to be a closed system, but the world believes differently, interjecting pictures of food with actual information of use. Like it or not, there’s data out there.

The expansion of data from a source over unit time, as mentioned in their paper on Crisis Informatics¬†, is not something ¬†I had thought of. I imagine they’re doing great work up there at the¬†Department of Information Science in the College of Media, Communication and Information at CU Boulder.

I’ll be keeping an eye out on what else they publish. Might be fun to toss a beowulf cluster at some data.

Better Mousetraps.

Head in HandsIn ‘Solving All The Wrong Problems‘, Allison Arief tackles something that has been bothering me for some time:

…We are overloaded daily with new discoveries, patents and inventions all promising a better life, but that better life has not been forthcoming for most. In fact, the bulk of the above list targets a very specific (and tiny!) slice of the population. As one colleague in tech explained it to me recently, for most people working on such projects, the goal is basically to provide for themselves everything that their mothers no longer do…

I’ve always wanted to work on things that matter, that actually have a positive impact on the world or society. Over the last 2 decades, I’ve had the opportunity to work on a few things that did.

It seems more rare to find work like that. And it seems like it’s not a pressing issue when it comes to business, either. The great revelation for me was the lawsuit between iFart and PullMyFinger back in 2009, where millions of dollars are spent on making sounds on demand that most mammals can make for free, but it’s more endemic and less obvious.

All those features that you don’t use are bought and paid for by someone – yes, even Free Software and Open Source.

Allison Arief is correct, and it’s something that is irksome. We’ve been solving all the wrong problems. It would be good to work on some of the right ones – but the risk of working on one of the right ones is high because people don’t necessarily want to pay for ‘useful’.

The market drives.

1 TANSTAAFL: There ain’t no such thing as a free lunch. Even if you pay nothing for software, it doesn’t write itself and basic amenities are required by humans to write it. Remember that the next time your company leverages open source and doesn’t give back to the project in some way.

On Diversity Training

Diversity training has recently become a topic of conversation again, mainly because a Harvard Business Review published, “Why Diversity Programs Fail“. I have my own thoughts on diversity programs as an INTJ third culture kid and as a multicultural.

Here’s a good quote from the above-mentioned article (emphasis mine):

…Despite a few new bells and whistles, courtesy of big data, companies are basically doubling down on the same approaches they‚Äôve used since the 1960s‚ÄĒwhich often make things worse, not better. Firms have long relied on diversity training to reduce bias on the job, hiring tests and performance ratings to limit it in recruitment and promotions, and grievance systems to give employees a way to challenge managers. Those tools are designed to preempt lawsuits by policing managers‚Äô thoughts and actions. Yet laboratory studies show that this kind of force-feeding can activate bias rather than stamp it out…

The article pretty much lays out the reasons for diversity training – avoidance of lawsuits – and the problems with it.

I remember the first time I suffered a formal diversity training was at Honeywell. A woman of European descent was telling us about how important diversity was, but kept dancing along the legal talking points and therefore wasn’t talking too much about diversity at all.

I chuckled through most of it because it was pretty dumb and was focusing on the differences – not the commonalities. As someone who had to forge his own identity instead of having one handed to him, this all seemed ludicrous to me. And, as usual in the United States, diversity was a black and white issue. Being a shade of brown, it was at times painful to watch since it did seem to create a bias when there wasn’t one.

To say that people don’t get promoted because they aren’t someone that their boss more easily relates to – I call it ‘one of us’ syndrome – is false. As much as a lot of us hate the suck-ups, we know that they get promoted by pretending to be like the boss. When you’re a different culture, that’s harder to do. When you’re a different color, it’s even more difficult. My response has been simply not to suck up, which is why I’m probably not a CTO somewhere now, but I’m perfectly OK with that.

Fast forward 20 years, and while at a non-profit that dealt with bipolar issues, I went through even more diversity training that went completely sideways. In their case, they were trying to deal with the accusation that they were a ‘white’ organization, so I heard a lot about ‘white privilege’ from our Afro-American facilitator. It went completely sideways; the hispanic woman and the two brown guys (I was one of them) were suddenly trying to keep everyone together. We kind of did that, sort of. It was messy.

And with those two as the extremes I’ve seen over the years – the rest were vanilla and boring – I’ve come to some conclusions about diversity training:

  • It doesn’t actually work. In fact, when you have a brown person in a room surrounded by those without pigmentation, that person pretty much feels singled out as the example. Some argue that this demonstrates why it is needed. It’s a self-serving argument; if you hire good people based on merit then you simply get what you get.
  • It becomes a glib talking point for the people who don’t understand diversity: They’ll talk about the pains of having to be politically correct and roll their eyes as they word their emails. In this way, diversity training reinforces the problem.
  • The people who feel singled out may either be apologetic or feel empowered, but neither is a proper outcome in a diverse environment. In fact, they may even feel more sensitive than they did before.
  • The business need for less legal costs (which is really what this is about) never had a great metric to start with, so tangible results are impossible. “We get sued less often” might be true, but it might not, and it has no bearing on the future.

Diversity is messy. It cannot be taught by training. Diversity requires empathy, not sympathy, and it requires people to have a world view beyond their own identity. As a TCK multicultural, I don’t understand why this is such a big issue from my own experience, but it¬†is a big issue for people.

And at the end of all of this – you don’t have to like people that you work with and you don’t have to be liked. You don’t have to have dinner with coworkers, you don’t have to sleep with them (HR might have something to say about that). What you have to do is respect people based on merits, and I’m boggled myself by the fact – and it is a fact –¬† that even people who speak highly of meritocracy themselves are poor at practicing it.

In the end, there’s only one way to allow for diversity – that’s what it is, it’s an allowance – and that is to build on commonalities. If it didn’t happen during the formative years, it’s unlikely that it will happen afterwards – no matter how much money you spend on the problem.