Understanding Frankensystems

Solar systemWhen I was just a software engineer in the eyes of managers, I usually got tossed into undocumented complex systems that nobody could figure out easily, largely because they were… undocumented. What overwhelmed and/or intimidated others was exciting for me – exploring something new, understanding how it worked. I’d developed a toolset for exactly that, finding and fixing memory leaks in things I’d never touched before, finding the circumstances that created specific bugs – but the real joy for me was in understanding systems. This transcended computing systems, but here I’ll write about computing systems.

My motto, adapted from something my paternal grandfather would mutter now and then, was, ‘Man made you, man will solve you.‘ I also tried to teach this to others. I recall telling one young intern, just spreading her wings on a GIS system, that I knew that she was smarter than the code she was looking at. I was right, she was – brilliant young woman handpicked from UCF – but she was one of the few; too often I said it and was wrong, probably inadvertently destroying someone’s confidence in what they did for a living. That was probably for the better, I hope – if you can’t hack it, you can’t hack it and you need to find something else to do that better suits you.

Not everyone can think on a systems level. There’s no shame in that. Not everyone can put the puzzle pieces together, even with well written documentation. Not everyone can eat elephants, particularly when they are afraid to get close to one. There’s a few of us that can. And it gets down to details, the moving parts of systems, understanding the principles of operation, understanding what and why things were done that way. It’s forensics, it’s imagination, and it’s also being able to understand the developers who wrote the code and their different styles or lack thereof.

When it comes to multiple software packages working together, the intrinsics of each interacting system are more important than people who create silos think; the more complex the system, the more personality it has for lack of a better word. Some software is plain grumpy, some is pretty and shallow, a reflection of the development cycle. If you’ve been around long enough, you can see something that was pushed out into production too early and abused over the years – scars of undocumented patches cover the code, each done differently as maintainers came and went, their lack of time afforded shown in how deep the scars run in the code. Frankensystems, held together by scar tissue. Let the healing begin.

Almost every time, this requires talking to people who have been around long enough and listening not just about the requirements of the system, and how things evolved from their perspective, but also understanding the developers involved. The brilliant developer who wrote undocumented code that never made sense to anyone else, who snuck things in when they could without others knowing. The plodder, who took their time and was always behind schedule. So many personalities, and all of that feeds into the Frankensystem in ways that defy management silos. How many times did I get in trouble with managers for talking to people outside of the development area to understand the system? Too many. I have references. But I always delivered, which probably boggled them.

There’s more to these systems than code, and the more complex the systems involved, the greater the interactions, the more one needs to see it from different angles. The metaphor of eating an elephant is scalar, from only one perspective, but to truly get into a system you need to eat that elephant from as many directions as you can.

Knowing only the code of a Frankensystem is failure. Sure, there will be landmarks, sprints, whatever – but those ‘successes’ are just small things created by people who just want their pain to go away and have been sold on the idea that these systems are solved that way. If you truly want to understand the systems, you have to mine all the data – from social engineering to code, from network diagnostics to documentation, and what is lacking you have to create.

The power of understanding is in the interactions, the intersections, and cannot be taught.

Linux Journal Ends.

existenceWhen I got the news that Linux Journal has ceased publication, I had to take a moment. I’d been published through them a few times in the early naughts, had worked for the parent company at LinuxGazette.com and A42.com, had met some of the coworkers over the distances and had hung out at Phil Hughes, the former publisher, in Estelli where I broke his carafe and had to get a new one… before I could speak Spanish well enough to know that carafe in Spanish is carafe.

2005 was a pretty good year for me, transformative in many ways. It was great to work with so many people rowing in the same direction; that I went off in my own direction again as I have done over the years speaks more to my own life. SSC is good people, and were the right team for a time for me.

It wasn’t that I expected Linux Journal to last forever. I’d visit the site now and again, but my own path had lead me into Drupal, back into C++, back into VB, into corporate .Net – or better, ugly stabs at using .Net for things that Linux would have been better at. Bills are bills. The down-turned economy (thank you, banks) had me doing things I didn’t like doing, where work was work to do.

I know people that I knew there had since moved on. My Costa Rican connection had gone on to Wisconsin, my Panamanian friends are half in Florida and half in parts unknown, last making biodiesel. It’s easy to wax nostalgic about there and then.

That’s the trouble with the digital age. Things change faster, and we have to change faster than them to stay afloat – and I know that LJ was trying to do just that while I was there (some of the stuff I worked on and advised the publisher on) just as we all do in our lives. And sometimes entropy catches up with us.

But there’s a glimmer of hope in there. Maybe they’ll find a way. It’s hard to say what will happen, but you don’t announce your own demise lightly.

Either way, it is the end of an era. It’s also the beginning of a new one, whatever it may be. That’s just writing and publishing in a digital age.

Net Neutrality and Trinidad and Tobago

Internet Innovation is The Goose That Laid the Golden EggThe global debate on Net Neutrality persists and the consequences are writ large for consumers everywhere.

Yet, there’s almost no interest in the Net Neutrality issues arising lately – I spent my time advocating for it to the uncaring masses who are more intent on other things that they believe are more important.

It is important though. Very important.

David Pogue of Scientific American has a great summarization of the net neutrality debate. I suppose the core of the problems happening over the heads of those in the world – not just the United States – is how the FCC classifies things in an antiquated system rather than create a new classification. The lines between being a provider of telecommunications and providing telecommunication services are said to be blurred, but the reality is that they no longer exist. The people who own the infrastructure are competing with those that don’t.

Businesses that do not own the infrastructure are at a disadvantage. They can be squeezed out by those who do own the infrastructure, effectively monopolizing user services. The end user – us, all of us – doesn’t get the variety that it would otherwise have when those that control the infrastructure can make sure that their services can run faster than their competitors. That’s the core of the issue for users.

The Local Context.

In Trinidad and Tobago and likely throughout the Caribbean, Digicel is cheering the repeal of what was called Network Neutrality (I could say it wasn’t for a variety of reasons). Their premise is actually quite real – they are rolling out infrastructure in the region at a cost, and shouldn’t they get their cost back? Of course they should, why would anyone invest in something that they don’t get paid back for? There is no digital philanthropy here, and Trinidad and Tobago’s National ICT Plans have always pushed broadband penetration, enough so that I’m surprised an over-exuberant feminist hasn’t accused the phrasing of misogyny.

Others in the local context of telecommunications have been relatively silent so far – perhaps wisely so – despite the fact that we’re connected to the U.S. pretty directly economically, through Internet connections, and through other things.

What it means, though, is that applications not owned by local providers – Digicel and bMobile mainly, since the most common data usage is through mobile phones, is that they could create competing applications to… for example… WhatsApp, and make WhatsApp undependable in the same twist. And of course, since these same providers have to report to the government by Law, will give away the encrypted anonymity that seems to only hide poor political commentary and bad pornography. Maybe there’s more to it than that, but I haven’t been added to the LOLCats group (please, don’t invite me).

It means local startup companies will have to get the blessings of those who own the infrastructure and maintain a relationship with them – practically at gunpoint.

Yet Digicel makes a valid point: Why invest in infrastructure if you don’t get paid for it?

The debate is much more nuanced than what is presented – arguably, by design. Show me a Member or Parliament, though, who has taken a stance on network neutrality. They don’t really care too much, and that National ICT plan doesn’t address it in any way. It makes us completely dependent on our own legislation, and our own legislation – despite some bold and debatable efforts by the present government – is not fast, is not as comprehensive as they might like to think, and is going to require more intellectual capital to debate and enforce than anyone seems interested in mustering.

We’re along for the ride in Trinidad and Tobago, hoping for a water taxi but instead getting a ferry to Tobago in a country that should be making strides toward progress… not stagnation. It should be making bold moves – debatable moves – that push the envelope for local talent to strike it’s colours boldly on an infrastructure being built rather than making a mockery of it by making it consumption only with no regard to using it for creating foreign exchange (we can talk about those banks another time, right?).

We could be using our infrastructure to leverage foreign exchange income, subsidizing it’s roll out while diversifying the economy. That’s not what we’re doing, and that’s why Network Neutrality is not even discussed outside of Digicel Press Releases.

Predicting Innovation

Dynamics on expanding spaces- modeling the emergence of noveltiesInnovation is something that has caught the fancy of businesses for some time, so of course anything that goes beyond the ‘common wisdom’ (which is typically neither) and the confusion with disruption is worth taking note of.

Such is the case with the paper, “Dynamics on expanding spaces: modeling the emergence of novelties“, by Vittorio LoretoVito D. P. ServedioSteven H. Strogatz and Francesca Tria.

Mathematics required for understanding the paper: If you don’t know much about Heaps Law or Zipf’s Law, you might want to read up on the underlying concepts.

The short of it is that there is now a mathematical model for innovation that has been used to predict how innovations appear in the real world, such as the emergence of tags in social annotation systems and how we discover new songs in online music catalogues.

Of course, the model doesn’t show people how to innovate, and there’s a lot to be learned by the masses when it comes to making innovation more possible – but we at least have a strong start at predicting when it will happen.

Introspection (Writing)

ThoughtReviewing the statistics between KnowProSE.com and RealityFragments.com has been a bit revealing – empirically.

Between the two sites, I’ve done about 300 posts in the last year.

KnowProSE.com has 27 followers and 50 likes (WordPress) with roughly 5,000 views, averaging 2.5 visitors a day with 100 posts over a year.

RealityFragments.com, on the other hand, has 89 followers and 750 likes (WordPress) with roughly 2,000 views, averaging 1.4 visitors a day with 200 posts over a year.

I did not make goals on these sites – I simply allowed myself to post as I wished to, when I wished to, as often as I wished to so that I could see what happened – because, despite what your goal oriented classes have told you, the best thing to do sometimes is to see what happens. That some more technology related writing has gone to TechNewsTT.com isn’t really worth factoring in – my contributions there, while appreciated, aren’t numerous enough to affect things.

My non-technology writing gets more interest than my technology writing – we could argue that I posted less tech, but there are some other factors: KnowProSE.com has been my domain for over 10 years, whereas RealityFragments is only a year old (and doesn’t suffer the history OpenDepth once did, which was messing with statistics).

So what does it mean? Nothing, really. But it’s interesting to look at. It’s a datapoint.

Artificial Intelligences and Responsibility.

AI-NYC_2017-1218MIT Technology Review has a meandering article, “A.I Can Be Made Legally Responsible for It’s Decisions“. In it’s own way, it tries to chart the territories of trade secrets and corporations, threading a needle that we may actually need to change to adapt to using Artificial Intelligence (AI).

One of the things that surprises me in such writing and conversations is not that it revolves around protecting trade secrets – I’m sorry, if you put your self-changing code out there and are willing to take the risk, I see that as part of it – is that it focuses on the decision process. Almost all bad decisions in code I have encountered have come about because the developers were hidden in a silo behind a process that isolated them… sort of like what happens with an AI, only two-fold.

If the decision process is flawed, the first thing to be looked at is the source data for the decisions – and in an AI, this can be a daunting task as it builds learning algorithms based on… data. And so, you have to delve into whether the data used to build those algorithms was corrupt or complete – the former is an issue we get better at minimizing, the latter cannot be solved if only because we as individuals and more so as a society are terrible at identifying what we don’t know.

So, when it comes to legal responsibility of code on a server, AI or not, who is responsible? The publishing company, of course, though if you look at software licensing over the decades you’ll find that software companies have become pretty good at divesting themselves of responsibility. “If you use our software we are not responsible for anything”, is a good short read that most end user license agreements and software licenses have in there, and by clicking through the OK, you’re basically indemnifying the publisher. That, you see, is the crux of of the problem when we speak of AI and responsibility.

In the legal frameworks, camped Armies of Lawyers wait on retainer for anything to happen so that they can defend their well paying client who by simply pointing at a contract that puts all responsibility on the user. Lawyers can argue that point, but they get paid to and I don’t. I’m sure there are some loopholes. I’m sure that when pushed into a corner by another company with similar or better legal resources, ‘settle’ becomes a word used more frequently.

So, if companies can’t be held responsible for their non-AI code, how can they be held responsible for their AI code?

Free Software and Open Source software advocates such as myself have made these points more often than not in so many ways – but this AI discussion extends into data as well, which pulls the Open Data Initiative into the spotlight as well.

The system is flawed in this regard, so to discuss whether an AI can be responsible for it’s decisions is silly. The AI won’t pay a fine, the AI won’t go to jail (what does ‘life’ mean for an AI, anyway?). Largely, it’s the court of public opinion that guides things – and that narrative is easily changed by PR people who have a side door to the legal department.

So let’s not discuss AI and responsibility. Let’s discuss code, data and responsibility – let’s go back to where the root of the problem exists. I’m not an MIT graduate, but I do understand Garbage In, Garbage Out (GIGO).

The Chromebook That Will Not Die.

ChromebookI can’t recall when exactly I got it – I think it was in 2013 – but the Acer Chromebook Model Q1VC I have will not die. Right now, as I type this, I’m on a beach in Trinidad – it has been a staple on trips anywhere. Small, light, and handles things like writing, as I am doing now, email, etc.

And it will not die. It’s dual booted with Linux if I want to get a little dirty in Python or, heaven forbid, PHP. It does just about everything I need it to – it’s scratched cover testament to my increasing abuses.

It’s not a machine I love, though. The right click using Alt is kludgy, the chicklet-style plastic keyboard is springy but hard for me to beat out all the words per minute I would like. It’s tinny internal speakers make Netflix something that requires headphones, and the ChromeOS means I can’t Skype – a throwover from Google’s attempt to break in with Google Groups, and maybe it does by now.

It will not die. And that makes it something special – in an age where technology has become as disposable as our ecosystems, the two tied together around the world by landfills, this simple little machine chugs along. It gets bogged down, it gets grumpy now and then with too many tabs open, but it… keeps going. And that, you see, is something that reviews of technology just don’t cater for.

Lasting. After all, what good is buying something that you have to replace every year? And why is it that, despite having looked at new Chromebooks and so forth, that I have not upgraded? Why don’t reviews say, “This will do you for at least 2 years, maybe more? Simply because we don’t know until enough time passes – we can’t chart what new and improved technologies will make what we purchase an antique. That’s part of the problem.

I recall when it was just geeks that got bragging rights for new tech. The gamers came along, demanding more of their machines, squeezing every last fps out of a system. Mobile phones came out and suddenly it wasn’t necessarily about ability as much as it did about status (Yes, I’m taking to you, iPeople).

But things that last? Like a pair of comfortable shoes, broken in just right, ugly, unfashionable… unfashionable. 

I’ll tell you what. My $199 US lasting 4 years is a marvel to me in an age where people spend 3 to 4 times as much to upgrade their phones annually.

Now, if you excuse me, I’m going to get back to torturing this little device.