deep learning

Rolling The Text Dice.

~ Taran ~ Leave a comment

I haven’t published any science fiction other than some things here and there on my blogs, so I wouldn’t say that I’m an expert on writing Science Fiction – but I most certainly would consider myself an expert on reading Science Fiction. And it’s Sunday, and this is what I feel like writing about today.

Presently, there’s an inflection in technology where technology can write like a science fiction author (poorly), however banal one might think it is. There’s a recurring theme right now about people worried about creative endeavors being taken over, but as far as I know nothing very impressive has come out in this regard and I don’t expect it anytime soon.

…There’s one barrier that AI can’t cross, as of now. And that’s creating new science fictional concepts. The writer who can take in everything that’s going on now and speculate about the near future in any coherent way has the edge – for the moment. AIs based on current training models are essentially limited to rearranging the deck chairs on the Carnival Cruise’s Mardi Gras. What’s disappointing, is that’s exactly what most current human science fiction writers are doing too…

“The Future of Science Fiction“, James Wallace Harris, 4/2/23

I fully endorse what he writes there. Most human science fiction writers these days seem to have Ye Olde CookeBooke of Formulaic Books which has been greatly disappointing when I explored bookstores, running my fingers gently across a creative cover of a book to flip it open, glance within, and see… the same formulae.

I’m not saying there isn’t good original science fiction out there. It’s just gotten so hard to find for me. The dwindling number of bookstores no longer seem to hold that many new ideas and thoughts, and Amazon, while convenient, lacks the tactile experience and probably has adversely affected any romantic relationships that blossomed over book choices in line. Don’t worry, I’m sure they’ll eventually create a dating app based on books you’ve purchased through Amazon, regardless of whether you read or understood them.

“We read, frequently if not unknowingly, in search of a mind more original than our own.”

Harold Bloom, How To Read and Why, 2000.

Harold Bloom’s quote has haunted me for at least a decade now, if not longer, because every time I go into a bookstore and see the same depressing stuff on the shelf that seems like corporate pulp fiction, I think that maybe it’s time I write something better. Something not formulaic.

Something that hits like Robert Heinlein’s Stranger in a Strange Land, an oddly poetic work considering how conservative the author was to create such an interestingly liberal book.

I’m starting a new project tomorrow, holding myself to landmarks and deadlines, so I’ve been roving around the Internet and reading up on things like these. The reason I’m writing about it here is because in it’s own way, it’s a system as hinted at here:

…Right now we have more science fiction of all kinds being produced than ever before. That production is going into overdrive with AI. And the percentage of it that’s readable and entertaining is increasing. That’s also part of the problem – there’s too much science fiction – at least for any one work to become widely known. Good science fiction novels have almost become a generic product line, in white boxes with black letter labeling…

“The Future of Science Fiction“, James Wallace Harris, 4/2/23

Anything sustainable in the world of our creation has to be fiscally sustainable. This is what happened with Web 2.0, this is what has happened with publishing in general, and so, it’s happened to the just about everything else.


Saying that there is a lack of originality in Science Fiction is very much like saying that there’s no real originality in modern music. It’s subjective.

As originality decreases, banality increases.

Now here’s the rub. Since bayesian probability, the core of just about every algorithm out there offering you suggestions on everything from Netflix to Facebook, targets the median, the median decides what is original or not.

As the average person experiences more original works, they increasingly see suggestions as banal. Bayesian probability only works based on what has happened.

Beating that system for marketing requires a mix of what people have liked with what is original, all of which are moving targets on a very small asteroid to be hit from a spaceship with really bad targeting.

Roll the dice, I suppose. I’d rather we had an infinite improbability machine.

Deep Learning, Information Bottlenecks – and Osmosis.

~ Taran ~ Leave a comment

I’ve experimented in the past with deep learning in a few different ways, coming up with my own thoughts on how things work and why they work. It was apparent to me when I stopped that in 2016 that I was missing something, and that I needed some distance between myself and the topic at hand. I gave up those Pine64s, and as it happened, moved away from where I was doing it – more importantly, divorcing me from a Software Engineering world where ‘solutions right now’ always trumped ‘solutions’, the former the harbinger of problems, the latter the Holy Grail of every software engineer who dare dream in a world that, except for the minority, requires lockstep precision within an industry that spends it’s time firefighting because of solutions-right-now.

It’s disenchanting. Being disenchanted allows for little in the way of real solutions, at least for myself.

And today I read, “New Theory Cracks Open The Black Box of Deep Neural Networks“. Of course, deep learning is not that new, and the ‘Information Bottleneck’ thought stems from the original work in 1999, the Information Bottleneck Method. That works perhaps in explaining things on a surface level and on an informational level – but as I read it, I was reminded of secondary school biology: Osmosis. No one has seemed to connect the two when they are so suitably connected, and I’d wager that Osmosis scales better since the information bottlenecks, when themselves in a matrix, pretty much would mimic a tunable osmosis.

That said, I’ve found the major problem with deep learning to be that we define inputs when, quite possibly, we should be more loose in our definitions of what we put in. This aligns better with chaos theory – something that the Wired article seems to dismiss:

…When Schwab and Mehta applied the deep belief net to a model of a magnet at its “critical point,” where the system is fractal, or self-similar at every scale, they found that the network automatically used the renormalization-like procedure to discover the model’s state. It was a stunning indication that, as the biophysicist Ilya Nemenman said at the time, “extracting relevant features in the context of statistical physics and extracting relevant features in the context of deep learning are not just similar words, they are one and the same.”

The only problem is that, in general, the real world isn’t fractal. “The natural world is not ears on ears on ears on ears; it’s eyeballs on faces on people on scenes,” Cranmer said…

Pragmatically, this is what we see when we work on projects – but the problem is not what we see, it’s what we don’t see. It’s the things we don’t intuitively connect ourselves because of our own limitations; with simple deep learning we may get away with what we see, but on a much larger scale, we may be looking at the motion of wings of a butterfly on the other side of the world causing a tipping point that creates a hurricane on the other.

Of course, this is all theory, and hardly some earth shattering change in the way we look at things – but a small change in how we approach things could well be what we need to move forward at various intersections. In this, I am trying to be a simple butterfly flapping his wings.

The Future Of Technology and Society (May 2016)

~ Taran ~ 1 Comment

FutureIf you’re one of those who likes tl;dr, skip this post and find a tweet to read.

It has been bothering me. There are a bunch of very positive articles out there that do not touch on the problems we face in technology.

What I mean by this is that, since the early 1980s, I have been voraciously reading up on the future and plotting my own course through it as I go through long, dark tea-times of my career. It allows me to land where things are interesting to me, or where I can make a living for a while as I watch things settle into place. I’ve never been 100% accurate, but I have never starved and have done well enough even in 3rd world countries without advanced infrastructure or policy. Over the course of decades, I have adapted and found myself attempting to affect policies that I found limiting – something most people don’t really care about.

Today, we’re in exciting times. We have the buzz phrases of big data, deep learning and artificial intelligence floating around as if they were all something new rather than things that have advanced and have been re-branded to make them more palatable. Where in the 1990s the joke was that, “We have a pill for that!”, these days the joke is, “We have an app for that!”. As someone who has always striven to provide things of use to the world, I shook my head when flatulence apps went to war for millions of dollars.

Social networks erupted where people willingly give up their privacy to get things for ‘free’. A read of Daniel Solove’s 10 year old book, The Digital Person: Technology and Privacy in the Information Age, should have woken people up in 2006, but by then everyone was being trained to read 140 characters at a time and ‘tl;dr’ became a thing. I am pleased you made it this far, gentle reader, please continue.

Big Data

All these networks collect the big data. They have predicted pregnancies from shopping habits and been sued for it (Feb 2012). There’s a pretty good list of 10 issues with Big Data and Privacy – here’s some highlights (emphasis mine):

1. Privacy breaches and embarrassments.
2. Anonymization could become impossible.
3. Data masking could be defeated to reveal personal information.
4. Unethical actions based on interpretations.
5. Big data analytics are not 100% accurate.
6. Discrimination.
7. Few (if any) legal protections exist for the involved individuals.
8. Big data will probably exist forever.
9. Concerns for e-discovery.
10. Making patents and copyrights irrelevant.

Item 4, to me, is the largest one – coupled with 5 and 7, it gets downright ugly. Do you want people to make judgements about you based on interpretations of the data that aren’t 100% accurate, and where you have no legal protections?

Instead, the legal framework is biased towards those that collect the data – entities known as corporations (you may have heard of them) – through a grouping of disparate ideas known as intellectual property. In fact, in at least one country I know of, a database can be copyrighted (Trinidad and Tobago) even though the information in it isn’t new. Attempts are being made by some to make things better, but in the end they become feeble – if not brittle – under a legal system that is undeniably swayed by whoever has the most money.

If it sounds like I’m griping – 10 years ago I would have been. This is just a statement of fact at this point. I did what I could to inform over the years, as did others, but ultimately the choice was not that of a well informed minority but that of a poorly informed majority.

Deep Learning / Artificial Intelligence

Deep learning allows amazing things to be done with data. There is no question of that; I’ve played with it myself and done my own analyses on things I have been working on in my ‘spare time’ (read: I have no life). There’s a lot of hypotheses that can come from big data, but it’s the outliers within the big data that are actually the meat of any hypothesis.

In English, the exceptions create the rules which further define what needs to be looked at. For outliers in the data can mean that another bit of data needs to be added to the mix.

Artificial Intelligence (AI), on the other hand, can incorporate deep learning and big data. While an AI may not be able to write a news article that can fool an editor, I imagine it could fool the reading public. This is particularly true since, because of the income issues related to the Internet, media outlets have gone to pulp opinionated pieces instead of the factual news that used to inform rather than attempt to sway or just get more reads by echoing a demographic’s sentiment. Then it is shared by people of like-minded people on social media. It’s an epic charlie-foxtrot. 

People worry about jobs and careers in all of this with robots and AI, and a lot of white collar folks are thinking it will affect those in the blue collar jobs alone. No, it will not. There is an evolution taking place (some call it a revolution), and better paid white collar jobs are much juicier for saving money for people who care only about their stock price. 5 white collar jobs are already under the gun.

KFC and McDonalds have already begun robotizing. More are coming.

And then let’s discuss Ethics in the implementation of AI – look at what Microsoft did with their Twitter-bot, Tay. We have a large corporation putting an alleged AI (chatbot, whatever you want to call it) into a live environment without a thought to the consequences. Granted, it seemed like a simple evolution of Eliza (click the link to see what that means), but you don’t just let your dog off it’s leash or your AI out in an uncontrolled environment. It’s just not done, particularly in an environment where kids need ‘safe places’ and others need trigger warnings. If they didn’t have an army of lawyers – another issue with technology – they probably would have had their pants shaken severely in Courts across the world. Ahh, but they do have an army of well paid lawyers – which leads us to Intellectual Property.
Space Marines: Into the Future

Copyrights, Patents and Trademarks (and Privacy)

If you haven’t read anything about Copyright by Lawrence Lessig in the past decade, or Privacy by Daniel Solove, you’re akin to an unlicensed, blindfolded teenager joy riding in your Mom’s Corvette ZR1. Sure, things might be fun, but it’s a matter of time unless you’re really, really lucky. You shouldn’t be allowed near a computing device without these prerequisites because you’re uninformed. This is not alarmist. This is your reality.

And anyone writing code without this level of familiarity is driving an 18 wheeler in much the same way.

You need a lawyer just to flush a virtual toilet these days. I exaggerate to make the point – but maybe not. It would depend on who owns the virtual toilet.

You can convert any text into a patent application. Really.

Meanwhile, Patent trolls are finally seen as harming innovation. The key point here is that the entire system is biased toward those with more in the bank – which means that small companies are destroyed while the larger companies, such as Google and Oracle, have larger legal battles that impact more people than even know about it. Even writing software tools has become a legal battle between the behemoths.

‘Fair Use’ – the ability to use things you bought in ways that allow you to keep copies of them – has all but been lost in all of this.

Meanwhile, Wounded Warrior – an alleged veteran’s non-profit – has been suing other non-profits because of use of the phrase, ‘Wounded Warrior’. If you want to take the nice approach, they’re trying to avoid dilution of their trademark… at the cost of veterans themselves, but that doesn’t explain them suing two of their own former employees with PTSD.

And Here I Am, Wondering About The Future.

There are a bunch of very positive articles out there that do not touch on the problems we face in technology. Our technology is presently being held for ransom by legal frameworks that do not fit well; this in turn means our ability to innovate, and by proxy entrepreneurship, are also being held ransom. Meanwhile we have people running around with Stockholm Syndrome waiting for the next iPhone hand built by suicidal workers, or the next application that they can open their private data to (hi, Google, Microsoft!), or…

I can’t predict anything at this point. It used to be much simpler and, by proxy, easily controlled. The questions of whether to do something used to be an ethical question, but now we go to lawyers for ethics (a group that is largely not known for ethics – apologies to those who do). The governments institute policies biased by whoever funds the campaigns of politicians, or gives United States congress people nice things. It affects the entire world, and every few years I think it won’t last – it continues.

Too big to fail.

But out of all of this, I don’t mean to stop trying. I don’t mean to stop innovating, starting new businesses, etc. What I mean is – we have a lot of things to do properly to assure a future that isn’t as dim as I see it now, to assure that the kids who are hooked on realities that someone else created rather than what they imagined. Imagination itself needs to be revisited, cultivated and unleashed against all of this like a cool wind across the desert.

It cannot be done blindly. People need to understand all of this. And if you made it this far – congratulations – I offer that you should, if not share this, share the ideas within it freely rather than simply clicking ‘like’ and hoping for the best.

We cannot change things on our own.

As for myself – just surfing the waves as they come in, but I fully intend to build my house on a distant shore at this point.