GIGO

Through A Blurry Looking Glass.

~ Taran ~ Leave a comment

_web_Reviewing Code Frosted GlassI’ve been spending, like so many, an inordinate amount of time considering the future of what we accuse of being artificial intelligence, particularly since I’ve been focusing on my writing and suddenly we have people getting things written for them by ChatGPT. I’ll add that the present quality doesn’t disturb me as much as the reliance on it.

Much of what these artificial intelligences pull from is on the Internet, and if you’ve spent much time on the Internet, you should be worried. It goes a bit beyond that if you think a bit ahead.

Imagine, if you would, artificial intelligences quoting artificial intelligences trained by artificial intelligences. It’s really not that far away and may have already begun as bloggers looking to capitalize on generating content quickly thrash their keyboards to provide prompts to ChatGPT and it’s ilk to create blog posts such that when they market their content it pops up in search engine results. Large language models (of which ChatGPT is one) suddenly think this is great content because what is repeated most makes predictive models say, “Aha! This must be what they mean or want!”.

long email ai

This cartoon at Marketoonist.com pokes at the problem with “AI Written, AI Read”, but not the recursive issue. Fortunately, on a personal blog I can use it otherwise it would take us down the path of copyright and the stable diffusion lawsuit.

This has the potential to be a scary future. People who look forward a few steps have started an open letter, which of course I signed, asking to Pause Giant AI Experiments. It’s a symbolic thing, I fully expect, because the people who are behind these artificial intelligences aren’t really too interested in much more than their bottom line, with the notable exception of DeepAI.org as far as I know.

Meanwhile, so many seem to think that these Large Language Models are a nice replacement for the nerds doing their homework because they don’t have to hang out with them. There’s a reason I bring this up. One of the reasons Artificial Intelligences, Large Language Model or otherwise, are considered creative is because they are basically well read people with mega-gobs of information at their digital synapses. They make it easier to find the intersectionality, a realm that has become more and more important and less and less thought about in our society where specializations require diplomas to do things that… one really doesn’t need a diploma to do.

And it doesn’t know everything. For example, I know a bit about Trinidad and Tobago, so I posed a question to ChatGPT related to the economy of Trinidad and Tobago (response in italics):

What are good exports that Trinidad and Tobago could focus on to help build the economy and better the socioeconomic status of it’s citizens?

Trinidad and Tobago has a diversified economy with a strong focus on the energy sector, particularly oil and gas production. However, there are several other sectors that the country could focus on to diversify its exports and boost its economy. Some potential export industries for Trinidad and Tobago include:

Tourism: Trinidad and Tobago has a rich cultural heritage, beautiful beaches, and unique natural attractions such as the Caroni Swamp and the Asa Wright Nature Centre. The country has the potential to further develop its tourism industry and attract visitors from around the world.

Agriculture: Trinidad and Tobago has a favorable climate for agriculture and could focus on exporting products such as cocoa, coffee, citrus fruits, and vegetables. The country could also explore the potential for organic farming and niche products such as spices and herbs.

Creative industries: Trinidad and Tobago has a vibrant cultural scene and could focus on exporting its music, film, and other creative products. The country has a rich tradition of Carnival, which could be further developed as a major tourist attraction and cultural export.

Information and communication technology (ICT): Trinidad and Tobago has a well-educated and tech-savvy workforce, and the country could focus on developing its ICT sector and exporting software, IT services, and other digital products.

Manufacturing: Trinidad and Tobago could focus on developing its manufacturing industry and exporting products such as chemicals, plastics, and processed foods.

Of course, developing any of these industries would require significant investment and effort, and there may be other factors to consider such as infrastructure, labor costs, and market demand. However, focusing on these industries could help diversify the economy and provide new opportunities for the citizens of Trinidad and Tobago.

Sounds very good, right? Devil is in the details.

Tourism is a definite possibility, but the decades long crime situation (which myself and others believe is because of socioeconomics related to the lack of diversity in the economy), as well as flash flooding and a focus on marketing rather than quality… no. I do like that it mentioned the Asa Wright center, and if anyone actually does come down this way, I can happily point you to other places that you won’t find in the tourist brochures.

Agricultural land has been used by the the government to build housing, so arable land use is decreasing with every the Housing Development Corporation creates, as well as with every agricultural plot of land converted to residential, commercial or industrial depending on who greases the wheels.

Manufacturing would be brilliant. Very little is made in Trinidad and Tobago, but if you’re going to be competing with robots and artificial intelligences in the developed world, we can throw that out.

ICT is my personal favorite, coming from a chatbot that has already got people generating code with it. Seriously, ChatGPT?

Everything ChatGPT has presented has been said more than once in the context of diversifying the economy of Trinidad and Tobago, and it’s a deep topic that most people only understand in a very cursory way. The best way to judge an economy is to observe it over time. In the grand scale of global discourse, the estimated population of 1.5 million people in a dual island nation is not as interesting to the rest of the world as Trinbagonians would like to think it is – like any other nation, most people think it’s the center of the universe – but it’s not a big market, for opportunities young intelligent people leave as soon as they can (brain drain), and what we are left with aspires to mediocrity while hiring friends over competency. A bit harsh, but a fair estimation in my opinion.

How did ChatGPT come up with this? With data it could access, and in that regard since it’s a infinitesimal slice of the global interest, not much content is generated about it other than government press releases by politicians who want to be re-elected so that they can keep their positions, a situation endemic to any democracy that elects politicians, but in Trinidad and Tobago, there are no maximum terms for some reason. A friend sailing through the Caribbean mentioned how hard it was to depart an island in the Caribbean, and I responded with, “Welcome to the Caribbean, where every European colonial bureaucracy has been perpetuated into stagnancy.“

The limitations using Trinidad and Tobago as a test case, an outlier of data in the global information database that we call the internet, can be pretty revealing in that there is a bias it doesn’t know about because the data it feeds on is in itself biased, and unlikely to change.

But It’s Not All Bad.
I love the idea that these large language models can help us find the intersectionality between specialties. Much of the decades of my life have been spent doing just that. I read all sorts of things, and much of what I have done in my lifetime has been cross referencing ideas from different specialties that I have read up on. I solved a memory issue in a program on the Microsoft Windows operating system by pondering Costa Rican addresses over lunch one day. Intersectionality is where many things wander off to die these days.

Sir Isaac Newton pulled from intersection. One biography describes him as a multilingual alchemist, whose notes were done in multiple languages which, one must consider, is probably a reflection of his internal dialogue. He didn’t really discover gravity – people knew things fell down well before him, I’m certain – but he was able to pull from various sources and come up with a theory that he could publish, something he became famous for, and something in academia that he was infamous for with respect to the politics of academia.

J.R.R Tolkien, who has recently had a great movie done on his life, was a linguist who was able to pull from many different cultures to put together fiction that has transcended beyond his death. His book, “The Hobbit”, and the later trilogy of “The Lord of the Rings” have inspired various genres of fantasy fiction, board games and much more. 

These two examples show how pulling from multiple cultures and languages, and specialties, are historically significant. Large Language Models are much the same.

Yet there are practical things to consider. Copyrights. Patents. Whether they are legal entities or not. The implicit biases on what they are fed, with the old software engineering ‘GIGO’ (Garbage in, garbage out) coming to mind with the potential for irrevocable recursion of supercharging that garbage and spewing it out to the silly humans who, as we have seen over the last decades, will believe anything. Our technology and marketing of it are well beyond what most people can comprehend.

We are sleeping, and our dreams of electric sheep come with an invisible electric fence with the capacity to thin the herd significantly.

 

Artificial Intelligences and Responsibility.

~ Taran ~ Leave a comment

AI-NYC_2017-1218MIT Technology Review has a meandering article, “A.I Can Be Made Legally Responsible for It’s Decisions“. In it’s own way, it tries to chart the territories of trade secrets and corporations, threading a needle that we may actually need to change to adapt to using Artificial Intelligence (AI).

One of the things that surprises me in such writing and conversations is not that it revolves around protecting trade secrets – I’m sorry, if you put your self-changing code out there and are willing to take the risk, I see that as part of it – is that it focuses on the decision process. Almost all bad decisions in code I have encountered have come about because the developers were hidden in a silo behind a process that isolated them… sort of like what happens with an AI, only two-fold.

If the decision process is flawed, the first thing to be looked at is the source data for the decisions – and in an AI, this can be a daunting task as it builds learning algorithms based on… data. And so, you have to delve into whether the data used to build those algorithms was corrupt or complete – the former is an issue we get better at minimizing, the latter cannot be solved if only because we as individuals and more so as a society are terrible at identifying what we don’t know.

So, when it comes to legal responsibility of code on a server, AI or not, who is responsible? The publishing company, of course, though if you look at software licensing over the decades you’ll find that software companies have become pretty good at divesting themselves of responsibility. “If you use our software we are not responsible for anything”, is a good short read that most end user license agreements and software licenses have in there, and by clicking through the OK, you’re basically indemnifying the publisher. That, you see, is the crux of of the problem when we speak of AI and responsibility.

In the legal frameworks, camped Armies of Lawyers wait on retainer for anything to happen so that they can defend their well paying client who by simply pointing at a contract that puts all responsibility on the user. Lawyers can argue that point, but they get paid to and I don’t. I’m sure there are some loopholes. I’m sure that when pushed into a corner by another company with similar or better legal resources, ‘settle’ becomes a word used more frequently.

So, if companies can’t be held responsible for their non-AI code, how can they be held responsible for their AI code?

Free Software and Open Source software advocates such as myself have made these points more often than not in so many ways – but this AI discussion extends into data as well, which pulls the Open Data Initiative into the spotlight as well.

The system is flawed in this regard, so to discuss whether an AI can be responsible for it’s decisions is silly. The AI won’t pay a fine, the AI won’t go to jail (what does ‘life’ mean for an AI, anyway?). Largely, it’s the court of public opinion that guides things – and that narrative is easily changed by PR people who have a side door to the legal department.

So let’s not discuss AI and responsibility. Let’s discuss code, data and responsibility – let’s go back to where the root of the problem exists. I’m not an MIT graduate, but I do understand Garbage In, Garbage Out (GIGO).

Deep Learning, Information Bottlenecks – and Osmosis.

~ Taran ~ Leave a comment

I’ve experimented in the past with deep learning in a few different ways, coming up with my own thoughts on how things work and why they work. It was apparent to me when I stopped that in 2016 that I was missing something, and that I needed some distance between myself and the topic at hand. I gave up those Pine64s, and as it happened, moved away from where I was doing it – more importantly, divorcing me from a Software Engineering world where ‘solutions right now’ always trumped ‘solutions’, the former the harbinger of problems, the latter the Holy Grail of every software engineer who dare dream in a world that, except for the minority, requires lockstep precision within an industry that spends it’s time firefighting because of solutions-right-now.

It’s disenchanting. Being disenchanted allows for little in the way of real solutions, at least for myself.

And today I read, “New Theory Cracks Open The Black Box of Deep Neural Networks“. Of course, deep learning is not that new, and the ‘Information Bottleneck’ thought stems from the original work in 1999, the Information Bottleneck Method. That works perhaps in explaining things on a surface level and on an informational level – but as I read it, I was reminded of secondary school biology: Osmosis. No one has seemed to connect the two when they are so suitably connected, and I’d wager that Osmosis scales better since the information bottlenecks, when themselves in a matrix, pretty much would mimic a tunable osmosis.

That said, I’ve found the major problem with deep learning to be that we define inputs when, quite possibly, we should be more loose in our definitions of what we put in. This aligns better with chaos theory – something that the Wired article seems to dismiss:

…When Schwab and Mehta applied the deep belief net to a model of a magnet at its “critical point,” where the system is fractal, or self-similar at every scale, they found that the network automatically used the renormalization-like procedure to discover the model’s state. It was a stunning indication that, as the biophysicist Ilya Nemenman said at the time, “extracting relevant features in the context of statistical physics and extracting relevant features in the context of deep learning are not just similar words, they are one and the same.”

The only problem is that, in general, the real world isn’t fractal. “The natural world is not ears on ears on ears on ears; it’s eyeballs on faces on people on scenes,” Cranmer said…

Pragmatically, this is what we see when we work on projects – but the problem is not what we see, it’s what we don’t see. It’s the things we don’t intuitively connect ourselves because of our own limitations; with simple deep learning we may get away with what we see, but on a much larger scale, we may be looking at the motion of wings of a butterfly on the other side of the world causing a tipping point that creates a hurricane on the other.

Of course, this is all theory, and hardly some earth shattering change in the way we look at things – but a small change in how we approach things could well be what we need to move forward at various intersections. In this, I am trying to be a simple butterfly flapping his wings.