Being a bit busy with other things, I didn’t get to write a little but about a new recommendation algorithm which involves artificial intelligence: Pyrorank. It has some lofty claims, largely hidden by academia and academic verbiage.

I initially read about it in Researchers devise algorithm to break through ‘search bubbles’ on July 10th and put it into the stack of things I consider interesting. ‘Search’ and ‘Bubbles’ mean something to some of us who, in the days of antiquity, wrote our own search and sort algorithms from scratch.

What this new algorithm is claimed to do is give you better recommendations by “reducing the impact of users’ profiles and broadening recommendations that still reflect the focus of the search, producing more diverse and useful results.”

In other words, recommendations on Netflix, or Amazon.com, or even advertising on Facebook could be less annoyingly predictable, showing us the same things – some of which we may have already seen, purchased, or passed over before. Personally, it wasn’t long before I started seeing present recommendation algorithms as a tyranny, and with my eclectic tastes that can be supremely annoying.

Recommendation systems, used by Google, Netflix, and Spotify, among others, are algorithms that use data to suggest or recommend products or choices to consumers based on the users’ past purchases, search history, and demographics. However, these parameters bias search outcomes because they put users in filter bubbles.

“The traditional way recommendation systems work is by basing recommendations on the notion of similarity,” explains Bari, who leads the Courant Institute’s Predictive Analytics and AI Research Lab. “This means that you will see similar items in the choice and recommended lists based on either users similar to you or similar items you have bought. For instance, if I am an Apple product user, I will increasingly see more and more Apple products in my recommendations.”

The limitations of existing recommendation systems have become evident in striking ways. For instance, political partisans may be largely directed to news content that aligns with their pre-existing views. More significantly, recommender systems have turned up self-harm videos to susceptible individuals.

Researchers devise algorithm to break through ‘search bubbles‘, New York University, 10 July 2023.

This sounds hopeful, particularly for social media algorithms which we have seen have reinforced polarized views. It could increase the size of the echo chambers (there are always echo chambers) while adding diversity to them.

Thus I did some digging this morning and found this paper, “Pyrorank: A Novel Nature-Inspired Algorithm to Promote Diversity in Recommender Systems“, which is unfortunately paywalled. I have requested the full text, and if I do manage to get it and I do find anything particularly interesting in it, I’ll post a bit more on it.

I also looked for different perspectives on it; others would have looked at the paper and found different things worth highlighting. My focus was on how it actually compares.

A comparison was made between Pyrorank and the traditional recommendation system to test the viability of each system. This experiment was carried out on large Movielens, Good Books, and Goodreads datasets. The objective of the testing was to find out which system stays true to the purpose of accurate recommendations while simultaneously providing diversification and unrepeated results.

The results were hugely in favor of Pyrorank, which not only stuck to genuine core recommendations but also gave mixed results that did not align with the past results of product purchases or to someone who is a similar user.

Pyrorank – A New Pathway For Your Search Engine, DigitalWorld, Web Desk, Wednesday, July 12, 2023

Unfortunately, it doesn’t say how rigorous the testing was, but it does sound a little promising.

What is interesting is that this is based on pyrodiversity, which is something that I had never considered, so to me this is a little new and exciting – once it lives up to it’s claimed results.

There may be hope for recommendations yet.