python

2019: New Year, Same Problems.

~ Taran ~ 1 Comment

Experimenting with proximity and remote control with @anki Vector.I’ve managed to avoid the deluge of end of year posts by people, as well as their bright and shiny posts of what they expect in 2019. After you’ve seen enough of them, you know the recipe and you can make your own – even if it’s not a very good recipe, even if it typically doesn’t stand the test of time.

A ‘New Year’ is just another date on the calendar for me these days – and truth be told, it has been for some time. So I spent this ‘holiday’ running some experimental code associated with the Anki Vector I picked up.

As a way of tracking what changes and what doesn’t, years are fickle. As an example, when it comes to code, the thing we sent that is furthest is still running 8-bit code, and it still seems to be working well. Looks like hunspell (that’s what you call it for pip) is the droid I was looking for, though the documentation on that… well…

Things that haven’t changed that much is the acceleration of technology – because it continues to accelerate, and documentation on it is simply horrible in some areas. I spent roughly an hour delving into replacements for PyEnchant, as an example, reading all sorts of the same thing that Google thought would be useful – and which wasn’t.

And this is, sadly, the sort of detritus that software projects leave behind. As a friend mentioned today, a lack of documentation is better than bad/misleading documentation – and when it comes to documentation, a lack of date tagging condemns people to whatever algorithm the search engine uses when college students are trying to find hardly known authors to plagiarize from.

It goes beyond that. There’s a trend where technology gets disposed of so fast that there is almost no documentation on any of it, or if there is, it’s dated and/or misleading.

This is why we’re not fixing things as much, those of us that have that mindset – because there are always a few people, statistically, that can fix things – remember repair shops? And then there are the people who pay to fix things. The way intellectual property – really, copyright – has gone in a legal sense keeps a space between people who would repair and the owners of copyright. And the contracts, threats about warranty… even more space, starving the ability for products to be supported by third parties.

Heaven forbid you reverse engineer something to fix it. That can get you in trouble with people have chain-linked bracelets and lawyers who love killing trees.

That’s where Open Source and Free Software were supposed to step in, at least in the context of software – but after a few decades, it’s all relatively young and the documentation is done largely in crayon hieroglyphics. The successful projects are documented, at least to some degree.

If there’s one thing that I’d like to see change this year, it’s people getting better at documentation. It’s as if they think what they do isn’t worth that investment.

And when they don’t, it isn’t.

Exploring the Anki Vector SDK Alpha.

~ Taran ~ 1 Comment
Installed the Vector SDK. @anki
Installing the SDK – which, fortunately, was easier since I am already running Anaconda (Python) for other things I’m fiddling with.

In my last post here, I said that the true value of the Anki Vector to me would be determined by the Software Development Kit (SDK), which wasn’t yet released.

I am a bit disappointed that no one at Anki answered my tweet on it to date – and so I used a Douglas Adams reference about hiding things when I tweeted them again.

A fair criticism of Anki is that they aren’t very good at organizing the information and updating customers when they’re doing pretty good things. Frankly, the beginning novelty of Vector and it’s potential is what seems to be allowing them not to pay as much for this faux pas. And too, I suspect, the project has grown faster than the company has – a testament to engineering. It has apparently sold well, a testament to their marketing. Yet when it comes to information on the product, it seems pretty hard to come by information users/expected are expected to have.

Installing the Vector SDK

I found the Vector SDK Alpha release note through an Anki blog entry not as easily found as I would have liked. Within it you’ll find the link to the SDK documentation, and within that you’ll find the actual downloads. I found this through force of will, largely because Vector was sitting impatiently on his charger for almost a week making R2D2-ish sounds while giving me the baleful look of Wall-E when I walked by.

It’s amazing how those eyes are really the center of how we see Vector.

I installed the Alpha SDK, and I configured Vector – which involves getting the IP address of Vector. It’s not available through the app on the phone, and there’s a trick to it (in case you’re looking for it yourself) – you have to tap Vector’s top button twice, then raise and lower his arm. Vetor’s IP address will then be shown where his eyes are. To get back to normal operation, raise and lower Vector’s arm again. Sacrificing a chicken is optional. Be careful with blood spatter; Vector is not fluid-proof.

After that, it was a simple matter of firing Spyder up – part of the Anaconda data science platform for Python, but available standalone – and ran some of the example code, tweaking it here and there to get a feel for the capabilities of the Vector SDK Alpha.

This is where they shine – when it comes to sharing the code. And the SDK documentation itself, so far, is pretty good.

The Reality of the SDK.

I think I was expecting a bit more from the SDK, which is my fault and I acknowledge that. I had expected more in the way of interacting with the cloud itself – for example, renaming Vector’s wake phrase/word, or allowing behavior change during normal operation. That’s presently not there, which effectively gives Vector a multiple personality disorder – with blackouts where, for better and worse, the SDK allows the hijacking of Vector.

Imagine waking up and not knowing how you got somewhere, what you just did, and where that eyebrow went. That’s a fair anthropomorphization.

The SDK works  through your wireless connection – the code/application has to be running on the same network as Vector, and your specific machine gets a certificate to run the code on Vector – a good security precaution or people would be hacking Vectors and checking out other people’s places.

It’s bad enough with the Alexa integration – I had an Alexa when they first came out but had enough creepy incidents with Amazon to get rid of mine. Still, the world of Amazonians wants it and it’s a good selling point for Anki, so I get it. That seems to be done well enough to please those that wanted it, so maybe they’ll focus on things other than that now.

In all, I’d like to transfer a version of what they have in the cloud into my personal systems and allow me to tinker with that as well.

Still, given what I have been playing with related to machine learning and natural language processing – it’s no mistake that I had the Anaconda distribution of Python installed already – I’m having a bit of fun playing with the SDK and testing the limitations of the hardware.

@anki Vector vide feed example. Rocking.Some things I noticed

The video from the Vector hardware platforms is good enough for some basic things, but lighting really does affect it. This is a limitation in it’s exploration, and it limits it’s facial recognition ability (the one thing I’ve found you can access from the cloud in a limited way).

I’ve been considering a polarizing film over the cameras for better images, and have even considered mounting a light source on Vector for darkness, which would have the misfortune of not being able to be controlled through Vector (but it could be controlled independently through code). I plan to play with the lights part of the SDK to see what I can get away with.

You don’t get to fiddle with facial recognition code, but there’s Python code for that – such as PyPi face_recognition.

The events ability does allow for more reactive things.

Making Vector use profanity is a must, if only once.

There are error codes that aren’t documented – I had the 915 error twice on Vector while I was writing this, and all I found was on Reddit. Without error codes, we don’t get error trapping with Vector – and that’s a problem that I hope they address in the Beta.

Overall – I’m happier with the SDK, which shows promise and a bit of effort on the part of Anki. The criticisms I have so far are of an Alpha SDK – which means that this will change in time.

They do need to get a bit better at the responsiveness, though – something I suspect that they are already aware of. To enjoy this level of success comes with painful growth. If only that were an engineering problem to solve.

Portable MariaDB and Python (USB drive)

~ Taran ~ Leave a comment

Schlüssel / Keys #apple #MacBook #MacBookPro #USB #Opener #Bottleopener #Prada #KeyringI’ve got a few projects rotating through the front burners, and one of them involves setting up a MariaDB instantiation on a USB key or drive or whatever you want to call it. And I want Python, at least for development. The nomenclature goes out the door when there’s more conflicted marketing than unconflicted engineering.

There’s some good stuff I found along the way that others might find useful. If you just want to get to the core of the solution, skip down to the ‘My Solution’ portion of this entry.

Stuff Found Along The Way.

I learned a few things along the way. For example, some people might be taken by this article on installing a web server on a USB key. Before you run off, you’ll might want to know about the Windows 10 Version of XAMPP available only on this German site (that I could find at the time of this writing).

A little rant here: Someone needs to do an AMP stack with Python. If I get bored enough and have enough time, maybe that will be something I fiddle with.

There is, of course, the ability to simply run a portable Linux install – I would historically go with Knoppix – but I opted out of doing that at this time because I just don’t want the overhead. No distro wars here, plenty of good distros here. In the end, when the time comes, I’ll research the new portable distros that have come out.

My Solution

It’s really simple. Just download MariaDB and drop the archive (zip, tar, whatever) onto your USB drive. Decompress it there. When you run it off the key, use the  “–no-defaults” option. If you’re running Windows, modify your shortcut with it.

“–no-defaults” keeps all paths relative. Bear that in mind if things get wonky. They haven’t for me, yet.

As far as Python,  WinPython is the answer for Windows. Runs right off the USB key. Read the documentation, someone wrote it for a reason.

Yeah, I know. Linux. Eventually, it will likely go that route, but for now it has to stay Windows 10 because of humans who use Windows 10.

Coding For Fun.

~ Taran ~ Leave a comment

Here I am, waiting for the errant landlord to show up at 7 a.m. when it’s now 8 a.m. (where is he? Who knows), to fix a breaker issue, when… suddenly… I think of something to code – not that I was looking anymore, but probably exactly because I wasn’t looking anymore.

It’s an exercise, I suppose – an idea – and following it up in Python is probably going to be more interesting than anything else you can do when stuck at home. In Trinidad and Tobago, if you have human dependencies, you’re pretty much stuck waiting on someone who is waiting on someone who is – well, you get the point.

And so – Python on Windows reinstalling; Anaconda of course (because you found this entry, I fully expect you can find Anaconda and the installation documentation).

Now, here’s the thing. Unlike when I was running my own company, and unlike when I was working for other companies, there’s no race. There’s no need for me to worry about whether or not a competitor will get to it in time, and whether they can do it better, or what have you.

And oddly, it allows me to leverage some pre-existing code I’d been working on. You’ll note I didn’t say re-use – that code reuse thing is a trap, much as using code from Stack Overflow can be for plug and pray ‘coders’.

Anyway, back to a little coding fun again after about a year without, nose firmly thumbed at corporate coding.

Pulling My Photos From Flickr: Lessons Learned, More.

~ Taran ~ 1 Comment

nsb SunriseI wrote ‘Risk and the Photo Cloud‘ as a first stab at identifying the problem I am having with mitigating risk with my Flickr stream as well as new needs I have from my collection of images.

Bear in mind, this is a ‘one off’ problem for me, and I approached it as such.

I was a bit more focused on what I wanted to do with them – I have people willing to buy prints – and I may not have made the best choice by paying $25 for Bulkr Pro. The trick was to get the tags, etc, and in doing the initial research on the Flickr forums, I saw nothing better. In writing this today, I found FlickrDownloader which I probably would have chosen given that it’s open source, available at no cost, etc. I may give it a spin anyway. Try it out and let me know how it differs. 

When using Bulkr Pro,  you can opt to have the full information being saved to text files.  This is generally not a bad idea, but it’s certainly not a great idea when you have over 19,000 files on Flickr as I do (only 18,000 or so are public). So, the general idea is to get them into a database.

I opted not to directly import them into the database after some thought because I need to do some manual editing of which files I want to keep, etc – and in doing that, a spreadsheet would be easier for that part of the process. So that’s what I did.

The text files generated by Bulkr are mildly annoying in this regard because their format isn’t meant for what I wanted to do. However, the line numbers for the specific information is constant, as is the labeling, so I was able to whip together a Python 3.x script that reads all the text files and writes them all to a CSV. I tossed it up on my GitHub; you can check out https://github.com/knowprose/BulkrTxtFilesToCSV

So now, I have the information in a CSV and, as time permits, I’m working on choosing which images I am getting rid of (with Flickr, I was lazy about space). Let’s just be clear and call it ‘dirty data’; when I uploaded I did it as a hobby and I now have a professional use.

From there, it’s a simple matter of uploading the CSV to MariaDB, which is extremely fast.

Having spoken to a few professional photographers I know, there are professional tools out there that do what I want with image management, etc, but I’m not too pleased with them. I may end up spinning my own image and file management system in Python, or not – I’m undecided at this point because I’m moving from a ‘one off’ to a more consistent system that will suit my personal needs.

Why Python? Honestly, I like coding in Python, but the job market has always been more interested in my C, C++, C#, VB, VB.Net, PHP/MySQL, etc. This is a low priority project for me, and it’s something I want to be fun while getting used to Python 3 a bit more. 

At this point, the system I’m thinking of will create resized images for the web, as well as allow me to edit based on tags. Pillow will allow for much of that, and more.