Revisiting my first web app

“There is no perfection, only beautiful versions of brokenness” - Shannon L. Alder

About one year ago I wrote my first web app, the Music Suggestions Ninja. It wasn’t fancy and it wasn’t fast. But it worked and . I wrote it in Python. It used very little dependencies, but only because I didn’t know about any of the great Python packages that I know of now. To make it work, I ended up writing a homegrown fuzzy string matching library and I learned a lot about NGINX loaded balancing. Still, it suffered massively in production.

But it worked. Its been running for about a year now on my schools fancy virtual machine - a 2-core Intel Xeon 2.70GHz loaded with Ubuntu, 2GB of memory and 20GB of disk space. Since I would eventually lose access to this fancy virtual machine, I decided to rewrite this program knowing what I know now - so that it functions perfectly on a Raspberry Pi 2 with a third the computing power, half the disk space and half the memory.

We can rebuild. We have the technology.

Where to start to transform my old web app into something leaner and faster? Well, when I took a look at the old site ( and ), it carries a bunch of junk no one cares about or uses. No one reads the blog, or the changelog, or the about page, or the terms of use, or the contact info. No one uses the API. So, out they go. Once I removed the junk, I realized there were only two components left - generating band names from searches and saving playlists. Simple.

I rewrote everything in my new favorite language, Go following some generic principles. It took about 3 hours. Luckily, I had just ported my homegrown fuzzy string matching library from Python to Go which had a 10x speed improvement already. From there I followed the same methods I used to write to add routes and process inputs. Everything else is basically the same, except the backend database is now BoltDB instead of MySQL.

Benchmarking the new vs old

I benchmarked both versions of Music Suggestions Ninja, the old and new, by searching for a band called “Coldpla” (which is fuzzy matched to “Coldplay”). The disk space was measured using ds -sh ./, the memory usage was measured using free -m before and after stopping program, and the speed was measured using by simulating 300 users hatching 10 users/second. Here are the results:

Basically, my new version is 20x faster and uses 70% less disk space and 97% less memory. Note: the improvement is not strictly due to Python vs Go. A lot of the improvement comes from implementing everything in a much smarter way (which comes from experience and is not language specific). E.g. I am using more DB files instead of loading into memory, I am skipping for loops that were unnecessary and using more REGEX.

tldr;

One year ago, it wasn’t possible for me to slim down my programs to allow the Raspberry Pi to run one web app. One year later, I am hosting 4 web apps on my Raspberry Pi thanks to knowledge and experience that comes with time (and lots of playing).