“There is no perfection, only beautiful versions of brokenness” - Shannon L. Alder
About one year ago I wrote my first web app, the Music Suggestions Ninja. It wasn’t fancy and it wasn’t fast. But it worked and people liked it. I wrote it in Python. It used very little dependencies, but only because I didn’t know about any of the great Python packages that I know of now. To make it work, I ended up writing a homegrown fuzzy string matching library and I learned a lot about NGINX loaded balancing. Still, it suffered massively in production.
But it worked. Its been running for about a year now on my schools fancy virtual machine - a 2-core Intel Xeon 2.70GHz loaded with Ubuntu, 2GB of memory and 20GB of disk space. Since I would eventually lose access to this fancy virtual machine, I decided to rewrite this program knowing what I know now - so that it functions perfectly on a Raspberry Pi 2 with a third the computing power, half the disk space and half the memory.
We can rebuild. We have the technology.
I rewrote everything in my new favorite language, Go following some generic principles. It took about 3 hours. Luckily, I had just ported my homegrown fuzzy string matching library from Python to Go which had a 10x speed improvement already. From there I followed the same methods I used to write my Golang wiki to add routes and process inputs. Everything else is basically the same, except the backend database is now BoltDB instead of MySQL.
Benchmarking the new vs old
I benchmarked both versions of Music Suggestions Ninja, the old and new, by searching for a band called “Coldpla” (which is fuzzy matched to “Coldplay”). The disk space was measured using
ds -sh ./, the memory usage was measured using
free -m before and after stopping program, and the speed was measured using
locustio by simulating 300 users hatching 10 users/second. Here are the results:
|Version||Disk space||Memory usage||Speed|
|Old||480 MB||580 MB||1 req/sec|
|New||140 MB||5-20 MB||20 reqs/sec|
Basically, my new version is 20x faster and uses 70% less disk space and 97% less memory. Note: the improvement is not strictly due to Python vs Go. A lot of the improvement comes from implementing everything in a much smarter way (which comes from experience and is not language specific). E.g. I am using more DB files instead of loading into memory, I am skipping
for loops that were unnecessary and using more REGEX.
One year ago, it wasn’t possible for me to slim down my programs to allow the Raspberry Pi to run one web app. One year later, I am hosting 4 web apps on my Raspberry Pi thanks to knowledge and experience that comes with time (and lots of playing).