— 3 min read

As you might expect, pypip.in employes a fair amount of caching in the backend to control load on the imaging API and servers.

For a long time, this cache was entirely managed by Varnish and was doing a fantastic job. Varnish has a hit:miss ratio of 10:1, for every 10 hits we get 1 miss. This is a fairly decent ratio when you consider where these images are displayed, how often they are viewed and that Varnish only caches the images for an hour.

The impact on PyPI

You will firstly need to understand how pypip.in used to work to understand the changes that were made and why they were made.

Let’s set up the request first - a request for a shield is made and it is not present in the Varnish cache.

Request received in API layer
              |
              v
    API layer queries PyPI
              |
              v
   PyPI …
 — 4 min read

While pypip.in is available under the MIT license on GitHub, it’s not explained how to really use it properly.

You can gather how to set-up the Python source of the project and get the Twisted process running, this is totally reliant on using the img.shields.io.

I decided to write this article explaining how to install your own copy of the shields nodejs code, pypipin itself and even cover off supervisord and Varnish too.

shields & nodejs

nodejs

First of all you’ll need to get the latest source code copy of nodejs from the nodejs download page.

Extract it.

tar -xvzf node-<VERSION>.tar.gz

cd node-<VERSION>

You’ll need to install the build tools, if you don’t have them already.

sudo apt-get install build-essential

And then make and install node.

make && sudo make install

Redis

Redis is used to temporarily store PyPI responses.

sudo …
 — < 1 min read

Supported Python versions

This one is generated from the list of classifiers you provide to PyPI.

If no Python version classifiers exist, it defaults to Python 2.7. This is because really, Python 3 is not widely used in production or supported by libraries.

Python implementation(s)

I think this one is really cool. Chances are you’re unlikely to get more than two supported implementations, like CPython and PyPy or CPython and Stackless.

The shield uses the Python implementation classifiers to generate this shield. It supports all classifiers that PyPI supports (CPython, Jython, Iron Python, PyPy and Stackless) and defaults to CPython is none are set.

Styling changes

This change is simply because of the upgrade of the shields library. This allows us to use the default rounded badges like below.

But also allow you to use a much nicer, cleaner, flat styling like the ones used on this …

 — 2 min read

Some time back in April 2013 I was bored and looking for a new project to keep my attention, if only for a short period of time.

My colleague @codeinthehole had an idea but no time to implement it, this idea was to have shields like those of travis-ci (shown below) but displaying package download counts.

Status of blackhole on Travis CI

Tech stack

From the very start I decided to use Tornado framework, although this may change in the future.

The original plan was to generate the images using Pillow (PIL) and then simply cache them to disk. I decided it would make far more sense to do this using Varnish and not have to worry about it working as expected.

Manually generating the images

The images were originally generated from a base template using Pillow, but sadly Python’s image manipulation is not very good, especially it’s text manipulation and the shields could …