— 3 min read

As you might expect, pypip.in employes a fair amount of caching in the backend to control load on the imaging API and servers.

For a long time, this cache was entirely managed by Varnish and was doing a fantastic job. Varnish has a hit:miss ratio of 10:1, for every 10 hits we get 1 miss. This is a fairly decent ratio when you consider where these images are displayed, how often they are viewed and that Varnish only caches the images for an hour.

The impact on PyPI

You will firstly need to understand how pypip.in used to work to understand the changes that were made and why they were made.

Let’s set up the request first - a request for a shield is made and it is not present in the Varnish cache.

Request received in API layer
              |
              v
    API layer queries PyPI
              |
              v
   PyPI …
 — 4 min read

While pypip.in is available under the MIT license on GitHub, it’s not explained how to really use it properly.

You can gather how to set-up the Python source of the project and get the Twisted process running, this is totally reliant on using the img.shields.io.

I decided to write this article explaining how to install your own copy of the shields nodejs code, pypipin itself and even cover off supervisord and Varnish too.

shields & nodejs

nodejs

First of all you’ll need to get the latest source code copy of nodejs from the nodejs download page.

Extract it.

tar -xvzf node-<VERSION>.tar.gz

cd node-<VERSION>

You’ll need to install the build tools, if you don’t have them already.

sudo apt-get install build-essential

And then make and install node.

make && sudo make install

Redis

Redis is used to temporarily store PyPI responses.

sudo …