A brief history of a tiny part of the Internet.

Blackhole 1 — Blackhole as it was originally known — was written on Python 2.7, briefly supporting Python 2.6 for a time and also supporting early version of Python 3, PyPy 2 and PyPy 3. Built on top of Tornado, it was asynchronous in a fashion and — quite simply — worked.

The original prototype that became Blackhole was SimpleMTA — a prototype that was created quickly, to serve a very simple testing purpose that I had for it.

As I needed SimpleMTA to do more, I wrote Blackhole to accomplish that task. I’d been using Tornado a bit and wanted to experiment with it more. Building on top of Tornado created some oddities in how the program was designed and that always irked me.

Between the time of the last 1.8.X and the 2.0 release, I experimented with …

Batfish is a Python client and API wrapper for the Digital Ocean V2 API. It can be used as a library module in your own Python code but also provides a CLI interface and a shell-like command interpreter.

Batfish is still under development and is considered in the Alpha stage. It is not yet available via PyPI but can be tried out using the code available on GitHub.

There is a small amount of documentation available on Read The Docs and tests are still being written to get as much coverage as possible and eaked out all of the bugs. You can find the latest test status on Travis CI.

>>> from batfish import Client
>>> client = Client()
>>> client.authorize("abcde12345")
>>> client.droplets
[<Droplet ego.kura.gg>, <Droplet fax.kura.gg>, <Droplet jet.kura.gg>, <Droplet ski.kura.gg>]
>>> client.droplet_reboot(1234)

$ batfish authorize
abcde12345
$ batfish droplets
ego …

Yarg is a PyPI client, it was written for pypip.in and can search packages as well as read the RSS feeds from PyPI for new packages and new package version releases.

>>> import yarg
>>> package = yarg.get("yarg")
>>> package.name
u'yarg'
>>> package.author
Author(name=u'Kura', email=u'kura@kura.gg')

>>> import yarg
>>> packages = yarg.newest_packages()
>>> packages
[<Package yarg>, <Package gray>, <Package ragy>]
>>> packages[0].name
u'yarg'
>>> packages.url
u'https://pypi.python.org/pypi/yarg

>>> import yarg
>>> packages = yarg.latest_updated_packages()
>>> packages
[<Package yarg>, <Package gray>, <Package ragy>]
>>> packages[0].name
u'yarg'
>>> packages[0].version
u'0.1.2'
>>> packages[0].url
u'https://pypi.python.org/pypi/yarg/0.1.2

As you might expect, pypip.in employes a fair amount of caching in the backend to control load on the imaging API and servers.

For a long time, this cache was entirely managed by Varnish and was doing a fantastic job. Varnish has a hit:miss ratio of 10:1, for every 10 hits we get 1 miss. This is a fairly decent ratio when you consider where these images are displayed, how often they are viewed and that Varnish only caches the images for an hour.

Request received in API layer
              |
              v
    API layer queries PyPI
              |
              v
   PyPI …

While pypip.in is available under the MIT license on GitHub, it’s not explained how to really use it properly.

You can gather how to set-up the Python source of the project and get the Twisted process running, this is totally reliant on using the img.shields.io.

I decided to write this article explaining how to install your own copy of the shields nodejs code, pypipin itself and even cover off supervisord and Varnish too.

tar -xvzf node-<VERSION>.tar.gz

cd node-<VERSION>

sudo apt-get install build-essential

make && sudo make install

sudo …

Pelican FontAwesome allows you to embed FontAwesome icons in your RST posts and pages.

$ pip install pelican-fontawesome

PLUGINS = [
    # ...
    'pelican_fontawesome',
    # ...
]

<link href="//netdna.bootstrapcdn.com/font-awesome/4.1.0/css/font-awesome.min.css" rel="stylesheet">

:fa:`fa-github`

<span class="fa fa-github"></span>

:fa:`fa-github fa-4x`

If you’re an Python developer that uses Ubuntu or even Debian, you have probably heard of Felix Krull’s deadsnakes PPA.

I find myself using it a lot and since I tend to destroy my environments quite frequently, I thought I’d save myself some time and write a simple shell script to install the PPA, the versions of Python I frequently use and, after messing up a machine — ignore the existing version of python.

Gist removed. Sorry.

Some time back in April 2013 I was bored and looking for a new project to keep my attention, if only for a short period of time.

My colleague @codeinthehole had an idea but no time to implement it, this idea was to have shields like those of travis-ci (shown below) but displaying package download counts.

Over the last week I’ve been doing a huge amount of refactoring of Blackhole as well as writing dozens of additional tests. To make Blackhole more testable I needed to make a big change to how the program is launched and controlled.

scripts=[
    'blackhole/bin/blackhole',
],

Pelican is a Python-powered static blog generator that processes ReStructuredText and Markdown articles and pages and converts them to HTML. I use Pelican to power this blog.

There is a YouTube RST directive built in to Pelican core but it really shouldn’t exist there.

I submitted a pull request for Pelican core to enable Vimeo videos in articles but that request was declined because it didn’t belong in the core. So I decided I would write it as a plugin instead and while I was doing it, also wrote a plugin for YouTube so that it could be removed from the core.

There is a decent amount of detail in the Pelican documentation on how to write plugins, I’m not going to cover the whole process but I thought I would cover a little of what I did.

Blackhole has always been able to handle unencrypted SMTP and for a long time it’s been able to handle encrypted SMTP via TLSv1.

One thing Blackhole hasn’t been able to do until the 1.7.0 release is handle STARTTLS.

In the past the STARTTLS command would cause Blackhole to return the standard 250 OK response but would continue to operate on unencrypted SMTP.

I wanted to fix this and do it properly, but this meant learning how to do so with Tornado, which itself proved to be tricky. I ended up deciding to go to my local coding spot - the pub and hash it out.

def connection_stream(connection):
    """
    Detect which socket the connection …

As part of my effort to make Blackhole as useful and usable as possible, I needed to be able to support SSL/TLS enabled connections.

Tornado itself has two built-in IOStreams that help us do the job; the first is the standard IOStream and the second is the SSLIOStream.

With this in mind we simply need to spawn two sockets, by default these listen on port 25 for standard SMTP and port 465 for SSL/TLS encrypted SMTP. With these two sockets bound we’re then very simply able to listen for incoming connections on either socket and use socket.socket.getsockname() to figure out if the connection is to the encrypted or unencrypted socket.

def connection_stream(connection):
    """
    Detect which socket the connection is being made on,
    create and iostream for the connection, wrapping it
    in SSL if connected over the SSL socket.

    The parameter 'connection' is an instance …

I have been a frequent audience member of DJUGL for a few years now and spend most of the time asking questions, playing devils advocate and generally being my annoying self.

I have repeatedly said I would do a talk but never got round to it until Jon basically forced me to get round to it.

My talk was on blackhole/blackhole.io and covered several topics including PyPy, SimpleMTA and moved on to talking about spamming and starting work on my honey pot suite called Nectar.

You can find the slides on Speaker Deck, sadly I ran out of time when creating them and although I was promised time to finish them at work, I got busy. So I replaced content with “Taylor Replacements(tm)”.

The event, attendees and other speakers are listen on the Lanyrd event page.

Several people took photos of the event, I don’t remember …

These videos are pretty old now and I had forgotten about them until a colleague tweeted about a Twitter code_swarm visualization which made me remember.

So here they are, best viewed in full screen mode.