— 3 min read

Mew, the Pokémon

Content-Security-Policy (CSP) is an HTTP header returned by servers that gives the client some information on where resources can be loaded. For example setting script-src 'self' in the CSP tells the client that it should only load script resources from the current origin. 'self' for this blog post would mean https://kura.gg.

To make loading those resources more secure you can use hashes or nonces within the CSP that the client can verify. Below I will show how I inject CSP nonces using Cloudflare Workers in to responses from my origin.

Security considerations

Think of this as an academic exercise rather than something you should do.

This article will only explain how I inject the nonces in to responses from the origin. This isn’t really secure given the method I use to do this is pretty dumb — it just adds a generated nonce to <script> and <link rel …

 — 10 min read

Eevee, the Pokémon

I was a little bored tonight and had a thought; “can I find a way to add a random Pokédex and it’s National Pokédex number as a header to requests on my website?”

The answer is - quite simply - “yes”.

Cloudflare Workers

This website runs on Cloudflare’s content delivery network and as such I have access to Cloudflare Workers, allowing me to run arbitrary code at the edge for incoming requests.

Setting up a worker that picks a random Pokemon and injects it as a header is very simple, add a worker, write the code and add some routing to it.

Try it yourself

curl -sI https://kura.gg/ | grep "x-pokemon"
x-pokemon: Eevee (#133)

The code

let pokemon = [
    "Bulbasaur (#1)", "Ivysaur (#2)", "Venusaur (#3)", "Charmander (#4)", "Charmeleon (#5)",
    "Charizard (#6)", "Squirtle (#7)", "Wartortle (#8)", "Blastoise (#9)", "Caterpie (#10)",
    "Metapod (#11)", "Butterfree (#12)", "Weedle (#13)", "Kakuna (#14)", "Beedrill (#15)",
    "Pidgey …
 — < 1 min read

Gmail provided by the paid Google Workspace service (formerly known as G Suite and Google Apps) has unofficial DNSSEC-signed MX records available for use. The officially supported ones that you’re told to configure do not offer DNSSEC signing.

These MX records have both IPv4 and IPv6 addresses, although the records are not officially supported or documented and may be unreliable or removed at any point. (I’ve been using them for a while now and they seem perfectly fine to me but use at your own risk.)

mx1.smtp.goog
mx2.smtp.goog
mx3.smtp.goog
mx4.smtp.goog

The table below has the MX record and the A and AAAA record values.

mx1.smtp.goog
    216.239.32.151
    2001:4860:4802:32::97

mx2.smtp.goog
    216.239.34.151
    2001:4860:4802:34::97

mx3.smtp.goog
    216.239.36.151
    2001:4860:4802:36 …
 — 10 min read

This article is a continuation of retrying dynamically configured upstreams that gives an example of how you can configure OpenResty to update your upstream backend servers dynamically with DNS.

Breaking down init_worker_by_lua_block

init_worker_by_lua_block can be used to make an nginx worker do some fun stuff. In this instance we’re going to use it in conjunction with ngx.timer.every and the resty.dns.resolver.

Here is the full example of my init_worker_by_lua_block.

init_worker_by_lua_block {
    _backend_servers = {}

    local function update_dns()
        -- Set up the resolver
        local resolver = require "resty.dns.resolver"
        local r, err = resolver:new{
            nameservers = {"1.1.1.1", {"1.0.0.1", 53} },  -- Cloudflare
            retrans = 5,  -- 5 retransmissions on receive timeout
            timeout = 1000,  -- 1 sec
        }
        if not r then
            ngx.log(ngx.ERR, "failed to instantiate resolver: ", err)
            return
        end

        -- Pull DNS records
        -- Use a hardcoded domain to make this example easier
        local answers, err, tries = r:query("kura.gg …
 — 3 min read

Preamble

OpenResty is a modified version of nginx with LuaJIT compiled in and many nginx options that can be controlled or modified via Lua. It is very commonly used in content delivery networks for it’s configurability.

As such, we use OpenResty and one of the features we use is the ability to dynamically modify upstream backends. To achieve this we use some logic within OpenResty to update upstreams based on DNS records. This means we can pull upstreams in and out of service via DNS records and have OpenResty update it’s upstream proxy passing configuration without needing to push configs out to hundreds of servers and reload daemons.

The logic behind how we update the upstream backends is beyond the scope of this post, so let’s just say we have a table of upstream servers.

local upstream_servers = {
    "10.0.0.1:443",
    "10.0.0.2:443 …
 — 3 min read

Eevee the Pokémon

Introduction

Eevee is a theme for Pelican, based on Google’s Material Design specification that I released in June 2016.

Schema.org is a collaborative effort, including private companies like Google and the open source community to promoter adding additional structure to data that indentifies what parts of HTML are being used for, for example it can help a search engine understand what content is more important to index or, it can help a user with a screen reader get to the important content more easily. These structures are more commonly known as microformats or microdata.

Structure is a beautiful thing

HTML — by it’s very design is structured — the difference is that the content defines or modifies that structure.

By using microformat data structures from schema.org you can define what elements are more prominent or important through a large set of additional vocabulary, improving the experience for everyone …

 — 1 min read

Eevee the Pokémon

This week I pushed the 0.0.3 update for my Pelican theme Eevee containing new and improved styling for share buttons and adding the ability to use Muut instead of Disqus for comments.

Share button changes

The original styling of the share buttons wasn’t particularly pretty.

Eevee's original share button styling

They badly needed a make over, and so they were restyled to be more prominent, look less out of place and randomly pasted in to the page and to have a colour scheme that matches the social media site they share to.

Eevee's new 0.0.3 share button styling

These buttons appear on articles and pages, but when viewing an article with DISQUS_SITENAME or MUUT_SITENAME enabled a fourth button is shown that links to the comments section of the current article.

Eevee's new 0.0.3 article share button styling

Comments powered by Muut

To enable commenting with Muut, simply modify pelicanconf.py and set the Muut site name.

MUUT_SITENAME = 'somethinghere'

This will automatically cause Eevee to enable …

 — < 1 min read

Lightbox

Introduction

With the release of my Eevee theme for Pelican, I realised displaying a thumbnail image of the theme that linked to a larger image wasn’t the most appealing design choice. I prefer to leave Javascript out of the equation where possible, being one of those weird people that have it disabled by default.

As such I sought out a way to create a pure CSS equivalent of a Lightbox and turn it in to an RST directive to plug directly in to Pelican.

 — 1 min read

Eevee the Pokémon

Introduction

Eevee is a theme for Pelican, based on Google’s Material Design specification that I released in June 2016.

Eevee allows configuring menu links in the header and footer of a template — including social links.

Eevee also includes Font Awesome by default, at time of release and writing of this article it provides Font Awesome version 4.6.3.

Because the links are totally customisable, it means you can inject HTML directly in to a link name and — because Font Awesome is included by default — you can inject Font Awesome icons in to link names using HTML.

Social icons in Eevee menu
 — 7 min read

A brief history of a tiny part of the Internet.

Blackhole 1 — Blackhole as it was originally known — was written on Python 2.7, briefly supporting Python 2.6 for a time and also supporting early version of Python 3, PyPy 2 and PyPy 3. Built on top of Tornado, it was asynchronous in a fashion and — quite simply — worked.

The original prototype that became Blackhole was SimpleMTA — a prototype that was created quickly, to serve a very simple testing purpose that I had for it.

As I needed SimpleMTA to do more, I wrote Blackhole to accomplish that task. I’d been using Tornado a bit and wanted to experiment with it more. Building on top of Tornado created some oddities in how the program was designed and that always irked me.

Between the time of the last 1.8.X and the 2.0 release, I experimented with …

 — < 1 min read

Ian Murdock

Image by Stephen Shankland

Today news arrived that broke my heart, Ian Murdock has passed away. I never met Ian but he had an enormous impact on my life, through his work.

Debian has been my operating system of choice for many years now, it runs on all of my servers and my home computers. Ian’s vision for the operating system is why I started using it and use it to this day. It’s why I’ve continued to work on the operating system and create packages of all of my work for it.

Ian is, without a doubt, one of the biggest influences on my career and on my life.

Farewell sir, you will be missed.

 — 2 min read

SSHFP records are a defense against people blindly typing ‘yes’ when asked if they want to continue connecting to an SSH host who’s authenticity is unknown.

$ ssh some.host.tld
The authenticity of host 'some.host.tld (123.456.789.10)' can't be established.
ED25519 key fingerprint is 69:76:51:39:a4:c6:de:15:7c:50:4b:4a:a7:98:40:5e.
Are you sure you want to continue connecting (yes/no)?

This prompt is likely to be extremely familiar to you and, most people seem to just type ‘yes’ to move on with their lives, which defeats the whole purpose of this prompt.

If you use DNSSEC you can bypass this prompt entirely by publishing your server’s key fingerprints via DNS and having SSH authenticate them for you.

Generating your SSHFP record

You can get SSH to generate the DNS records for you, log in …

 — 1 min read

I currently use name.com as my registrar and I use Rage4 because Rage4 are awesome, they also support TLSA and SSHFP records and of course, DNSSEC.

I’m writing this up because I found getting DNSSEC from Rage4 to work with name.com as my registrar was a pain and the name.com support were not very helpful, linking me to a support article that I’d already read and did not help at all.

Rage4

I’m going to assume you’ve already got your records in Rage4, if not, the interface is really easy so you’ll figure it out.

Within the management section for your domain’s zone, there is a menu bar of icons, the icon pictured below enabled DNSSEC.

Enabled DNSSEC

Clicking this will turn on DNSSEC. You will then have a new icon that will allow you to display your DNSSEC information.

Display DNSSEC info

Clicking this icon …

 — 3 min read

There is a variety of information out there about being a Tor exit node operator. Articles like this one make the thought of running a Tor exit as scary prospect, it’s understandable, some countries have laws that make running an exit scary too.

I run a variety of relays in various countries in this crazy World and thought I’d share my experiences.

Choosing a hosting partner

I personally choose to use a third party hosting provider for my relays, rather than using colocation. I just find this is easier and I don’t have to think about the hardware much at all.

Finding a provider can be a pain, there is a decent list on the Tor wiki. I use some of the providers on this list but I’ve also found it can be a really good idea to just contact a provider and talk to them …

 — 3 min read

Public Key Pinning is a security feature that tells a web browser to associate a public cryptographic key with a server or servers. When a web browser visits a website for the first time, it will read the HPKP header and store the hashes for the certificates that are provided. Each time the browser then revisits that website, the hash from the provided public key is compared against the stored keys, if the hashes do not match, the web browser should display a warning.

The HPKP header adds protection against man-in-the-middle (MITM) attacks but, if incorrectly configured can make your website display a TLS error for a long period of time.

Here’s a look at what this website publishes as it’s HKPK header.

Public-Key-Pins: pin-sha256="cYf9T3Il8DaCnaMaM0LatIAru1vqmcu2JSwS7uvyEB0=";
                 pin-sha256="u2q8QZ8Hjp3o/efZjsch9NKjnZmrISJQjwoi/rmsKLU=";
                 max-age=15768000; includeSubDomains

To explain it, the first pin-sha265 key is the hash of the public key that …

 — 3 min read

This is really a follow up article to one I wrote earlier this year but is really applicable to any similar set-up, with some modifications. The only configuration similarity this requires is that mail for all users is stored on the filesystem in the same place, rather than to separate locations i.e. each user having ~/.Maildir.

EncFS

sudo apt-get install encfs

Once installed, you’ll need to make a directory for encrypted and decrypted mail to live.

sudo mkdir /var/mail/encrypted /var/mail/decrypted

You’ll need to set up permissions so your mail user can access the fuse device and the new directories.

For me, this user and group are called vmail but yours may be different.

sudo chgrp mail /var/mail/decrypted
sudo g+rw /var/mail/decrypted
sudo usermod -a -g fuse vmail
sudo chgrp fuse /dev/fuse
sudo chmod g+rw /dev/fuse

Next …

 — < 1 min read

I think most of us have been in a position where we really shouldn’t continue communicating with someone or contact that person when drunk… You know what I mean, ex relationships etc (it happens.)

With Postfix you can block yourself from emailing that person again, which is quite useful.

In /etc/postfix/main.cf add make the start of your smtpd_recipient_restrictions look like below.

smtpd_recipient_restrictions =
    check_recipient_access hash:/etc/postfix/recipient_access,

Create a new file /etc/postfix/recipient_access and add the email address you wish to block, the word REJECT in capitals and optionally; a reason. Example below.

test@example.com REJECT Don't be silly... You're probably drunk.

For every address you wish to block yourself from emailing, simply add them on a new line.

You can see the email is blocked from being sent in /var/log/mail.log.

NOQUEUE: reject: RCPT from 123.123.123.123: 554 5 …
 — 9 min read

This mail platform does use a fair amount of memory, the memory usage is ClamAV and Solr, the latter being used for IMAP SEARCH. I personally use 2 GB.

I’ll warn you all now, this is a long article.

SSL

sudo openssl genrsa -out /etc/ssl/private/mail.key 4096
sudo openssl req -new -key /etc/ssl/private/mail.key -out /tmp/mail.csr
sudo openssl x509 -req -days 365 -in /tmp/mail.csr -signkey /etc/ssl/private/mail.key -out /etc/ssl/certs/mail.crt

MySQL

sudo apt-get install mysql-server

You’ll be prompted several times for a password for MySQL during the installation, just come up with something nice and secure.

The first thing to set-up will be the MySQL database and schema.

mysql -u root -p

Next up, create the database.

CREATE DATABASE mailserver CHARACTER SET utf8 COLLATE utf8_general_ci;

And grant some privileges, you’ll need …

 — < 1 min read

Mirror

With the DDoS attacks on the torproject.org website over the mid to end of December, I decided it would be prudent to join the relatively small list of mirrors.

Access

+——- —-+—————————+ | Website | Dist / Downloads | +=========+==================+ | HTTP | HTTP DIST | +————-+—————————+ | HTTPS | HTTPS DIST | +————-+—————————+ | FTP | FTP DIST | +————-+—————————+ | RSYNC | RSYNC DIST | +————-+—————————+

Note: This mirror is now dead and links have been removed.

 — < 1 min read

Exiting relays

Both SpunkWeasel (865A408E2B1EA3E18C9A12E80A8D458F9C985C16) and AnorexicSquirrel (B8E6FFEB6F91FA3D26BC572836FB0ABBD142DC87) have been given additional horsepower in terms of CPU and memory and both have been allowed to exit. Additionally, both are now capable of IPv6 connectivity as guards, relays and exits.

New relays

A new exit relay has been put online called VivaciousAlpaca (24B1783665A9B0A4BF640A1CD02F685C0CA098ED). It has the same CPU and memory as SpunkWeasel and AnorexicSquirrel and also has full IPv6 capabilities.

A full list of my public Tor relays can be found on the tor page.

 — 3 min read

Privacy is key

I am a big fan of keeping my browsing habits and other personal information private. As such, I use a VPN service provided by proxy.sh (affiliate link), I also use their proxies, Tor and various other proxies, usually from online lists, should I feel the need.

Anonymous nodes

I’ve run some Tor nodes for quite a while now, two Exits and five relays to be precise. They are all listed as being part of the same family and have never had any reference to me being their operator.

Sadly, these seven nodes will always remain a secret due to the hassle that inherently comes with running Tor Exit nodes. These issues include some mean emails and IP addresses and CIDR blocks being blacklisted by services like Netflix.

New nodes!

The good news is, I have recently launched three more nodes! These nodes belong to a …

 — 1 min read

Batfish is a Python client and API wrapper for the Digital Ocean V2 API. It can be used as a library module in your own Python code but also provides a CLI interface and a shell-like command interpreter.

Batfish is still under development and is considered in the Alpha stage. It is not yet available via PyPI but can be tried out using the code available on GitHub.

There is a small amount of documentation available on Read The Docs and tests are still being written to get as much coverage as possible and eaked out all of the bugs. You can find the latest test status on Travis CI.

Module interface

>>> from batfish import Client
>>> client = Client()
>>> client.authorize("abcde12345")
>>> client.droplets
[<Droplet ego.kura.gg>, <Droplet fax.kura.gg>, <Droplet jet.kura.gg>, <Droplet ski.kura.gg>]
>>> client.droplet_reboot(1234)

CLI interface

$ batfish authorize
abcde12345
$ batfish droplets
ego …
 — < 1 min read

Yarg is a PyPI client, it was written for pypip.in and can search packages as well as read the RSS feeds from PyPI for new packages and new package version releases.

Search interface

>>> import yarg
>>> package = yarg.get("yarg")
>>> package.name
u'yarg'
>>> package.author
Author(name=u'Kura', email=u'kura@kura.gg')

Newest packages interface

>>> import yarg
>>> packages = yarg.newest_packages()
>>> packages
[<Package yarg>, <Package gray>, <Package ragy>]
>>> packages[0].name
u'yarg'
>>> packages.url
u'https://pypi.python.org/pypi/yarg

Updated packages interface

>>> import yarg
>>> packages = yarg.latest_updated_packages()
>>> packages
[<Package yarg>, <Package gray>, <Package ragy>]
>>> packages[0].name
u'yarg'
>>> packages[0].version
u'0.1.2'
>>> packages[0].url
u'https://pypi.python.org/pypi/yarg/0.1.2

Documentation

Full documentation is at <https://yarg.readthedocs.org>.

 — 3 min read

As you might expect, pypip.in employes a fair amount of caching in the backend to control load on the imaging API and servers.

For a long time, this cache was entirely managed by Varnish and was doing a fantastic job. Varnish has a hit:miss ratio of 10:1, for every 10 hits we get 1 miss. This is a fairly decent ratio when you consider where these images are displayed, how often they are viewed and that Varnish only caches the images for an hour.

The impact on PyPI

You will firstly need to understand how pypip.in used to work to understand the changes that were made and why they were made.

Let’s set up the request first - a request for a shield is made and it is not present in the Varnish cache.

Request received in API layer
              |
              v
    API layer queries PyPI
              |
              v
   PyPI …