This article is a continuation of retrying dynamically configured upstreams that gives an example of how you can configure OpenResty to update your upstream backend servers dynamically with DNS.

Breaking down init_worker_by_lua_block

init_worker_by_lua_block can be used to make an nginx worker do some fun stuff. In this instance we’re going to use it in conjunction with ngx.timer.every and the resty.dns.resolver.

Here is the full example of my init_worker_by_lua_block.

init_worker_by_lua_block {
    _backend_servers = {}

    local function update_dns()
        -- Set up the resolver
        local resolver = require "resty.dns.resolver"
        local r, err = resolver:new{
            nameservers = {"1.1.1.1", {"1.0.0.1", 53} },  -- Cloudflare
            retrans = 5,  -- 5 retransmissions on receive timeout
            timeout = 1000,  -- 1 sec
        }
        if not r then
            ngx.log(ngx.ERR, "failed to instantiate resolver: ", err)
            return
        end

        -- Pull DNS records
        -- Use a hardcoded domain to make this example easier
        local answers, err, tries = r:query("kura.gg …

Preamble

OpenResty is a modified version of nginx with LuaJIT compiled in and many nginx options that can be controlled or modified via Lua. It is very commonly used in content delivery networks for it’s configurability.

As such, we use OpenResty and one of the features we use is the ability to dynamically modify upstream backends. To achieve this we use some logic within OpenResty to update upstreams based on DNS records. This means we can pull upstreams in and out of service via DNS records and have OpenResty update it’s upstream proxy passing configuration without needing to push configs out to hundreds of servers and reload daemons.

The logic behind how we update the upstream backends is beyond the scope of this post, so let’s just say we have a table of upstream servers.

local upstream_servers = {
    "10.0.0.1:443",
    "10.0.0.2:443 …

Ian Murdock

Image by Stephen Shankland

Today news arrived that broke my heart, Ian Murdock has passed away. I never met Ian but he had an enormous impact on my life, through his work.

Debian has been my operating system of choice for many years now, it runs on all of my servers and my home computers. Ian’s vision for the operating system is why I started using it and use it to this day. It’s why I’ve continued to work on the operating system and create packages of all of my work for it.

Ian is, without a doubt, one of the biggest influences on my career and on my life.

Farewell sir, you will be missed.

There is a variety of information out there about being a Tor exit node operator. Articles like this one make the thought of running a Tor exit as scary prospect, it’s understandable, some countries have laws that make running an exit scary too.

I run a variety of relays in various countries in this crazy World and thought I’d share my experiences.

Choosing a hosting partner

I personally choose to use a third party hosting provider for my relays, rather than using colocation. I just find this is easier and I don’t have to think about the hardware much at all.

Finding a provider can be a pain, there is a decent list on the Tor wiki. I use some of the providers on this list but I’ve also found it can be a really good idea to just contact a provider and talk to them …

pypip.in

pypip.in is a website I have written about before. I decided I would look at the shields and see which ones were the most requested so far this year.

Due to the volumn of requests, I only keep 90 days of logs from nginx and no logs from Varnish, pypipin or the local version of buckler. nginx sits in front of Varnish so, even if Varnish responds with a cached version of the shield, a log line is still written to say it was requested.

Top 20

  1. requests [downloads]
  2. Pillow [version]
  3. Pillow [downloads]
  4. Theano [version]
  5. Theano [downloads]
  6. fake-factory [downloads]
  7. livestreamer [downloads]
  8. django-cms [downloads]
  9. django-cms [version]
  10. speedtest-cli [version]
  11. speedtest-cli [downloads]
  12. boto [downloads]
  13. thumbor [downloads]
  14. thumbor [version]
  15. pip [version]
  16. tweepy [version]
  17. tweepy [downloads]
  18. django-allauth [downloads]
  19. pymssql [downloads]
  20. pymssql [version]

The first thing that struck me is that I actually hadn’t heard of or used quite a lot of …

Today I can happily announce that the Google Analytics tracking code has been removed from this website.

Goodybye Google Analytics & hello Open Web Analytics

I’ve been planning on doing it from a while because I do not like Google Analytics, I don’t like being tracked and I actively stopped trying to use Google services for my own reasons.

The company I work for uses Piwik for some of our clients, I am not a fan of Piwik or how it works and does things. I did some research and found some service providers but their free options were limited or I felt they weren’t a good fit, eventually I stumbled upon Open Web Analytics and decided that it not only suited my purposes, but it meant servers I control would hold the analytical data, rather than some third party.

Hello DuckDuckGo

After launching the new version of …

24 hours of SSH attacks against a single server, visualised on a world map using Python.

When a country stays lit up for more than 1 tick of the clock in the left hand corner it means that multiple attacks are happening from different IP addresses. An attacker is banned after;

  • 1 failed root login,
  • 3 failed user logins (including invalid users) and
  • 3 failed system logins.