Recently I found that one of the servers I look after that runs a high profile site was generating semi-high load at traffic peaks. You could generally say that this would be understandable but the server was shooting up to a load of around 10 for a few seconds and with that load jump I was able to graph an increase of Apache processes on top of it. Again though, this would generally be considered normal, but knowing how well the server performs and having nginx sitting on top handling all static content I knew something wasn’t quite right.

Looking through the logs I found quite a lot of requests from a badly written spider which was generating a lot of server load when it hit the server, but after IP banning the culprit I also found several instances of Apache waking it’s child processes.

127.0.0 …

I’ll assume you already have Nagios installed and configured and have an understanding of actually configuring and using Nagios.

Remote server — the server to be monitored

First we’ll install the needed plugins and daemon on the remote server.

sudo apt-get install nagios-plugins nagios-nrpe-server

Once installed, open up /etc/nagios/nrpe_local.cfg

And place the following in it

allowed_hosts=NAGIOS.SERVER.IP,127.0.0.1

command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_all_disks]=/usr/lib/nagios/plugins/check_disk -w 20 -c 10
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 150 -c 200
command[check_swap]=/usr/lib/nagios/plugins/check_swap -w 20 -c 10

Save and exit.

Commands need to explicitly be enabled on the …

This is a very quick blog to show you how to show a users IP address in your Apache access logs when the site in question is being reverse proxied to Apache through nginx.

You need the rpaf module for Apache, on Debian and Ubuntu this is simple to install

sudo apt-get install libapache2-mod-rpaf
sudo a2enmod rpaf
sudo /etc/init.d/apache2 restart

This set of commands will do the following;

  1. Update apt package list
  2. Install libapache2-mod-rpaf
  3. Enable mod-rpaf
  4. Gracefully restart Apache (doesn’t kill connections)

Once installed you simple need to be sure to pass the correct headers through, so open up one of your nginx site configuration files and add the following within the server definition.

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

So you should have something that looks like this, but without the “… snip …”

server {
    # ...snip...
    location / {
        # ...snip...
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # ...snip...
    }
}

Today I ran in to something I’d never seen before when configuring nginx.

I ran the nginx config test, as I usually do before I restart it.

nginx -t

But, the response I got was interesting

2010/03/18 21:16:09 [emerg] 12299#0: could not build the server_names_hash, you should increase server_names_hash_bucket_size: 32
2010/03/18 21:16:09 [emerg] 12299#0: the configuration file /etc/nginx/nginx.conf test failed

I found that one of the domain names I was using was over 32 characters in length, nginx’s default max length.

Thankfully the fix was simple.

http {
    # ...snip...
    server_names_hash_bucket_size 64;
    # ...snip...
}

This is a really great simple way to find files on the filesystem that are over 200k in size.

find /path/to/directory/ -type f -size +200k -exec ls -lh {} ; | awk '{ print $NF ": " $5 }'

You can use the output of this to either store in a file, or pipe to wc for a count of lines

find /path/to/directory/ -type f -size +200k -exec ls -lh {} ; | awk '{ print $NF ": " $5 }' | wc -l

You can also use egrep before wc to look for specific filetypes

find /path/to/directory/ -type f -size +200k -exec ls -lh {} ; | awk '{ print $NF ": " $5 }' | egrep '(jpg|bmp|gif|tiff|jpeg)' | wc -l

Some times as an administrator you will be given a certificate from a third party that will be in the DER format, which cannot be loaded in to Apache.

Converting it is a simple process:

openssl x509 -in certificate.crt -inform DER -out certificate.pem -outform PEM

To my surprise I have found that there are still people out there who use “more”, this has shocked me.

So this is a very, very short blog post to tell those who visit that less is more and more is less.

What?

less is a command that comes as standard in almost all Linux distros now, and unlike more it actually has the ability to do backwards and forwards scrolling with Page Up, Page Down, arrow keys and spacebar. It’s a fantastic little command!

less FILE

Very simple to use and an all round great tool. The best thing about less is it doesn’t need to read the whole file in one go, it reads in chunks. Opening a 100MB log file is simple with less!

Useful options

-gHighlights just the current match of any searched string,
-ICase-insensitive searches,
-MShow a more detailed prompt …