Contents

I have several servers powering syslog including it’s Raspberry Pi mirror, load balancer and email servers. All of my servers are hosted using Linode in their London data centre and have Linode’s back-up system doing both daily and weekly snapshots.

For the app and database servers I do server-side backups storing each website and it’s database in it’s own folder within /backup in case I require a quick back-up to fix something, rather than the server has died.

This is all well and good but I like having an off-site backup too and for that I use S3

About S3

Amazon’s S3 is pretty cheap and very easy to use. Because only data is going in you don’t pay a transfer fee and the cost of storage is very affordable, you can see a pricing list here.

To do the backup I use a daily cron job which then uploads the data to S3 using s3cmd.

Installation

Download the S3 tools package list in to apt

sudo wget -O- -q http://s3tools.org/repo/deb-all/stable/s3tools.key | sudo apt-key add -
sudo wget http://s3tools.org/repo/deb-all/stable/s3tools.list -O /etc/apt/sources.list.d/s3tools.list

Update your package list and install s3cmd

sudo apt-get update && apt-get install s3cmd

Configuration

You’ll need to configure the tool to work with your AWS account, so run

sudo s3cmd --configure

When prompted, fill in your access and secret key which you can find on the Amazone website.

When asked to provide an encryption password, I choose yes but you can say no.

When asking if you want to use HTTPS, I choose yes but again, you can say no, it really depends on how secure you want the data transfer.

I would suggest using an encryption password and enabling HTTPS.

Using s3cmd

Now that s3cmd is installed and configured you can use it.

You can create a bucket using the s3cmd command below, but as far as I know you can’t select a location so I create my buckets manually on the web interface.

s3cmd mb s3://your-bucket-name

Once done you can see a list of available buckets with

s3cmd ls

As shown below

s3cmd ls

2012-02-29 20:28 s3://kura-linode-test

Now that this is done we can put some data in there, create a test file

echo "this is a test" > test.file

And put it in S3

s3cmd put test.file s3://your-bucket-name/

You can see it using

s3cmd ls s3://your-bucket-name

Download it with

s3cmd get s3://your-bucket-name/test.file

And delete it with

s3cmd del s3://your-bucket-name/test.file

Once satisfied with this you can create a shell script to automate some backups for you, I’ll provide a simple one below that uploads my home directory.

Example

#!/bin/sh
s3cmd sync --recursive --skip-existing /home/kura
s3://kura-linode-test/

Kura

Anarchist. Pessimist. Bipolar. Hacker. Hyperpolyglot. Musician. Ex pro gamer. Cunt. They/Them.

Kura

Join the discussion

View Source