Quick and dirty script to backup EC2 sites to S3 Storage

May 8th, 2012 · · Uncategorized No comments - Tags: , ,

Having now set up several sites on an Amazon EC2 Ubuntu instance, I wanted a way to ensure regular backups of the sites were sent somewhere else for disaster recovery. It’s a variety of different sites, three WordPress blogs, one Webtrees genealogy site and a ZenPhoto library. However they are all of the same typical architecture… php apps with a mysql database backing it. Zen photo also has the big filesystem dump of images arranged into albums (directories).

Fortunately Amazon makes it relatively cheap and cost effective! S3 storage is both cheap and has redundancy that should make even the most paranoid people sleep well at night! And the s3cmd package makes it a simple task to script syncronisation of an ec2 filesystem (or part of it) to S3 storage which is independent of the EC2 instance.

So assuming you have the s3cmd package installed and configured, he is a very quick and very dirty first draft of a script I wrote to backup the sites to S3. I also have an AMI made of my current working ubuntu install, so disaster recovery should be really simple…

  1. Create new instance from the AMI
  2. Restore the databases from these backups
  3. Restore the webdocs from these backups
  4. Switch elastic IP to new instance

Anyway, as promised here is the very rough script, I am no bash master so I am sure you may be able to make many suggestions on how to improve it!

#!/bin/bash

## LOG FILE
LOGFILE=/home/ubuntu/s3_backup.log
exec > $LOGFILE 2>&1

## ROOT DIR FOR BACKUPS
BAKROOT=/mnt/public_html/backups/

## CLEAN UP THE OLD BACKUPS

cd $BAKROOT
echo “You are In Backup Directory” $BAKROOT
find -mindepth 2 -delete
echo “Old files deleted”
DSTAMP=$(date +%Y-%m-%d-%H-%M)

## CREATE THE DB BACKUPS

mysqldump -h localhost -u USER1 -p’SOMEPASS’ DB1 > $BAKROOT/SITE1.COM/SITE1.COM_$DSTAMP.sql
mysqldump -h localhost -u USER2 -p’SOMEPASS’ DB2 > $BAKROOT/SITE2.COM/SITE2.COM_$DSTAMP.sql
mysqldump -h localhost -u USER3 -p’SOMEPASS’ DB3 > $BAKROOT/SITE3.COM/SITE3.COM_$DSTAMP.sql
mysqldump -h localhost -u USER4 -p’SOMEPASS’ DB4 > $BAKROOT/SITE4.COM/SITE4.COM_$DSTAMP.sql
mysqldump -h localhost -u USER5 -p’SOMEPASS’ DB5 > $BAKROOT/SITE5.COM/SITE5.COM_$DSTAMP.sql

echo “Your Database Backups Successfully Completed”

## CREATE THE WEBDOCS BACKUP
tar -cvf $BAKROOT/SITE1.COM/SITE1.COM-$DSTAMP.tar /mnt/public_html/SITE1.COM
tar -cvf $BAKROOT/SITE2.COM/SITE2.COM-$DSTAMP.tar /mnt/public_html/SITE2.COM
tar -cvf $BAKROOT/SITE3.COM/SITE3.COM-$DSTAMP.tar /mnt/public_html/SITE3.COM
tar -cvf $BAKROOT/SITE4.COM/SITE4.COM-$DSTAMP.tar /mnt/public_html/SITE4.COM
tar -cvf $BAKROOT/SITE5.COM/SITE5.COM-$DSTAMP.tar /mnt/public_html/SITE5.COM

echo “Your WebDocs Backups Successfully Completed”

## Send backups to S3

s3cmd put –recursive $BAKROOT* s3://SITE_BACKUPS/
echo “Files sent to S3”

## Sync the albums to S3 Storage
echo “Photo sync starting at ”
echo $DATE
s3cmd sync /mnt/photos/ s3://SITE_PHOTOS/
echo “Photo sync finished at ”
echo $DATE

Building a LAMP Server on Amazon EC2 with S3 Storage & EBS volumes mounted!

April 24th, 2012 · · Uncategorized No comments - Tags: , , , , , , , ,
1. Launch an EC2 Instance
I chose micro, ubuntu 11.04 32bit
2. Install Basic Lamp
sudo apt-get update
sudo apt-get install lamp-server^
sudo apt-get install php5 php5-gd php5-mysql php5-curl php5-cli php5-cgi php5-dev
Note at this point some builds from AMI’s have no root password set for mysql, this can be set by the following
shell> mysql -u root
mysql> SET PASSWORD FOR ‘root’@'localhost’ = PASSWORD(‘newpwd’);
After checking the root password is set, continue to install myphpadmin
sudo apt-get install phpmyadmin
When prompted during the phpmyadmin config I chose apache and yes.
3. Other Stuff
sudo apt-get install proftpd
I chose standalone and will update later with other config
sudo apt-get install memcached php-pear
sudo pecl install memcache
sudo vi /etc/php5/apache2/php.ini
extension=memcache.so
sudo apt-get install imagemagick
sudo apt-get install zip
sudo /etc/init.d/apache2 restart
4. Install S3fs
cd /home/ubuntu/ 
sudo apt-get install build-essential libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support
wget http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz
tar xvzf s3fs-1.61.tar.gz
cd s3fs-1.61.tar.gz
./configure –prefix=/usr
make
sudo make install
sudo vi /etc/passwd-s3fs
and add accesskey:secretkey
5. Create Storage
optional if new ebs -> sudo mkfs -t ext4 /dev/xvdf
sudo mkdir /mnt/public_html/
sudo mkdir /mnt/s3/
6. Mount Storage
sudo vi /etc/fstab
s3fs#mybucket /mnt/s3 fuse allow_other,url=https://s3.amazonaws.com 0 0
/dev/xvdf /mnt/public_html ext4 noatime 0 0
Reboot
cd /var/www
sudo ln -s /mnt/public_html public_hmtl
sudo ln -s /mnt/s3 s3
sudo chmod -R 775 /var/www/
sudo chown -R www-data:www-data /var/www/
Good to go!

Obiligatory Welcome Post

April 24th, 2012 · · Uncategorized No comments - Tags: , ,

I guess it’s tradition but they are boring. So I’ll just say permalinks aren’t working without index.php before the link… strange

UA-31158184-1