So, you have some really important data, so much so that 99.99% redundancy is not enough for you. One solution to this is to use multiple copies in multiple datacentres. Most enterprise backup will have on-site, an off-site, and an archival copy. What I’m going to show here is how to make 4 different copies of your data, in 4 different datacentres around the world. This will provide a very high redundancy of storage, and greatly reduce the likelihood of data loss. Although it costs a bit more, this kind of solution may be suitable for many small, medium and large businesses. Naturally, depending on the size of the data, and the importance of redundancy. You might not have many files to backup, perhaps a small cd worth.. it will be very inexpensive if you have a small backup to make. However, due to the way that cloud files is billed, copying data to cloud files costs money in bandwidth when writing from a server in London to a cloud files in Sydney, Chicago or Dallas for instance, so it’s very important to consider the impact of bandwidth costs when utilizing an additional 3 cloud files endpoints that are not in the local datacentre region. Which, is essentially what we are doing in this guide.
Setup swiftly
yum install python-devel python-pip -y pip install swiftly eventlet
Create your swiftly environments (setting the name for each file)
==> /root/.swiftly-dfw.conf <== [swiftly] auth_user = myusername auth_key = censored auth_url = https://identity.api.rackspacecloud.com/v2.0 region = dfw ==> /root/.swiftly-iad.conf <== [swiftly] auth_user = myusername auth_key = censored auth_url = https://identity.api.rackspacecloud.com/v2.0 region = iad ==> /root/.swiftly-ord.conf <== [swiftly] auth_user = myusername auth_key = censored auth_url = https://identity.api.rackspacecloud.com/v2.0 region = ord ==> /root/.swiftly-syd.conf <== [swiftly] auth_user = myusername auth_key = censored auth_url = https://identity.api.rackspacecloud.com/v2.0 region = syd
Create your Script
# Adam Bull # Adam Bull, Rackspace UK # May 17, 2016 # This can be sequential or, it can be parallel, not sure which is better yet use & for parallel # This backs up /documents file and puts it in the 'managed_backup' cloud files container at the following 4 datacentres ,DFW, IAD, ORD and SYD swiftly --verbose --conf ~/.swiftly-dfw.conf --concurrency 100 put -i /documents /managed_backup swiftly --verbose --no-snet --conf ~/.swiftly-iad.conf --concurrency 100 put -i /documents /managed_backup swiftly --verbose --no-snet --conf ~/.swiftly-ord.conf --concurrency 100 put -i /documents /managed_backup swiftly --verbose --no-snet --conf ~/.swiftly-syd.conf --concurrency 100 put -i /documents /managed_backup
Because the other 3 endpoints are in different datacentres, we can't use servicenet, so we defined --no-snet option for swiftly as above.
Execute your script
chmod +x multibackup.sh ./multibackup.sh
This obviously is a basic system and script of taking backups, and it is not for production use (yet). This is an alpha project I started today. The cool thing is that it works, and quite nicely. Although it is far from finished yet as a workable script.
Once the script is made, you can simply add it to crontab -e as you would usually. Make sure the user you execute with cron has access to the .conf files in their home directory!