Are you ready for Amazon S3...Maybe?

This is sort of a continuation of my previous post about Amazon S3.

While was looking for a way to use Amazon S3 without doing any coding, I ran into a couple of beta programs. Some of which just didn't seem to work right. I was looking for a program that would copy the files to S3 daily and wouldn't change the name of the files. JungleDisk was one of the first I looked at and it would constantly be doing a get and put while uploading the data along with changing the file name (if you used only JungleDisk you won't realize this). As mentioned in my previous post, S3 backup and S3interface.com were good candidates but they didn't do automated backups. S3interface limits the file size to 10MB and you can only upload from within the web GUI. S3 backup on the other hand with the release of S3 backup beta 12 the automated feature now works and I have been using it successfully for 2 months.

I have S3 backup installed on my "monitoring" server (which monitors all of my web/sql servers) where the external HD is directly connected. So after my backups run I scheduled an upload within S3 backup to upload the changed files. I also installed this program on my desktop. Since I can connect to the same bucket I can see if, what time and which files where uploaded. It works great!

The only problem I had with this program was when I created two jobs. I was trying to use multiple buckets to store the files for the different servers that I backup. When I would run the second job it would put all of the files in the first bucket and overwrite the files from the previous job. I worked around this by creating a single job using one bucket with multiple folders. As a consequence, this is much easier to view all of the files at once. So when reviewing my backup files I can quickly look at all of the files quickly.

Since this has been working so well I have started to think about cancelling my other online off-site storage provider. With Amazon S3, their costs are so cheap that I am paying about $60-70 per month for about 30GB. This includes the daily uploads of the differential files (between 200MB and 3GB) and full backups of about 26GB. My current provider gives me 500GB for $600 per month. Although this is more space the cost savings is nearly 10 fold since with Amazon I only pay for what I use and I don't use all 500GB.

In summary, I have found that S3 backup is the best program (so far) out there to upload files to S3 without having to create your own. Also a great way to take advantage of Amazon S3's cost savings and data center reliability.

Are You Ready for Amazon S3?...Probably Not

Having only discovered Amazon S3 a couple of months ago I was totally blown away by the idea. For those of you who don't know what this it, it is storage space in Amazon's data center that is rented out on a monthly fee (and transfer fee). As soon as I read about this I started thinking about the current online/off site storage vendors that I was using. I couldn't believe that Amazon would offer this "service" at such a cheap price.

I started researching everything about it and how I could start using it. The catch is that you must use it by utilizing their APIs. Meaning that you just don't sign up for the service and download their program, you need to write you own program or use someone else's open-source program which is what I did. Since there are so many open-source type programs out there you can just pick the one that you think is the best for your needs and use it. (On a side note, there are also some "pay-for" programs and some online storage companies have moved their infrastructure over to Amazon and are already offering this as a "storage backed by Amazon" type of service.)

The problem I see is the maturity of these open source programs. If I use one now for the next year and the person (or company) stops development of it, then what? A program like JungleDisk changes the file names for some reason when the file is uploaded. If development stops on this program you may need another program to interact with Amazon S3, renaming all of your files could be an arduous task. The other programs that I have used S3backup and www.S3Interface.com both keep the file name unchanged. But I want to be able to automate the file transfer and with these programs this task is more of a manual one.

On another note, I believe that S3 will revolutionize the way we think about storage all together. It will turn storage into a commodity and something where a physical location is not important. Backing up data in one location and restoring it at another may not be a problem in the not too distant future.

Here is the link to Amazon's site if you want to read more about it. www.amazon.com/s3

Data Centers Use 90% More Electricity

Data Centers are using 90% more electricity. Although it's not because of the price of electricity, it is the amount of servers that each data center houses. The main factor as to why such as large increase can be attributed to the amount of smaller low end servers. These lower end servers are used for VoIP, video and music. A way to stop such an increase from continuing is using virtualization software. This will reduce the physical number of servers needed thus reducing the amount of electricity needed.

[Via InformationWeek]