Hosting Your Images With Amazon S3

Images make just about everything more interesting, which is why I do my best to include at least one with each blog post even if for nothing more than comic relief.

I run a few different websites, and a while ago I decided to host the images for all of them in the cloud using Amazon S3 (Simple Storage Service).  I've been a very satisfied customer since then.  Not only is it incredibly easy to use, it's also rather simple to "mask" it so your images look like they're coming from somewhere else.  If you look at the properties of any image on this page, you'll see that it's coming from "img.bobpusateri.com".  In reality, it's coming from S3 thanks to the magic of DNS aliasing.  In this post I'll show you how to do that.

Why'd I Do This?

My original website was hosted out of my house on a single Linux box residing under my desk.  This machine was a real powerhouse at 300Mhz and 128MB RAM (and I was using it until the end of 2009!), so I wanted to move the load of hosting images off that box and into the cloud.    After considering a few different options I decided to use S3 based on both their 99.9% uptime guarantee and what I consider to be very reasonable pricing.  The fact that I'm not paying for anything I don't use was also rather attractive.

The downsides?  It's not completely free, but I've always believed that you get what you pay for.  You're charged based on how much data you're storing and how many times it's accessed.  Hopefully nobody decides to keep clicking "refresh" forever with the hopes of bankrupting me (don't worry, it won't take you too long!)   At the time I did the switch it also required some coding changes to my sites, but that was a one-time expense.

Accessing S3

Once you create an S3 account, you'll want to get it set up so you can start uploading your awesome images.  Since S3 lives in the cloud, there are a wide variety of clients available for uploading & managing your data.  My personal favorite is Amazon's recently-released web console.  Actually the console has been around for a while as it supports several Amazon cloud products but it only recently started supporting S3.  Another client I like is S3Fox, an add-on for Mozilla Firefox.  There are many others out there as well depending on your needs.

The Bucket

The basis of all thing in S3 is the "bucket" and very object you store in S3 will be in a bucket.  Buckets store key/object pairs, the keys being a string and the object being whatever type you like (in this case, an image file).  Keys are unique within a bucket.  Buckets cost nothing by themself (you're only billed by what's stored in them) and each account can contain up to 100 buckets, though you probably won't need anywhere near that many.  Of particular importance is the bucket's name, as it determines how you access the objects contained inside.  Bucket names must be unique across all buckets stored in S3, so if you try to create a bucket named "images" it will probably fail because someone else has likely already thought of that.

The other choice you have when creating a bucket is the region its data is stored in.  Amazon currently has 4 different regions to choose from, two in the U.S., Ireland, and Singapore.  In general you'll probably want to pick the reason that's closest to your target audience, but you may have reasons for storing it elsewhere (legal compliance, etc.)  The price you pay depends on the region you store the data in.

If you want to "mask" S3 so that it appears as another domain, you'll need to give your bucket the name of whatever domain you want to mask it as.  This means that for my "img.bobpusateri.com" domain I have a bucket named "img.bobpusateri.com" in my S3 account.

Objects & Permissions

Once you've created your bucket, you'll want to put stuff in there.  Uploading instructions will vary depending on your client, but most of them utilize a standard FTP-type manager allowing you to create folders and copy local files to S3.

By default a bucket's contents are shared with nobody except its creator.  To allow the world to see an object, you'll need to alter its permissions so that "everyone" can read it.  This again varies by client, but generally you can right-click on an object, select "Edit Permissions" or "Edit ACLs" (Access Control List) and grant read/view rights to everyone.  This is possible on a per-object basis through S3 (though some clients will recurse through buckets and/or folders) or a "canned access" policy may be applied to an entire bucket.

Accessing Objects

To access an object, you can formulate a URL from its key and the name of the bucket it's stored in.  For a bucket named "bucket" containing an object with a key of "key", the URL would be as follows:

http://bucket.s3.amazonaws.com/key
or
http://s3.amazonaws.com/bucket/key

In the case of the image in this post, the bucket is named "img.bobpusateri.com" and the key is "bc/2010/07/Buckets.jpg", which means you can access it from URLs:

https://s3.amazonaws.com/img.bobpusateri.com.s3.amazonaws.com/bc/2010/07/Buckets.jpg
or
http://s3.amazonaws.com/img.bobpusateri.com/bc/2010/07/Buckets.jpg

But neither of those are all that good looking, are they?  The purpose of this post is to be able to mask the Amazon part of the URL so that it will also work like this:

https://s3.amazonaws.com/img.bobpusateri.com/bc/2010/07/Buckets.jpg

This is done with a DNS command called CNAME, which more or less creates an alias for a subdomain.  This needs to be done at your hosting provider and will probably be under an advanced options menu somewhere.  You'll want to set your desired subdomain ("img" in my case) to point to "s3.amazonaws.com".  Once that's set up you should be good to go.