(mis-)Adventures with Amazon Simple Email Services (SES)

Amazon has recently launched a new service as part of its web-services offerings – Amazon Simple Email Service (or SES for short), an HTTP API for sending Emails. The main selling points are Amazon’s usual scalability power, as well as a relatively low price point for sending Emails (10 cents per 1000 Emails, not counting bandwidth). They also promise to improve deliverability using filters and feedback loops.

Having used Sendgrid for a while at our startup, Binpress, we were overall satisfied with their service. We did have some problems with seemingly innocent Emails ending up in spam boxes, and combined with the better pricing we thought Amazon SES deserved a shot.

This ain’t no rocket science

Amazon’s claims of simplicity would be justified indeed if you compare it with building a complete Email sending infrastructure with similar scaling power as their own offerin, but well, if you are tired of verifying email addresses one by one, you should take a look at one of these tools. However, compared to integrating competitors such as Sendgrid or any SMTP based service, SES integration is anything but simple.

Using any SMTP abstraction class, integrating an SMTP service involves passing the credentials and service endpoint IP. That’s it. We use the Zend_Mail component from the Zend Framework and our code looked like this:

$config = array(
	'auth' => 'login',
	'username' => 'user@domain.com',
	'password' => 'ourpassword'
);
$transport = new Zend_Mail_Transport_Smtp('smtp.sendgrid.net', $config);

$mail = new Zend_Mail();
// Prepare mail
$mail -> send($transport);

Pretty straightforward. Getting Amazon SES up and running was anything but. I ended up writing a nice SES abstraction and even a Zend_Mail transport to go with it (you can check out the code here if you are not interested in the details).

Requests and authentication

The first hurdle to overcome is using Amazon’s authentication scheme. Unlike most HTTP APIs, Amazon doesn’t pass a request signature as a request parameter. Instead, it uses a custom HTTP header called ‘X-Amzn-Authorization’ – the documentation of which is hidden in one of the inner pages of the docs as a seemingly meaningless footnote. You cannot do anything without authorizing first! this section should’ve been front and center and explained much more clearly.

Since we need custom headers, that immediately rules out something simple like file_get_contents() (which probably should be avoided anyway for it’s simplistic error handling). My first thought was to use cURL, like I do with most HTTP APIs – however after struggling with it for around an hour, I came to realize that cURL (at least for PHP) doesn’t allow you to specify custom headers that are not a part of the HTTP standard – like Amazon’s authentication header.

I briefly considered using http_request(), however that requires installing a PECL extension that is not bundled with the default installation, so that was scraped. The only viable option left for me was to construct the HTTP request myself from scratch and negotiate with the SES API using socket functions. Simple, right?

Luckily, SES docs do provide example requests. Being this was the first time in a while for me to experiment with raw HTTP requests and socket connections, it took me some time to get my bearings.

Socked up and raring to go

So was I ready to rock my Email world with Amazon SES at this point? not quite. Amazon’s example POST request looks like this –

POST / HTTP/1.1
Host: email.us-east-1.amazonaws.com
Content-Type: application/x-www-form-urlencoded
Date: Tue, 25 May 2010 21:20:27 +0000
X-Amzn-Authorization: AWS3-HTTPS AWSAccessKeyId=AKIADQKE4EXAMPLE,Algorithm=HMACSHA256,Signature=lBP67vCvGl ...

The sample request was failing with a cryptic “Unable to determine service/operation name to be authorized” error. Can you tell what’s missing? it’s the Content-Length header. Sure enough, adding it finally made my first request to go through to the API. More time wasted because of a bad usage example in the docs.

Obviously, there’s more

We’ve gone this far without mentioning one of the basic annoyances inherent to Amazon SES. You must verify each and every Email address you want to send to and from before you can actually do that. I’m sorry, come again? yes, in order to help protect itself from spammers, Amazon implemented yet another barrier to adoption. Here, user experience means something a user experiences as he contemplates smashing his screen and not a craft used to improve costumer satisfaction.

I mean, I understand the underlying reasoning, but this is going a bit far. Fortunately, requesting “production access” can alleviate verifying recipient addresses leaving the restriction only on source (from) addresses. Seeing that the service is basically useless without it, we sent out a request and put this integration project aside until we got it (which took about a week). It’s a good thing they’re cheap and scalable (at least, by reputation), cause simple it hasn’t been so far.

Finally, up and running

After we got our production access and verified some of our Emails, we could finally put Amazon SES to use. The code provided in their SDK for PHP is simply terrible, and it doesn’t even work for SES (UPDATE: while this was true when I tested it a few weeks ago, one of the commentators has clued me in to that it has been updated and it does work as of now), so I’ve completed my own API wrapper and even wrote a transport for Zend_Mail that we could plug instead of our previous SMTP transport (shown at the beginning of the post).

I even added a simple control panel for listing and editing verified Email addresses as well as viewing usage statistics and quotas. You can download it for free for non-commercial use (there are also licenses for commercial applications if you need it).

A couple of days ago we sent out a newsletter to about 350 recipients and in the middle Amazon SES crapped on us by throwing a “sending rate quota exceeded” exception. Since we don’t have an internal queue (we were counting on theirs), we basically had no way of knowing which Emails got sent and which didn’t.

It’s been almost a month since we were granted production access, and according to their quota growth table, our rate should have been 90 Emails per second. However, it was still at the start value of 1 per second, which we exceeded apparently. I would’ve expected it to go into a queue and be sent slowly instead of throwing an error, but that’s just me I guess.

Another issue that has popped up lately, is one of their API actions returning a malformed XML after the information grew beyond a certain size. Since there is no official technical support for non premium users like me, even for reporting bugs relating to their beta service, I posted the error on their forums and hoped for the best. I have received no response on it so far.

So that’s the state of Amazon SES for you. A very cheap and apparently scalable service with some kinks that need to be ironed out. The API is nothing to write home about as well, but with some abstractions that is acceptable.

I hope this article has given you some sense what it takes to integrate Amazon SES and how to do it. If you have any particular questions, don’t hesitate to probe me in the comments.

To know when the next article is published, please subscribe to new articles using your Email below or follow me on Twitter.

Subscribe to Blog via Email

Enter your email address to receive notification about new posts.