john spurlock 2008


baltic-avenue: an open source clone of the amazon s3 rest api for google app engine

I've been recently experimenting with the idea of creating an open source S3 rest api clone that runs as a GAE application. This would allow for "private S3 instances" that leverage the google hosted infrastructure (big-table etc) and your existing S3 tools/applications/processes.

The project has reached a minimum level of functionality as a proof of concept, and is now hosted out on google code.

code.google.com/p/baltic-avenue

The core bucket and object-level operations are implemented, as well as support for multiple custom user accounts, ACLs and public-reads. More info on the site on how to configure third-party clients (smoke-tested with s3sync, boto, jets3t/cockpit, resourceful/spaceblock).

There are some obvious limitations right now due to the preview release quotas on GAE - e.g. object size limited to 1MB, and total storage limited to 500MB. The assumption is that these will be lifted or relaxed once the service is pay-as-you-go.

Would love to get people involved in testing and making the implementation more robust. I'm not necessarily a pythonista, so I attempted to write as little of it as possible...