Tech Blog

Amazon S3 Storage Plug-In

Rosetta allows using different storage solutions, implemented as instances of the Storage Handler plug-in. Each plug-in instance uses a list of methods that can implement HTTP requests for exchanging files, between Rosetta and the storage layer.

The plug-in parameters are the unique attributes of each storage type (e.g. bucket name. secret access key in S3)

The storage entity ‘inherits’ the plug-in parameters with their default values and users can create multiple storages with different parameters (e.g. different buckets in S3). It should only be used for permanent storage (and not operational).

The plug-in for using Amazon S3 as a storage solution is given below. Users can use it as an example for implementing other cloud solutions.

The plug-in and its source code are available on github. Users should note there is no fixity check implementation in this plugin (and the bundled version) as it relies on Amazon to maintain data integrity.

See also this post on Rosetta and Amazon Storage.

Leave a Reply