An object store is a simple web based storage solution focused on reliability, scalability, and security. It is best suited for public content storage/distribution, archiving data, or secure data sharing between users. Our Object Storage can be used through this web interface, the command line umobj utilities, third-party graphical clients, and even programmatically using many popular programming languages. We support a subset of the Amazon Simple Storage Services (S3) API, built around a technology called Ceph.
To start, let's introduce a little bit of terminology. S3-like storage thinks in terms of buckets and keys. Keys are analogous to files. A bucket is simply a container to a set of keys. There is no actual hierarchy inside of a bucket, but the standard UNIX path separator, a forward slash (
/) at the end of a Key name, is interpreted by many clients (including this web site and our umobj utilities) as being a directory delimiter. This allows you to copy data from your local filesystems to your buckets through umobj or third-party clients. Finally, you may specify who has what types of access to your buckets via Access Control Lists (ACLs) at the bucket level or the individual key level.
Your data is protected from individual machine failure via replication within the cluster. All data is checksummed in accordance with the Amazon S3 protocol to ensure that data in transit is valid before it is accepted by the cluster. However, there are no backups or snapshots of this data in the cluster, so if a user deletes a key or bucket in the object store, there is no way to restore that information.
You can create and browse your buckets (containers that hold data) by visiting your buckets page. You can also set bucket-level Access Control Lists (ACLs) from this page. Bucket-level ACLs get implicitly inherited to all the keys within the bucket. However, individual keys can have additional specific ACLs applied for more granular control.
Buckets names must be unique. When you create a bucket it will notify you if the name is already taken.
After selecting a bucket, you will be able to create folders and upload files within that bucket. Listed files can be downloaded, deleted, or assigned a specific ACL by the key owner/creator.
Please note: Local file system ownership and permissions, and special files (such as symlinks) can not be represented in the object store. We highly suggest that if you are securing data into the object store and need these to be faithfully maintained that you use a local archive tool (tar, zip, etc..) to collect the data and then upload the resulting archive file(s).
Within the web interface you can delete files one-by-one. If you want to remove a bunch of files, you will need to use a different client as described below.
This is dangerous as there are no backups of files in the object store. Be careful to only delete the data you intend to delete.
There are several clients that can be used (sometimes with a limited set of features) on your desktop to gain access to the Object Store. All supported UMIACS systems have a copy of our umobj utilities which provide command line access to the object store. We also have an article in our wiki on 3rd party clients that lists and explains the details. These clients need to be configured with your Access and Secret Keys as described below.
Each user has one or more pairs of Access Keys and Secret Keys that are used as a credential to not expose your password when using the object store. These can be obtained by clicking on your username in the upper right-hand corner. You'll use these to identify and authenticate yourself to the Object Store.
When using the umobj utilities, you will need to make sure you have added these credentials in your local shell initialization files. There are links to files that have these automatically generated for the 3 most popular UNIX shell families (bash/sh, csh/tcsh, and zsh). Please make sure that whatever file(s) you copy these credentials into can not be read by other users (eg.
chmod 600 filename).
Note: Each Access Key and Secret Key are specific to a particular object store, so if you are accessing multiple object stores you may want to write the credentials for each to separate files and then
source each file when you want to use the associated object store. Please contact email@example.com if you have any questions.
Lab Groups allow a group of users to share data while avoiding the need for complex ACLs by maintaining group ownership. Each user in a Lab Group has read and write access to the buckets owned by the Lab Group. All objects owned by a Lab Group count against the group quota. Lab Groups can be navigated using the menu with username in the top right corner of the page. Note: this will only appear if you are a member of at least one Lab Group. At this point, you can browse the Object Store as the Lab Group and obtain your unique Access Key and Secret Key pair using the instructions above. To switch to another Lab Group or your own buckets, simply click the menu and select another user or group.
Lab Groups have two different types of membership, Managers and Members. The primary difference in roles is that Managers can add or remove users while Members cannot. If you hold the Manager role in a Lab Group, you can add and remove users using the Manage LabGroups page, which is available under the Manage menu at the top of the page. After selecting a Lab Group, you can add users by typing their username into the search field and selecting a membership role.
To request a Lab Group for your project, please contact firstname.lastname@example.org