Hi. So, when I started working at Rackspace I didn’t know very much about API. I know what it is, what it does, and why it’s used, but my experience was rather limited. So I was understandably a little bit concerned about using cloud files and API. Specifically using POST and GET thru CURL, and simple things such as authorization and identification thru header tokens.<\/p>\n
First of all to use API, and make API requests we need an access token. The access token is totally different to my mycloud username, password and API Key itself. The token is a bit like a session, wheras the API key is a bit like a form username and password. It’s also worth noting, just like a http session, the $TOKEN will expire every now and then requiring you to authorise yourself to get a new token.<\/p>\n
When you authorise yourself, you will get a token and a list of all the possible endpoints to GET and POST data, that is to say to retrieve or store records or query against particular search pattern, etc. <\/p>\n
Step 1: Get User Token Thru Identity API using JSON auth structure<\/strong><\/p>\n File: auth.json<\/p>\n It is possible to do this without the auth.json file and just use the string, like so:<\/p>\n It is also possible to connect to API using just USERNAME and API KEY:<\/p>\n After running one of these commands you will receive a large, and something like the below will be inside it:<\/p>\n This includes your full ‘token’ ID. In this case ‘BBBySDLKsxkj4CXdioidkj_a-vHqc4k6PYjxM2fu6D57Bf0dP0-Su6OO2beafdzoKDavyw32Sjd6SpiMhI-cUb654odmeiglz_2tsplnZ26T2Vj2h3LF-vwXNBEYS1IXvy7ZpARRMVranXw’ <\/p>\n This is the token we need to use to authenticate against the API.<\/p>\n Step 2. Uploading a File with CURL to a Cloud File Container<\/p>\n Output<\/strong><\/p>\n In this case I had 3 txt files, each approximately 1MB, that I uploaded to cloud files using CURL and the commandline. The pertinent part is “X-Auth-Token: $TOKEN”, which is the header which contains my authorisation key which was set at the commandline previously with TOKEN= line above it.<\/p>\n Now, it’s possible using a manifest to append these 3 files together into a single file. For instance if these files were larger, say 5GB and at the maximum limit for a file, such as a large dvd-iso, to exceed the 5GB limit we’d need to split that large file up into smaller files. But we want the cloud files container to send the whole file, all 3 5GB parts as if it were one file, this can be achieved with manifests. Here is how to do it!<\/p>\n Step 1: Create Manifest File <\/strong> Step 2: Assign Manifest <\/strong><\/p>\n As you can see, it’s really quite simple, by giving the path, etag and size_bytes, it’s possible to append these 3 file references to a single file association. Now when downloading mylargeappendedfile from the cloud storage container, you’ll get \/meh\/1.txt \/meh\/2.txt and \/meh\/3.txt all appended to a single file. This is pretty handy when dealing with files over 5GB, as the maximum limit for an individual file is 5GB, but that doesn’t stop you from spreading some size across multiple files, just like for instance in a rar archive.<\/p>\n Generated Output:<\/strong><\/p>\n Hi. So, when I started working at Rackspace I didn’t know very much about API. I know what it is, what it does, and why it’s used, but my experience was rather limited. So I was understandably a little bit … Continue reading \r\n{\r\n \"auth\": {\r\n \"RAX-KSKEY:apiKeyCredentials\": {\r\n \"username\": \"mycloudusernamehere\",\r\n \"apiKey\": \"mycloudapikeyhere\"\r\n }\r\n }\r\n}\r\n<\/pre>\n\r\ncurl https:\/\/identity.api.rackspacecloud.com\/v2.0\/tokens -X POST -d @auth.json -H \"Content-type: application\/json\"\r\n<\/pre>\n
\r\n curl https:\/\/identity.api.rackspacecloud.com\/v2.0\/tokens -X POST -d '{\"auth\":{\"RAX-KSKEY:apiKeyCredentials\":{\"username\":\"yourUserName\",\"apiKey\":\"yourAPIPassword\"}}}' -H \"Content-type: application\/json\" | python -m json.tool\r\n<\/pre>\n\r\ncurl https:\/\/identity.api.rackspacecloud.com\/v2.0\/tokens -X POST -d '{\"auth\":{\"passwordCredentials\":{\"username\":\"yourUserName\",\"password\":\"yourmycloudpassword\"}}}' -H \"Content-type: application\/json\"\r\n<\/pre>\n \r\n\r\n \"token\": {\r\n \"RAX-AUTH:authenticatedBy\": [\r\n \"APIKEY\"\r\n ],\r\n \"expires\": \"2015-09-29T17:04:56.092Z\",\r\n \"id\": \"BBBySDLKsxkj4CXdioidkj_a-vHqc4k6PYjxM2fu6D57Bf0dP0-Su6OO2beafdzoKDavyw32Sjd6SpiMhI-cUb654odmeiglz_2tsplnZ26T2Vj2h3LF-vwXNBEYS1IXvy7ZpARRMVranXw\",\r\n \"tenant\": {\r\n \"id\": \"1000000\",\r\n \"name\": \"1000000\"\r\n }\r\n },\r\n<\/pre>\n\r\n# Set the TOKEN Variable in the BASH SHELL\r\nTOKEN='BBBySDLKsxkj4CXdioidkj_a-vHqc4k6PYjxM2fu6D57Bf0dP0-Su6OO2beafdzoKDavyw32Sjd6SpiMhI-cUb654odmeiglz_2tsplnZ26T2Vj2h3LF-vwXNBEYS1IXvy7ZpARRMVranXw'\r\n\r\n# CURL REQUEST TO API\r\nCURL -i -X PUT https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh\/1.txt -T \/Users\/adam9261\/1.txt -H \"X-Auth-Token: $TOKEN\"\r\ncurl -i -X PUT https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh\/2.txt -T \/Users\/adam9261\/2.txt -H \"X-Auth-Token: $TOKEN\"\r\ncurl -i -X PUT https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh\/3.txt -T \/Users\/adam9261\/3.txt -H \"X-Auth-Token: $TOKEN\"\r\n<\/pre>\n
\r\n\r\nHTTP\/1.1 100 Continue\r\n\r\nHTTP\/1.1 201 Created\r\nLast-Modified: Mon, 28 Sep 2015 17:34:45 GMT\r\nContent-Length: 0\r\nEtag: 8aec10927922e92a963f6a1155ccc773\r\nContent-Type: text\/html; charset=UTF-8\r\nX-Trans-Id: txca64c51affee434abaa79-0056097a34lon3\r\nDate: Mon, 28 Sep 2015 17:34:45 GMT\r\n\r\nHTTP\/1.1 100 Continue\r\n\r\nHTTP\/1.1 201 Created\r\nLast-Modified: Mon, 28 Sep 2015 17:34:46 GMT\r\nContent-Length: 0\r\nEtag: 83736eb7b9eb0dce9d11abcf711ca062\r\nContent-Type: text\/html; charset=UTF-8\r\nX-Trans-Id: tx6df90d2dede54565bccfe-0056097a35lon3\r\nDate: Mon, 28 Sep 2015 17:34:46 GMT\r\n\r\nHTTP\/1.1 100 Continue\r\n\r\nHTTP\/1.1 201 Created\r\nLast-Modified: Mon, 28 Sep 2015 17:34:47 GMT\r\nContent-Length: 0\r\nEtag: 42f2826fff018420731e7bead0f124df\r\nContent-Type: text\/html; charset=UTF-8\r\nX-Trans-Id: tx79612b28754c4bb3aedd6-0056097a36lon3\r\nDate: Mon, 28 Sep 2015 17:34:46 GMT<\/pre>\n
Creating a Manifest file, defining 3 file parts as one single file download in the data stream<\/h2>\n
\nIn my case I created a manifest file first which tells cloud files which individual files make up the large file<\/p>\n\r\n[\r\n\r\n {\r\n \"path\": \"\/meh\/1.txt\",\r\n \"etag\": \"8aec10927922e92a963f6a1155ccc773\",\r\n \"size_bytes\": 1048585\r\n },\r\n\r\n {\r\n \"path\": \"\/meh\/2.txt\",\r\n \"etag\": \"83736eb7b9eb0dce9d11abcf711ca062\",\r\n \"size_bytes\": 1048585\r\n },\r\n\r\n {\r\n \"path\": \"\/meh\/3.txt\",\r\n \"etag\": \"42f2826fff018420731e7bead0f124df\",\r\n \"size_bytes\": 1048587\r\n }\r\n]\r\n<\/pre>\n\r\n\r\ncurl -i -X PUT https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh\/mylargeappendedfile?multipart-manifest=put -d @manifest -H \"X-Auth-Token: $TOKEN\"\r\n\r\n<\/pre>\n
\r\n\r\nHTTP\/1.1 201 Created\r\nLast-Modified: Mon, 28 Sep 2015 17:32:18 GMT\r\nContent-Length: 0\r\nEtag: \"8f49c8861f0aef2eb750099223050c27\"\r\nContent-Type: text\/html; charset=UTF-8\r\nX-Trans-Id: tx64093603d9444e088f242-00560979a0lon3\r\nDate: Mon, 28 Sep 2015 17:32:32 GMT\r\n\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"