So, we had a lot of customers asking for ways to delete all of their cloud files in a single container, instead of having to do manually. This is possible using the bulk delete function defined in the rackspace docs. Find below the steps required to do this.<\/p>\n
Step 1: Make an auth.json file (for simplicity)<\/p>\n
\r\n{\r\n \"auth\": {\r\n \"RAX-KSKEY:apiKeyCredentials\": {\r\n \"username\": \"mycloudusername\",\r\n \"apiKey\": \"mycloudapikey\"\r\n }\r\n }\r\n}\r\n<\/pre>\nIt’s quite simple and nothing intimidating.<\/p>\n
For step 2 I’m using an application called jq, to install it do<\/p>\n\r\n\r\nwget https:\/\/github.com\/stedolan\/jq\/releases\/download\/jq-1.5\/jq-linux64\r\nmv jq-linux64 \/bin\/jq\r\nalias jq = '\/bin\/jq'\r\n<\/pre>\nNow you can use jq at the commandline.<\/p>\n
Step 2: Set variable called $TOKEN that can store the api token password output, the nice thing is there is no token stored in the script so its kind of secure<\/p>\n
\r\nTOKEN=`curl https:\/\/identity.api.rackspacecloud.com\/v2.0\/tokens -X POST -d @auth.json -H \"Content-type: application\/json\" | jq .access.token.id | sed 's\/\"\/\/g'`\r\necho $TOKEN\r\n<\/pre>\nStep 3: Set a variable for the container name<\/p>\n
\r\n# Container to Upload to\r\nCONTAINER=meh2\r\n<\/pre>\nStep 4: Populate a List of all the files in the $CONTAINER variable, in this case ‘meh2’. <\/p>\n
\r\n# Populate File List\r\necho \"Populating File List\"\r\ncurl -X GET https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh2 -H \"X-Auth-Token: $TOKEN\" > filelist.txt\r\n<\/pre>\nStep 5: Add container name to the file listing by rewriting the output file filelist.txt to a deletelist.txt<\/p>\n
\r\nsed -e \"s\/^\/\\\/$CONTAINER\\\/\/\" < filelist.txt > deletelist.txt\r\n<\/pre>\nStep 6: Bulk Delete Files thru API<\/p>\n
\r\necho \"Deleting Files..\"\r\ncurl -i -v -XDELETE -H\"x-auth-token: $TOKEN\" https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\\?bulk-delete -T .\/deletelist.txt\r\n<\/pre>\nStep 7: Confirm the deletion success<\/p>\n
\r\n# Confirm Deleted\r\necho \"Confirming Deleted in $CONTAINER..\"\r\ncurl -i -X GET https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh2 -H \"X-Auth-Token: $TOKEN\"\r\n<\/pre>\nThe completed script looks like this:<\/p>\n
\r\n\r\n Mass Delete Container\r\n\r\n# Get Token\r\n\r\nTOKEN=`curl https:\/\/identity.api.rackspacecloud.com\/v2.0\/tokens -X POST -d @auth.json -H \"Content-type: application\/json\" | jq .access.token.id | sed 's\/\"\/\/g'`\r\necho $TOKEN\r\n\r\n# Container to Upload to\r\nCONTAINER=meh2\r\n\r\n\r\n# Populate File List\r\necho \"Populating File List\"\r\ncurl -X GET https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh2 -H \"X-Auth-Token: $TOKEN\" > filelist.txt\r\n\r\n\r\n# Add Container Prefix\r\necho \"Adding Container Prefix..\"\r\nsed -e \"s\/^\/\\\/$CONTAINER\\\/\/\" < filelist.txt > deletelist.txt\r\n\r\n\r\n# Delete Files\r\necho \"Deleting Files..\"\r\ncurl -i -v -XDELETE -H\"x-auth-token: $TOKEN\" https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\\?bulk-delete -T .\/deletelist.txt\r\n\r\n# Confirm Deleted\r\necho \"Confirming Deleted in $CONTAINER..\"\r\ncurl -i -X GET https:\/\/storage101.lon3.clouddrive.com\/v1\/MossoCloudFS_10045567\/meh2 -H \"X-Auth-Token: $TOKEN\"\r\n\r\n<\/pre>\nPretty simple!<\/p>\n
Running it.. <\/p>\n
\n* About to connect() to storage101.lon3.clouddrive.com port 443 (#0)
\n* Trying 2a00:1a48:7900::100...
\n* Connected to storage101.lon3.clouddrive.com (2a00:1a48:7900::100) port 443 (#0)
\n* Initializing NSS with certpath: sql:\/etc\/pki\/nssdb
\n* CAfile: \/etc\/pki\/tls\/certs\/ca-bundle.crt
\n CApath: none
\n* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
\n* Server certificate:
\n* \tsubject: CN=storage101.lon3.clouddrive.com
\n* \tstart date: May 18 00:00:00 2015 GMT
\n* \texpire date: Nov 17 23:59:59 2016 GMT
\n* \tcommon name: storage101.lon3.clouddrive.com
\n* \tissuer: CN=thawte DV SSL CA - G2,OU=Domain Validated SSL,O=\"thawte, Inc.\",C=US
\n> DELETE \/v1\/MossoCloudFS_10045567?bulk-delete HTTP\/1.1
\n> User-Agent: curl\/7.29.0
\n> Host: storage101.lon3.clouddrive.com
\n> Accept: *\/*
\n> x-auth-token: AAA7uz-F91SDsaMOCKTOKEN-gOLeB5bbffh8GBGwAPl9F313Pcy4Xg_zP8jtgZolMOudXhsZh-nh9xjBbOfVssaSx_shBMqkxIEEgW1zt8xESJbZLIsvBTNzfVBlTitbUS4RarUOiXEw
\n> Content-Length: 515
\n> Expect: 100-continue
\n>
\n< HTTP\/1.1 100 Continue\nHTTP\/1.1 100 Continue\n\n* We are completely uploaded and fine\n< HTTP\/1.1 200 OK\nHTTP\/1.1 200 OK\n< Content-Type: text\/plain\nContent-Type: text\/plain\n< X-Trans-Id: tx010194ea9a104443b89bb-00161f7f1dlon3\nX-Trans-Id: tx010194ea9a104443b89bb-001611f7f1dlon3\n< Date: Thu, 15 Oct 2015 10:25:35 GMT\nDate: Thu, 15 Oct 2015 10:25:35 GMT\n< Transfer-Encoding: chunked\nTransfer-Encoding: chunked\n\n<\nNumber Deleted: 44\nNumber Not Found: 0\nResponse Body:\nResponse Status: 200 OK\nErrors:\n* Connection #0 to host storage101.lon3.clouddrive.com left intact\n\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"
So, we had a lot of customers asking for ways to delete all of their cloud files in a single container, instead of having to do manually. This is possible using the bulk delete function defined in the rackspace docs. … Continue reading