Perform a traceroute thru the Rackspace monitoring API

So, I was thinking about the Rackspace traceroute monitoring API and wondering what I could do with it, when I come across this gem

/monitoring_zones/mzsyd/traceroute

Well what is it you ask. Well it’s an API path for performing a traceroute on the 6 different region endpoints. This means you can use an API call to run traceroutes (for free!) thru the Rackspace cloud monitoring API. This would be pretty handy at testing connectivity around the world to your chosen destination from each datacentre. Handy Andy.

Then you ask what does the mzsyd mean? That’s a region ID: Let’s see about putting together a script to list the region ID’s we can run the traceroutes on first of all:

File: list-monitoring-zones.sh

!/bin/bash

USERNAME='mycloudusername'
APIKEY='mycloudapikey'
ACCOUNTNUMBER='10010110'
API_ENDPOINT="https://monitoring.api.rackspacecloud.com/v1.0/$ACCOUNTNUMBER"


TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d '{ "auth":{"RAX-KSKEY:apiKeyCredentials": { "username":"'$USERNAME'", "apiKey": "'$APIKEY'" }} }' -H "Content-type: application/json" |  python -mjson.tool | grep -A5 token | grep id | cut -d '"' -f4`




curl -s -v  \
-H "X-Auth-Token: $TOKEN"  \
-H "X-Project-Id: $ACCOUNTNUMBER" \
-H "Accept: application/json"  \
-X GET  \
"$API_ENDPOINT/monitoring_zones"

Lets take a look at the response when I run this monitoring zone list.


chmod +x list-monitoring-zones.sh
./list-monitoring-zones.sh

Response

< Content-Type: application/json; charset=UTF-8
< Via: 1.1 Repose (Repose/7.3.0.0)
< Vary: Accept-Encoding
< X-LB: api1.dfw1.prod.cm.k1k.me
< Transfer-Encoding: chunked
<
{
    "values": [
        {
            "id": "mzdfw",
            "label": "Dallas Fort Worth (DFW)",
            "country_code": "US",
            "source_ips": [
                "2001:4800:7902:0001::/64",
                "50.56.142.128/26"
            ]
        },
        {
            "id": "mzhkg",
            "label": "Hong Kong (HKG)",
            "country_code": "HK",
            "source_ips": [
                "2401:1800:7902:0001::/64",
                "180.150.149.64/26"
            ]
        },
        {
            "id": "mziad",
            "label": "Northern Virginia (IAD)",
            "country_code": "US",
            "source_ips": [
                "2001:4802:7902:0001::/64",
                "69.20.52.192/26"
            ]
        },
        {
            "id": "mzlon",
            "label": "London (LON)",
            "country_code": "GB",
            "source_ips": [
                "2a00:1a48:7902:0001::/64",
                "78.136.44.0/26"
            ]
        },
        {
            "id": "mzord",
            "label": "Chicago (ORD)",
            "country_code": "US",
            "source_ips": [
                "2001:4801:7902:0001::/64",
                "50.57.61.0/26"
            ]
        },
        {
            "id": "mzsyd",
            "label": "Sydney (SYD)",
            "country_code": "AU",
            "source_ips": [
                "2401:1801:7902:0001::/64",
                "119.9.5.0/26"
            ]
        }
    ],
    "metadata": {
        "count": 6,
        "limit": 100,
        "marker": null,
        "next_marker": null,
        "next_href": null
    }
* Connection #0 to host monitoring.api.rackspacecloud.com left intact

We can see many zones available to run our traceroute to;

id 'mzsyd' for Sydney SYD.
id 'mzdfw' for Dallas Fort Worth DFW
id 'mzhkg' for Hong Kong HKG
id 'mziad' for Northern Viginia IAD
id 'mzord' for Chicago ORD
id 'mzlon' for London LON

So now I know what the zone id's are, as defined above here. Now time to use them and run a traceroute to haxed.me.uk. Lets see;

File: perform-traceroute-from-monitoring-zone.sh

!/bin/bash

USERNAME='mycloudusernamehere'
APIKEY='apikeyhere'
ACCOUNTNUMBER=10010110
API_ENDPOINT="https://monitoring.api.rackspacecloud.com/v1.0/$ACCOUNTNUMBER"



TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d '{ "auth":{"RAX-KSKEY:apiKeyCredentials": { "username":"'$USERNAME'", "apiKey": "'$APIKEY'" }} }' -H "Content-type: application/json" |  python -mjson.tool | grep -A5 token | grep id | cut -d '"' -f4`




curl -s -v  \
-H "X-Auth-Token: $TOKEN"  \
-H "X-Project-Id: $ACCOUNTNUMBER" \
-H "Accept: application/json"  \
-d @ip.json -H "content-type: application/json" -X POST  \
"$API_ENDPOINT/monitoring_zones/mzsyd/traceroute"

You also need the ip.json file. It's easy to make, put it in the same dir as the shellscript.

File: ip.json

{
        "target":               "haxed.me.uk",
        "target_resolver":      "IPv4"
}

We're going to refer to ip.json file which contains our destination data. You can do this with IPv6 IP's too if you wanted! That is pretty cool!
It is possible to do this without including the file, and actually just pass the json directly, with -d { "target": "haxed.me.uk", "target_resolver": "IPv4"} , but lets do it properly 😀


chmod +x perform-traceroute-from-monitoring-zone.sh
./perform-traceroute-from-monitoring-zone

the response, a nice traceroute of course from syd to my lon server.

Response

 Accept: application/json
> content-type: application/json
> Content-Length: 55
>
* upload completely sent off: 55 out of 55 bytes
< HTTP/1.1 200 OK
< Date: Wed, 13 Jan 2016 11:19:14 GMT
< Server: Jetty(9.2.z-SNAPSHOT)
< X-RateLimit-Type: traceroute
< X-RateLimit-Remaining: 296
< X-RateLimit-Window: 24 hours
< x-trans-id: eyJyZXF1ZXN0SWQiOiI5MTNhNTY1Mi05ODAyLTQ5MmQtOTAwYS05NDU1M2ZhNDJmNzUiLCJvcmlnaW4
< X-RateLimit-Limit: 300
< X-Response-Id: .rh-TI8E.h-api1.ord1.prod.cm.k1k.me.r-4RFTh9up.c-28452540.ts-1452683954386.v-91eaf0a
< Content-Type: application/json; charset=UTF-8
< Via: 1.1 Repose (Repose/7.3.0.0)
< Vary: Accept-Encoding
< X-LB: api0.ord1.prod.cm.k1k.me
< Transfer-Encoding: chunked
<
{
    "result": [
        {
            "ip": "119.9.5.2",
            "hostname": null,
            "number": 1,
            "rtts": [
                0.421,
                0.384,
                0.442,
                0.457,
                0.455
            ]
        },
        {
            "ip": "119.9.0.30",
            "hostname": null,
            "number": 2,
            "rtts": [
                1.015,
                0.872,
                0.817,
                1.014,
                0.926
            ]
        },
        {
            "ip": "119.9.0.109",
            "hostname": null,
            "number": 3,
            "rtts": [
                1.203,
                1.179,
                1.185,
                1.232,
                1.182
            ]
        },
        {
            "ip": "202.84.223.2",
            "hostname": null,
            "number": 4,
            "rtts": [
                3.53,
                5.301,
                3.975,
                5.772,
                3.804
            ]
        },
        {
            "ip": "202.84.223.1",
            "hostname": null,
            "number": 5,
            "rtts": [
                3.437,
                3.522,
                2.837,
                4.274,
                2.805
            ]
        },
        {
            "ip": "202.84.140.206",
            "hostname": null,
            "number": 6,
            "rtts": [
                141.198,
                140.746,
                143.871,
                140.987,
                141.545
            ]
        },
        {
            "ip": "202.40.149.238",
            "hostname": null,
            "number": 7,
            "rtts": [
                254.354,
                175.559,
                176.787,
                176.701,
                175.634
            ]
        },
        {
            "ip": "134.159.63.18",
            "hostname": null,
            "number": 8,
            "rtts": [
                175.302,
                175.299,
                175.183,
                175.146,
                175.149
            ]
        },
        {
            "ip": "64.125.26.6",
            "hostname": null,
            "number": 9,
            "rtts": [
                175.395,
                175.408,
                175.469,
                175.49,
                175.475
            ]
        },
        {
            "ip": "64.125.30.184",
            "hostname": null,
            "number": 10,
            "rtts": [
                285.818,
                285.872,
                285.801,
                285.835,
                285.887
            ]
        },
        {
            "ip": "64.125.29.52",
            "hostname": null,
            "number": 11,
            "rtts": [
                285.864,
                285.938,
                285.826,
                285.922,
                303.125
            ]
        },
        {
            "ip": "64.125.28.98",
            "hostname": null,
            "number": 12,
            "rtts": [
                284.711,
                284.865,
                284.73,
                284.697,
                284.713
            ]
        },
        {
            "ip": "64.125.29.48",
            "hostname": null,
            "number": 13,
            "rtts": [
                287.341,
                310.82,
                287.33,
                287.359,
                287.455
            ]
        },
        {
            "ip": "64.125.29.130",
            "hostname": null,
            "number": 14,
            "rtts": [
                286.168,
                286.012,
                286.108,
                286.105,
                286.168
            ]
        },
        {
            "ip": "64.125.30.235",
            "hostname": null,
            "number": 15,
            "rtts": [
                284.61,
                284.681,
                284.667,
                284.892,
                286.069
            ]
        },
        {
            "ip": "64.125.20.97",
            "hostname": null,
            "number": 16,
            "rtts": [
                287.516,
                287.435,
                287.557,
                287.581,
                287.438
            ]
        },
        {
            "ip": "94.31.42.254",
            "hostname": null,
            "number": 17,
            "rtts": [
                288.156,
                288.019,
                288.034,
                288.08
            ]
        },
        {
            "ip": null,
            "hostname": null,
            "number": 18,
            "rtts": []
        },
        {
            "ip": "134.213.131.251",
            "hostname": null,
            "number": 19,
            "rtts": [
                292.687,
                293.72,
                295.335,
                293.981
            ]
        },
        {
            "ip": "162.13.232.1",
            "hostname": null,
            "number": 20,
            "rtts": [
                293.295,
                293.738,
                295.46,
                294.301
            ]
        },
        {
            "ip": "162.13.232.103",
            "hostname": null,
            "number": 21,
            "rtts": [
                294.733,
                294.996,
                298.884,
                295.056
            ]
        },
        {
            "ip": "162.13.136.211",
            "hostname": null,
            "number": 22,
            "rtts": [
                294.919,
                294.77,
                298.956,
                296.481
            ]
        }
    ]
* Connection #0 to host monitoring.api.rackspacecloud.com left intact

This is pretty cool. If we want to run a traceroute from lets say chicago, we just swap out the 'mzsyd' variable to show 'mziad', wow thats simple 🙂

Generate SSH Keys pairs and copy public key to guests the fast way

What it says on the tin!

 ssh-keygen -t dsa
ssh-copy-id root@iporhostnamehere
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

So simple. Thanks to my colleague Jan for this.

4 way NORAID mirror using ZFS

So I thought about a cool way to backup my files without using anything too fancy and I started to think about ZFS. Don’t know why I didn’t before because it’s ultra ultra resilient. Cheers Oracle. This is in Debian 7 Wheezy.

Step 1 Install zfs

# apt-get install lsb-release
# wget http://archive.zfsonlinux.org/debian/pool/main/z/zfsonlinux/zfsonlinux_6_all.deb
# dpkg -i zfsonlinux_6_all.deb

# apt-get update
# apt-get install debian-zfs

Step 2 Create Mirrored Disk Config with Zpool.
Here i’m using 4 x 75GB SATA Cloud Block Storage Devices to have 4 copies of the same data with ZFS great error checking abilities

zpool create -f noraidpool mirror xvdb xvdd xvde xvdf

Step 3. Write a little disk write utility

#!/bin/bash


while :
do

        echo "Testing." $x >> file.txt
        sleep 0.02
  x=$(( $x + 1 ))
done

Step 4 (Optional). Start killing the Disks with fire, kill iscsi connection etc, and see if file.txt is still tailing.

./write.sh & ; tail -f /noraidpool/file.txt

Step 5. Observe that as long as one of the 4 disks has it’s virtual block device connection your data is staying up. So it will be OK even if there is 3 or less I/O errors simultaneously. Not baaaad.


root@zfs-noraid-testing:/noraidpool# /sbin/modprobe zfs
root@zfs-noraid-testing:/noraidpool# lsmod | grep zfs
zfs                  2375910  1
zunicode              324424  1 zfs
zavl                   13071  1 zfs
zcommon                35908  1 zfs
znvpair                46464  2 zcommon,zfs
spl                    62153  3 znvpair,zcommon,zfs
root@zfs-noraid-testing:/noraidpool# zpool status
  pool: noraidpool
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        noraidpool  ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            xvdb    ONLINE       0     0     0
            xvdd    ONLINE       0     0     0
            xvde    ONLINE       0     0     0
            xvdf    ONLINE       0     0     0

errors: No known data errors

Step 6. Some more benchmark tests

time sh -c "dd if=/dev/zero of=ddfile bs=8k count=250000 && sync"

Step 7. Some concurrent fork tests

#!/bin/bash

while :
do

time sh -c "dd if=/dev/zero of=ddfile bs=8k count=250000 && sync" &
        echo "Testing." $x >> file.txt
        sleep 2
  x=$(( $x + 1 ))
 zpool iostat
clear
done

or better

#!/bin/bash

time sh -c "dd if=/dev/zero of=ddfile bs=128k count=250000 && sync" &
time sh -c "dd if=/dev/zero of=ddfile bs=24k count=250000 && sync" &
time sh -c "dd if=/dev/zero of=ddfile bs=16k count=250000 && sync" &
while :
do

        echo "Testing." $x >> file.txt
        sleep 2
  x=$(( $x + 1 ))
 zpool iostat
clear
done

bwm-ng ‘elegant’ style output of disk I/O using zpool status


#!/bin/bash

time sh -c "dd if=/dev/zero of=ddfile bs=8k count=250000 && sync" &
while :
do
clear
 zpool iostat
sleep 2
clear
done

To test the resiliency of ZFS I removed 3 of the disks, completely unlatching them


        NAME                      STATE     READ WRITE CKSUM
        noraidpool                DEGRADED     0     0     0
          mirror-0                DEGRADED     0     0     0
            1329894881439961679   UNAVAIL      0     0     0  was /dev/xvdb1
            12684627022060038255  UNAVAIL      0     0     0  was /dev/xvdd1
            4058956205729958166   UNAVAIL      0     0     0  was /dev/xvde1
            xvdf                  ONLINE       0     0     0

And noticed with just one remaining Cloud block storage device I was still able to access the data on the disk as well as create data:

cat file.txt  | tail
Testing. 135953
Testing. 135954
Testing. 135955
Testing. 135956
Testing. 135957
Testing. 135958
Testing. 135959
Testing. 135960
Testing. 135961
Testing. 135962

# mkdir test
root@zfs-noraid-testing:/noraidpool# ls -a
.  ..  ddfile  file.txt  forktest.sh  stat.sh  test  writetest.sh


That’s pretty flexible.

Configuring a Console Prompt for BASH Linux

In BASH it’s pretty simple to customize the console prompt. There are a few good reasons for doing this, for instance if you are pulling out data from the commandline or running automation tasks and want to know when each section was executed. Here is how I did it:

Edit .bash_profile

cd
vi .bash_profile

Insert this line into .bash_profile

PS1='bash-\v \d \t \H \w\$ '

source it

$source .bash_profile
bash-4.9 Tue Nov 24 11:32:09 pirax-test ~#

Nice.

Using SNI with Rackspace Cloud Load Balancer and adding upto 20 SSL Certificates on single LB

This is going to be a short and dirty documentation on how to add multiple SSL certificates to a Rackspace Load Balancer.

1. Authorise with rackspace auth api (Get a token with user and api key credentials)
x-auth-key is apikey and x-auth-user is the mycloud username

curl -D - -H "x-auth-user: myusername" -H "x-auth-key: 1c989d8f89dfd87f3df3dff3d6f7fgf" https://auth.api.rackspacecloud.com/v1.0


HTTP/1.1 204 No Content
Server: nginx
Date: Thu, 19 Nov 2015 15:41:38 GMT
Connection: keep-alive
X-Storage-Token: AAA98345kdfg893DFGDF43iudng39dfgjkdfgDFI$JUIDFJGDFJGDFGDJJHDFGJHIfdg34dfgkdfjgiodfgiodfDFGDdg323
X-Storage-Url: https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_1001001
X-NewRelic-App-Data: PxQGUF9aDwETVlhSBQUP
X-CDN-Management-Url: https://cdn3.clouddrive.com/v1/MossoCloudFS_1001001
X-Auth-Token: AAA98345kdfg893DFGDF43iudng39dfgjkdfgDFI$JUIDFJGDFJGDFGDJJHDFGJHIfdg34dfgkdfjgiodfgiodfDFGDdg323
vary: Accept, Accept-Encoding, X-Auth-Token, X-Auth-Key, X-Storage-User, X-Storage-Pass, X-Auth-User
Cache-Control: s-maxage=86319
Front-End-Https: on

Now you can copy and paste the X-Auth-Token. It is needed for the next step

2. Configure the JSON file to upload an additional certificate and private key via API for a domain hostname. Here I am configuring domain.com

file: lb.json

{
  "certificateMapping": {
     "hostName": "domain.com",
     "certificate": "-----BEGIN CERTIFICATE-----\nMIIC/TCCAeWgAwIBAgIJAOjRMYJKDeryMA0GCSqGSIb3DQEBBQUAMBUxEzARBgNV\nBAMMCmRvbWFpbi5jb20wHhcNMTUxMTE5MTQzMjE3WhcNMjUxMTE2MTQzMjE3WjAV\nMRMwEQYDVQQDDApkb21haW4uY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB\nCgKCAQEAvHTjzWQchX+Gyl/No+ABR9R+F65rJmEPBEutjgWUynOir7ZYu5vmFol8\nhF054W5Xv3Ii4oYJjDJingOqQUBBxJD4jXx8H79y04JGXl8BBrG7azbRbowc4HoP\nRUiVTNaCPgYAGTreiRXmYKb/beotlGDvl0HQQLeDh4iq1X1E8R/lkFRHVAu0rEgC\nIeuJZ2L3Qu06A5yTCwdTJnZmviLmuDQtkfLDqTA8N67U8zjBgKGsj9t7GDSQ7zGp\n6JbTSJXqsXvd7XMLm2Ns2UelVUToxBTwgOIBn0XzZLCIOIlbIn0LHBk8oYEA4JDF\n1mXeqdsFOCtYvFcQBoUihiDjwDdTNQIDAQABo1AwTjAdBgNVHQ4EFgQU1wBZxNte\n9Q//UOl7ZMUvtsXghPEwHwYDVR0jBBgwFoAU1wBZxNte9Q//UOl7ZMUvtsXghPEw\nDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQUFAAOCAQEAUNM56u/cc56ESZY4gubX\nh0UQ8TjVbV2G4EkbBkNnm7RgNK48lFIxc55tshawhdN01JH5ZIgB1RvO1/lqouVs\nJrXwnPULBb4M5FcrjjBVu3bIvOjAUVDogOm7pKP/hJALM9CWMuZcXr5C+sYFczaB\nA7uDuMuQoTZBIGF1NyzfO7vmHT5QbEA/1ZYISWrVFNt8g2oxJY+jdgKacxVujWIs\nFpuiCCdvFVI05wCjj3C8BIN/EAcRIqe5gwr5oI+AtwK7fjK5K47/sREMI+W6Bj1w\nZEDz92S+dNtoSPJTBWiIQFLslTPiaDAu1EjJO1+YRXG7LANdxpQrogvDG1l9VpDW\nRg==\n-----END CERTIFICATE-----",
  "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEAvHTjzWQchX+Gyl/No+ABR9R+F65rJmEPBEutjgWUynOir7ZY\nu5vmFol8hF054W5Xv3Ii4oYJjDJingOqQUBBxJD4jXx8H79y04JGXl8BBrG7azbR\nbowc4HoPRUiVTNaCPgYAGTreiRXmYKb/beotlGDvl0HQQLeDh4iq1X1E8R/lkFRH\nVAu0rEgCIeuJZ2L3Qu06A5yTCwdTJnZmviLmuDQtkfLDqTA8N67U8zjBgKGsj9t7\nGDSQ7zGp6JbTSJXqsXvd7XMLm2Ns2UelVUToxBTwgOIBn0XzZLCIOIlbIn0LHBk8\noYEA4JDF1mXeqdsFOCtYvFcQBoUihiDjwDdTNQIDAQABAoIBAQCSEJr7d0tv4P6s\n3gI5sIXtkXHFkwczcOi9sJYszICdRXDjdZZimpuD/j3HLaaN5gMWvDTzk2XVBrxO\nspKEDnSrEJ3Es6ZUyQMLkh5OSJ43/QtBNvSuFOTQy2oIjhBBxMSfo/DxnSIb6CBt\n6yFwpJ99MICioHzznAjSxId7/qKvq294emBGwpyP6JbCEtrM6rsnBO4J/uHUDLRj\nlU0zLFwFHNQnhnfIuxOoUZthyCSzZgUquC7C52qIPTZxqCydSi045pDoymn6pT43\n5YdafzWarmEqBGcyqDOyjOz01IEicrmFW7e2+DICIOTOvTSeFQtHbO4Rn2VE2V+x\nGNJY3DoFAoGBAORqB6gFlLUKBXdmP1VcEifjwcVtBaY9QwehbH8En6O0N1t5bKFx\nTBaShm2El+7UCeeSz9hx3vmV/4gn9amJnu6stOEUfjbfxe6mw8OtR13g5iSAI9TQ\nXesf1HoCrUsljzAPvBAKxWSQl9e6fYBxmB1IvFvd4n9uvoNWr/lOfbe3AoGBANM3\neddZYHBB0PhgiJ9aq7QkgqUSdv5JlBdtGdPDr3cpIx9QmXMtf+wc8vZ6CSvC3EIn\npADRt3QAIzxQLpXb3ADjBCwwsFCu27IXlVkvxD+yvqaLbAjB/LgbKqt5wR6YAarj\nDQzNzxhGvrCS+CvYSKospY6UK5+V0nuhuPVcuJRzAoGAAPHLTE+RmNoMwbyjgGfc\nD1wqvfVAc7qHH230c+YB/vxMyk0LPPOp++HpOmS0+CDaVaHOyDdYU7HiF58KrgPK\nq3P9X3zlNLbiK6V248VAqUu3x+jbvRKLgOBl0YdXThs+p1U5Utuoi0zpw9Oal0Bg\n/6YAWWTmfd5oXUSrf51qeasCgYEAgMahBZgbgTXPh6+rfKTWbQWZlbU1UYJgxQui\npIb5cwhkvpHwjNWf2cAorffnoYOzsK3kgw9Z72KqGPq1/G5Iq0293Idu6DJEBkf0\nqaTC3SdIr9fvbUOApmsBz/xyrwl0ctDtwvG0IxP27UceAfVjEEYaRly2YB0DcJdA\nYnA+pVsCgYEAoHfkw/ZPmB7r8LesF0+N93AErJ/IiPoCBFNKijVDplzLQbMeWyxL\njcnFdq8vQT0Os4qzRNCR5QbMcprJIh4LC96OIlGWz5NhKCWbGsKxA8N7YoWGYy9Z\nmRkVP6peBU2cGdXRWjCrxkKR+uJM9BCG0Ix3BOPy29nWaCEl+5wjBEc=\n-----END RSA PRIVATE KEY-----"
  }

3. Call API to add certificatemapping json lb hostname configuration file. This just allows example.com to have SSL on the Load Balancer.
(you can add up to 20 Domains). It’s lots cheaper and not as hard as I might have initially thought!!

curl -v -H "X-Auth-Token: $TOKEN" -d @lb.json -X POST -H "content-type: application/json"  https://lon.loadbalancers.api.rackspacecloud.com/v1.0/1001001/loadbalancers/157089/ssltermination/certificatemappings

It’s also possible to update the Load Balancer Certificates via the API, please see https://developer.rackspace.com/docs/cloud-load-balancers/v1/developer-guide/#update-certificate-mapping for more information

4. Confirm the certificate mappings are added (please note 1001011 is the customer DDI and 157090 is the Load Balancer ID).

curl -v -H "X-Auth-Token: $TOKEN" -X GET https://lon.loadbalancers.api.rackspacecloud.com/v1.0/1001011/loadbalancers/157090/ssltermination/certificatemappings


< HTTP/1.1 200 OK
< Content-Type: application/json
< Via: 1.1 Rackspace Cloud Load Balancer API v1.25.11 (Repose/2.11.0)
< Content-Length: 83
< Date: Thu, 19 Nov 2015 15:49:24 GMT
* Server Jetty(8.0.y.z-SNAPSHOT) is not blacklisted
< Server: Jetty(8.0.y.z-SNAPSHOT)
<
* Connection #0 to host lon.loadbalancers.api.rackspacecloud.com left intact
{"certificateMappings":[{"certificateMapping":{"id":999,"hostName":"domain.com"}}]}

You may note that the lb.json file has the certificate all on one line! how to do this? It's not that hard. Here is how I did it:

cat domain.com.cert | sed ':a;N;$!ba;s/\n/\\n/g'
cat domain.com.key  | sed ':a;N;$!ba;s/\n/\\n/g'
-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEAvHTjzWQchX+Gyl/No+ABR9R+F65rJmEPBEutjgWUynOir7ZY\nu5vmFol8hF054W5Xv3Ii4oYJjDJingOqQUBBxJD4jXx8H79y04JGXl8BBrG7azbR\nbowc4HoPRUiVTNaCPgYAGTreiRXmYKb/beotlGDvl0HQQLeDh4iq1X1E8R/lkFRH\nVAu0rEgCIeuJZ2L3Qu06A5yTCwdTJnZmviLmuDQtkfLDqTA8N67U8zjBgKGsj9t7\nGDSQ7zGp6JbTSJXqsXvd7XMLm2Ns2UelVUToxBTwgOIBn0XzZLCIOIlbIn0LHBk8\noYEA4JDF1mXeqdsFOCtYvFcQBoUihiDjwDdTNQIDAQABAoIBAQCSEJr7d0tv4P6s\n3gI5sIXtkXHFkwczcOi9sJYszICdRXDjdZZimpuD/j3HLaaN5gMWvDTzk2XVBrxO\nspKEDnSrEJ3Es6ZUyQMLkh5OSJ43/QtBNvSuFOTQy2oIjhBBxMSfo/DxnSIb6CBt\n6yFwpJ99MICioHzznAjSxId7/qKvq294emBGwpyP6JbCEtrM6rsnBO4J/uHUDLRj\nlU0zLFwFHNQnhnfIuxOoUZthyCSzZgUquC7C52qIPTZxqCydSi045pDoymn6pT43\n5YdafzWarmEqBGcyqDOyjOz01IEicrmFW7e2+DICIOTOvTSeFQtHbO4Rn2VE2V+x\nGNJY3DoFAoGBAORqB6gFlLUKBXdmP1VcEifjwcVtBaY9QwehbH8En6O0N1t5bKFx\nTBaShm2El+7UCeeSz9hx3vmV/4gn9amJnu6stOEUfjbfxe6mw8OtR13g5iSAI9TQ\nXesf1HoCrUsljzAPvBAKxWSQl9e6fYBxmB1IvFvd4n9uvoNWr/lOfbe3AoGBANM3\neddZYHBB0PhgiJ9aq7QkgqUSdv5JlBdtGdPDr3cpIx9QmXMtf+wc8vZ6CSvC3EIn\npADRt3QAIzxQLpXb3ADjBCwwsFCu27IXlVkvxD+yvqaLbAjB/LgbKqt5wR6YAarj\nDQzNzxhGvrCS+CvYSKospY6UK5+V0nuhuPVcuJRzAoGAAPHLTE+RmNoMwbyjgGfc\nD1wqvfVAc7qHH230c+YB/vxMyk0LPPOp++HpOmS0+CDaVaHOyDdYU7HiF58KrgPK\nq3P9X3zlNLbiK6V248VAqUu3x+jbvRKLgOBl0YdXThs+p1U5Utuoi0zpw9Oal0Bg\n/6YAWWTmfd5oXUSrf51qeasCgYEAgMahBZgbgTXPh6+rfKTWbQWZlbU1UYJgxQui\npIb5cwhkvpHwjNWf2cAorffnoYOzsK3kgw9Z72KqGPq1/G5Iq0293Idu6DJEBkf0\nqaTC3SdIr9fvbUOApmsBz/xyrwl0ctDtwvG0IxP27UceAfVjEEYaRly2YB0DcJdA\nYnA+pVsCgYEAoHfkw/ZPmB7r8LesF0+N93AErJ/IiPoCBFNKijVDplzLQbMeWyxL\njcnFdq8vQT0Os4qzRNCR5QbMcprJIh4LC96OIlGWz5NhKCWbGsKxA8N7YoWGYy9Z\nmRkVP6peBU2cGdXRWjCrxkKR+uJM9BCG0Ix3BOPy29nWaCEl+5wjBEc=\n-----END RSA PRIVATE KEY-----

Notice the extra \n's after the processing.

Don't be intimidated by the sed line, it just replaces the \n newline with the character \n instead, so the json file is easier to lay out the cert as a 'string'.

Important notes on SNI:

We know we can add certificate mappings on the Load Balancer.
the Load Balancer has been configured for Allowing secure and insecure traffic, Port 443. SSL is terminated at the load balancer. This is what is known as OFFLOADING, it just means the SSL encryption is seen at the load balancer. Behind the load balancer, there is no encryption between it and the server. This allows the SNI hostname to be forwarded to the server, without it being in an encrypted form within the TCP packet.

5. Now lets install apache2 and configure some virtualhosts, at the most basic level. This is for an example and not a perfect setup

apt-get update
apt-get install httpd
vi /etc/apache2/httpd.conf



ServerName example.com
Documentroot /var/www/example.com/html




ServerName domain.com
Documentroot /var/www/domain.com/html


mkdir -p /var/www/domain.com/html
mkdir -p /var/www/example.com/html
echo 'example.com page body testing' > /var/www/example.com/html/index.html
echo 'domain.com page body testing' > /var/www/domain.com/html/index.html
vi /etc/apache2/apache2.conf

add one line in the file like:

Include /etc/apache2/httpd.conf

I just like to configure apache2 this way.

/etc/init.d/apache2 restart

6. Confirm both websites are working thru LB with SNI

# Curl domain
$ curl domain.com
domain.com page body testing

# curl domain 2
$ curl example.com
example.com page body testing

# curl IP address
curl https://194.213.79.117
someotherdefaultpage

# what happened when curling the IP address? Well..There was no TCP servername/hostname forwarded in the header for SNI support to detect the domain x-forwarded-for

# Lets provide the header
curl https://194.213.79.117 -H "host: example.com"

Testing SSL on the hostnames

openssl s_client -connect domain.com:443
openssl s_client -connect domain.com:443 -host domain.com
openssl s_client -connect domain.com:443 -servername domain.com

List all Cloud Server Details thru the API

Well, this one is a bit cheeky because I borrowed it from a colleague of mine David Coon. Thanks David, I appreciate your assistance!

#!/bin/bash


auth() {
    read -p "What is your Account Number: " ddi
    read -p "Whats your username:" username    
    read -p "Whats your APIkey:" APIkey
    read -p "Which Datacenter are your servers in? " dc
}

token() {
    
    token=`curl -s https://identity.api.rackspacecloud.com/v2.0/tokens -X POST \
    -d '{"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"'$username'", "apiKey":"'$APIkey'"}}}' \
    -H "Content-Type: application/json" | python -m json.tool  | sed -n '/expires/{n;p;}' |sed -e 's/^.*"id": "\(.*\)",/\1/'`
    echo "Your API Token is ---->  $token"
}

listservers() {
    curl -s -H "X-Auth-Token: $token" "https://$dc.servers.api.rackspacecloud.com/v2/$ddi/servers" | python -m json.tool
}

getservers() {
    read -p "What is the server id?" id
    curl -s -H "X-Auth-Token: $token" "https://$dc.servers.api.rackspacecloud.com/v2/$ddi/servers/$id" | python -m json.tool
}

auth
token
listservers
getservers

Testing the consistency of response time on a website

So, we had some customers today complaining about inconsistent page load times. So I taken a look at the hypervisor they were on and I could see it was really quite busy. In the sense that all 122GB of RAM available was being used by server instances. Ironically though it wasn’t that busy, but I live-migrated the customer anyway to a much quieter server, but the customer saw no change whatsoever.

In this case it indicated already that it wasn’t a network infrastructure or hardware issue and likely the increase in latency they saw over the last so many days was being caused by something else. Most likely the growing size of their database, not being reflected by their static amount of ram and the variables set for their tablesize cache, and etc in MySQL.

So my friend kindly put together this excellent oneliner. Check it out!

$ for i in $(seq 50); do curl -sL http://www.google.com/ -o /dev/null -w %{time_total}\\n; sleep 1; done
0.698
0.493
0.365
0.293
0.326
0.525
0.342
0.527
0.445
0.263
0.493

Pretty neat eh

Cloud Files Container Bulk Deletion Script written in BASH

So, we had a lot of customers asking for ways to delete all of their cloud files in a single container, instead of having to do manually. This is possible using the bulk delete function defined in the rackspace docs. Find below the steps required to do this.

Step 1: Make an auth.json file (for simplicity)

{
    "auth": {
        "RAX-KSKEY:apiKeyCredentials": {
            "username": "mycloudusername",
            "apiKey": "mycloudapikey"
        }
    }
}

It’s quite simple and nothing intimidating.

For step 2 I’m using an application called jq, to install it do


wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64
mv jq-linux64 /bin/jq
alias jq = '/bin/jq'

Now you can use jq at the commandline.

Step 2: Set variable called $TOKEN that can store the api token password output, the nice thing is there is no token stored in the script so its kind of secure

TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d @auth.json -H "Content-type: application/json" | jq .access.token.id | sed 's/"//g'`
echo $TOKEN

Step 3: Set a variable for the container name

# Container to Upload to
CONTAINER=meh2

Step 4: Populate a List of all the files in the $CONTAINER variable, in this case ‘meh2’.

# Populate File List
echo "Populating File List"
curl -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN" > filelist.txt

Step 5: Add container name to the file listing by rewriting the output file filelist.txt to a deletelist.txt

sed -e "s/^/\/$CONTAINER\//" <  filelist.txt > deletelist.txt

Step 6: Bulk Delete Files thru API

echo "Deleting Files.."
curl -i -v -XDELETE -H"x-auth-token: $TOKEN" https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567\?bulk-delete -T ./deletelist.txt

Step 7: Confirm the deletion success

# Confirm Deleted
echo "Confirming Deleted in $CONTAINER.."
curl -i -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN"

The completed script looks like this:


 Mass Delete Container

# Get Token

TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d @auth.json -H "Content-type: application/json" | jq .access.token.id | sed 's/"//g'`
echo $TOKEN

# Container to Upload to
CONTAINER=meh2


# Populate File List
echo "Populating File List"
curl -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN" > filelist.txt


# Add Container Prefix
echo "Adding Container Prefix.."
sed -e "s/^/\/$CONTAINER\//" <  filelist.txt > deletelist.txt


# Delete Files
echo "Deleting Files.."
curl -i -v -XDELETE -H"x-auth-token: $TOKEN" https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567\?bulk-delete -T ./deletelist.txt

# Confirm Deleted
echo "Confirming Deleted in $CONTAINER.."
curl -i -X GET https://storage101.lon3.clouddrive.com/v1/MossoCloudFS_10045567/meh2 -H "X-Auth-Token: $TOKEN"

Pretty simple!

Running it..

* About to connect() to storage101.lon3.clouddrive.com port 443 (#0)
* Trying 2a00:1a48:7900::100...
* Connected to storage101.lon3.clouddrive.com (2a00:1a48:7900::100) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject: CN=storage101.lon3.clouddrive.com
* start date: May 18 00:00:00 2015 GMT
* expire date: Nov 17 23:59:59 2016 GMT
* common name: storage101.lon3.clouddrive.com
* issuer: CN=thawte DV SSL CA - G2,OU=Domain Validated SSL,O="thawte, Inc.",C=US
> DELETE /v1/MossoCloudFS_10045567?bulk-delete HTTP/1.1
> User-Agent: curl/7.29.0
> Host: storage101.lon3.clouddrive.com
> Accept: */*
> x-auth-token: AAA7uz-F91SDsaMOCKTOKEN-gOLeB5bbffh8GBGwAPl9F313Pcy4Xg_zP8jtgZolMOudXhsZh-nh9xjBbOfVssaSx_shBMqkxIEEgW1zt8xESJbZLIsvBTNzfVBlTitbUS4RarUOiXEw
> Content-Length: 515
> Expect: 100-continue
>
< HTTP/1.1 100 Continue HTTP/1.1 100 Continue * We are completely uploaded and fine < HTTP/1.1 200 OK HTTP/1.1 200 OK < Content-Type: text/plain Content-Type: text/plain < X-Trans-Id: tx010194ea9a104443b89bb-00161f7f1dlon3 X-Trans-Id: tx010194ea9a104443b89bb-001611f7f1dlon3 < Date: Thu, 15 Oct 2015 10:25:35 GMT Date: Thu, 15 Oct 2015 10:25:35 GMT < Transfer-Encoding: chunked Transfer-Encoding: chunked < Number Deleted: 44 Number Not Found: 0 Response Body: Response Status: 200 OK Errors: * Connection #0 to host storage101.lon3.clouddrive.com left intact