Checking a crashing / unstable Server

So, what to do if a customer has a server which is frequently crashing? Well, important things to check is open files, to look at all the users in /etc/passwd and substitute the username to check each of their cron jobs and check the files which are open using the apache process id.

This will help rule out a lot of common issues being seen on servers, and may even be of use for checking whether the server has been hacked.

netstat -ntulp
for i in $(awk -F: '{print $1}' /etc/passwd); do crontab -l -u $i ;done
lsof -p $(cat /var/run/apache2/apache2.pid) | grep log

This is a nice one liner, thanks to my colleague Aaron for providing this, well, actually it was so awesome I stole it 😛

Perform a traceroute thru the Rackspace monitoring API

So, I was thinking about the Rackspace traceroute monitoring API and wondering what I could do with it, when I come across this gem

/monitoring_zones/mzsyd/traceroute

Well what is it you ask. Well it’s an API path for performing a traceroute on the 6 different region endpoints. This means you can use an API call to run traceroutes (for free!) thru the Rackspace cloud monitoring API. This would be pretty handy at testing connectivity around the world to your chosen destination from each datacentre. Handy Andy.

Then you ask what does the mzsyd mean? That’s a region ID: Let’s see about putting together a script to list the region ID’s we can run the traceroutes on first of all:

File: list-monitoring-zones.sh

!/bin/bash

USERNAME='mycloudusername'
APIKEY='mycloudapikey'
ACCOUNTNUMBER='10010110'
API_ENDPOINT="https://monitoring.api.rackspacecloud.com/v1.0/$ACCOUNTNUMBER"


TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d '{ "auth":{"RAX-KSKEY:apiKeyCredentials": { "username":"'$USERNAME'", "apiKey": "'$APIKEY'" }} }' -H "Content-type: application/json" |  python -mjson.tool | grep -A5 token | grep id | cut -d '"' -f4`




curl -s -v  \
-H "X-Auth-Token: $TOKEN"  \
-H "X-Project-Id: $ACCOUNTNUMBER" \
-H "Accept: application/json"  \
-X GET  \
"$API_ENDPOINT/monitoring_zones"

Lets take a look at the response when I run this monitoring zone list.


chmod +x list-monitoring-zones.sh
./list-monitoring-zones.sh

Response

< Content-Type: application/json; charset=UTF-8
< Via: 1.1 Repose (Repose/7.3.0.0)
< Vary: Accept-Encoding
< X-LB: api1.dfw1.prod.cm.k1k.me
< Transfer-Encoding: chunked
<
{
    "values": [
        {
            "id": "mzdfw",
            "label": "Dallas Fort Worth (DFW)",
            "country_code": "US",
            "source_ips": [
                "2001:4800:7902:0001::/64",
                "50.56.142.128/26"
            ]
        },
        {
            "id": "mzhkg",
            "label": "Hong Kong (HKG)",
            "country_code": "HK",
            "source_ips": [
                "2401:1800:7902:0001::/64",
                "180.150.149.64/26"
            ]
        },
        {
            "id": "mziad",
            "label": "Northern Virginia (IAD)",
            "country_code": "US",
            "source_ips": [
                "2001:4802:7902:0001::/64",
                "69.20.52.192/26"
            ]
        },
        {
            "id": "mzlon",
            "label": "London (LON)",
            "country_code": "GB",
            "source_ips": [
                "2a00:1a48:7902:0001::/64",
                "78.136.44.0/26"
            ]
        },
        {
            "id": "mzord",
            "label": "Chicago (ORD)",
            "country_code": "US",
            "source_ips": [
                "2001:4801:7902:0001::/64",
                "50.57.61.0/26"
            ]
        },
        {
            "id": "mzsyd",
            "label": "Sydney (SYD)",
            "country_code": "AU",
            "source_ips": [
                "2401:1801:7902:0001::/64",
                "119.9.5.0/26"
            ]
        }
    ],
    "metadata": {
        "count": 6,
        "limit": 100,
        "marker": null,
        "next_marker": null,
        "next_href": null
    }
* Connection #0 to host monitoring.api.rackspacecloud.com left intact

We can see many zones available to run our traceroute to;

id 'mzsyd' for Sydney SYD.
id 'mzdfw' for Dallas Fort Worth DFW
id 'mzhkg' for Hong Kong HKG
id 'mziad' for Northern Viginia IAD
id 'mzord' for Chicago ORD
id 'mzlon' for London LON

So now I know what the zone id's are, as defined above here. Now time to use them and run a traceroute to haxed.me.uk. Lets see;

File: perform-traceroute-from-monitoring-zone.sh

!/bin/bash

USERNAME='mycloudusernamehere'
APIKEY='apikeyhere'
ACCOUNTNUMBER=10010110
API_ENDPOINT="https://monitoring.api.rackspacecloud.com/v1.0/$ACCOUNTNUMBER"



TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d '{ "auth":{"RAX-KSKEY:apiKeyCredentials": { "username":"'$USERNAME'", "apiKey": "'$APIKEY'" }} }' -H "Content-type: application/json" |  python -mjson.tool | grep -A5 token | grep id | cut -d '"' -f4`




curl -s -v  \
-H "X-Auth-Token: $TOKEN"  \
-H "X-Project-Id: $ACCOUNTNUMBER" \
-H "Accept: application/json"  \
-d @ip.json -H "content-type: application/json" -X POST  \
"$API_ENDPOINT/monitoring_zones/mzsyd/traceroute"

You also need the ip.json file. It's easy to make, put it in the same dir as the shellscript.

File: ip.json

{
        "target":               "haxed.me.uk",
        "target_resolver":      "IPv4"
}

We're going to refer to ip.json file which contains our destination data. You can do this with IPv6 IP's too if you wanted! That is pretty cool!
It is possible to do this without including the file, and actually just pass the json directly, with -d { "target": "haxed.me.uk", "target_resolver": "IPv4"} , but lets do it properly 😀


chmod +x perform-traceroute-from-monitoring-zone.sh
./perform-traceroute-from-monitoring-zone

the response, a nice traceroute of course from syd to my lon server.

Response

 Accept: application/json
> content-type: application/json
> Content-Length: 55
>
* upload completely sent off: 55 out of 55 bytes
< HTTP/1.1 200 OK
< Date: Wed, 13 Jan 2016 11:19:14 GMT
< Server: Jetty(9.2.z-SNAPSHOT)
< X-RateLimit-Type: traceroute
< X-RateLimit-Remaining: 296
< X-RateLimit-Window: 24 hours
< x-trans-id: eyJyZXF1ZXN0SWQiOiI5MTNhNTY1Mi05ODAyLTQ5MmQtOTAwYS05NDU1M2ZhNDJmNzUiLCJvcmlnaW4
< X-RateLimit-Limit: 300
< X-Response-Id: .rh-TI8E.h-api1.ord1.prod.cm.k1k.me.r-4RFTh9up.c-28452540.ts-1452683954386.v-91eaf0a
< Content-Type: application/json; charset=UTF-8
< Via: 1.1 Repose (Repose/7.3.0.0)
< Vary: Accept-Encoding
< X-LB: api0.ord1.prod.cm.k1k.me
< Transfer-Encoding: chunked
<
{
    "result": [
        {
            "ip": "119.9.5.2",
            "hostname": null,
            "number": 1,
            "rtts": [
                0.421,
                0.384,
                0.442,
                0.457,
                0.455
            ]
        },
        {
            "ip": "119.9.0.30",
            "hostname": null,
            "number": 2,
            "rtts": [
                1.015,
                0.872,
                0.817,
                1.014,
                0.926
            ]
        },
        {
            "ip": "119.9.0.109",
            "hostname": null,
            "number": 3,
            "rtts": [
                1.203,
                1.179,
                1.185,
                1.232,
                1.182
            ]
        },
        {
            "ip": "202.84.223.2",
            "hostname": null,
            "number": 4,
            "rtts": [
                3.53,
                5.301,
                3.975,
                5.772,
                3.804
            ]
        },
        {
            "ip": "202.84.223.1",
            "hostname": null,
            "number": 5,
            "rtts": [
                3.437,
                3.522,
                2.837,
                4.274,
                2.805
            ]
        },
        {
            "ip": "202.84.140.206",
            "hostname": null,
            "number": 6,
            "rtts": [
                141.198,
                140.746,
                143.871,
                140.987,
                141.545
            ]
        },
        {
            "ip": "202.40.149.238",
            "hostname": null,
            "number": 7,
            "rtts": [
                254.354,
                175.559,
                176.787,
                176.701,
                175.634
            ]
        },
        {
            "ip": "134.159.63.18",
            "hostname": null,
            "number": 8,
            "rtts": [
                175.302,
                175.299,
                175.183,
                175.146,
                175.149
            ]
        },
        {
            "ip": "64.125.26.6",
            "hostname": null,
            "number": 9,
            "rtts": [
                175.395,
                175.408,
                175.469,
                175.49,
                175.475
            ]
        },
        {
            "ip": "64.125.30.184",
            "hostname": null,
            "number": 10,
            "rtts": [
                285.818,
                285.872,
                285.801,
                285.835,
                285.887
            ]
        },
        {
            "ip": "64.125.29.52",
            "hostname": null,
            "number": 11,
            "rtts": [
                285.864,
                285.938,
                285.826,
                285.922,
                303.125
            ]
        },
        {
            "ip": "64.125.28.98",
            "hostname": null,
            "number": 12,
            "rtts": [
                284.711,
                284.865,
                284.73,
                284.697,
                284.713
            ]
        },
        {
            "ip": "64.125.29.48",
            "hostname": null,
            "number": 13,
            "rtts": [
                287.341,
                310.82,
                287.33,
                287.359,
                287.455
            ]
        },
        {
            "ip": "64.125.29.130",
            "hostname": null,
            "number": 14,
            "rtts": [
                286.168,
                286.012,
                286.108,
                286.105,
                286.168
            ]
        },
        {
            "ip": "64.125.30.235",
            "hostname": null,
            "number": 15,
            "rtts": [
                284.61,
                284.681,
                284.667,
                284.892,
                286.069
            ]
        },
        {
            "ip": "64.125.20.97",
            "hostname": null,
            "number": 16,
            "rtts": [
                287.516,
                287.435,
                287.557,
                287.581,
                287.438
            ]
        },
        {
            "ip": "94.31.42.254",
            "hostname": null,
            "number": 17,
            "rtts": [
                288.156,
                288.019,
                288.034,
                288.08
            ]
        },
        {
            "ip": null,
            "hostname": null,
            "number": 18,
            "rtts": []
        },
        {
            "ip": "134.213.131.251",
            "hostname": null,
            "number": 19,
            "rtts": [
                292.687,
                293.72,
                295.335,
                293.981
            ]
        },
        {
            "ip": "162.13.232.1",
            "hostname": null,
            "number": 20,
            "rtts": [
                293.295,
                293.738,
                295.46,
                294.301
            ]
        },
        {
            "ip": "162.13.232.103",
            "hostname": null,
            "number": 21,
            "rtts": [
                294.733,
                294.996,
                298.884,
                295.056
            ]
        },
        {
            "ip": "162.13.136.211",
            "hostname": null,
            "number": 22,
            "rtts": [
                294.919,
                294.77,
                298.956,
                296.481
            ]
        }
    ]
* Connection #0 to host monitoring.api.rackspacecloud.com left intact

This is pretty cool. If we want to run a traceroute from lets say chicago, we just swap out the 'mzsyd' variable to show 'mziad', wow thats simple 🙂

Custom Error pages for Linux Apache2 and the Rackspace Load Balancer

This article describes how to configure custom error pages for Apache2 and Zeus Load Balancer thru Rackspace API.

I have noticed that every now and then this question comes up. For instance, for most people there will define the errorpage within apache2, but if your not able to do that, and want a more helpful error page to be custom set, perhaps in the same xhtml layout as on your website, then custom error page for Load Balancer may be useful to you.

In apache2 this is traditionally set using the ErrorDocument directive.

The most common error pages are:

400 — Bad Request
The server did not understand the request due to bad syntax.

401 — Unauthorized
The visitor must by authorized (e.g., have a password) to access the page.

403 — Forbidden
The server understood the request but was unable to execute it. This could be due to an incorrect username and/or password or the server requires different input.

404 — Not Found
The server cannot find a matching URL.

500 — Internal Server Error
The server encountered an unexpected condition which prevented it from fulfilling the request.

In your apache2 httpd.conf you will need to add some directives to configure custom error pages. These custom error-pages will be forwarded to the Load Balancer in the case you have servers behind the Load Balancer, and the cloud server behind it encounters an error, it will be sent to the LB and then relayed to the customer, obviously. Instead of being sent directly to the customer as with a traditional apache2 webserver. To form your error page directive edit your httpd conf file either in sites-enabled or httpd.conf or similar, and merely add the following:

ErrorDocument 404 /my404page.html
ErrorDocument 500 /myphppage.php

ErrorDocument 403 /my-custom-forbidden-error-page.html
ErrorDocument 400 /my-bad-request-error-page.html

Then ensure that the error page defined (i.e. my404page.html my-custom-forbidden-error-page.html is placed in the correct directory i.e. the websites root /).

I.e. if your Documentroot is /var/www/html then your my404page.html should go in there.

For some people, for example who are using Load Balancer ACL, the blocked customer/client/visitor won’t be able to see/contact your server when it’s added to the LB DENY list. Therefore you might want to setup another error page on the LB to help customers that are accidentally/wrongly blocked on what process to carry out to remove the block, i.e. contact admin email, etc, etc.

To set custom error page on the load balancer you can use the API like so. You will need two files to achieve this, which will be detailed below:

errorpage.json (file)

{"errorpage":
{"content":"\n\n   Warning- Your IP has been blocked by the Load Balancer ACL, or there is an error contacting the servers behind the load balancer. Please contact [email protected] if you believe you have been blocked in error. \n\n"}
}

customerror.sh (file)

#!/bin/bash
# Make sure to set your customer account number, if you don't know what it is, it's the number that appears in the URL
# after logging in to mycloud.rackspace.com
# ALSO set your username and apikey, visible from the 'account settings' part of rackspace mycloud control panel

USERNAME='mycloudusernamehere'
APIKEY='mycloudapikeygoeshere'
ACCOUNTNUMBER='1001100'

# The load balancer ID, shown in the load balancer details page in mycloud contrl panel
LOADBALANCERID='157089'


API_ENDPOINT="https://lon.loadbalancers.api.rackspacecloud.com/v1.0/$ACCOUNTNUMBER"


# Store the API auth password response in the variable TOKEN

TOKEN=`curl https://identity.api.rackspacecloud.com/v2.0/tokens -X POST -d '{ "auth":{"RAX-KSKEY:apiKeyCredentials": { "username":"'$USERNAME'", "apiKey": "'$APIKEY'" }} }' -H "Content-type: application/json" |  python -mjson.tool | grep -A5 token | grep id | cut -d '"' -f4`

# Execute the API call to update the load balancer error page, submitting as a file the errorpage.json to be used

curl -s -v  \
-H "X-Auth-Token: $TOKEN"  \
-H "X-Project-Id: $ACCOUNTNUMBER" \
-H "Accept: application/json"  \
-d @errorpage.json -X PUT -H "content-type: application/json" \
"$API_ENDPOINT/loadbalancers/$LOADBALANCERID/errorpage"

That it, this concludes the discussion/tutorial on how to set basic error pages both in apache2 (without load balancer) , and with the load balancer only as described above. Please note, it appears you can only set a single error page for the load balancer, as opposed to one error page for each HTTP code.

If anyone see’s any differences or errors though, please let me know!

Best wishes,
Adam

Securing your segmented Network, and interpretation of tcpdump on dual NIC segmented network

Howdy! So, here is a real life example of some basic networking security analysis.

Without giving too much away about the way my own home network works. I have a basic firewall-like setup segmenting my red internet routers network offering from the nic port handling my local network. This offers a level of separation or isolation, that prevents any-old-packet reaching the local network. By default I am allowing services to be routed out to internets only when it is requested first by an application running on boxes inside the local network. If any external client attempts to connect to my router on a particular port without it being SYN ACK, then it won’t be accepted.

Another good thing is it’s easy to find out what destination and source is because eth1 shows me primarily all internet bound and internet from traffic, and the eth0 adapter shows me primarily all local traffic on it’s way to be internet bound thru the firewall and then out the other NIC.

Today I saw some worrying traffic coming thru on eth1, something called teamview;

Here is what it looked like:

# tcpdump -i eth1 | grep teamview
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
16:07:05.641711 IP firewall.home.50490 > server19703.teamviewer.com.https: Flags [P.], seq 2473010817:2473010841, ack 788873912, win 255, length 24
16:07:05.676803 IP server19703.teamviewer.com.https > firewall.home.50490: Flags [P.], seq 1:25, ack 24, win 251, length 24
16:07:05.891233 IP firewall.home.50490 > server19703.teamviewer.com.https: Flags [.], ack 25, win 255, length 0

As we can see the firewall just ‘dialed’ out to a remote server19703, and I am like ‘wtf’ is this? So I start to panic, then I run:

]# tcpdump -i eth0 | grep teamview
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:07:05.641686 IP 192.168.0.120.50490 > server19703.teamviewer.com.https: Flags [P.], seq 2473010817:2473010841, ack 788873912, win 255, length 24
16:07:05.676846 IP server19703.teamviewer.com.https > 192.168.0.120.50490: Flags [P.], seq 1:25, ack 24, win 251, length 24
16:07:05.891204 IP 192.168.0.120.50490 > server19703.teamviewer.com.https: Flags [.], ack 25, win 255, length 0

This allows me to see, what the nature of the request was, just before the firewall started to route it out on the eth1 adapter. As such it shows me that the local network machine devices ip address on eth0, which is 192.168.0.120, or in other words, my parentals new computer, specifically the one I bought them for christmas from ebay.

What does this mean? It means I’ll be paying a visit to their box to make sure that this is disabled.

Checking for Network packet Retransmission , troubleshooting network card & switches

So, you might want to test whether the NIC of your box is ‘bad’, one way to do this is looking at the retransmissions.

netstat -s | grep retransmits
   3535665 fast retransmits
   3920918 forward retransmits
   122319 retransmits in slow start
   3652 sack retransmits failed
netstat -s | grep transmit
    10512472 segments retransmited
    733 times recovered from packet loss due to fast retransmit
    Detected reordering 73 times using reno fast retransmit
    TCPLostRetransmit: 196
    400 timeouts after reno fast retransmit
    3535665 fast retransmits
    3920918 forward retransmits
    122319 retransmits in slow start
    13652 sack retransmits failed

This isn’t much use though, because you need to see how many total packets come in:

netstat -s | grep total
    23799703342 total packets received

It’s possible to get the full details with netstat -s , naturally.

Checking Forward and Reverse Connectivity of a Linux Server

A good friend of mine is to thank for this excellent pair of one liners. One is to be executed on source, and the other on destination target.

Testing forward route


# Machine Source
root@iup2-web01:/mnt/www# dd if=/dev/zero bs=1024K count=1024 | nc -v 10.181.164.100 23
Connection to 10.181.164.100 23 port [tcp/telnet] succeeded!
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 11.0009 s, 97.6 MB/s
---

# Machine Destination
root@iup2-nfs:~# nc -v -l 23 > /dev/null
Listening on [0.0.0.0] (family 0, port 23)
Connection from [10.181.162.15] port 23 [tcp/telnet] accepted (family 2, sport 54373)

However when we try this in reverse, we see a major degradation in the network speed. Here is test with 20MB transfer instead of 1000MB:

Testing Reverse Route

---

# Machine Source
root@iup2-nfs:~# dd if=/dev/zero bs=1024K count=20 | nc -v 10.181.162.15 23
Connection to 10.181.162.15 23 port [tcp/telnet] succeeded!
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 144.327 s, 145 kB/s
---
# Machine Destination
root@iup2-web01:/mnt/www#  nc -v -l 23 > /dev/null 
Listening on [0.0.0.0] (family 0, port 23)
Connection from [10.181.164.100] port 23 [tcp/telnet] accepted (family 2, sport 56072)

As we can see one of the machines has some difficulty. The issue at hand was that there was some problems with the virtual switch daemon on the hypervisor. Thanks to my friend Gospodin for documenting this one and sharing with me how he tested it,

PCI Compliance on Rackspace public cloud

So you wanna be PCI compliant on the public cloud? You server can’t be. But YOU CAN BE PCI COMPLIANT whilst still using your server on the cloud! By using a payment gateway with tokenisation like worldpay or paypal you won’t have to change all your infrastructure. Here is why:

The public cloud, because of the shared environment makes it more difficult to be PCI compliant than in dedicated solutions Rackspace offer, mainly because of the shared environment. Specifically, on the Load Balancers, all of the ciphers are shared, so removing TLS 1.0 for you would affect all other customers load balancers.

You could potentially run your own load balancer, and/or even if Rackspace were able to remove the TLS 1.0 cipher from the load balancer for you, because of the nature of the shared hypervisor, and the shared network it would be really difficult for your rackspace public cloud server to be PCI compliant on public cloud. For you to achieve this I would recommend considering private cloud or dedicated, as this would be practical to accommodate such requirements. It would however cost more though.

Just to re-emphasise, removing TLS 1.0 from the Load Balancer will not make you PCI compliant because of the nature of shared cloud infrastructure, one other important aspect of compliance is to ensure that payment information is tokenised. For instance one alternative that is available to you, so that you could continue to use the cloud as you are and be PCI compliant without making too many large changes, is to use a 3rd party payment gateway service, and tokenisation, that way you should satisfy the compliance, without having to move your website infrastructure. Worldpay and paypal are examples of payment gateways.

I hope this helps clarify PCI compliance on the public rackspace cloud, and how to do it right. Use a 3rd party internet gateway, if you really must keep your ecommerce store on the public cloud.

Deploying your own cloud API using Keystone Openstack

Just a quick one. There are a lot of things that aren’t complete, but this is mostly for my reference and to make writing an Ansible playbook massively easier of course!

For the full guide you will want the link at the bottom of the page.

Outlay

openstack-101-update-25-638

Operation

SCH_5002_V00_NUAC-Keystone

Deployment

# EPEL Not Needed for CENTOS 7 on RS Cloud, included for detail
yum install http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm -y

# Install Openstack Liberty repo
yum install centos-release-openstack-liberty

# Upgrade dist packages
yum upgrade -y

# Install openstack client
yum install python-openstackclient -y

# Update selinux policies for Openstack
yum install openstack-selinux -y

# Configure SQL
yum install mariadb mariadb-server MySQL-python -y

# Configure and enable mariadb bind and utf8 settings etc
vi /etc/my.cnf.d/mariadb_openstack.cnf

systemctl enable mariadb.service
systemctl start mariadb.service

# Prepare database privileges ____________TODO_______
# mysql_secure_installation _____TODO______


# Prepare mongodb nosqli set controller address, set start and enabled

yum install mongodb-server mongodb -y
vi /etc/mongod.conf


systemctl enable mongod.service
systemctl start mongod.servicei

# Queuing Install , enable start rabbitmq, add user and set permissions for openstack user
yum install rabbitmq-server -y
systemctl enable rabbitmq-server.service
systemctl start rabbitmq-server.service
rabbitmqctl add_user openstack somepasswordhere
rabbitmqctl set_permissions openstack ".*" ".*" ".*"

# Generate admin_token
openssl rand -hex 15

# Install openstack keystone, httpd and memcached, set to start, enable
yum install openstack-keystone httpd mod_wsgi memcached python-memcached -y
systemctl enable memcached.service
systemctl start memcached.service

# Complete Keystone [Default], [database] connection, [memcache] servers, [token] provider and driver = memcache [revoke] driver = sql [default] verbose = True
vi /etc/keystone/keystone.conf

# Populate the keystone database
su -s /bin/sh -c "keystone-manage db_sync" keystone

# (re)configure httpd
vi /etc/httpd/conf.d/wsgi-keystone.conf
systemctl enable httpd.service
systemctl start httpd.service

# Update environment variable exports for OS_TOKEN=admintoken, OS_URL=http://snetip:35357/v3 OS_IDENTITY_API_VERSION=3 and source it

vi .bash_profile
source .bash_profile

# Create Service entity and API endpoints
openstack service create   --name keystone --description "OpenStack Identity" identity



# API Endpoints
openstack endpoint create --region RegionOne identity public http://10.179.1.188:5000/v2.0
openstack endpoint create --region RegionOne identity internal http://10.179.1.188:5000/v2.0
openstack endpoint create --region RegionOne identity admin http://10.179.1.188:35357/v2.0

# Create project; admin
openstack project create --domain default   --description "Admin Project" admin

# Create admin user for project
openstack user create --domain default   --password-prompt admin


# Create admin's role
openstack role create adminn

# Add admin role to admin project & it's admin user
openstack role add --project admin --user admin admin

# Create Service Project

openstack project create --domain default   --description "Service Project" service

# Create demo project
openstack project create --domain default   --description "Demo Project" demo

# Create the demo user
openstack user create --domain default   --password-prompt demo

# and user role for demo user
openstack role create user

# Add the user role to the demo project and user
openstack role add --project demo --user demo user

# SKIPPED remove keystone-dist-paste.ini

# Unset the OS_TOKEN and OS_URL environment variables
unset OS_TOKEN OS_URL

# Request token for admin user
openstack --os-auth-url http://10.179.1.188:35357/v3  --os-project-domain-id default --os-user-domain-id default   --os-project-name demo --os-username demo --os-auth-type password   token issue

# Verify operation toadd check verification status function
touch demo-openrc.sh
touch admin-openrc.sh
cat /etc/keystone/keystone.conf | grep admin_token

# Test admin api credentials
source admin-openrc.sh
opentack token issue

# Test demo api credentials
source demo-openrc.sh
openstack token issue
http://docs.openstack.org/liberty/install-guide-rdo/environment.html

Using Python with nova-client to list servers

A customer came to me today complaining about his code not working. He’d forgot to include the ‘account-number’, also refered to as the project_id in openstack. Without it, your going to get HTTP 405, i.e. MethodNotAllowed: Method Not Allowed (HTTP 405)

from novaclient import client

nova = client.Client ("2" ,"username", "password", "account-number", "https://lon.identity.api.rackspacecloud.com/v2.0")

list = nova.servers.list()

print list

This does what it says on the tin, queries the API using nova python module to extract server list.

Fail2ban on CentOS 7 not working [and solution]

because configuration settings in fail2ban 0.9.0 having been completely re-factored, CentOS7 fail2ban hardening automation now is not safe by merely running an yum install fail2ban.

It will also apparently no longer work if you uncomment the sshd enabled jail in local.conf or jail.conf.

The newer re-factored configuration suggests to use a dedicated file for this to prevent being overwritten as I have now set in my /etc/fail2ban/jail.d/sshd.local

[sshd] enabled = true
port = ssh
#action = firewallcmd-ipset
logpath = %(sshd_log)s
maxretry = 5
bantime = 86400

Do note firewallcmd-ipset needs to be commented out or fail2ban will not start.

Once it has been configured like this, it is happy again. And worked straight away banning my home IP! Whilst before it was quite literally failing to ban :- )

Of course you might need to install it first:

yum install -y epel-release
yum install -y fail2ban fail2ban-systemd

You might also want to start fail2ban, and also set it to run on startup:

systemctl enable fail2ban
systemctl start fail2ban

If you run selinux, then you’ll need (running this command may have security implications)

yum update selinux-policy*