Automatically loading iptables in Debian and Ubuntu Linux

Debian doesn’t have an automatic way of loading up iptables, so if you use Debian, or Ubuntu you might be wondering why everytime you restart you lose your iptables rules. And if you got that far your probably wondering where the default iptables file is stored. Here is news, there is no file store! You have to make it! Here is how I did it.

1. Save your existing firewall rules to a file for later execution

iptables-save > /etc/firewall.conf 

2. Configure a firewall load command in if-up.d network startup script folder.

 
iptables-restore < /etc/firewall.conf

3. Make sure that the script you've put in the if up auto configuration is executable

 
chmod +x /etc/network/if-up.d/iptables

4. Now when you add new iptables rules you can run a save command

 
iptables-save > /etc/firewall.conf

or you can just edit the startup file, which will become active next reboot

vi /etc/firewall.conf 

This is one of the most elegant and simple ways to configure iptables, thanks to Major Hayden, a fellow Rackspace employee for this tip.

Viewing your servers processor details

Last week, I had an internal phone call, where a customer at their end of the line was asking for the specific hardware used in our openstack cloud solution. I told them that although there were no statistics available, and that most of the servers had a similar performance hardware configuration that the type of processors and their information can still be seen from the Guest virtual Machine. This means if the customer is running Linux they can run a command to return that information, and if they are running Windows they can run an application like CPU-Z to read the type of memory, processor etc being used.

Of course the command was simple

cat /proc/cpuinfo 

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4521.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           L5520  @ 2.27GHz
stepping	: 5
cpu MHz		: 2261.060
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4522.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

Booting a Virtual Machine using the Openstack API using supernova

It’s pretty damn simple to boot a virtual machine using the Openstack API. In this case I want to boot an image that is the same identical reference as the base image that a customer of ours is using that is getting some bad Cloud Block Storage performance. Being able to replicate the circumstances precisely is important when trying to recreate the symptoms of the customer. Configuring a new boot is really easy to do thru the API using supernova.

supernova customer boot testing-cbs-lon --image 9g46rb-79d6-45eb-9505-fb0bf31556d7 --flavor performance1-4

The above command is using my ‘customer’ auth user configuration (single API user, as opposed to admin API). I have asked API to boot a performance server with 4GB ‘performance1-4’ Using the image id ‘9g46rb-79d6-45eb-9505-fb0bf31556d7’ which is a customers image reference running windows. I tell the API I want to call the server testing-cbs-lon, and here is what I get back after running it


+--------------------------------------+-------------------------------------------------------------------+
| Property                             | Value                                                             |
+--------------------------------------+-------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                            |
| OS-EXT-STS:power_state               | 0                                                                 |
| OS-EXT-STS:task_state                | -                                                                 |
| OS-EXT-STS:vm_state                  | building                                                          |
| RAX-PUBLIC-IP-ZONE-ID:publicIPZoneId | e1cc4036fc52d50ca8ac2e90ff57e6b8c38d850bf6e3bd608e800971          |
| accessIPv4                           |                                                                   |
| accessIPv6                           |                                                                   |
| adminPass                            | censoredforsecurity                                               |
| config_drive                         |                                                                   |
| created                              | 2015-09-17T08:23:25Z                                              |
| flavor                               | 4 GB Performance (performance1-4)                                 |
| hostId                               |                                                                   |
| id                                   | d23cc039-ecc7-442a-81ad-b6bbb7a8c8e9                              |
| image                                | Windows Server 2008 R2 SP1 (8f39aeb0-79d6-45eb-9505-fb0bf31556d7) |
| key_name                             | -                                                                 |
| metadata                             | {}                                                                |
| name                                 | testing-cbs-lon                                                   |
| progress                             | 0                                                                 |
| status                               | BUILD                                                             |
| tenant_id                            | 10045567                                                          |
| updated                              | 2015-09-17T08:23:26Z                                              |
| user_id                              | 05b18e859cad42bb9a5a35ad0a6fba2f                                  |
+--------------------------------------+-------------------------------------------------------------------+

So it’s building, lets presume I want to check on the status of the server now. The id of the server, we can see is ‘d23cc039-ecc7-442a-81ad-b6bbb7a8c8e9’. So lets ask supernova to show the details for this server again for us.

supernova customer show d23cc039-ecc7-442a-81ad-b6bbb7a8c8e9
[SUPERNOVA] Running nova against customer...
+--------------------------------------+-------------------------------------------------------------------+
| Property                             | Value                                                             |
+--------------------------------------+-------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                            |
| OS-EXT-STS:power_state               | 0                                                                 |
| OS-EXT-STS:task_state                | spawning                                                          |
| OS-EXT-STS:vm_state                  | building                                                          |
| RAX-PUBLIC-IP-ZONE-ID:publicIPZoneId | e1cc4036fc52d50ca8ac2e90ff57e6b8c38d850bf6e3bd608e800971          |
| accessIPv4                           |                                                                   |
| accessIPv6                           |                                                                   |
| config_drive                         |                                                                   |
| created                              | 2015-09-17T08:23:25Z                                              |
| flavor                               | 4 GB Performance (performance1-4)                                 |
| hostId                               | d9a81c37f8f8aba51cd30a04151d99d2bdd60ad4504d4c95f4a3f609          |
| id                                   | d23cc039-ecc7-442a-81ad-b6bbb7a8c8e9                              |
| image                                | Windows Server 2008 R2 SP1 (8f39aeb0-79d6-45eb-9505-fb0bf31556d7) |
| key_name                             | -                                                                 |
| metadata                             | {}                                                                |
| name                                 | testing-cbs-lon                                                   |
| private network                      | 10.1.1.1                                                          |
| progress                             | 80                                                                |
| public network                       | 134.1.1.1     , 2a00:1a48:7810:101:be76:4eff:fe08:1f5d            |
| status                               | BUILD                                                             |
| tenant_id                            | 10045567                                                          |
| updated                              | 2015-09-17T08:28:28Z                                              |
| user_id                              | 05b18e859cad42bb9a5a35ad0a6fba2f                                  |
+--------------------------------------+-------------------------------------------------------------------+

As we can see the server is still building but it has started to get it’s network added.

If I wanted to see all the different flavors available (GB ram , disk, etc) we could run a

supernova customer flavor-list
[SUPERNOVA] Running nova against customer...
+------------------+-------------------------+-----------+------+-----------+------+-------+-------------+-----------+
| ID               | Name                    | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+------------------+-------------------------+-----------+------+-----------+------+-------+-------------+-----------+
| 2                | 512MB Standard Instance | 512       | 20   | 0         | 512  | 1     | 80.0        | N/A       |
| 3                | 1GB Standard Instance   | 1024      | 40   | 0         | 1024 | 1     | 120.0       | N/A       |
| 4                | 2GB Standard Instance   | 2048      | 80   | 0         | 2048 | 2     | 240.0       | N/A       |
| 5                | 4GB Standard Instance   | 4096      | 160  | 0         | 2048 | 2     | 400.0       | N/A       |
| 6                | 8GB Standard Instance   | 8192      | 320  | 0         | 2048 | 4     | 600.0       | N/A       |
| 7                | 15GB Standard Instance  | 15360     | 620  | 0         | 2048 | 6     | 800.0       | N/A       |
| 8                | 30GB Standard Instance  | 30720     | 1200 | 0         | 2048 | 8     | 1200.0      | N/A       |
| compute1-15      | 15 GB Compute v1        | 15360     | 0    | 0         |      | 8     | 1250.0      | N/A       |
| compute1-30      | 30 GB Compute v1        | 30720     | 0    | 0         |      | 16    | 2500.0      | N/A       |
| compute1-4       | 3.75 GB Compute v1      | 3840      | 0    | 0         |      | 2     | 312.5       | N/A       |
| compute1-60      | 60 GB Compute v1        | 61440     | 0    | 0         |      | 32    | 5000.0      | N/A       |
| compute1-8       | 7.5 GB Compute v1       | 7680      | 0    | 0         |      | 4     | 625.0       | N/A       |
| general1-1       | 1 GB General Purpose v1 | 1024      | 20   | 0         |      | 1     | 200.0       | N/A       |
| general1-2       | 2 GB General Purpose v1 | 2048      | 40   | 0         |      | 2     | 400.0       | N/A       |
| general1-4       | 4 GB General Purpose v1 | 4096      | 80   | 0         |      | 4     | 800.0       | N/A       |
| general1-8       | 8 GB General Purpose v1 | 8192      | 160  | 0         |      | 8     | 1600.0      | N/A       |
| io1-120          | 120 GB I/O v1           | 122880    | 40   | 1200      |      | 32    | 10000.0     | N/A       |
| io1-15           | 15 GB I/O v1            | 15360     | 40   | 150       |      | 4     | 1250.0      | N/A       |
| io1-30           | 30 GB I/O v1            | 30720     | 40   | 300       |      | 8     | 2500.0      | N/A       |
| io1-60           | 60 GB I/O v1            | 61440     | 40   | 600       |      | 16    | 5000.0      | N/A       |
| io1-90           | 90 GB I/O v1            | 92160     | 40   | 900       |      | 24    | 7500.0      | N/A       |
| memory1-120      | 120 GB Memory v1        | 122880    | 0    | 0         |      | 16    | 5000.0      | N/A       |
| memory1-15       | 15 GB Memory v1         | 15360     | 0    | 0         |      | 2     | 625.0       | N/A       |
| memory1-240      | 240 GB Memory v1        | 245760    | 0    | 0         |      | 32    | 10000.0     | N/A       |
| memory1-30       | 30 GB Memory v1         | 30720     | 0    | 0         |      | 4     | 1250.0      | N/A       |
| memory1-60       | 60 GB Memory v1         | 61440     | 0    | 0         |      | 8     | 2500.0      | N/A       |
| performance1-1   | 1 GB Performance        | 1024      | 20   | 0         |      | 1     | 200.0       | N/A       |
| performance1-2   | 2 GB Performance        | 2048      | 40   | 20        |      | 2     | 400.0       | N/A       |
| performance1-4   | 4 GB Performance        | 4096      | 40   | 40        |      | 4     | 800.0       | N/A       |
| performance1-8   | 8 GB Performance        | 8192      | 40   | 80        |      | 8     | 1600.0      | N/A       |
| performance2-120 | 120 GB Performance      | 122880    | 40   | 1200      |      | 32    | 10000.0     | N/A       |
| performance2-15  | 15 GB Performance       | 15360     | 40   | 150       |      | 4     | 1250.0      | N/A       |
| performance2-30  | 30 GB Performance       | 30720     | 40   | 300       |      | 8     | 2500.0      | N/A       |
| performance2-60  | 60 GB Performance       | 61440     | 40   | 600       |      | 16    | 5000.0      | N/A       |
| performance2-90  | 90 GB Performance       | 92160     | 40   | 900       |      | 24    | 7500.0      | N/A       |
+------------------+-------------------------+-----------+------+-----------+------+-------+-------------+-----------+

On the left are the names to use for the first command we entered. So we can spin up a whole host of different servers. If we wanted to know details about a specific flavor, then we can run a flavor-show command like so:

 
[SUPERNOVA] Running nova against customer...
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property                          | Value                                                                                                                                                            |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-EXT-DATA:ephemeral         | 900                                                                                                                                                              |
| OS-FLV-WITH-EXT-SPECS:extra_specs | {"number_of_data_disks": "3", "resize_policy_class": "performance_flavor", "class": "performance2", "disk_io_index": "70", "policy_class": "performance_flavor"} |
| disk                              | 40                                                                                                                                                               |
| extra_specs                       | {"number_of_data_disks": "3", "resize_policy_class": "performance_flavor", "class": "performance2", "disk_io_index": "70", "policy_class": "performance_flavor"} |
| id                                | performance2-90                                                                                                                                                  |
| name                              | 90 GB Performance                                                                                                                                                |
| ram                               | 92160                                                                                                                                                            |
| rxtx_factor                       | 7500.0                                                                                                                                                           |
| swap                              |                                                                                                                                                                  |
| vcpus                             | 24                                                                                                                                                               |
+-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+

A lot of information about the avaialble ram, 90GB, The primary disk and ephemeral (data) disk are shown. This is handy way to get information about virtualised servers, and ends this brief tutorial on using the rackspace openstack API with supernova.

Live Migrating a server with the openstack API

Yesterday was a very long night, I didn’t leave work till 9:30pm. It was because I was trying to figure out how to live migrate a cloud server from one hypervisor to another with a targeted destination. This ensures that any customer using the openstack for cloud reaches a hypervisor in the same cell:

 
supernova lon live-migration --block-migrate uuidgoeshere c-10-1-1-1

Where ‘uuidgoeshere’ is the uuid of the server to migrate, and c-10-1-1-1 is the target of the hypervisor. It was as simple as that, during the live migration, only a few packets are dropped by the server. This ensures very good continuity and minimizes the disruption that would have otherwise been experienced had you imaged, and then rebuilt the server that way! In our case, the lon part is just the configuration file used by supernova. supernova is a wrapper for nova.

Using CURL to get Autoscale groups via Openstack API

So it has been a little while since I last posted something. Recently I have been learning the openstack API in cloud environments at Rackspace. We had a customer who wanted to check the servers in an autoscaling group. This is really easy and API is not as difficult as it may sound, and it was simply by running a:

curl -X GET "X-Auth-Token: $token" https://lon.autoscale.api.rackspacecloud.com/v1.0/$ddi/groups/$uuid | python -mjson.tool

What the above command does is simply provide the header X-Auth-Token, which is the users API Key from the control panel, or an impersonation key. You can set the $token, $ddi and $uuid variables used by the command, where the $ddi is the Rackspace customer account number and $uuid is the id of the autoscaler.

All the API calls are pretty much as simple as that! Nothing too crazy hard, simples!

Mounting A disk in Linux Operating Systems

So, apparently, some people aren’t sure how to properly add a new disk to a Linux server. Well, it’s pretty simple if you are using something like Cloud Block Storage and have ondemand. But even if you don’t, after adding the disk into your Linux machine, here is the process of going about adding a standard disk, fdisking (partitioning), formating (mkfs), and the mounting process itself, including that in fstab.

Step 1. List the disk’s on the box

 
$ fdisk -l

Device     Boot Start      End  Sectors Size Id Type
/dev/xvda1 *     2048 41940991 41938944  20G 83 Linux

Disk /dev/xvdb: 75 GiB, 80530636800 bytes, 157286400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

As we can see the new 75GiB drive I added hasn’t any file system on it yet.

Step 2. Let’s start partitioning the disk

$ fdisk /dev/xvdb

Device     Boot Start      End  Sectors Size Id Type
/dev/xvda1 *     2048 41940991 41938944  20G 83 Linux

Disk /dev/xvdb: 75 GiB, 80530636800 bytes, 157286400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Step 3. Let’s create a new partition using ‘n’ key, and then let’s state we want a primary partition using ‘p’.
Step 4: Let’s set the first and last sectors of the disk (you can type enter and the machine will normally chose something sane for you). It’s also possible to add multiple partitions of a single device, but for this tutorial we won’t be covering that.


Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)

Select (default p): p

Partition number (1-4, default 1): 1
First sector (2048-157286399, default 2048): 
Last sector, +sectors or +size{K,M,G,T,P} (2048-157286399, default 157286399): 

Created a new partition 1 of type 'Linux' and of size 75 GiB.

Step 4. Write the disk using the ‘w’ key, and then after it’s finished confirm with fdisk that new partition is present.


Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

root@tesladump:/home# fdisk -l

Disk /dev/xvda: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0c708d98

Device     Boot Start      End  Sectors Size Id Type
/dev/xvda1 *     2048 41940991 41938944  20G 83 Linux

Disk /dev/xvdb: 75 GiB, 80530636800 bytes, 157286400 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x07c2e0a6

Device     Boot Start       End   Sectors Size Id Type
/dev/xvdb1       2048 157286399 157284352  75G 83 Linux

Now the disk has been partitioned.

Step 5. So lets create an (EXT3) file system on the device.


$ mkfs -t ext3 /dev/xvdb1

mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 19660544 4k blocks and 4915200 inodes
Filesystem UUID: c717ddbd-d5c9-4bb1-a8af-b521d38cbb14
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

So that is the file system done. Now all we need to do is mount it. We can do that thru the /etc/fstab file, and also using the mount -a command.

Step 6. Simply we edit our /etc/fstab to accommodate the new disk

 

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
#                
/dev/xvda1      /               ext3    errors=remount-ro,noatime,barrier=0 0       1


/dev/xvdb1      /home/thetesladump ext3 defaults,noatime,nofail 0    0

Here we choose to mount the partition1 /dev/xvdb (/dev/xvdb1) on the symlink directory /home/thetesladump . A little FTP I temporarily wanted to host.

So we setup the new 75Gig cloud block device to be mounted at the ftp user thetesladump’s root home user directory. All ready to go. wooo.

Step 7. Complete hte process by running a mount -a

 

mount -a

Step 8. Confirm your disks are mounted where you wanted


root@tesladump:/home/theteslasociety# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       20G  1.1G   18G   6% /
udev             10M     0   10M   0% /dev
tmpfs           199M   21M  179M  11% /run
tmpfs           498M     0  498M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           498M     0  498M   0% /sys/fs/cgroup
/dev/xvdb1       74G  178M   70G   1% /home/thetesladump

Upgrading PHP 5.3.3 to 5.3.17+ or 5.4.44 in CENTOS6

A customer of ours was looking to upgrade their php to 5.3.17+ following the guide:
http://www.rackspace.com/knowledge_center/article/how-to-installupgrade-php-53-for-centos-5x

Unfortunately this guide is specifically tailored for CentOS 5 instead of CentOS 6, which really broke the customers dependencies.

when running the command ‘yum upgrade php’ there were getting the following error:
Error: Package: php-common-5.3.29-4.w5.x86_64 (webtatic) Requires: libcurl.so.3()(64bit) Error: Package: php-snmp-5.3.29-4.w5.x86_64 (webtatic) Requires: libnetsnmp.so.10()(64bit) You could try using –skip-broken to work around the problem You could try running: rpm -Va –nofiles –nodigest

The first thing to do was to

Step 1.

run a "CREATE IMAGE". Just in case.

Step 2.

 
sudo vim /etc/yum.repos.d/webtatic.repo

The file should look like , with enabled=0

[webtatic]
name=Webtatic Repository $releasever - $basearch
#baseurl=http://repo.webtatic.com/yum/centos/5/$basearch/
mirrorlist=http://mirror.webtatic.com/yum/centos/5/$basearch/mirrorlist
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-webtatic-andy

Again, make sure that enabled is now 0. This disables the webtatic repo.

Then re-run:
Step 3:

sudo yum install php php-cli php-gd php-mysql php-mbstring

This fixed all their badly borked deps. Then the next step was to get the latest PHP 5 support , and show the customer how to use the correct repository, and even after overcoming the dependency issues, there was a new issue now,

Error: php54w-common conflicts with php-common-5.3.3-46.el6_6.x86_64

Solving this was quite simple too, after an hour or so of research.

Step 1: Make Backup

please make a backup image before carrying out this procedure to ensure you have a roll back point.

Step 2 : Check PHP Packages

 yum list installed | grep php

Step 3: Use correct webtatic repository

 rpm -Uvh http://mirror.webtatic.com/yum/el6/latest.rpm

Step 4: Install yum replace package addin and replace php-common conflicted package with php54w-common

yum install yum-plugin-replace
yum replace php-common --replace-with=php54w-common

Step 5:

yum install php54w.x86_64 php54w-cli.x86_64 php54w-common.x86_64 php54w-gd.x86_64 php54w-ldap.x86_64 php54w-mbstring.x86_64 php54w-mcrypt.x86_64 php54w-mysql.x86_64 php54w-pdo.x86_64

I have tested this and it does seem to work. However, again, please make a backup image before carrying out this procedure to ensure you have a roll back point. I also don’t recommend doing it.

Well, it worked, yay

[root@centos6test ~]# php -v
PHP 5.4.44 (cli) (built: Aug  9 2015 13:45:34)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies

Installing Webmin on Linux Ubuntu LTS, Debian or other aptitude based OS

So, I had a customer yesterday that was unhappy about the fact that Rackspace were offering no GUI management panel install on their cloud servers as a default. I thought that what was being asked was not a literally-difficult-hard-to-do-thing. So, I stepped the customer thru how to install Webmin, and it could not have been simpler for him. We could even have a special image that supported cpanel to help placate such customers.

Here’s how we did it:

 
sudo apt-get install perl libnet-ssleay-perl libauthen-pam-perl libpam-runtime openssl libio-pty-perl apt-show-versions python

wget http://www.webmin.com/download/deb/webmin-current.deb

sudo dpkg --install webmin-current.deb

Simple as that. Now your running webmin on port 10000. If you run IPTABLES or any kind of firewall you must also open port 10000 for use on your server.

You should know be able to access webmin thru http://youripaddress:10000 or http://yourhostname.com:10000. Simples!

Playing with Xenstore

So, I have been playing around with xenstore-ls and xenstore-read commands on my Virtual Machine in the cloud. Basically xenstore-ls and xenstore-read are used to retrieve variable information about the network settings which are passed to a vdi when it is being built. Also if the network configuration breaks there is a way to use xenstore-write to read the vm-data read only network variables and the nova agent will be called to reset it.

I am still familiarizing myself with this so apologies if there are any mistakes. This article will be updated as I learn more about it.

Commands available on Rackspace Virtual Machines

xenstore         xenstore-chmod   xenstore-exists  xenstore-list    xenstore-ls      xenstore-read    xenstore-rm      xenstore-watch   xenstore-write

There are several things we can do

1. Show the Data associated with the VM, including mem_free, mem_total of the instance, the OS version (8), the os_name, distro, and the uname for the kernel.

 root@dingdong:~# xenstore-ls data
host = ""
meminfo_total = "1018872"
meminfo_free = "560180"
os_name = "Debian GNU/Linux 8.1 (jessie)"
os_majorver = "8"
os_minorver = "1"
os_uname = "3.16.0-4-amd64"
os_distro = "debian"
updated = "Fri Aug 28 17:30:00 BST 2015"
guest = ""
 9cab4aed-0d29-4c7e-be2f-e15f1ed33231 = "{"message": "1.39.1", "returncode": "0"}"
 6f38e0e1-6606-4245-8c8f-560c0204b419 = "{"message": "109108621899310233456141728258155", "returncode": "D0"}"
 c96d2b6e-31fb-489f-882c-790da25dbe1a = "{"message": "", "returncode": "0"}"
 be73ed2e-5c5a-4183-861b-2b6faaf8b09b = "{"message": "", "returncode": "0"}"
 ad730c7c-3c5e-4ba7-bfcc-8400a2675566 = "{"message": "75660051671748071924891088737764", "returncode": "D0"}"
 d3536286-4447-4f5b-84cc-8b6b1f61989c = "{"message": "", "returncode": "0"}"
 PresentationForAdamOfHowXenstoreWork = "{"message": "", "returncode": "0"}"

As you can see from above my colleague was helping make a presentation for me as to how this actually works!

Listing all of the current vm-data

 

xenstore-ls vm-data

networking = ""
 BC764E08E370 = "{"label": "private", "broadcast": "10.179.255.255", "ips": [{"ip": "10.179.197.101", "netmask": "255.255.192.0", "enabled": "1", "gateway": null}], "mac": \..."
 BC764E086A56 = "{"ip6s": [{"ip": "2a00:1a48:7806:115:be76:4eff:fe08:6a56", "netmask": 64, "enabled": "1", "gateway": "fe80::def"}], "label": "public", "broadcast": "162.13\..."
meta = "{"rxtx_cap": 120.0}"
hostname = "dingdong"
auto-disk-config = "False"
provider_data = ""
 ip_whitelist = ""
  54 = "10.182.5.215"
  53 = "134.213.147.236"
  52 = "10.182.5.234"
  51 = "134.213.148.114"
  50 = "10.179.0.222"
  49 = "10.179.75.22"
  48 = "162.13.1.53"
  47 = "95.138.174.55"
  46 = "162.13.5.15"
  45 = "10.177.132.233"
  44 = "31.222.169.12"
  43 = "10.179.0.234"
  42 = "10.177.199.231"
  41 = "10.179.0.159"
  40 = "10.176.3.232"
  39 = "10.176.3.236"
  38 = "10.176.3.239"
  37 = "10.176.3.235"
  36 = "10.177.5.90"
  35 = "10.177.5.89"
  34 = "10.177.5.88"
  33 = "10.177.1.73"
  32 = "10.176.3.158"
  31 = "10.177.0.105"
  30 = "162.13.5.96"
  29 = "5.79.25.90"
  28 = "162.209.3.51"
  27 = "162.13.22.243"
  26 = "162.13.22.242"
  25 = "166.78.7.98"
  24 = "166.78.17.140"
  23 = "166.78.24.91"
  22 = "31.222.184.215"
  21 = "31.222.184.38"
  20 = "46.38.166.180"
  19 = "46.38.160.93"
  18 = "31.222.157.156"
  17 = "31.222.177.183"
  16 = "31.222.177.167"
  15 = "31.222.164.168"
  14 = "31.222.180.84"
  13 = "31.222.161.245"
  12 = "173.203.157.20"
  11 = "119.9.12.98"
  10 = "119.9.12.91"
  9 = "162.13.1.53"
  8 = "95.138.174.55"
  7 = "162.209.4.155"
  6 = "166.78.107.18"
  5 = "50.56.249.239"
  4 = "166.78.7.146"
  3 = "89.234.21.64/28"
  2 = "67.192.155.96/27"
  1 = "173.203.5.160/27"
  0 = "173.203.32.136/29"
 roles = "["object-store:default", "compute:default", "identity:user-admin"]"
 region = "lon"
 provider = "Rackspace"
user-metadata = ""
 build_config = ""monitoring_defaults,monitoring_agent_only,auto_updates""
 rax_service_level_automation = ""Complete""
allowvssprovider = "false"


It is possible to retrieve specific information like networking configuration

xenstore-ls vm-data/networking
BC764E08E370 = "{"label": "private", "broadcast": "10.179.255.255", "ips": [{"ip": "10.179.197.101", "netmask": "255.255.192.0", "enabled": "1", "gateway": null}], "mac": "\..."
BC764E086A56 = "{"ip6s": [{"ip": "2a00:1a48:7806:115:be76:4eff:fe08:6a56", "netmask": 64, "enabled": "1", "gateway": "fe80::def"}], "label": "public", "broadcast": "162.13.\..."

The format here is kind of nasty, so there is a tool that we can use called jq

Filtering Network Data for MAC interfaces


apt-get update
apt-get install jq

xenstore-read vm-data/networking/BC764E086A56 | jq .

{
  "ip6s": [
    {
      "ip": "2a00:1a48:7806:115:be76:4eff:fe08:6a56",
      "netmask": 64,
      "enabled": "1",
      "gateway": "fe80::def"
    }
  ],
  "label": "public",
  "broadcast": "162.13.86.255",
  "ips": [
    {
      "ip": "162.13.86.79",
      "netmask": "255.255.255.0",
      "enabled": "1",
      "gateway": "162.13.86.1"
    }
  ],
  "mac": "BC:76:4E:08:6A:56",
  "gateway_v6": "fe80::def",
  "dns": [
    "83.138.151.81",
    "83.138.151.80"
  ],
  "gateway": "162.13.86.1"
}

Lots of cool stuff there. Including the Rackspace Service Net and Rackspace Public Net configurations. It’s possible to use jq to filter the JSON output of the xenstore-read command by filtering less.

Filtering ips network data with JQ

 

 xenstore-read vm-data/networking/BC764E086A56 | jq .ips
[
  {
    "ip": "162.13.86.79",
    "netmask": "255.255.255.0",
    "enabled": "1",
    "gateway": "162.13.86.1"
  }
]

 

There are a lot more things that can be done, however this is all I have time for , today.

Port Scanning with NMAP

The other day a customer was asking me what services were running on their machine. In the case of not being able to login to the customers servers due to security and policy reasons the only way for us to be able to verify the services they were running was by using NMAP; the free open source network exploitation utility for network discovery and security auditing. There are several types of scans you can do, such as discovery for the top most common 1000 ports. Full TCP or UDP scans of all 65535 ports, discovery of alive IP’s in a range or block of IP’s, and much much more. Find below a summary of the most frequent type of NMAP scans;

Discover IP’s in a Subnet thru Ping Scan

 $ nmap -sP 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2015-08-28 11:47 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0010s latency).
 Nmap scan report for 192.168.0.33
 Host is up (0.0051s latency).
 Nmap scan report for 192.168.0.105
 Host is up (0.0018s latency).

A quick and dirty command known as a ping scan. This tells nmap to send an ICMP echo request (ping), TCP SYN to port 443, TCP ACK to port 80 and ICMP timestamp request to all hosts in the specified subnet. nmap will return a list of the ip’s in the range that responded, (are up/alive) in that range.

Scan for 1000 most common ports on a Subnet

 $ nmap 192.168.0.0/24
 Starting Nmap 5.21 ( http://nmap.org ) at 2015-08-28 11:00 MST
 Nmap scan report for 192.168.0.1
 Host is up (0.0043s latency).
 Not shown: 998 closed ports
 PORT STATE SERVICE
 80/tcp open http
 443/tcp open https

If specifying no flags with NMAP it will scan using TCP SYN for the typical 1000 most common ports. In this case because we’re scanning for a /24 IP block, it includes the whole 255 ip’s in the range too. It can take at least a few minutes to perform this.

Identify Operating Systems with NMAP

$ nmap -O 162.13.136.212

Starting Nmap 6.47 ( http://nmap.org ) at 2015-08-28 06:50 UTC
Nmap scan report for 162.13.136.211
Host is up (0.00080s latency).
Not shown: 998 filtered ports
PORT    STATE  SERVICE
80/tcp  open   http
443/tcp closed https
MAC Address: BC:76:4E:08:95:6D (Rackspace US)
Device type: general purpose
Running: Linux 2.6.X|3.X
OS CPE: cpe:/o:linux:linux_kernel:2.6 cpe:/o:linux:linux_kernel:3
OS details: Linux 2.6.32 - 3.10
Network Distance: 1 hop

OS detection performed. 
Nmap done: 1 IP address (1 host up) scanned in 22.55 seconds

It’s possible to detect the operating system of IP addresses, in the security and hacking industry, when looking for exploits, or vulnerabilities in company servers, it’s important to identify the operating system correctly. This allows one to mount attack, or pentests with the correct rootkits, exploits and tools. Correctly identifying a machine is critical to security.

Find DNS Hostnames in a Range

nmap -sL 162.13.136.0/24

Starting Nmap 6.47 ( http://nmap.org ) at 2015-08-28 06:54 UTC
Nmap scan report for 162.13.136.0
Nmap scan report for 162.13.136.1
Nmap scan report for 162.13.136.2
Nmap scan report for 162.13.136.3
Nmap scan report for 162.13.136.4
Nmap scan report for 162.13.136.5
Nmap scan report for 162.13.136.6
Nmap scan report for 162.13.136.7
Nmap scan report for rack-pfc-acc-02.somedreams.com (162.13.136.8)

It’s possible to determine the hostname of all the IP addresses in a range. When performing security scans, exploits or just general pentesting, one can gather and collect information regarding the domains associated with a machine. By doing this one is able to understand what DNS server or record is vulnerable, and, allows the security analyst to find out more about the company and it’s associated infrastructure. For instance when combined with dig and whois it’s possible to reveal more information about any given ip and domain.

Pending completion.