Tag Archives: Openstack

Openstack Kilo on home network – part 2

In my previous post, I outlined the steps followed to install OpenStack Kilo on CENTOS7.  In this post I show the steps to setup networking, routing, and spinning up a couple of test instances.

Create Networks

In this section, I list the steps to create two OpenStack networks:

  1. Homelan. This network connects the OpenStack VMs to an external network, which in my case is my home LAN.
  2. Tenant.  This network connects the OpenStack VMs internally.

From the root login,  run the following neutron commands:

Create network homelan-net, subnet homelan-subnet

My OpenStack Kilo server sits in my home network,  The server is .154 and gateway .162.  The homelan-net network gateways to my home LAN and through my cable modem, to the internet.   The virtual machines that sit inside the OpenStack cloud on a different network and will map their IP addresses to one of the IP addresses in the allocation pool .70 thru .99.  This mapping is assigned one at a time as the VMs are created.

# source keystonerc_admin
# neutron net-create homelan-net --router:external
# neutron subnet-create homelan-net \
   --name homelan-subnet \
   --allocation-pool start=,end= \
   --disable-dhcp --gateway
Create network tenant-net, subnet tenant-subnet

The neutron commands below create a tentant-net that has a DHCP-based address pool  in the address block.  Virtual machines will be created  with an address from this DHCP address pool.

# neutron net-create tenant-net
# neutron subnet-create tenant-net \
    --name tenant-subnet --gateway --dns-nameservers list=true\

Verify the setup by going to the OpenStack  dashboard. It displays the networks:



Create Router

After creating the networks, connect them together through a router, using the following neutron commands:

# neutron router-create router1
# neutron router-interface-add router1 tenant-subnet
# neutron router-gateway-set router1 homelan-net

The router setup can be verified at the OpenStack dashboard:



The OpenStack Dashboard displays the same thing, but graphically.



The tenant-net is where the instances will connect.  Let’s bring up two.

Spin-up Cirros Instance

WIth the networks and router setup, we have the base for spinning up the default image supplied with the packstack installation.

Cirros is a barebones linux distribution well-suited for verifying OpenStack installations.

You can see the cirros image in the OpenStack dashboard:



To spin-up cirros, first setup a key-pair and some access control rules:

# nova keypair-add 080315-key > 080315-key.key
# chmod 600 080315-key.key
# nova keypair-list
| Name       | Fingerprint                                     |
| 080315-key | c9:02:a4:18:f5:32:f4:3c:64:5a:0e:e1:5e:be:f8:39 |
# nova secgroup-add-rule default icmp -1 -1
# nova secgroup-add-rule default tcp 22 22
# nova secgroup-add-rule default tcp 80 80


Now use the nova command to boot-up a cirros instance.  The nova command needs to know the id of the tenant-net.  I can get this with the neutron net-list command.

# neutron net-list
| id                                   | name        | subnets                                               |
| f8f04abf-dcba-46b7-9094-910e58530f0f | tenant-net  | 4a626740-7a14-443a-873d-d490132bbe10      |
| dbc40c3e-0175-4774-8b63-20896990c806 | homelan-net | 81f1eb33-0265-4d23-b668-d69b7a29c4b1 |

# nova boot --flavor m1.tiny --image cirros 
       --nic net-id=f8f04abf-dcba-46b7-9094-910e58530f0f   
       --security-group default --key-name 080315-key cirros081315

The new image, named  cirros081315, interfaces to the tenant-net network; it is allocated a DHCP-based address from the pool.  The virtual machine needs to have it’s IP address mapped through the router to a floating IP address on  the homelan interface of the router.

The following commands allocates a floating IP address,  then the IP address gets associated with the cirros instance.

# neutron floatingip-create homelan-net
Created a new floatingip:
| Field               | Value                                |
| fixed_ip_address    |                                      |
| floating_ip_address |                       |
| floating_network_id | dbc40c3e-0175-4774-8b63-20896990c806 |
| id                  | f104ccad-7227-4bc8-8bc5-e74a9656cfd5 |
| port_id             |                                      |
| router_id           |                                      |
| status              | DOWN                                 |
| tenant_id           | 0487893a0cdb4ac18c8837f9e7e177ab     |
# nova floating-ip-associate cirros081315

The new instance can be found in the OpenStack dashboard:



Login to the instance using the following:

ssh -i 080315-key.key cirros@

Verify that  pings to  the internet work, verify ssh to other servers on the homelan. If you get this far, things are in pretty good shape.

Spin-up a Fedora instance

With cirros working, I want to verify that I can import my usual image and spin it up.    And, I haven’t used Fedora 22 yet and this will give me a chance to see what’s changed.

The following commands imports a Fedora image, spins it up into instance, then using a floating IP address, maps the instance to my home LAN.

# glance image-list
| ID                                   | Name                   |
| 571a3d1f-afce-4e8b-b180-acf11903ada2 | cirros                 |
| e6c70bd3-dd5c-4687-ba5f-2bf6c6196b22 | Fedora 22 Cloud x86_64 |

# nova boot --flavor m1.small --image "Fedora 22 Cloud x86_64" \
  --nic net-id=f8f04abf-dcba-46b7-9094-910e58530f0f \  
  --security-group default \
  --key-name 080315-key fedora22080315
# neutron floatingip-create homelan-net
Created a new floatingip:
| Field               | Value                                |
| fixed_ip_address    |                                      |
| floating_ip_address |                       |
| floating_network_id | dbc40c3e-0175-4774-8b63-20896990c806 |
| id                  | d23783fe-18bc-44b8-9ba6-299c7917046c |
| port_id             |                                      |
| router_id           |                                      |
| status              | DOWN                                 |
| tenant_id           | 0487893a0cdb4ac18c8837f9e7e177ab     |
# nova floating-ip-associate fedora22080315

Note that this new Fedora instance uses the same key-name and net-id as the cirros instance.  Login to the instance and verify connectivity:

# ssh -i 080315-key.key fedora@
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is 1a:60:ef:e1:9b:f1:78:72:61:47:01:54:ab:21:6e:5d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
$ ping yahoo.com
PING yahoo.com ( 56(84) bytes of data.
64 bytes from ir1.fp.vip.gq1.yahoo.com ( icmp_seq=1 ttl=47 time=107 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com ( icmp_seq=2 ttl=47 time=135 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com ( icmp_seq=3 ttl=47 time=120 ms

The OpenStack dashboard screen caps below show the Fedora and Cirros instances:

Projects->Network->Network Topology

Web Server on Fedora 22

Just to confirm end-to-end networking, the following steps setup a web server on the Fedora 22 instance.  Continuing from the Fedora ssh session above, run the following:

$ sudo su -
# yum groupinstall "Web Server"
# systemctl enable httpd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
# systemctl start  httpd.service

The IP address for the Fedora instance is .72.  Go to another PC on the homelan and verify that the newly installed web server works. The default apache test page looks like the following:



In my write-up here, I setup the networks, router, image, and instances using OpenStack command lines.  In past installations, I used the OpenStack dashboard.  This time around I took the extra effort to learn the command lines.   I would say that the web interface was easier and probably what I will use going forward.

For my records, and the potential benefit of others, here’s the entire sequence of commands all in one big gist:



OpenStack Kilo on CENTOS7 using packstack on home server

I have a home network server (with one NIC) that I dedicate to OpenStack testing. It is time for me to migrate to OpenStack Kilo on CENTOS7. As with my previous OpenStack setup, I am using the RDO community’s packstack tool. This note logs my experience, for my notes, and the potential benefit of others.

I generally followed the RDO Quickstart instructions, so chapeau to the community for laying out the steps for me.

CENTOS7 with Static IP.

Centos7logo072215I picked CENTOS7 and did a bare-metal install usinga DVD I burned from the iso image named CentOS-7-x86_64-LiveKDE-1503.  I installed new disk partitions, reformatting my drives.

As I learned from previous packstack installs, the networking needs to be configured with a static IP address. This can be done at CENTOS7 install time.  Look for the “Networking & Hosts” button. Click it. On the next screen, there will be an icon at the bottom right; click it and edit the interface: change it from DHCP to manual.

The following screen cap from CENTOS7 install gives  an idea of what I did.



My passed Fedora/Centos install experience is to let the default DHCP networking  get installed, then manually change things to static IP address. Not this time; here, I found setting the static IP at install time was easier for me.

Setup network, ssh, hosts and selinux

/etc/hosts and hostname

The packstack scripts need the hosts file and the hostname setup correctly.

# hostnamectl set-hostname kozik4.lan
# vi /etc/hostname
# vi /etc/hosts kozik4.lan kozik4 localhost.localdomain localhost kozik4.lan kozik4


I turn SELinux to permissive mode ; edit a line in SElinux config file:

# setenforce permissive
# vi /etc/selinux/config

sshd for root login

OpenStack’s install script requires root access for ssh login. To setup ssh:

# vi /etc/ssh/sshd_config
PermitRootLogin yes
# systemctl  enable sshd.service
# systemctl  start sshd.service

When done with the install, I will turned off root login access.

Network Manager

Following the RDO Quickstart guide, turn off NetworkManager and turn on network

# systemctl stop NetworkManager.service
# systemctl disable NetworkManager.service
# systemctl enable network.service
# systemctl start network.service

Then reboot. Make sure basic networking is running. Verify ssh for root login works.  Use another PC on the same LAN and verify ssh into the server works.

Install OpenStack Kilo on CENTOS7

In the root login, run the following steps. Make sure each one runs successfully before starting the next.

# yum update -y
# yum install -y https://rdoproject.org/repos/rdo-release.rpm
# yum install -y openstack-packstack

At this point, I insert a work around that is specific to my setup. This particular CENTOS7 distribution pulls in mariaDB as part of KDE. I found that this created a conflict with the packstack scripts. As I documented in a previous post, uninstall mariaDB as follows.

# yum remove mariadb-server

Run packstack

Now run the main install script, packstack. Note, I am adding orchestration in addition to the default install.

# packstack --allinone --os-heat-install=y

Note that this last step is the big installation step. It runs for a long time, and if it fails, it will give detailed error messages and pointers to log files. If you run into troubles, apply workarounds and to re-try, run packstack with an answer file, as follows:

# packstack  --answer-file=packstack-answers-20150704-200908.txt

Assuming the packstack install works, verify the basic dashboard works by bringing-up the login screen. For my home server,


Setup Bridging

In order for the OpenStack VM to network with my home lan, I needed to get bridging setup. First, I made the br-ex bridge points to the IP address/gateway that I originally setup for the NIC (in CENTOS7, the NIC is assigned a name enp3s0).

I edited ifcfg-br-ex as follows:

# vi /etc/sysconfig/network-scripts/ifcfg-br-ex

By default, the NIC (enp3s0) has the external IP address and gateway. Since we moved that to the external bridge, we need to make the NIC card look like a member of the bridge. I edited the ifcfg-enp3s0 as follows:

# vi /etc/sysconfig/network-scripts/ifcfg-enp3s0

At this point, I rebooted. When done, check everything. Verify ping to the Internet, verify the same, inbound. Verify ssh, both inbound and outbound.

For reference, I record my network setup here:

[root@kozik4 ~]# ifconfig
br-ex: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet  netmask  broadcast
        inet6 fe80::4216:7eff:feb1:64cd  prefixlen 64  scopeid 0x20
        ether 40:16:7e:b1:64:cd  txqueuelen 0  (Ethernet)
        RX packets 142320  bytes 27075628 (25.8 MiB)
        RX errors 0  dropped 15257  overruns 0  frame 0
        TX packets 3632  bytes 1435026 (1.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::4216:7eff:feb1:64cd  prefixlen 64  scopeid 0x20
        ether 40:16:7e:b1:64:cd  txqueuelen 1000  (Ethernet)
        RX packets 149724  bytes 27519634 (26.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4016  bytes 1460686 (1.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet  netmask
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 0  (Local Loopback)
        RX packets 2627327  bytes 289453679 (276.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2627327  bytes 289453679 (276.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
# ip route show
default via dev br-ex dev enp3s0  scope link  metric 1002 dev br-ex  scope link  metric 1004 dev br-ex  proto kernel  scope link  src

OpenStack Dashboard

openstack-cloud-softwareThe dashboard is found at The User Name is admin, the password for the dashboard is found in the /root/keystone_admin file. By default, I was able to access the dashboard from any PC on my home lan. For access from outside of my home LAN, I setup a NAT mapping in my home gateway, but I had to setup an alias in the OpenStack horizon apache configuration.

# vi /etc/httpd/conf.d/15-horizon_vhost.conf
ServerName kozik4.lan
ServerAlias os.mydomain.net # Add this line
# systemctl restart httpd.service

The name os.mydomain.net is a sub-domain name that I own, where I map it to the Internet routable IP address my ISP gives me. Verifiy that http://os.mydomain.net/dashboard works.

Once  logged in,  the dashboard shows the network setup that packstack installs for us by default.  It sets up a private and public networks IP addresses that don’t match my needs.  So I need to clear this out.   The way OpenStack works, you cannot delete everything in one hammer-style step; you need to disassemble the networking one step at a time.

First, you need to delete the router, then delete the networks.

Remove the packstack default router and networks

Delete the pre-installed router

This step deletes the default router that packstack setup for us. The dashboard gives a nice GUI to do this.  Navigate to Admin->Routers.


Before we can delete router1, we need to delete its interfaces.  Click on “router1″

AdminRouterDetails072215Using he GUI, we can delete these two interfaces, and then we can delete the router.

But for my own education and the potential automation of future installs, I figured out how to do this with  OpenStack command line scripts.  The easiest way to display this is to screen scrape my putty commands and insert it inline below:

Go back to the dashboard Admin->Routers page, verify that the router has been cleared. I know, the command line steps are alot more difficult than clicking the “Delete Router” button, but this is how I learn.


Delete Pre-Installed Networks

From the dashboard Admin->Networks. We want to delete the two networks packstack setup for us. In packstack (OpenStack?) terminology, the private network is the range of IP addresses used by the virtual machines used by the admin project within this OpenStack installation. The public network maps to the interface that routes packets to outside networks… in this case, the public network will eventually be assigned addresses that fit within my home lan (not Internet addressable IP address).

The Networks dashboard shows two networks. I want to remove them. It turns out you need to click on each of the networks and delete the subnets attached to the network.  On the dash board it is pretty easy to do.



Here’s the command lines that will delete these two networks.  Note, again, I typed the commands into my putty screen, and scraped them and inserted them inline below.

Verify that the Networks are gone in the dashboard.



Ok, to this point we have a clean slate ready to bring in images, spin up instances, and network them together.


Openstack RDO packstack Kilo install stuck on CENTOS 7 – MariaDB conflict

kilologoBackground: I got stuck on RDO packstack Kilo install onto CENTOS7. I got some strange log entries I didn’t understand – I post the problem and the solution here for my records and maybe to help someone else.

For my RDO packstack Kilo install setup, I followed the normal Quick Start  steps on a bare metal installed version of CENTOS 7 (CentOS-7-x86_64-LiveKDE-1503).

I ran the following step in the install guide:

# packstack --allinone

The packstack installed failed with the following error on the console:

Error: Execution of '/usr/bin/rpm -e mariadb-server-5.5.41-2.el7_0.x86_64' returned 1: error: Failed dependencies:
You will find full trace in log /var/tmp/packstack/20150704-200908-

This is an error pointing to the following log file entry:

Error: Execution of '/usr/bin/rpm -e mariadb-server-5.5.41-2.el7_0.x86_64' returned 1: error: Failed dependencies:
        mariadb-server is needed by (installed) akonadi-mysql-1.9.2-4.el7.x86_64

This is only my second install using packstack, and it is my first attempt at installing RDO packstack Kilo; so, I wasn’t familiar with the troubleshoot this kind of problem. It didn’t make sense to me that packstack would be trying to uninstall (rpm -e) MariaDB. The error indicates that the uninstall wouldn’t work because another package akonadi-mysql was using it — right? is that how I read this?  FYI:  Akonadi is a storage service that runs under KDE… I wasn’t familiar with it.

Browsing the RDO Workaround archives , I found a short article on this topic: Packstack fails when mariadb-server is installed. The body of the article shows a log file message that is different enough from mine that I didn’t find it right away.

I tried the recommended fix to uninstall MariaDB and re-run RDO packstack Kilo:

# yum remove mariadb-server
# packstack  --answer-file=packstack-answers-20150704-200908.txt

This fixed my problem!

I was able to run packstack to completion and bring up the openstack horizon dashboard. Note: I didn’t re-run packstack –allinone; I followed the packstack workflow requirement to re-run an install using the answer file previously generated.

Since the workaround was written for Juno and Fedora 20, I thought the fix was just a coincidence, but in reading the details, the original problem was triggered by installing KDE ontop of Fedora 20. Since I picked a KDE distribution of CENTOS7, maybe my problem and the original workaround write-up are related?!

Respectfully submitted,

Setting up Virtual Servers in Openstack environment

Background on Setting up Virtual Servers in Openstack environment: Once I got my Openstack environment setup, and I was able to create a couple of instances, I had to figure out the easiest way of managing IP Addresses and sub-domain names for web access to each of my instances.

I needed web access to my openstack host. I needed web access to each of my instances, which are running virtually on the same host. Further, since I am running all of this on one server in my home network, I need to somehow map all of this to one external IP address.

This is nothing too new to me. I have lots of vintages of Linux servers in my basement, and I sort of know the ropes around setting up NAT-ing, Virtual Servers, and proxies. My question was: what’s the best practice? What would be the easiest?

I couldn’t find anything directly on this (let me know if you have a reference). So here’s what I decided to do.

Enable Openstack Dashboard Network Access

By default, the Openstack Horizon Django configuration strictly controls who can get access. It’s roughly localhost only. For testing purposes, I went into the settings file and removed all restrictions:

$ cd /etc/openstack-dashboard
$ vi local_settings
#ALLOWED_HOSTS = ['horizon.example.com', ]
$ systemctl restart httpd.service

I tried making a restrictive list, but it kept getting in my way. When done setting up, I will lock this up.

I then verified from a different PC in the same subnet that works.

Map Openstack Host to External IP Address

Using my home router, I configured an address mapping between port 80 and my Openstack host. Here’s the screen shot:


Port Forwarding Table from my home router

Now verify that http://my.external.IP.address/dashboard works.

Setup an Openstack Subdomain Name

One of my domain names, jackkozik.net points to my home router’s IP address. I setup an Openstack subdomain name, using my zoneedit account — I used os.jackkozik.net. Sorry no screen shot. I am (perhaps too liberally) showing my domain name, but I am reluctant to show my IP addresses. Zoneedit is pretty quick, but distributing a new subdomain address takes anywhere from 0 to 60 minutes.

I then edit Openstack’s virtual server configuration adding os.jackkozik.net as a ServerAlias, as follows:

$ cd /etc/httpd/conf.d
$ vi 15-horizon_vhost.conf
<VirtualHost *:80>
ServerName kozik4.lan
ServerAlias os.jackkozik.net # Add this line
$ systemctl restart httpd.service

Then assuming the subdomain has had a chance to get distributed, verify http://os.jackkozik.net/dashboard.

Openstack’s install scripts automatically setup this VirtualHost.

Create Subdomains for Each of My Instances

Within Openstack, I create instances that are automatically assigned IP addresses from a pool in the range of Of course these instances are accessible from my home network (eg displays a nice Apache default screen). But I only have one external IP address and I need a mechanism to for external web access.

Absent any better approach (I hope to find one!), I am using Apache’s ProxyPass capability. I have used this for physical servers, why not use it for virtual machines?!

For starters, I created another subdomain in zoneedit. I decided to name each external instance with a letter followed by the least significant digits from the IP address. My first subdomain is named f100: That is, it is a Fedora instance and it’s running an instance mapped to In zoneedit, I enter the subdomain f100.jackkozik.net, and I put the same external IP address that I used for os.jackkozik.net.

Within the Apache configuration files, I created a virtual server named f100.jackkozik.net, and used ProxyPass to map it to (redirect it to?) the web server running on See the following config file:

$ cd /etc/httpd/conf.d
$ vi openstackInstances_vhost.conf
# This file configures all the proxy modules:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
<VirtualHost *:80>
ServerName f100.jackkozik.net
ProxyPreserveHost On
ProxyPass /
ProxyPassReverse /
$ systemctl restart httpd.service

I created this file and put it in the conf.d directory. It automatically gets read whenever the apache web server starts.

From here, allowing enough time for zoneedit to work, I verified that http://f100.jackkozik.net worked from both my home network and from an outside network (I sometime use my desktop PC at work test this; more commonly I use the Chrome browser on my Android phone).

I edit this file for each new instance I setup.

So each of my instances think they are sitting on the internet, but really the Openstack host Apache server and my home network router’s NAT function are fooling it.


Finally, once I got everything working, I fixed ALLOWED_HOSTS to permit any traffic from my home subnet and only allow requests from URL os.jackkozik.net from the Internet. See following:

$ cd /etc/openstack-dashboard
$ vi local_settings
ALLOWED_HOSTS = ['localhost', 'os.jackkozik.net', 'kozik4.lan', ]
$ systemctl restart httpd.service