Background: I got stuck on RDO packstack Kilo install onto CENTOS7. I got some strange log entries I didn’t understand – I post the problem and the solution here for my records and maybe to help someone else.
For my RDO packstack Kilo install setup, I followed the normal Quick Start steps on a bare metal installed version of CENTOS 7 (CentOS-7-x86_64-LiveKDE-1503).
I ran the following step in the install guide:
# packstack --allinone
The packstack installed failed with the following error on the console:
Error: Execution of '/usr/bin/rpm -e mariadb-server-5.5.41-2.el7_0.x86_64' returned 1: error: Failed dependencies:
You will find full trace in log /var/tmp/packstack/20150704-200908-
This is an error pointing to the following log file entry:
Error: Execution of '/usr/bin/rpm -e mariadb-server-5.5.41-2.el7_0.x86_64' returned 1: error: Failed dependencies:
mariadb-server is needed by (installed) akonadi-mysql-1.9.2-4.el7.x86_64
This is only my second install using packstack, and it is my first attempt at installing RDO packstack Kilo; so, I wasn’t familiar with the troubleshoot this kind of problem. It didn’t make sense to me that packstack would be trying to uninstall (rpm -e) MariaDB. The error indicates that the uninstall wouldn’t work because another package akonadi-mysql was using it — right? is that how I read this? FYI: Akonadi is a storage service that runs under KDE… I wasn’t familiar with it.
I was able to run packstack to completion and bring up the openstack horizon dashboard. Note: I didn’t re-run packstack –allinone; I followed the packstack workflow requirement to re-run an install using the answer file previously generated.
Since the workaround was written for Juno and Fedora 20, I thought the fix was just a coincidence, but in reading the details, the original problem was triggered by installing KDE ontop of Fedora 20. Since I picked a KDE distribution of CENTOS7, maybe my problem and the original workaround write-up are related?!
Background on Setting up Virtual Servers in Openstack environment: Once I got my Openstack environment setup, and I was able to create a couple of instances, I had to figure out the easiest way of managing IP Addresses and sub-domain names for web access to each of my instances.
I needed web access to my openstack host. I needed web access to each of my instances, which are running virtually on the same host. Further, since I am running all of this on one server in my home network, I need to somehow map all of this to one external IP address.
This is nothing too new to me. I have lots of vintages of Linux servers in my basement, and I sort of know the ropes around setting up NAT-ing, Virtual Servers, and proxies. My question was: what’s the best practice? What would be the easiest?
I couldn’t find anything directly on this (let me know if you have a reference). So here’s what I decided to do.
Enable Openstack Dashboard Network Access
By default, the Openstack Horizon Django configuration strictly controls who can get access. It’s roughly localhost only. For testing purposes, I went into the settings file and removed all restrictions:
$ cd /etc/openstack-dashboard
$ vi local_settings
...
#ALLOWED_HOSTS = ['horizon.example.com', ]
ALLOWED_HOSTS = ['*']
ZZ
$ systemctl restart httpd.service
I tried making a restrictive list, but it kept getting in my way. When done setting up, I will lock this up.
I then verified from a different PC in the same subnet that http://192.168.100.154/dashboard works.
Map Openstack Host to External IP Address
Using my home router, I configured an address mapping between port 80 and my Openstack host. Here’s the screen shot:
Port Forwarding Table from my home router
Now verify that http://my.external.IP.address/dashboard works.
Setup an Openstack Subdomain Name
One of my domain names, jackkozik.net points to my home router’s IP address. I setup an Openstack subdomain name, using my zoneedit account — I used os.jackkozik.net. Sorry no screen shot. I am (perhaps too liberally) showing my domain name, but I am reluctant to show my IP addresses. Zoneedit is pretty quick, but distributing a new subdomain address takes anywhere from 0 to 60 minutes.
I then edit Openstack’s virtual server configuration adding os.jackkozik.net as a ServerAlias, as follows:
$ cd /etc/httpd/conf.d
$ vi 15-horizon_vhost.conf
...
<VirtualHost *:80>
ServerName kozik4.lan
ServerAlias os.jackkozik.net # Add this line
...
ZZ
$ systemctl restart httpd.service
Then assuming the subdomain has had a chance to get distributed, verify http://os.jackkozik.net/dashboard.
Openstack’s install scripts automatically setup this VirtualHost.
Create Subdomains for Each of My Instances
Within Openstack, I create instances that are automatically assigned IP addresses from a pool in the range of 192.168.100.100-119. Of course these instances are accessible from my home network (eg http://192.168.100.100 displays a nice Apache default screen). But I only have one external IP address and I need a mechanism to for external web access.
Absent any better approach (I hope to find one!), I am using Apache’s ProxyPass capability. I have used this for physical servers, why not use it for virtual machines?!
For starters, I created another subdomain in zoneedit. I decided to name each external instance with a letter followed by the least significant digits from the IP address. My first subdomain is named f100: That is, it is a Fedora instance and it’s running an instance mapped to 192.168.100.100. In zoneedit, I enter the subdomain f100.jackkozik.net, and I put the same external IP address that I used for os.jackkozik.net.
Within the Apache configuration files, I created a virtual server named f100.jackkozik.net, and used ProxyPass to map it to (redirect it to?) the web server running on 192.168.100.100. See the following config file:
$ cd /etc/httpd/conf.d
$ vi openstackInstances_vhost.conf
# This file configures all the proxy modules:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
<VirtualHost *:80>
ServerName f100.jackkozik.net
ProxyPreserveHost On
ProxyPass / http://192.168.100.100/
ProxyPassReverse / http://192.168.100.100/
</VirtualHost>
ZZ
$ systemctl restart httpd.service
I created this file and put it in the conf.d directory. It automatically gets read whenever the apache web server starts.
From here, allowing enough time for zoneedit to work, I verified that http://f100.jackkozik.net worked from both my home network and from an outside network (I sometime use my desktop PC at work test this; more commonly I use the Chrome browser on my Android phone).
I edit this file for each new instance I setup.
So each of my instances think they are sitting on the internet, but really the Openstack host Apache server and my home network router’s NAT function are fooling it.
Fix ALLOWED_HOSTS
Finally, once I got everything working, I fixed ALLOWED_HOSTS to permit any traffic from my home subnet and only allow requests from URL os.jackkozik.net from the Internet. See following:
I was in Las Vegas for the week. My meetings started at 8am and most nights ended at 10pm, so I didn’t get alot of time to really do anything.
My last night, after 10pm, I went for a walk on the Las Vegas Strip. I exited through the taxi stand area, turned right / West and walked along Tropicana Ave.
At the light with Las Vegas Blvd, I crossed the street and turned right. From there I walked North passing Excalibur, New York-New York, Monte Carlo, Mandarin Oriental, City Center Place, Planet Hollywood, Harley Davidson Cafe, Cosmopolitan, Aria, Bellagio (saw the nice fountain show!), Paris, Caesar’s Palace, Flamingo, and Bally’s.
From Bally’s I took the Monorail ($5) back to the MGM. This was a fun way to see the city lights!
This posting is part 2. With the installation from part 1, I can now use the Openstack Dashboard. It is a nice web interface that contains straight forward menus. In the sections below, I will show screen captures from a web browser connected to the Dashboard; I will also show some command line text from a putty window.
This writeup will show steps for setting up a public subnet that links Openstack to my home network and setting up a private subnet for my guest instances to use. Then I will show how to create an instance using the bare-bones cirros image. And finally, I’ll install into the image repository a Fedora 20 cloud image and spin it up with a basic web server.
Since Openstack and the Openstack Dashboard are new to me, I do all of this with the generous help of the references listed below, especially recognizing Seth Jenning’s excellent Openstack Icehouse on Fedora 20 using RDO video.
Create a public network.
Starting with the Openstack Dashboard, logged in as admin, create a public network. My home network is 192.168.100.0/24. The IP address 192.168.100.163 is the address of my home router, gateway to the internet. My Openstack Icehouse host is 192.168.100.154. And Openstack needs a subnet, referred to as public, that sits in this address range.
The naming convention of calling the network 192.168.100.0/24 public started with the packstack install scripts. In the context of Openstack, this network is the one with a gateway to the internet and thus it is referred to as public, even though we know the internet defines 192.168.x.x IP addresses as private.
Next, create a subnet that is private to the Openstack host. The addresses must be different from the public_subnet. These addresses will never leave the Openstack host and it’s underlying Open vSwitch network address space. The references use 10.0.0.0/24 as the network and 10.0.0.1 as the gateway address — and that’s what I use below. Further, the guest instances each need to be given an IP address, and 10.0.0.2, 10.0.0.20 address are what I choose to be the range for a DHCP address pool.
Ok, there’s a public and a private subnet defined. Openstack Dashboard has a really simple way to connect them together with a router function. Create a router, define a default gateway and then add interfaces to private subnets.
All the steps upto this point were building a network into which virtual machines connect to the home LAN. Run the Dashboard command below to see two subnets connect to a router. The public subnet is on the home LAN. The private subnet is the address space where the guests instances will connect.
The references suggest that for trial/learning purposes, the Security Group Rules should be wide open. The idea is while learning the technology, the security settings can obscure basic setup issues. In the long run, this needs to be managed more carefully.
First, remove the default rules that packstack setup, then install rules that permit incoming and outgoing TCP/UDP/ICMP — all ports.
Openstack Dashboard Menu: Project->Compute->Access and Security->default
Delete all the default rules and rebuild the rules so that the Security Group Rules table looks as follows:
Setup an ssh key pair
Using the normal ssh tools, make an ssh public and private key pair. The Openstack Dashboard lets you cut / paste your own public key into the project. The instances that get created will have the public key pre-installed into it. To access instances that Openstack creates, use the private key as an option in an ssh command line.
Go to the root login of the Openstack host, at the command prompt create a key and copy the public key into the clipboard.
[root@kozik4 ~]# ssh-keygen -t rsa -f cloud.key
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in cloud.key.
Your public key has been saved in cloud.key.pub.
The key fingerprint is:
36:37:4d:d3:5b:82:45:53:48:98:24:00:8b:0b:3b:06 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
| .......==o.|
| . . .o+.. |
|E . . . + o .|
| . o . o . + |
| + . S o . . |
| . . . o . |
| |
| |
| |
+-----------------+
[root@kozik4 ~]# ls
anaconda-ks.cfg ifconfig5.out packstack-answers-20140803-201418.txt
cloud.key installpackstack.log packstack.log
cloud.key.pub keystonerc_admin rdorelease.log
ifconfig1.out keystonerc_demo runpackstack.log
ifconfig2.out ovs1.out yumupdate.log
ifconfig3.out ovs2.out
ifconfig4.out ovs3.out
[root@kozik4 ~]# cat cloud.key.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDijKn/k5ejNii3SaNugO75Njz1LQHyDDwI5blZO4+CZTRL/O/8czffrUUfK8+j3QjAx7MByNJVkj8YCGtOAYv5wCFEhzkRhqNNJlH235L++QV6ai/XPD7b0VcqhCjQTkDIfyBMp7fZO+D0BGdvTjBiQXIJdZLqZWV2j9qH8EHHS55OlOXpAAcMHvRRgWFtMdn5YSLUcq8X5HRtvfesLL7quJmNDc8/rS6mhmL/NFU56r+SJpHvr7N59U7ywNejLgFp6hfz4zZw3nWDH9y+by1zdWbNfATIO362SRue+FvuF060ss4Ciesuqw5v3tJMeyq9JM41lu8fQaIeBqoJTB43 [email protected]
[root@kozik4 ~]#
In putty screen like the above example, select the text output from the ‘cat cloud.key.pub’ command and paste into the Openstack Dashboard as follows.
Launch Test Instance ‘cirros’ and assign floating IP
To help verify that the Openstack packstack installed correctly, spin-up the barebones cirros image. This image is a really small linux distribution. I’ve never heard of cirros, but I get the purpose of it. My initial setup had troubles, and cirros helped me trouble shoot basic setup problems. I was glad the initial install pulled it in.
The following steps startup an instance, link it to the private subnet, and map the private IP address of the instance to a floating IP address on the public subnet. Floating IP addresses were new to me, and it wasn’t obvious how they should be used, at first. I think of it as a generalized NATing function, that lets me hide my home network topology from the Openstack instances.
Connect the instance to the private subnet. Note: the web page below requires you to drag the private line and drop it into the Selected Networks cyan-colored bar.
Verify that the instance has two IP addresses and is running ok.
Openstack Dashboard Menu: Project->Instance
Go back to the 192.168.100.154 host putty prompt (root login, home directory). Verify that we can setup an ssh connection to the new instance. The default login id is cirros. The Instances web page above tells us to use the 192.168.100.101.
# ssh -i cloud.key [email protected]
The authenticity of host '192.168.100.101 (192.168.100.101)' can't be established.
RSA key fingerprint is 34:51:4c:22:c3:67:d3:47:38:83:c2:ee:55:0f:4b:e5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.100.101' (RSA) to the list of known hosts.
$ ifconfig
eth0 Link encap:Ethernet HWaddr FA:16:3E:62:7D:36
inet addr:10.0.0.3 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe62:7d36/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
$ ping yahoo.com
PING yahoo.com (98.139.183.24): 56 data bytes
64 bytes from 98.139.183.24: seq=0 ttl=47 time=49.484 ms
64 bytes from 98.139.183.24: seq=1 ttl=47 time=121.366 ms
64 bytes from 98.139.183.24: seq=2 ttl=47 time=81.164 ms
Note: the ifconfig shows that the instance only knows about the private_subnet address 10.0.0.3. Also, an important test to verify: check that the instance can access the internet, I used ping yahoo.com.
Create a Fedora 20 instance
The cirros instance installation steps above helped to verify that basic functionality worked. But cirros is not a linux distribution I want to use; I want to use the latest version of Fedora. In this section I repeat some of the steps from the previous section to get a Fedora 20 instance started. There’s enough different here that I wanted to document it.
From the Fedora In the Cloud web page, right click the 64-bit qcow2 image and “Copy Link Address.” The Images page has an option to import new images using a URL.
Create an image from this URL:
Openstack Dashboard Menu: Project->Create Image
Following the same steps as the cirros image, launch the Fedora 20 image.
Be sure to click the Networking tab and select private subnet, then click on Launch.
Once the instance is running, allocate a Floating IP. Go to the Projects->Access & Security->Floating IP menu. First run Allocate IP to Project then Manage Floating IP Associations for the Fedora 20 instance… just like we did for the cirros instance.
The Instance Dashboard page now shows 2 instances.
Openstack Dashboard Menu: Project->Instance
And the Network Topology page gives a nice picture of how everything is wired together.
So, just like with cirros, go to the Openstack host root login prompt and ssh to the Fedora instance. The Instance page above show 192.168.100.102 as the IP address.
# ssh -i cloud.key [email protected]
The authenticity of host '192.168.100.102 (192.168.100.102)' can't be established.
RSA key fingerprint is 2e:b3:7b:6b:06:43:cf:d5:95:95:49:38:5f:ab:20:39.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.100.102' (RSA) to the list of known hosts.
[fedora@fedora-20 ~]$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.0.4 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::f816:3eff:fe9b:baf5 prefixlen 64 scopeid 0x20
ether fa:16:3e:9b:ba:f5 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 0 (Local Loopback)
[fedora@fedora-20 ~]$ ping yahoo.com
PING yahoo.com (206.190.36.45) 56(84) bytes of data.
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=1 ttl=49 time=1 06 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=2 ttl=49 time=1 56 ms
64 bytes from ir1.fp.vip.gq1.yahoo.com (206.190.36.45): icmp_seq=3 ttl=49 time=1 02 ms
Note: the Fedora guest instance only knows the private address 10.0.0.4. The default login id, fedora, has sudo permissions and access to the root login is done with a ‘sudo su -’ command. It is important also to verify that it can talk to the outside world, and thus verify that ping yahoo.com works.
To further test my Fedora 20 instance, I switched over to the Fedora 20 instance root login and installed apache:
Background: I got stuck on RDO packstack Kilo install onto CENTOS7. I got some strange log entries I didn’t understand – I post the problem and the solution here for my records and maybe to help someone else.
