Razor Chef Broker

Update: there is a newer, official version of razor chef broker. Please refer to this post for more details.

Project Razor is a power control, provisioning, and management application designed to deploy both bare-metal and virtual computer resources. It also provides broker plugins to integrate with third party systems. If such broker is provided as part of the policy for deployments, the broker will be used to hand off the newly deployed node to a DevOps system. Up until now, only Puppet broker was available.  However, if you are already using Chef as your chosen DevOps tool, that should not prevent you from trying out Razor!

For the last week or so I been spending my evenings working on a Chef broker for Razor. It still needs a bit of work done, however, right now my broker successfully registers nodes with the Chef server.

Lets take a look at the cli usage for adding Chef broker.  First, determine what brokers are available:

root@ubuntu:/opt/razor/bin# razor broker get plugins
Available Broker Plugins:
 Plugin Description
 puppet PuppetLabs PuppetMaster
 chef OpsCode Chef

Great, chef is one of the options! Try and add one. If not sure what the parameters are, try this:

root@ubuntu:/opt/razor/bin# razor broker add
[Broker] [add_broker] <-Must Provide: [The broker plugin to use.]
Command help:
razor broker add (options...)
-p, --plugin BROKER_PLUGIN The broker plugin to use. 
-n, --name BROKER_NAME The name for the broker target. 
-d, --description DESCRIPTION A description for the broker target. 
-s, --servers SERVER_LIST A comma-separated list of servers for this broker target 
-c, --certificate CERTIFICATE Full path to the Chef server certificate file 
-v, --version VERSION A target broker version (used in gem install) 
-h, --help Display this screen.

The options for adding Chef broker differ only slighly from the ones used for Puppet broker (current documentation for it is here).  The main difference is “-c” option, to add a path to the Chef server certificate.  The certificate usually be found on your server in /etc/chef/validation.pem file.   Make a local copy of this file so that it can be used to register a new node with the server.

Lets go ahead and add a new broker:

root@ubuntu:/opt/razor/bin# razor broker add -p chef -n Chef_2 -d Chef -s -c /opt/razor/bin/validation.pem 
Name => Chef_2
 Description => Chef
 Plugin => chef
 Servers => []
 UUID => 2B0KgW2xCleWreET16WI4p
 Certificate => /opt/razor/bin/validation.pem
 Version => Default

Associate a current policy with the new broker:

root@ubuntu:/opt/razor/bin# razor policy update 4ZtLkicLls6isgee91JCMN
 [Policy] [update_policy] 4ZtLkicLls6isgee91JCMN
 Line Number => 3
 Label => precise
 Enabled => true
 Template => linux_deploy
 Description => Policy for deploying a Linux-based operating system.
 Tags => [memsize_1GiB, vmware_vm]
 Model Label => install_precise
 Broker Target => Chef_2
 Currently Bound => 26
 Maximum Bound => 0
 Bound Counter => 27

Now, any node deployed using this policy, will be handed off to Chef server.  For hand off,  broker follows these steps:

  • installs chef on the node
  • creates basic /etc/chef/client.rb file with client settings
  • creates /etc/chef/validation.pem file
  • installs ohai
  • calls home (registers itself with the server and provides personal details)

In it’s current form, this broker is already pretty useful!  Next, I will try and provide Chef server with Razor’s custom metadata, so that there is feature parity between Puppet and Chef brokers.  If you would like to try it out, checkout the working branch: https://github.com/eglute/Razor/tree/feature/master/chef_broker


Update: there is a newer, official version of razor chef broker. Please refer to this post for more details.

GitHub Tips for Beginners

Q: How often should I checkin code into github?
A: More often is better than less often, if you write a lot of code, do it at least 2-4 times a day.

Q: Should I branch/fork?
A: branching and forking is there to help you better organize your code by feature. If you are working on an open source project, you will need to fork it first so that you have access to checkin code into it. After you forked a project, you may want to create branches for different features that you will be working on. If you are part of one organization where you have access to the repo, you may want to forgo the forking and go straight to branching. This is a decision that each group needs to make.

Q: Can I work from master?
A: Yes you can, but you probably should not! Think of your master branch as a golden copy of your code. Merge to it after you thoroughly tested your new features or bug fixes.

Q: What if command line scares me?
A: Use GUI tools built just for you!  There are lots of them out there, my favorite one is: http://mac.github.com/


While looking up some documentation for git, I came across this awesome cheatsheet:

Try clicking on it!

You may be aware of github documentation, but do visit git docs once in a while as well to get very clear and detail answers to all your git questions:


Tutorial: Adding Existing Project to Github

If you are currently working on a project that you would like to check into github, it may not be very obvious at first. This step by step tutorial assumes that you already setup your github account.  If you have not, follow these instructions first: https://help.github.com/articles/set-up-git. Once your github is setup, create a new repo trough UI: https://github.com/new

In this example, I will use repo “testRepo”, just to keep it simple!

Now, on your command line, go to your existing project. For the sake of this tutorial, the names of existing project and git repo are the same, but they do not have to be.

My current project has 2 files:

egle@ubuntu:~/testRepo$ ls -la
total 16
drwxrwxr-x  2 egle egle 4096 Nov 16 19:41 .
drwxr-xr-x 25 egle egle 4096 Nov 16 19:40 ..
-rw-rw-r--  1 egle egle   14 Nov 16 19:41 file1.txt
-rw-rw-r--  1 egle egle   15 Nov 16 19:41 file2.txt

To turn this directory into a git repository, I need to initialize it:

egle@ubuntu:~/testRepo$ git init
Initialized empty Git repository in /home/egle/testRepo/.git/

This creates a local git repository on your machine. That’s right, you can version files locally without ever needing to connect to github or other git server! Note that currently your new git repo contains no files.

Add all the files to the repository:

egle@ubuntu:~/testRepo$ git add .

Once the files are added, they need to be committed:

egle@ubuntu:~/testRepo$ git commit -m "first commit"
[master (root-commit) b9c1002] first commit
Committer: Egle <egle@ubuntu.(none)>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
git config --global user.name "Your Name"
git config --global user.email you@example.com

After doing this, you may fix the identity used for this commit with:

git commit --amend --reset-author
2 files changed, 2 insertions(+)
create mode 100644 file1.txt
create mode 100644 file2.txt

In this case, I have not setup my git settings, so will need to fix that later. The important part here is that files got committed to the local repo.  However, local repositories are a little hard to share. Lets add a remote repository, named origin, and push files to it:

egle@ubuntu:~/testRepo$ git remote add origin https://github.com/eglute/testRepo.git
egle@ubuntu:~/testRepo$ git push -u origin master
Username for 'https://github.com': eglute
Password for 'https://eglute@github.com': 
To https://github.com/eglute/testRepo.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

Now your current project is in github, ready for pulls, pushes, merges, branches, and forks!



Fun with Git Config

It seems like everyone is using github these days, and it is easy to fork and branch trough it’s UI. But what if you are just starting with using git?

I will reveal a few not so hidden secrets of git. First, github is based on git, so if you just been reading github documentation, you are missing out on a lot of great documentation that can be found here: http://git-scm.com/doc.

Second, when you are using github, you are really working with two repositories, not one. The first one is on your local machine, set up to point to the second, remote repo.

Here are a few tips on how to configure your local repo:

You will want to start with git config tool to configure your git profile. git config will setup your identity and some preferences.

Start with setting your name:

egle@ubuntu:~/testRepo$ git config --global user.name "Any Stacker"
 egle@ubuntu:~/testRepo$ cat ~/.gitconfig
 name = Any Stacker

A few more settings:

egle@ubuntu:~/testRepo$ git config --global merge.tool vimdiff
 egle@ubuntu:~/testRepo$ cat ~/.gitconfig
 name = Any Stacker
 tool = vimdiff
 egle@ubuntu:~/testRepo$ git config --global core.editor vi
 egle@ubuntu:~/testRepo$ git config --list
 user.name=Any Stacker

Don’t be a git, check in your code often!


Book: OpenStack Cloud Computing Cookbook, by Kevin Jackson

If you are following the cloudscape, you may have noticed that OpenStack is getting a lot of attention right now.  If you never even heard of OpenStack, cookbook by Kevin Jackson is not the right place for you to start. For the very green I would suggest http://devstack.org/.

For those that want something a little more advanced, I would recommend picking up a copy OpenStack Cloud Computing Cookbook.  Whether you are building your very own private cloud or maintaining one, this book is right for you.

Recipes start with setting up sand box environment on VirtualBox, followed by Essex install on Ubuntu Precise (12.04).  After basic install, the book covers installing, configuring, and administering all of the components of OpenStack.  Chapters 2 and 3 cover compute and keystone components.  Chapter 4 starts out with a setup of swift (storage component) sand box environment. Chapters 5 and 6 are more Swift recipes. Glance, Nova, Horizon and Networking get the next 4 chapters, while 11 and 12 cover practical details like installing OpenStack on bare metal (MAAS) and monitoring. The last chapter delves into troubleshooting, logging, submitting bug reports, and getting help from community.

What this book is not: an in-depth explanation of OpenStack components. It is also not OpenStack for Dummies.  However, if you just want to get things working, this is a great reference book.


Step by Step WordPress Install on Rackspace Cloud Server

Time for a new blog and already have a Rackspace cloud account?  If you are going with a brand new server, you will need to set up some basic things first.  Lets get started:

Login to Rackspace’s cloud control panel. Create a server: pick a server name, something short and sweet!  I picked OS: Centos 6.3, because it is the closest to RedHat and because it does not require RedHat license fee. Region: the default is probably fine!

Pick a size: smallest should work.

Click “Create Server”.  You will be presented with “Root Admin Password”. Copy the password, and store it in a safe place.  You will need it later.

Go get coffee, it will take a few minutes for the server to be created.

Copy the ip of your server and ssh into it: root@your.ip.  It will ask you a yes/no question, if it is the first time you are logging in.  Say yes!

computer:~ eglute$ ssh root@
The authenticity of host ' (' can't be established.
RSA key fingerprint is 27:e4:7f:5e:25:f8:e6:90:e4:c3:1b:4d:6f:c6:f2:b3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (RSA) to the list of known hosts.
root@'s password:

Since you just created this server, it is missing a few things. Fun with updates:

Check whether server needs updating:

[root@blog ~]# yum check-update

It probably needs updating:

[root@blog ~]# yum update

Don’t want to do updates manually? Well, you are in luck! Just configure cron to take care of this for you:

[root@blog ~]# yum -y install yum-cron
[root@blog ~]# chkconfig --list yum-cron
yum-cron 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@blog ~]# service yum-cron status
Nightly yum update is disabled.
[root@blog ~]# service yum-cron start
Enabling nightly yum update: [ OK ]
[root@blog ~]#

Install mysql server:

[root@blog ~]# yum -y install mysql-server

Start mysql:

[root@blog ~]# service mysqld start

Secure mysql installation:

[root@blog ~]# mysql_secure_installation
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none): 
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.
Set root password? [Y/n] Y
New password: 
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
 ... Success!
By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] Y
 ... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] Y
 ... Success!
By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] Y
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] Y
 ... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MySQL
installation should now be secure. Thanks for using MySQL!

Install apache:

[root@blog ~]# yum -y install httpd
[root@tempserver ~]# service httpd status
httpd is stopped
[root@blog ~]# chkconfig --list httpd
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@blog ~]# service httpd start

Create user for database:

[root@blog ~]# useradd press
[root@blog ~]# passwd press
Changing password for user press.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@blog ~]# mysql -p
 Enter password:
 Welcome to the MySQL monitor. Commands end with ; or \g.
 Your MySQL connection id is 12
 Server version: 5.1.61 Source distribution
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
 affiliates. Other names may be trademarks of their respective
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE DATABASE blog;
 Query OK, 1 row affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON blog.* TO press@localhost IDENTIFIED BY 'verysecretpassword';
 Query OK, 0 rows affected (0.00 sec)
 Query OK, 0 rows affected (0.00 sec)
mysql> EXIT
 [root@tempserver ~]#

Install php:

yum -y install php php-mysql
cd /var/www/html/

Download wordpress:

wget http://wordpress.org/latest.tar.gz


tar -xzvf latest.tar.gz

Move all files a directory up:

mv wordpress/* .
[root@blog html]# ls
 index.php license.txt wordpress wp-admin wp-blog-header.php wp-config-sample.php wp-cron.php wp-links-opml.php wp-login.php wp-settings.php wp-trackback.php
 latest.tar.gz readme.html wp-activate.php wp-app.php wp-comments-post.php wp-content wp-includes wp-load.php wp-mail.php wp-signup.php xmlrpc.php

You currently have all the pieces installed, yet while trying to go to http://example.com/wp-admin/install.php page, your browser won’t be able to connect to your server.  Check your iptables:

[root@blog html]# cat /etc/sysconfig/iptables
 # Firewall configuration written by system-config-firewall
 # Manual customization of this file is not recommended.
 -A INPUT -p icmp -j ACCEPT
 -A INPUT -i lo -j ACCEPT
 -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
 -A INPUT -j REJECT --reject-with icmp-host-prohibited
 -A FORWARD -j REJECT --reject-with icmp-host-prohibited

The problem is that port 80 is not accesible!

Edit iptables to open port 80. Since you are there, you might need 443 as well:

vi /etc/sysconfig/iptables
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

You should have ipv6 address, so as such, change your ip6tables:

vi /etc/sysconfig/ip6tables
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
-A INPUT -p ipv6-icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp6-adm-prohibited
-A FORWARD -j REJECT --reject-with icmp6-adm-prohibited

Restart services:

[root@blog html]# service iptables restart
iptables: Flushing firewall rules: 
 [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
[root@blog html]# service ip6tables restart
ip6tables: Flushing firewall rules: [ OK ]
ip6tables: Setting chains to policy ACCEPT: filter [ OK ]
ip6tables: Unloading modules: [ OK ]
ip6tables: Applying firewall rules: 
 [ OK ]
[root@blog html]#

At this point, if you check http://example.com/wp-admin/install.php page, your browser will be connecting to the server, but you will be getting “Error: PHP is not running” message.  Still a couple steps left!

[root@blog html]# pwd
[root@blog html]# ls
index.php license.txt wordpress wp-admin wp-blog-header.php wp-config-sample.php wp-cron.php wp-links-opml.php wp-login.php wp-settings.php wp-trackback.php
latest.tar.gz readme.html wp-activate.php wp-app.php wp-comments-post.php wp-content wp-includes wp-load.php wp-mail.php wp-signup.php xmlrpc.php

Make a copy of wp-config-sample.php:

[root@blog html]# cp wp-config-sample.php wp-config.php

Edit wp-config.php.  For this example, it would look like this:

// ** MySQL settings - You can get this info from your web host ** //
 /** The name of the database for WordPress */
 define('DB_NAME', 'blog');
/** MySQL database username */
 define('DB_USER', 'press');
/** MySQL database password */
 define('DB_PASSWORD', 'verysecretpassword');
/** MySQL hostname */
 define('DB_HOST', 'localhost');

The rest of the values can stay as is in the file.

One last time:

service httpd restart

Proceed to the browser http://example.com/wp-admin/install.php , and follow directions on the screen.  Happy blogging!