Posts tagged with: security

Kerberizing my cluster

In a customer perspective, security is not an option.

You can hear about cryptography, firewalling, etc. but no security will take place without the 2 main components of security : authentication and authorization.
The former takes place for insuring that you are the person you pretend to be, the latter that you have the right to access a resource.

In this post we’ll talk about authentication, and the best and only existing way to implement that is Kerberos, a 30+ years protocol from MIT.

First let’s talk about glossary :

– Client has its secret key
– Server has its secret key
– TGS (Ticket Granting System) has its secret key and knows the Server key
– KDC (Key Distribution Center) knows secret keys of Client and TGS

Steps are :

– Client identifies itself with KDC, which gives back a ticket authorizing him to request the TGS
– Client asks TGS for a ticket
– Client get his ticket and send his id with the ticket, sever checks the ticket validity and authorize the access.


Now, let’s get to work :)

On my 3 machines (vagrant powered) cluster, first let’s install free-ipa server, which is a really great/simple/robust way to have Kerberos on your system.
This cluster is under HDP 2.1.3 and Ambari 1.6.1.

The ipa-server package (provided by RedHat) should be in your repos.

On the KDC machine :

[vagrant@gw ~]$ sudo yum install -y ipa-server
[vagrant@gw ~]$ sudo ipa-server-install

Since my cluster is virtual machines with not “real” IP addresses and FQDN, I updated the /usr/lib/python2.6/site-packages/ipapython/ to get rid of the IANA Ip address check.

So you can now add users & groups in IPA, after doing a kinit admin. Now on every other hosts, install the client IPA package :

$ sudo yum install -y ipa-client
$ sudo ipa-client-install --enable-dns-updates

Now let’s kerberize our cluster ! In Ambari, let’s go to Security
Enable Security

Let’s get started.
step 01

Click Next and provide the information requested. Basically, the only information you need to provide is the realm.
step 02

Now Ambari proposes a smart way to generate all keytabs : download a CSV file that will be used with a script, which will take care of all that stuff.
step 03

Download that CSV file and put it on your KDC machine. For using with IPA we have to make a slight modification into Hortonwork’s script : add the -x ipa-setup-override-restrictions parameter after the kadmin.local -q “addprinc -randkey $principal” command.
Now let’s make all that keytabs.

Please note that on this version there is no rm (ResourceManager) keytab generated, so add the following line (assuming host is in before executing it :
kadmin.local -q "addprinc -randkey rm/" -x ipa-setup-override-restrictions

[vagrant@gw ~]$ /var/lib/ambari-server/resources/scripts/ /vagrant/host-principal-keytab-list.csv > ./
[vagrant@gw ~]$ chmod +x ./
[vagrant@gw ~]$ sudo su -
[root@gw ~]$ cd /home/vagrant
[root@gw ~]$ kinit admin
[root@gw ~]$ ./

This will generate all keytabs in a tar archive we copy to each host.

Then on each machine, extract & copy the keytabs

[vagrant@gw ~]$ sudo tar xpvf --strip=1
[vagrant@dn1 ~]$ sudo tar xpvf --strip=1
[vagrant@nn ~]$ sudo tar xpvf --strip=1

Please note the p option to preserve ownership, and sudo to make that option works. The –strip=1 is to avoid the ./ extraction which will make the current directory unbrowsable. Now we have all keytabs in /etc/security/keytabs directory.

It’s now time to activate our kerberized cluster by clicking the Apply button in Ambari interface.

Note : Generating a keytab will invalidate all that related keytabs on the realm !
As an example, if you re-run the script, this will ask new keytabs so you’ll got to copy it on all the servers.
Note 2 : If you want to enable HA on your cluster, you’ll need new keytabs because of the new components. The easiest way is to redownload the csv and regenerate all the keytabs.