In a customer perspective, security is not an option.
You can hear about cryptography, firewalling, etc. but no security will take place without the 2 main components of security : authentication and authorization.
The former takes place for insuring that you are the person you pretend to be, the latter that you have the right to access a resource.
In this post we’ll talk about authentication, and the best and only existing way to implement that is Kerberos, a 30+ years protocol from MIT.
First let’s talk about glossary :
– Client has its secret key
– Server has its secret key
– TGS (Ticket Granting System) has its secret key and knows the Server key
– KDC (Key Distribution Center) knows secret keys of Client and TGS
Steps are :
– Client identifies itself with KDC, which gives back a ticket authorizing him to request the TGS
– Client asks TGS for a ticket
– Client get his ticket and send his id with the ticket, sever checks the ticket validity and authorize the access.
Now, let’s get to work 
On my 3 machines (vagrant powered) cluster, first let’s install free-ipa server, which is a really great/simple/robust way to have Kerberos on your system.
This cluster is under HDP 2.1.3 and Ambari 1.6.1.
The ipa-server package (provided by RedHat) should be in your repos.
On the KDC machine :
[vagrant@gw ~]$ sudo yum install -y ipa-server
[vagrant@gw ~]$ sudo ipa-server-install
Since my cluster is virtual machines with not “real” IP addresses and FQDN, I updated the /usr/lib/python2.6/site-packages/ipapython/ipautil.py to get rid of the IANA Ip address check.
So you can now add users & groups in IPA, after doing a kinit admin. Now on every other hosts, install the client IPA package :
$ sudo yum install -y ipa-client
$ sudo ipa-client-install --enable-dns-updates
Now let’s kerberize our cluster ! In Ambari, let’s go to Security
Let’s get started.
Click Next and provide the information requested. Basically, the only information you need to provide is the realm.
Now Ambari proposes a smart way to generate all keytabs : download a CSV file that will be used with a script, which will take care of all that stuff.
Download that CSV file and put it on your KDC machine. For using with IPA we have to make a slight modification into Hortonwork’s script : add the -x ipa-setup-override-restrictions parameter after the kadmin.local -q “addprinc -randkey $principal” command.
Now let’s make all that keytabs.
Please note that on this version there is no rm (ResourceManager) keytab generated, so add the following line (assuming host is nn.example.com) in generate_keytab.sh before executing it :
kadmin.local -q "addprinc -randkey rm/nn.example.com@EXAMPLE.COM" -x ipa-setup-override-restrictions
[vagrant@gw ~]$ /var/lib/ambari-server/resources/scripts/keytabs.sh /vagrant/host-principal-keytab-list.csv > ./generate_keytabs.sh
[vagrant@gw ~]$ chmod +x ./generate_keytabs.sh
[vagrant@gw ~]$ sudo su -
[root@gw ~]$ cd /home/vagrant
[root@gw ~]$ kinit admin
[root@gw ~]$ ./generate_keytabs.sh
This will generate all keytabs in a tar archive we copy to each host.
Then on each machine, extract & copy the keytabs
[vagrant@gw ~]$ sudo tar xpvf keytabs_gw.example.com.tar --strip=1
[vagrant@dn1 ~]$ sudo tar xpvf keytabs_dn1.example.com.tar --strip=1
[vagrant@nn ~]$ sudo tar xpvf keytabs_nn.example.com.tar --strip=1
Please note the p option to preserve ownership, and sudo to make that option works. The –strip=1 is to avoid the ./ extraction which will make the current directory unbrowsable. Now we have all keytabs in /etc/security/keytabs directory.
It’s now time to activate our kerberized cluster by clicking the Apply button in Ambari interface.
Note : Generating a keytab will invalidate all that related keytabs on the realm !
As an example, if you re-run the generate_keytabs.sh script, this will ask new keytabs so you’ll got to copy it on all the servers.
Note 2 : If you want to enable HA on your cluster, you’ll need new keytabs because of the new components. The easiest way is to redownload the csv and regenerate all the keytabs.