Le Cluster de Laurent Just another Hadoop blog

When you enable security on your HDP cluster, the wizard deletes the ATS (App Timeline Server) which is useful to follow YARN applications history.

Re-enabling it is not very difficult :

1. Installation through Ambari API

[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components/APP_TIMELINE_SERVER

HTTP/1.1 201 Created

2. Component activation

[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Install AppTimelineServer via REST","query":"HostRoles/component_name.in('APP_TIMELINE_SERVER')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components

HTTP/1.1 202 Accepted
Set-Cookie: AMBARISESSIONID=1dkx65ox0k0vx1vigxw7r0i4aw;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 170
Server: Jetty(7.6.7.v20120910)

{
"href" : "http://gw.example.com:8080/api/v1/clusters/hdp-cluster/requests/260",
"Requests" : {
"id" : 260,
"status" : "InProgress"
}
}

3. Keytab for ATS creation

[vagrant@gw ~]# ipa service-add ats/gw.example.com@EXAMPLE.COM

4. Copy

[vagrant@gw ~]# ipa-getkeytab -s nn.example.com -p ats/gw.example.com@EXAMPLE.COM -k /etc/security/keytabs/ats.service.keytab

Keytab successfully retrieved and stored in: /etc/security/keytabs/ats.service.keytab
[vagrant@gw ~]# chown yarn:hadoop /etc/security/keytabs/ats.service.keytab && chmod 400 /etc/security/keytabs/ats.service.keytab

5. Add parameters in YARN

yarn.timeline-service.principal = ats/_HOST@EXAMPLE.COM
yarn.timeline-service.keytab = /etc/security/keytabs/ats.service.keytab
yarn.timeline-service.http-authentication.type = kerberos
yarn.timeline-service.http-authentication.kerberos.principal = HTTP/_HOST@EXAMPLE.COM
yarn.timeline-service.http-authentication.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab

6. Restart YARN + ATS

The UI is enabled by default on port 8188 : http://gw.example.com:8188/applicationhistory