enabling Application Timeline Server on a kerberized cluster

When you enable security on your HDP cluster, the wizard deletes the ATS (App Timeline Server) which is useful to follow YARN applications history.

Re-enabling it is not very difficult :

1. Installation through Ambari API

[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components/APP_TIMELINE_SERVER

HTTP/1.1 201 Created

2. Component activation

[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Install AppTimelineServer via REST","query":"HostRoles/component_name.in('APP_TIMELINE_SERVER')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components

HTTP/1.1 202 Accepted
Set-Cookie: AMBARISESSIONID=1dkx65ox0k0vx1vigxw7r0i4aw;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 170
Server: Jetty(7.6.7.v20120910)

{
"href" : "http://gw.example.com:8080/api/v1/clusters/hdp-cluster/requests/260",
"Requests" : {
"id" : 260,
"status" : "InProgress"
}
}

3. Keytab for ATS creation

[vagrant@gw ~]# ipa service-add ats/gw.example.com@EXAMPLE.COM

4. Copy

[vagrant@gw ~]# ipa-getkeytab -s nn.example.com -p ats/gw.example.com@EXAMPLE.COM -k /etc/security/keytabs/ats.service.keytab

Keytab successfully retrieved and stored in: /etc/security/keytabs/ats.service.keytab
[vagrant@gw ~]# chown yarn:hadoop /etc/security/keytabs/ats.service.keytab && chmod 400 /etc/security/keytabs/ats.service.keytab

5. Add parameters in YARN

yarn.timeline-service.principal = ats/_HOST@EXAMPLE.COM
yarn.timeline-service.keytab = /etc/security/keytabs/ats.service.keytab
yarn.timeline-service.http-authentication.type = kerberos
yarn.timeline-service.http-authentication.kerberos.principal = HTTP/_HOST@EXAMPLE.COM
yarn.timeline-service.http-authentication.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab

6. Restart YARN + ATS

The UI is enabled by default on port 8188 : http://gw.example.com:8188/applicationhistory


2 Comments

  • Reply Hoang Le |

    Hi,
    Your instruction is very helpful for me. But I got a problem, which is related to token renewal stuff.
    I set up yarn-site.xml like this:

    yarn.timeline-service.delegation.key.update-interval
    600000

    yarn.timeline-service.delegation.token.max-lifetime
    900000

    yarn.timeline-service.delegation.token.renew-interval
    600000

    Adding this into yarn-site will reduce the time period for the token, because I want to test whether or not Yarn will automatically renew the token for client application to connect to Yarn Timeline Server. But my application fail to connect to Yarn Timeline Server when the token is expired. So enabling Application Timeline Server on a kerberized cluster is not enough, you also need to implement something to renew the token. Am I right?
    Could you please give some comments?

    Thanks,
    Hoang

    • Reply administrator |

      Hi, the variables you mention exist already so adding them would only override their default values.

      You can ensure that by getting it with hdfs getconf -confKey yarn.timeline-service.delegation.token.renew-interval

So, what do you think ?