When you enable security on your HDP cluster, the wizard deletes the ATS (App Timeline Server) which is useful to follow YARN applications history.
Re-enabling it is not very difficult :
1. Installation through Ambari API
[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components/APP_TIMELINE_SERVER
HTTP/1.1 201 Created
2. Component activation
[vagrant@gw ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Install AppTimelineServer via REST","query":"HostRoles/component_name.in('APP_TIMELINE_SERVER')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components
HTTP/1.1 202 Accepted
Set-Cookie: AMBARISESSIONID=1dkx65ox0k0vx1vigxw7r0i4aw;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 170
Server: Jetty(7.6.7.v20120910)
{
"href" : "http://gw.example.com:8080/api/v1/clusters/hdp-cluster/requests/260",
"Requests" : {
"id" : 260,
"status" : "InProgress"
}
}
3. Keytab for ATS creation
4. Copy
5. Add parameters in YARN 6. Restart YARN + ATS The UI is enabled by default on port 8188 : http://gw.example.com:8188/applicationhistory[vagrant@gw ~]# ipa service-add ats/gw.example.com@EXAMPLE.COM
[vagrant@gw ~]# ipa-getkeytab -s nn.example.com -p ats/gw.example.com@EXAMPLE.COM -k /etc/security/keytabs/ats.service.keytab
Keytab successfully retrieved and stored in: /etc/security/keytabs/ats.service.keytab
[vagrant@gw ~]# chown yarn:hadoop /etc/security/keytabs/ats.service.keytab && chmod 400 /etc/security/keytabs/ats.service.keytab
yarn.timeline-service.principal = ats/_HOST@EXAMPLE.COM
yarn.timeline-service.keytab = /etc/security/keytabs/ats.service.keytab
yarn.timeline-service.http-authentication.type = kerberos
yarn.timeline-service.http-authentication.kerberos.principal = HTTP/_HOST@EXAMPLE.COM
yarn.timeline-service.http-authentication.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab
Hi,
Your instruction is very helpful for me. But I got a problem, which is related to token renewal stuff.
I set up yarn-site.xml like this:
yarn.timeline-service.delegation.key.update-interval
600000
yarn.timeline-service.delegation.token.max-lifetime
900000
yarn.timeline-service.delegation.token.renew-interval
600000
Adding this into yarn-site will reduce the time period for the token, because I want to test whether or not Yarn will automatically renew the token for client application to connect to Yarn Timeline Server. But my application fail to connect to Yarn Timeline Server when the token is expired. So enabling Application Timeline Server on a kerberized cluster is not enough, you also need to implement something to renew the token. Am I right?
Could you please give some comments?
Thanks,
Hoang
Hi, the variables you mention exist already so adding them would only override their default values.
You can ensure that by getting it with
hdfs getconf -confKey yarn.timeline-service.delegation.token.renew-interval