Posts tagged with: ambari

get metrics with Ambari API

[vagrant@gw ~]$ curl -u admin:admin -X GET http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/nn.example.com/host_components/NAMENODE?fields=metrics/jvm
{
 "href" : "http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/nn.example.com/host_components/NAMENODE?fields=metrics/jvm",
 "HostRoles" : {
 "cluster_name" : "hdp-cluster",
 "component_name" : "NAMENODE",
 "host_name" : "nn.example.com"
 },
 "host" : {
 "href" : "http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/nn.example.com"
 },
 "metrics" : {
 "jvm" : {
 "HeapMemoryMax" : 1052770304,
 "HeapMemoryUsed" : 56104392,
 "NonHeapMemoryMax" : 318767104,
 "NonHeapMemoryUsed" : 49148216,
 "gcCount" : 190,
 "gcTimeMillis" : 4599,
 "logError" : 0,
 "logFatal" : 0,
 "logInfo" : 16574,
 "logWarn" : 2657,
 "memHeapCommittedM" : 1004.0,
 "memHeapUsedM" : 53.473206,
 "memMaxM" : 1004.0,
 "memNonHeapCommittedM" : 133.625,
 "memNonHeapUsedM" : 46.87139,
 "threadsBlocked" : 0,
 "threadsNew" : 0,
 "threadsRunnable" : 7,
 "threadsTerminated" : 0,
 "threadsTimedWaiting" : 54,
 "threadsWaiting" : 7
 }
 }
}

The metrics you may want to watch are HeapMemoryMax and HeapMemoryUsed


Ambari tips & tricks

Restarting some components

(including clients, which you can’t put to another state than “INSTALLED”) :

curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '
{
"RequestInfo":{
"command":"RESTART",
"context":"Restart ZooKeeper Client and HDFS Client",
"operation_level":{
"level":"HOST",
"cluster_name":"hdp-cluster"
}
},
"Requests/resource_filters":[
{
"service_name":"ZOOKEEPER",
"component_name":"ZOOKEEPER_CLIENT",
"hosts":"gw.example.com"
},
{
"service_name":"ZOOKEEPER",
"component_name":"ZOOKEEPER_SERVER",
"hosts":"gw.example.com,nn.example.com,dn1.example.com"
}
]
}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/requests

As indicated in the wiki, the RESTART refreshs the configs.

 

Delete a host from Ambari

// get all COMPONENTS for the host

[root@uabdfes03 ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X GET http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/hosts/$HOSTNAME/host_components

// delete all COMPONENTS for this HOST
[root@host ~]# for COMPONENT in ZOOKEEPER_CLIENT YARN_CLIENT PIG OOZIE_CLIENT NODEMANAGER MAPREDUCE2_CLIENT HIVE_CLIENT HDFS_CLIENT HCAT HBASE_REGIONSERVER HBASE_CLIENT GANGLIA_MONITOR DATANODE; do curl -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/hosts/$HOSTNAME/host_components/$COMPONENT; done
// delete HOST
[root@host ~]# curl -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER/hosts/$HOSTNAME

Delete a service

(for example STORM)

// get the components for that service
[vagrant@gw ~]$ curl -u admin:admin -X GET  http://gw.example.com:8080/api/v1/clusters/hdp-cluster/services/STORM
// stop the service
[vagrant@gw ~]$ curl -u admin:admin -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo":{"context":"Stop Service"},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/services/STORM
//stop each component on each host
[vagrant@gw ~]$ for COMPONENT_NAME in DRPC_SERVER NIMBUS STORM_REST_API STORM_UI_SERVER SUPERVISOR; do curl -u admin:admin -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components/${COMPONENT_NAME}; done
// stop service components
[vagrant@gw ~]$ for COMPONENT_NAME in DRPC_SERVER NIMBUS STORM_REST_API STORM_UI_SERVER SUPERVISOR; do curl -u admin:admin -H 'X-Requested-By: ambari' -X PUT -d '{"RequestInfo":{"context":"Stop All Components"},"Body":{"ServiceComponentInfo":{"state":"INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/services/STORM/components/${COMPONENT_NAME}; done
// delete the service
[vagrant@gw ~]$ curl -u admin:admin -H 'X-Requested-By: ambari' -X DELETE http://gw.example.com:8080/api/v1/clusters/hdp-cluster/services/STORM

 Add a component

For example we want to add a HBase RegionServer

// add the component
[vagrant@gw ~]$ curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components/HBASE_REGIONSERVER

// then install
[vagrant@gw ~]$ curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo": {"context": "Install RegionServer","query":"HostRoles/component_name.in('HBASE_REGIONSERVER')"}, "Body":{"HostRoles": {"state": "INSTALLED"}}}' http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts/gw.example.com/host_components

 Get host components for a service

[vagrant@gw ~]$ curl -u admin:admin -H "X-Requested-By:ambari" -i -X GET http://gw.example.com:8080/api/v1/clusters/hdp-cluster/hosts?host_components/HostRoles/service_name=HDFS&fields=host_components/HostRoles/service_name

 


My Ambari Notes

When trying to enable HA NameNode wizard, I encountered this weird exception in Install JournalNodes step.

When looking in the logs, I noticed this

07:57:09,617 ERROR [qtp1251571107-6336] AbstractResourceProvider:244 - Caught AmbariException when creating a resource
org.apache.ambari.server.ServiceComponentNotFoundException: ServiceComponent not found, clusterName=hdp-cluster, serviceName=HDFS, serviceComponentName=JOURNALNODE

exception in the ambari-server.log

So the solution was to “manually” install the ServiceComponent with the Ambari API :

[vagrant@gw ~]$ curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST http://gw.example.com:8080/api/v1/clusters/hdp-cluster/services/HDFS/components/JOURNALNODE

HTTP/1.1 201 Created
Set-Cookie: AMBARISESSIONID=1frjfvu8yjb5fylf4w5m449c7;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 0
Server: Jetty(7.6.7.v20120910)