Posts tagged with: ambari api

one-shot backup all config files with Ambari API

Before performing an upgrade for example, or in a more general manner, it’s a good thing to get backups of your config files.

In an upgrade situation, this allows you to perform a quick diff to see if there are parameters which had been reset, or new parameters to take care of.

Ambari API allows you to get all config files, and we’ll use the amazing configs.sh script provided by Hortonworks in HDP to perform backups.

First, config types : when looking at the output of http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/, we can find them on Clusters / desired_config.

Ambari API result

So let’s dig into that part : go to http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME/?fields=Clusters/desired_configs

Ambari API desired_configs

Now let’s get only file names :

[root@sandbox ~]# curl -s -u admin:admin http://sandbox.hortonworks.com:8080/api/v1/clusters/Sandbox/?fields=Clusters/desired_configs | grep '" : {' | grep -v Clusters | grep -v desired_configs | cut -d'"' -f2
ams-env
ams-hbase-env
ams-hbase-log4j
ams-hbase-policy
ams-hbase-security-site
ams-hbase-site
ams-log4j
ams-site
capacity-scheduler
cluster-env
core-site
...

Now for each file we can get a backup with the following command :

[root@sandbox ~]# /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8080 get $AMBARI_HOST $CLUSTER_NAME $CONFIG_TYPE | grep '^"' | grep -v '^"properties" : {'

where CONFIG_TYPE = core-site for example.

So we can now have a complete backup:

#!/bin/bash
AMBARI_HOST=sandbox.hortonworks.com
CLUSTER_NAME=Sandbox
AMBARI_USER=admin
AMBARI_PASSWORD=admin
AMBARI_PORT=8080
timeNow=`date +%Y%m%d_%H%M%S`
RESULT_DIR=/root/migrationHDP/configs.sh/$timeNow
mkdir -p $RESULT_DIR
for CONFIG_TYPE in `curl -s -u $AMBARI_USER:$AMBARI_PASSWORD http://$AMBARI_HOST:$AMBARI_PORT/api/v1/clusters/$CLUSTER_NAME/?fields=Clusters/desired_configs | grep '" : {' | grep -v Clusters | grep -v desired_configs | cut -d'"' -f2`; do
echo "backuping $CONFIG_TYPE"
/var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PASSWORD -port $AMBARI_PORT get $AMBARI_HOST $CLUSTER_NAME $CONFIG_TYPE | grep '^"' | grep -v '^"properties" : {' > $RESULT_DIR/$CONFIG_TYPE.conf
done

 

Note than you can also output to a single file to make the diff easier, adding the CONFIG_TYPE to have a better view :

#!/bin/bash
AMBARI_HOST=sandbox.hortonworks.com
CLUSTER_NAME=Sandbox
AMBARI_USER=admin
AMBARI_PASSWORD=admin
AMBARI_PORT=8080
timeNow=`date +%Y%m%d_%H%M%S`
RESULT_DIR=/root/migrationHDP/configs.sh/$timeNow
mkdir -p $RESULT_DIR
for CONFIG_TYPE in `curl -s -u $AMBARI_USER:$AMBARI_PASSWORD http://$AMBARI_HOST:$AMBARI_PORT/api/v1/clusters/$CLUSTER_NAME/?fields=Clusters/desired_configs | grep '" : {' | grep -v Clusters | grep -v desired_configs | cut -d'"' -f2`; do
echo "backuping $CONFIG_TYPE"
/var/lib/ambari-server/resources/scripts/configs.sh -u $AMBARI_USER -p $AMBARI_PASSWORD -port $AMBARI_PORT get $AMBARI_HOST $CLUSTER_NAME $CONFIG_TYPE | grep '^"' | grep -v '^"properties" : {' | sed "1i ##### $config_type #####" >> $RESULT_DIR/all.conf
done