install a new ShareLib for Oozie

The latest versions of Oozie are dealing with a new schema concerning the ShareLib.
The ShareLib is used by Oozie to pickup the necessary JARs for the jobs to run in their containers; To be able to let old jobs finish while upgrading the ShareLib, Oozie now use a timestamped version of the ShareLib on HDFS.

[root@localhost ~]# su - oozie
[oozie@localhost ~]# export OOZIE_URL=http://OOZIE_SERVER_FQDN:11000/oozie
[oozie@localhost ~]# /usr/hdp/current/oozie-server/bin/oozie-setup.sh sharelib create -fs hdfs://MY_DEFAULT_FS_NAME

You’ll get the new ShareLib : 

–> the destination path for sharelib is: /user/oozie/share/lib/lib_20151207105251

Let’s have a look on what we got :

[root@localhost ~]# hdfs dfs -ls /user/oozie/share/lib
Found 11 items
drwxr-xr-x - oozie hadoop 0 2015-10-01 12:54 /user/oozie/share/lib/distcp
drwxr-xr-x - oozie hadoop 0 2015-10-05 19:05 /user/oozie/share/lib/hbase
drwxr-xr-x - oozie hadoop 0 2015-10-01 12:54 /user/oozie/share/lib/hcatalog
drwxr-xr-x - oozie hadoop 0 2015-10-01 12:54 /user/oozie/share/lib/hive
drwxr-xr-x - oozie hadoop 0 2015-11-26 19:31 /user/oozie/share/lib/lib_20151126193054
drwxrwx--- - oozie hadoop 0 2015-12-07 10:53 /user/oozie/share/lib/lib_20151207105251
drwxr-xr-x - oozie hadoop 0 2015-10-01 12:54 /user/oozie/share/lib/mapreduce-streaming
drwxr-xr-x - oozie hadoop 0 2015-10-19 14:41 /user/oozie/share/lib/oozie
drwxr-xr-x - oozie hadoop 0 2015-10-01 12:54 /user/oozie/share/lib/pig
-rwxr-xr-x 3 oozie hadoop 1393 2015-10-01 12:54 /user/oozie/share/lib/sharelib.properties
drwxr-xr-x - oozie hadoop 0 2015-10-12 15:28 /user/oozie/share/lib/sqoop

So we found our old ShareLib (lib_20151126193054) and our new ShareLib (lib_20151207105251) populated with the HDP current version.

Remember all modifications in the previous ShareLib are not automagically copied to the new ShareLib, so take a close look on the JARs.

Finally, you can tell Oozie to switch to the latest ShareLib (it will take the latest timestamp) :

[oozie@localhost ~]$ oozie admin -sharelibupdate

You can eventually check if the path has been updated by listing a jar to see its path :

[oozie@localhost ~]$ oozie admin -shareliblist hive | head


So, what do you think ?