Dr Elephant Installation on Linux - Cloudera - Part 4

Dr Elephant Installation 

·        Clone Dr.elephant :

[mani@node1.manilab.com drelephant]$ pwd

/opt/drelephant

mkdir dr-ele       

git clone https://github.com/linkedin/dr-elephant.git

Now Inside à /dr-ele/app/com/linkedin/drelephant/analysis/AnalyticJobGeneratorHadoop2.java change the following values of Resource Manager from  http: to https:/

yarn.resourcemanager.webapp.http.address to yarn.resourcemanager.webapp.https.address

mani@node1.manilab analysis]$ cat AnalyticJobGeneratorHadoop2.java | grep https

  private static final String RESOURCE_MANAGER_ADDRESS = "yarn.resourcemanager.webapp.https.address";

  private static final String RM_NODE_STATE_URL = "https://%s/ws/v1/cluster/info";

    URL succeededAppsURL = new URL(new URL("https://" + _resourceManagerAddress), String.format(

    URL failedAppsURL = new URL(new URL("https://" + _resourceManagerAddress), String.format(

·        Compile Dr.elephant

[mani@node1.manilab.com dr-ele]$ pwd

/opt/drelephant/dr-ele

[mani@node1.manilab.com dr-ele]$ ./compile.sh ./compile.conf

·        Now inside ‘dist’ folder dr-elephant-2.0.13.zip will be created .Extract zip file inside /dist.

mani@node1.manilab.com dist]$ pwd

/opt/drelephant/dr-ele/dist

[mani@node1.manilab.com dist]$ ll

total 88148

drwxr-xr-x 8 mani manidevl      131 Jan 18 07:42 dr-elephant-2.0.13

-rw-r--r-- 1 mani manidevl 90259919 Jan 18 07:33 dr-elephant-2.0.13.zip

Extract it.

Everything happens inside the extracted folder now.

·        Now inside /dr-ele/dist/dr-elephant-2.0.13/app-conf/  in the elephant.conf change the following values :

port=8083

# Database configuration

db_url=node1.manilab.com

db_name=drelephant

db_user=admin

db_password="*****"

keytab_user="mani"

keytab_location="/home/mani/mani.keytab"

jvm_args="-Devolutionplugin=disabled -DapplyEvolutions.default=false -mem 1024 -J-Xloggc:$project_root../logs/elephant/dr-gc.`date +'%Y%m%d%H%M'` -J-XX:+PrintGCDetails"

give permission 777 to hadoop fs –ls /user/history/done

·        Start Dr.elephant

./start.sh ../app-conf/

Make sure dr.elephant started without any errors, check the dr.log

·        Go to Dr.elephant UI

Change the hostname/ip according to your env, you should be able to see the Dr.elephant dashboard

Hostname:<port>

In our case it is: http://node1.manilab.com:8083/

·        Run a sample Hadoop job

 After the job completes you could see the analysis on Dr.elephant UI