Dr Elephant Installation on Linux - Cloudera - Part 4

Dr Elephant Installation 

·        Clone Dr.elephant :

[mani@node1.manilab.com drelephant]$ pwd

mkdir dr-ele       

Now Inside à /dr-ele/app/com/linkedin/drelephant/analysis/AnalyticJobGeneratorHadoop2.java change the following values of Resource Manager from  http: to https:/

yarn.resourcemanager.webapp.http.address to yarn.resourcemanager.webapp.https.address

mani@node1.manilab analysis]$ cat AnalyticJobGeneratorHadoop2.java | grep https
  private static final String RESOURCE_MANAGER_ADDRESS = "yarn.resourcemanager.webapp.https.address";
  private static final String RM_NODE_STATE_URL = "https://%s/ws/v1/cluster/info";
    URL succeededAppsURL = new URL(new URL("https://" + _resourceManagerAddress), String.format(
    URL failedAppsURL = new URL(new URL("https://" + _resourceManagerAddress), String.format(

·        Compile Dr.elephant

[mani@node1.manilab.com dr-ele]$ pwd

[mani@node1.manilab.com dr-ele]$ ./compile.sh ./compile.conf

·        Now inside ‘dist’ folder dr-elephant-2.0.13.zip will be created .Extract zip file inside /dist.

mani@node1.manilab.com dist]$ pwd

[mani@node1.manilab.com dist]$ ll
total 88148
drwxr-xr-x 8 mani manidevl      131 Jan 18 07:42 dr-elephant-2.0.13
-rw-r--r-- 1 mani manidevl 90259919 Jan 18 07:33 dr-elephant-2.0.13.zip
Extract it.

Everything happens inside the extracted folder now.

·        Now inside /dr-ele/dist/dr-elephant-2.0.13/app-conf/  in the elephant.conf change the following values :


# Database configuration


jvm_args="-Devolutionplugin=disabled -DapplyEvolutions.default=false -mem 1024 -J-Xloggc:$project_root../logs/elephant/dr-gc.`date +'%Y%m%d%H%M'` -J-XX:+PrintGCDetails"
give permission 777 to hadoop fs –ls /user/history/done

·        Start Dr.elephant

./start.sh ../app-conf/
Make sure dr.elephant started without any errors, check the dr.log

·        Go to Dr.elephant UI

Change the hostname/ip according to your env, you should be able to see the Dr.elephant dashboard

In our case it is: http://node1.manilab.com:8083/

·        Run a sample Hadoop job

 After the job completes you could see the analysis on Dr.elephant UI

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.