Container [pid=26551,containerID=container_1437800838385_0177_01_000002] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 3.3 GB of 2.1 GB virtual memory used. Killing container.

Container [pid=26551,containerID=container_1437800838385_0177_01_000002] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 3.3 GB of 2.1 GB virtual memory used. Killing container.

  Dump of the process-tree for container_1437800838385_0177_01_000002 :


From the error message, you can see that you're using more virtual memory than your current limit of 1.0gb. This can be resolved in two ways:

From the error message, you can see that you're using more virtual memory than your current limit of 1.0gb. This can be resolved in two ways:
Disable Virtual Memory Limit Checking
YARN will simply ignore the limit; in order to do this, add this to your yarn-site.xml:
<property>
  <name>yarn.nodemanager.vmem-check-enabled</name>
  <value>false</value>
  <description>Whether virtual memory limits will be enforced for containers.</description>
</property>

The default for this setting is true.

Increase Virtual Memory to Physical Memory Ratio

In your yarn-site.xml change this to a higher value than is currently set
<property>
  <name>yarn.nodemanager.vmem-pmem-ratio</name>
  <value>5</value>
  <description>Ratio between virtual memory to physical memory when setting memory limits for containers. Container allocations are expressed in terms of physical memory, and virtual memory usage is allowed to exceed this allocation by this ratio.</description>
</property>
The default is 2.1
You could also increase the amount of physical memory you allocate to a container.

Make sure you don't forget to restart yarn after you change the config.

Clock skew too great while getting initial credentials error - Kerberos / AD Integration step

Issue  :  Clock skew too great while getting initial credentials error  - Kerberos / AD Integration step

Sol : machines are not in sync with AD server


Update ntp server with AD server 

Failing Oozie Launcher, Output data size [7,692] exceeds maximum [2,048]

    Issue :   Failing Oozie Launcher, Output data size [7,692] exceeds maximum [2,048]

Solution :

Add the property in oozie-site.xml

<property>
     <name>oozie.action.max.output.data</name>
     <value>8192</value>
</property>



After upgrade from cloudera 5.3.3 to 5.4.3 hive cli is throwing the below error

Error:

ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
                at jline.TerminalFactory.create(TerminalFactory.java:101)
                at jline.TerminalFactory.get(TerminalFactory.java:158)
                at org.apache.hive.beeline.BeeLineOpts.<init>(BeeLineOpts.java:73)
                at org.apache.hive.beeline.BeeLine.<init>(BeeLine.java:117)
                at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
                at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:606)
                at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
                at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
                at org.apache.hive.beeline.BeeLineOpts.<init>(BeeLineOpts.java:101)
                at org.apache.hive.beeline.BeeLine.<init>(BeeLine.java:117)
                at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
                at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:606)
                at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

                at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 


Reason:  Hive has upgraded to Jline2 but jline 0.94 exists in the Hadoop lib.

Resolution:  Delete jline from the Hadoop lib directory