好不容易把Hive装完了,结果一执行Hive的查询语句运行MapReduce程序立马报错。。。
log详细信息如下:
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 0
2017-10-21 21:54:15,503 null map = 0%, reduce = 0%
2017-10-21 21:54:44,930 null map = 100%, reduce = 0%
Ended Job = job_1508590228879_0004 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1508590228879_0004_m_000000 (and more) from job job_1508590228879_0004
Exception in thread "Thread-19" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: localat org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)at org.apache.hadoop.hive.ql.exec.JobTrackerURLResolver.getURL(JobTrackerURLResolver.java:42)at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:209)at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:92)at java.lang.Thread.run(Thread.java:744)
Execution failed with exit status: 2
Obtaining error information
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
看了网上的许多方法,有让在hive-site日志下加入这个属性:
<property><name>hive.exec.show.job.failure.debug.info</name><value>false</value>
</property>
然而并没有效果。刚好最近开十九大,整个VPN被封完了,也没法翻墙找答案,只能在内网大海捞针(答案千篇一律)。然而皇天不负有心人,最终总算在一个算是能访问的国外论坛中找到了答案,(同样的问题,十几个回答中找到了唯一正确的解决方法。。。)。至于原因,我还没搞懂(根据这个属性的设置,个人猜测应该通过添加这个设置的属性值,将程序源码执行中要抛出的异常给忽略了)。
废话少说直接上解决方案:
直接再hive-site下添加如下属性:
<property><name>mapreduce.jobtracker.address</name><value>ignorethis</value>
</property>
这样直接就OK了。
就因为一个这样的小问题就花了我快一天的学习时间,真TM要命。。。
希望后能为后来者提供下帮助。