- 在配置namenode和jobtracker的ip时使用的是localhost ，将这两个改成namenode以及jobtracker本机的实际ip后，问题得以解决 。 具体的原因我也不太明白
- 确保master（namenode） 、slaves（datanode）的防火墙已经关闭
- 确保DFS空间的使用情况
- Hadoop默认的hadoop.tmp.dir的路径为/tmp/hadoop-${user.name},而有的linux系统的/tmp目录文件系统的类型往往是Hadoop不支持的。（ 本人在Ubuntu 10.10下测试伪分布式时就遇到了这种情况 ）
- 先后启动namenode、datanode（本人没有碰到过这样的情况 ）
• $hadoop-daemon.sh start namenode
$hadoop-daemon.sh start datanode- 查看hosts文件，仔细一看，好像是多了一个配置192.168.1.131 ubuntu add by netmanager ，把这个注释掉果然好了。ps：重启ubuntu后发现问题又出现了，应该是NetManager在重启后自动修改了hosts文件。关于自动修改hosts这个问题好像暂时无法 修复，是ubuntu系统的一个bug。后来问了下同事，发现将127.0.1.1多余的行注释掉就可以了（127.0.1.1 机器名）。
• 。。。在hadoop的日常输出会有一下信息 /************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = davidbeckham-ubuntu11/127.0.1.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
这里便是我出错的地方 我在master里边是localhost的 但是这里取到的信息在hosts里边换成了如上的信息。于是/etc/hosts速度修改之。
果然安好。这个错误都快把我折磨的没脾气了。• File /home/hexianghui/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 由这个可以认为是没有找到datanode。我用jps查看进程都正常。但是用web查看的话，live nodes为0. 这说明datanode没有正常启动，但是datanode进程又启动了，这是为何?网友可以跟帖提出意见或者指导…… 2009-12-30 22:02:19,190 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting JobTracker STARTUP_MSG: host = hexianghui/192.168.0.4 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.1 STARTUP_MSG: build =http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1-rc1 -r 810220; compiled by 'oom' on Tue Sep 1 20:55:56 UTC 2009 ************************************************************/ 2009-12-30 22:02:19,280 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) 2009-12-30 22:02:19,322 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=JobTracker, port=9001 2009-12-30 22:02:24,544 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2009-12-30 22:02:24,691 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50030 2009-12-30 22:02:24,692 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 2009-12-30 22:02:24,692 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50030 2009-12-30 22:02:24,692 INFO org.mortbay.log: jetty-6.1.14 2009-12-30 22:04:01,351 INFO org.mortbay.log: StartedSelectChannelConnector@0.0.0.0:50030 2009-12-30 22:04:01,353 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 2009-12-30 22:04:01,353 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 9001 2009-12-30 22:04:01,353 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030 2009-12-30 22:04:01,468 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory 2009-12-30 22:04:01,511 INFO org.apache.hadoop.mapred.CompletedJobStatusStore: Completed job store is inactive 2009-12-30 22:04:01,555 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hexianghui/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) at org.apache.hadoop.ipc.Client.call(Client.java:739) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.addBlock(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at $Proxy4.addBlock(Unknown Source) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2904) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2786) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262) 2009-12-30 22:04:01,555 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null2009-12-30 22:04:01,555 WARN org.apache.hadoop.hdfs.DFSClient: Could not get block locations. Source file "/home/hexianghui/tmp/mapred/system/jobtracker.info" - Aborting... 2009-12-30 22:04:01,556 WARN org.apache.hadoop.mapred.JobTracker: Writing to file hdfs://hexianghui:9000/home/hexianghui/tmp/mapred/system/jobtracker.info failed! 2009-12-30 22:04:01,556 WARN org.apache.hadoop.mapred.JobTracker: FileSystem is not ready yet! 2009-12-30 22:04:01,568 WARN org.apache.hadoop.mapred.JobTracker: Failed to initialize recovery manager. org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hexianghui/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) at org.apache.hadoop.ipc.Client.call(Client.java:739) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy4.addBlock(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at $Proxy4.addBlock(Unknown Source) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2904) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2786) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076) at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262) 2009-12-30 22:04:11,569 WARN org.apache.hadoop.mapred.JobTracker:- Retrying... 以下是我在网上找的解决方法集锦：方案1： 是否是防火墙未关闭，查看。确实忘记关闭防火墙了，因为我换了几台机器，以前是在虚拟机下用的是redhat，现在用的是ubuntu8.0.4。所以跟这个很可能相关。关闭iptables后，出现的错误信息没那么多了，但是还有错误。如下：
• 2010-01-03 22:08:25,073 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 9000, call addBlock(/user/hexianghui/input/file01, DFSClient_105866075) from 192.168.0.4:53604: error: java.io.IOException: File /user/hexianghui/input/file01 could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /user/hexianghui/input/file01 could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2010-01-03 22:08:25,087 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=hexianghui,hexianghui,adm,dialout,cdrom,floppy,audio,dip,video,plugdev,fuse,lpadmin,admin ip=/192.168.0.4 cmd=create src=/user/hexianghui/input/file02 dst=null perm=hexianghui:supergroup:rw-r--r--
2010-01-03 22:08:25,088 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9000, call addBlock(/user/hexianghui/input/file02, DFSClient_105866075) from 192.168.0.4:53604: error: java.io.IOException: File /user/hexianghui/input/file02 could only be replicated to 0 nodes, instead of 1
java.io.IOException: File /user/hexianghui/input/file02 could only be replicated to 0 nodes, instead of 1
虽然有类似的信息：could only be replicated to 0 nodes, instead of 1。但情况不一样了，可以对照上面的错误信息。继续接着找……方案2：Hadoop DFSClient警告NotReplicatedYetException信息
有时，当你申请到一个HOD集群后马上尝试上传文件到HDFS时，DFSClient会警告NotReplicatedYetException。通常会有一个这样的信息 -
WARN hdfs.DFSClient: NotReplicatedYetException sleeping <filename> retries left 3 |
08/01/25 16:31:40 INFO hdfs.DFSClient: org.apache.hadoop.ipc.RemoteException:java.io.IOException: File <filename>could only be replicated |
当你向一个DataNodes正在和NameNode联络的集群上传文件的时候，这种现象就会发生。在上传新文件到HDFS之前多等待一段时间就可以解决这个问题，因为这使得足够多的DataNode启动并且联络上了NameNode。
PS:我的实践：等待了几分钟，还是依然报错，此法不通。
方案3：http://trac.nchc.org.tw/cloud/wiki/waue/2009/0709
這個錯誤訊息意思是，他想要放檔案，但沒半個node可以給存取，因此我們需要檢查：
- 系統或hdfs是否還有空間 （像我就是）
- datanode數是否正常
- 是否在safemode
- 讀寫的權限
- 什麼都檢查過都正常的話，也只好砍掉重練了
PS:检查上面几个，1）系统空间够。df -hl查看。 2）datanode数是2.datanode用jps查看进程，都启动了。3，是否在safemode下。hadoop dfsadmin -safemode leave.使用后，可以正常拷贝了。也许之前的操作和这一步操作起作用了。
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
