Hadoop/spark安装实战（系列篇4） Hadoop MapReduce词频统计之小试牛刀
运行hadoop 自带的例子的MapReduce 计算
1 上传文件到hadoop的hdfs的根目录
[root@localhost hadoop-1.2.1]# hadoop fs -put README.txt /
检验
[root@localhost hadoop-1.2.1]# hadoop fs -ls /
Found 2 items
-rw-r--r-- 1 root supergroup 1366 2015-09-12 06:45 /README.txt
drwxr-xr-x - root supergroup 0 2015-09-12 06:34 /home
[root@localhost hadoop-1.2.1]#
2 运行MapReduce 分词统计
[root@localhost hadoop-1.2.1]# hadoop jar hadoop-examples-1.2.1.jar wordcount /README.txt /wordcountoutput
Warning: $HADOOP_HOME is deprecated.15/09/12 06:48:00 INFO input.FileInputFormat: Total input paths to process : 1
15/09/12 06:48:00 INFO util.NativeCodeLoader: Loaded the native-hadoop library
15/09/12 06:48:00 WARN snappy.LoadSnappy: Snappy native library not loaded
15/09/12 06:48:03 INFO mapred.JobClient: Running job: job_201509120634_0001
15/09/12 06:48:04 INFO mapred.JobClient: map 0% reduce 0%
15/09/12 06:48:27 INFO mapred.JobClient: map 100% reduce 0%
15/09/12 06:48:34 INFO mapred.JobClient: map 100% reduce 33%
15/09/12 06:48:36 INFO mapred.JobClient: map 100% reduce 100%
15/09/12 06:48:36 INFO mapred.JobClient: Job complete: job_201509120634_0001
15/09/12 06:48:36 INFO mapred.JobClient: Counters: 29
15/09/12 06:48:36 INFO mapred.JobClient: Job Counters
15/09/12 06:48:36 INFO mapred.JobClient: Launched reduce tasks=1
3/运行结果
[root@localhost hadoop-1.2.1]# hadoop fs -ls /wordcountoutput
Warning: $HADOOP_HOME is deprecated.Found 3 items
-rw-r--r-- 1 root supergroup 0 2015-09-12 06:48 /wordcountoutput/_SUCCESS
drwxr-xr-x - root supergroup 0 2015-09-12 06:48 /wordcountoutput/_logs
-rw-r--r-- 1 root supergroup 1306 2015-09-12 06:48 /wordcountoutput/part-r-00000 [root@localhost hadoop-1.2.1]# hadoop fs -cat /wordcountoutput/part-r-00000
Warning: $HADOOP_HOME is deprecated.(BIS), 1
(ECCN) 1
(TSU) 1
(see 1
5D002.C.1, 1
740.13) 1
<http://www.wassenaar.org/> 1
Administration 1
Apache 1
BEFORE 1
BIS 1
Bureau 1
Commerce, 1
Commodity 1
Control 1
Core 1
Department 1
ENC 1
Exception 1
Export 2
For 1
。。。。。
hadoop小试牛刀 OK
