Face Detection(OpenCV) Using Hadoop Streaming API 如何使用Python为Hadoop编写一个简单的MapReduce程序,请参考： Here.

MachineLP

发布于 2018-01-09 07:36:24

1K00

代码可运行

文章被收录于专栏：小鹏的专栏小鹏的专栏

运行总次数：0

代码可运行

如何使用Python为Hadoop编写一个简单的MapReduce程序,请参考： Here.

Face Detection(OpenCV) Using Hadoop Streaming API: 代码.

cd /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/
rm -f deploy/Imageprocessing.zip
zip -q -r deploy/Imageprocessing.zip Imageprocessing/

python DataPreProcessing.py /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/images/ SerializedImages.txt

cp SerializedImages.txt /tmp/liupeng/

cd ~

# 脚本权限
chmod +x /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/*

cd /usr/local/hadoop
 
# 在我们运行MapReduce job 前，我们需要将本地的文件复制到HDFS中
# /tmp/liupeng/SerializedImages.txt
liupeng@1:/usr/local/hadoop$ bin/hadoop dfs -copyFromLocal /tmp/liupeng liupeng 

# 执行 MapReduce job
liupeng@1:/usr/local/hadoop$ bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.8.1.jar 
    -file /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/FaceDetectorRun.py 
    -file /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/deploy/Imageprocessing.zip 
    -file /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/Classifier/haarcascade_frontalface_default.xml 
    -mapper /home/liupeng/hadoop/Face_Detection_Using_Hadoop_Streaming_API/FaceDetectorRun.py 
    -input liupeng/* -output liupeng-output