Hadoop Word Count - step by step execution part 2

Hadoop Word Count - step by step execution part 2

  10:08 PM    ssaikgame

---

Check the part 1 here, about how to write and generate *.jar file

Step 1: Download hortonworks sandbox
Get the Hortonworks Sandbox virtual box image from here,
note: i'm not using Hortonworks sandbox, here i have installed hadoop in my virtual Operating system. in your case you can use Hortonworks sandbox.

Open virtual box and select the image (Downloaded from Hortonworks) and click on start.
use credentials to login to the system.



you can connect to the image by using putty, here in my case i configured to 127.0.0.1 port 2222, and create a new directory



Step 2: Creating text file and copying jar file to virtual system.
create a text file that contains the any text, or just copy some code from wikipedia.com and save it as <filename>.txt




use winscp to copy the *.jar file from local system to virtual system, just use drag and drop, it's pretty easy to use.

Change the permission to 755, using chmod command.

Step 3: Copying text(input) file to hdfs
Create a new directory in Hadoop file system by using $hdfs dfs -mkdir ssaik
Check the available directories in the HDFS by using $hdfs dfs -ls

Now it's time to put the text file into the HDFS, using $hdfs dfs -put <filename.txt> /user/hadoop/ssaik/
<filename>.txt is the file, that created in ssaik directory in virtual system.



cat command is used to check the content of the file that is copied to hadoop file system.
$hdfs dfs -cat /user/hadoop/ssaik/<filename>.txt



Step 4: Running map reduce
$hadoop jar <filename>.jar <packagename>.<filename> <filename>.txt /user/hadoop/ssaik/output
/user/hadoop/ssaik/output is used for the output results.

the mapping and reducing progress can be seen on the screen.



Job tracker interface is used to check the state of the application.



Finally the output is



Every hadoop program is gives two files named, part-r-00000, _SUCCESS, can be check through hadoop interface 

Please like and comment.