Hadoop (setup in standalone)

 Pre-setup
1) install Hadoop
2) setup environment variable

• JAVA_HOME
• HADOOP_HOME

standalone
setup SSH for a Hadoop cluster

1) Define a common account
create a user level account with no Hadoop management privileges.  Assume it is "hadoopUser"

2) Generate SSH key pair
execute this command and following the prompts for additional inputs "ssh-keygen -t rsa"
the public keys are stored in location you have specified

3) Distribute the public key to all nodes (master and slaves)
scp <the location of your public key> hadoopUser@<hostname>:<new location>/master_key

on the target host, execute the following commands
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
$ mv ~/master_key ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys

4) Hadoop configuration
cd $HADOOP_HOME
In "hadoop-env.sh" add "export JAVA_HOME=/usr/share/jdk"

5) the 3 main configuration files should be empty

1. core-site.xml
2. hdfs-site.xml
3. mapred-site.xml

Hadoop runs completely on local machine and it doesn't launch any of the Hadoop daemons.