SSH Key Authentication for Hadoop

Hadoop cluster setup requires SSH key based authentication among master and slave nodes. Using SSH key based authentication, master node can connect to slave nodes or secondary nodes to start/stop the daemons\processes without any password.

If password-less SSH is not setup, user has to manually specify the password on each individual machine to start all the processes.

Steps for Password-Less SSH in Hadoop

1. Create an SSH

$ ssh-keygen -t rsa -P “”

Use the ssh-keygen command to create an SSH key

Here:

-t: type of key to be created  

-P: indicates an empty password

Alternatively you can use directly “ssh-keygen -t rsa”

Go to .ssh directory and list the files.

[/home/username]$ cd .ssh

[/home/username/.ssh]$ ls

Here you will find id_rsa.pub (public key) and id_rsa (private key) files.

2. Copy the SSH Public Key to the Remote Host

[/home/username/.ssh] $ cp id_rsa.pub authorized_keys

3. Give access to SSH key

[/home/username/.ssh] $ chmod 755 authorized_keys

Now if you need to setup SSHless between multiple host in distributed environment you need to copy ‘authorized_keys' to ‘~/.ssh’ location with permission 700 on all host, this can be archive in single line like beow

[hasnain@host1 ~]$ cat ~/.ssh/id_rsa.pub | ssh host2 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 640 ~/.ssh/authorized_keys'

There is short cut command to do this is below:

[hasnain@host1 ~]$ ssh-copy-id host2

To validate login host 2 from host1, it should ask for password anymore.

[hasnain@host1 ~]$ ssh -i ~/.ssh/id_rsa host2

In Hadoop installation preparation it will ask for sudo less password to root for this you need to simply append user which you are using for hadoop installation (hasnain here) as no password in ‘/etc/sudoers’ like below at the end of the file.

echo "hasnain        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers

alternately you can use visudo and add below line at the end of the file

hasnain        ALL=(ALL)       NOPASSWD: ALL

Now run your hadoop, it will not ask for password.