Install Cloudera step by step

We need at least 4 vm. 1 namenode and 3 datanodes.

In this tutorial I’m using Centos 7.0 and cloudera 5.x free edition.

first, create run.sh that contains command like this:

#!/bin/sh
echo "SELINUX=disabled" > /etc/selinux/config
echo "SELINUXTYPE=targeted" >> /etc/selinux/configchkconfig iptables off
service iptables stop
setenforce 0
yum update -y
yum install -y yum-utils
yum install -y epel-release
yum install -y htop
yum install -y wget
yum install -y unzip
yum install -y nano
wget https://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
ssh-keygen -t rsa -q -f "$HOME/.ssh/id_rsa" -N ""
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
sestatus
cat ~/.ssh/id_rsa.pub
init 6

After restart, kindly run another command to reduce warning when installing cloudera manager via wizard:

echo "10" > /proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

install cloudera manager from namenode (node01.hadoop.com)

wait until you see the installation is complete, so we can move on to web UI wizard

installation complete

After that, let’s move to web UI. http://<your ip address>:7180

cloudera manager login page

default login is admin/admin. make sure you change the password right after login.

List all the hostname nodes, and cloudera will automatically detect the IP.

node01.hadoop.com
node02.hadoop.com
node03.hadoop.com
node04.hadoop.com
node05.hadoop.com

Leave all configuration by default and click continue

cluster installation

Wait until cluster installation is success.

cluster installation success

If you find this message,

warning message

please kindly run this commands on every host

echo "10" > /proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Make sure all validations are green:

green validations

Later you can also select which host to install particular service. By default, namenode will also install all service at once in one server and datanodes will only have datanode service and node manager.

You can use internal database for metadata, or you can choose another existing database. Make sure you success to connect to your db before go to next part.

choose internal database

For cluster setup, you can also set the minimum datanode machine failure. The default datanode number are 3, so if you have more than three you can define number_of_datanode-3 .
For example, I have 4 datanodes, so, I can set datanode tolerated failure by (4–3) = 1. So, if you have only 3 datanodes, just fill it with 0.

wait until cloudera deployment is done

you’ll get this warm welcome message after done installing.

success

here’s the main dashboard of Cloudera Manager

main dashboard

You can check warning message in left side. As long as it’s warning, you don’t have to worry.

For example the warning message for HDFS is java heap size. You should increase java heap size memory to get better performance. But if you don’t it still can run as is.