Setting up Namenode metadata location
The most critical component of Hadoop is Namenode, and it is important to safeguard the information it stores. It stores metadata, which is a combination of namespace and inode structure.
All the steps are to be performed as the hadoop user. It is expected that the user has gone through Chapter 1, Hadoop Architecture and Deployment and understands the uses and function of Namenode.
Getting ready
You are going to need a preinstalled Hadoop as discussed in Chapter 1, Hadoop Architecture and Deployment. In the following recipes, we will configure the parameters for a copy of Hadoop that is already installed.
How to do it...
sshto the Namenode, which in this case isnn1.cluster1.com.- Navigate to the
/opt/cluster/hadoop/etc/hadoopdirectory. This is the directory where we installed Hadoop in the first chapter. If the user has installed it at a different location, then navigate to this directory. - Configure the
dfs.namenode.name.dirparameter, which defines the location where Namenode will persist its metadata. - See the following screenshot for the contents of the file:
- It is good practice to have the
/data/namenodemount point on a different disk drive, rather than from a disk on which the OS is installed. - In production, there will always be more than one directory to store the metadata, as shown in the following screenshot:
- Once the changes are made, save the file and restart the
namenodedaemon. - We need to format the Namenode so that all the directories listed are in sync and initialized.
- If the user already had a running cluster, then copy the contents across the directories and do not format the Namenode, as it will result in the loss of data.
How it works...
With steps 3 to 5, the user will be able to configure the Namenode metadata store location. Usually, in production, one of the common separated lists of directories is a cross mount from the NFS server. This is done to make sure that, if the Namenode server goes down, we can still access the data by quickly mounting it on another machine.
Please start the Datanode daemons on all nodes configured to run as Datanodes. If the Datanodes were initially connected to another Namenode, then the Datanode directories specified using dfs.datanode.data.dir must be cleaned.