Recently I have formatted namenode and while starting hadoop daemon datanode was failing and giving error as below
2019-01-11 10:39:15,449 WARN org.apache.hadoop.hdfs.server.common.Storage: Failed to add storage directory [DISK]file:/app/hadoop/tmp/dfs/data/ java.io.IOException: Incompatible clusterIDs in /app/hadoop/tmp/dfs/data: namenode clusterID = CID-76c39119-061a-4ecf-9de1-3a6610ca57dd; datanode clusterID = CID-90359d7f-b1a5-431e-8035-bc4b9e2ea8b9
As a resolution I have deleted tmp and created again and its working file, also copying the namenode CID to datanode CID in version file it started working.
While looking my backups I can see CID was different for datanode and namenode and that was working earlier.
Can someone provide some detail what each value represent and both CID need to be same or different?