Apache Zookepeer Tutorial: Centralized Configuration Management
Establishing effective configuration management is an important step for building a distributed system. It is a complex process which helps in planning, identifying, tracking and verifying changes in the software. It is important to maintain configuration integrity throughout the life cycle of the system, this can be done by having good configuration management system.
Generally, configurations are divided into following groups:
- Global: configurations that are same for whole system.
- Environment specific: configurations that differ from environment to environment like they are different for development, test, etc.
- Service specific: configurations that are specific for a particular service.
- Instance specific: configurations that are responsible for identification of specific instances.
The divisions above do not have any fixed boundaries and often overlap each other. Usually this configuration data is stored in files that have some common and some individual sections. As the size of the system grows the file becomes difficult to maintain. If there are several platforms involved in the system then it gets all the more difficult to maintain as all the platforms should have same file copied and updated all the time. Continuously increasing size and variability of configuration data makes the task of maintaining the integrity and security of configuration file difficult.
Zookeeper is the easiest way for effective configuration management. It has two main benefits. First, it can be accessed from anywhere as it is stored centrally. This also reduces the issue with data integrity. Second, dynamic configuration management can be done as configuration data is stored centrally. This allows adjusting the system settings without restarting the system. Thus creating “znode” and storing configuration data is a handy way for configuration management.
Apache Zookeeper 3.4.6 with minimum 3 servers and 1 client.
Example below gives details of how to create znode, its subnodes, store configuration data, read the data and finally delete the node.
Basic initial steps:
- Start all the Zookeeper servers with the following command zkServer.sh start
- To confirm that the Zookeeper has started use the command jps and check for QuorumPeerMain.
- To start a client use command zkCli.sh -server Slave1:2181 on any of the server machines
- To understand who is the leader and follower in the Zookeeper use command zkServer.sh status on the console
Creating a directory or “znode”:
Create znode using:
For example here we create new znode with name dezyre and we add data loveZookeeper to it:
create /dezyre loveZookeeper
Reading data from znode:
Data can be read using simple get command
example: get /dezyre
This command will retrieve data that was stored in the znode dezyre and along with that it will also return some metadata like ctime i. e. time znode was created, mtime i.e. the time znode was last modified, dataVersion and dataLength. It also returns numChildren that shows the number of subdirectories or subnodes created in that particular znode. We will create subnode in further part.
Creating subdirectory or “subnode”:
To create a subnode it is important to have parent znode. Subnode can be created using:
create / /
create /dezyre /dezyreZookeeper lovelearningZookeeper
Znode can be deleted either individually or they can be deleted recursively. On deleting it individually only the nodes that are mentioned after the delete command gets deleted whereas in case of recursive removal, rmr is used to delete the znode as well all its subnodes recursively. To delete a znode:
To delete a subnode in znode:
Deletion of all the subnodes along with the parent node can be done using rmr command:
rmr can also be used to deleted znodes that do not have a child. Hence using rmr to delete node is safer option.