zookeeper05 - using ZooKeeper

3. Start using ZooKeeper

3.2. The first ZooKeeper session

Use zkServer and zkCi tools under bin / directory in ZooKeeper for debugging and management.

Use the client to establish a session

]# zkCli.sh    --
...
--The client starts the program to establish a session.
2021-12-22 09:37:37,464 [myid:] - INFO  [main:ZooKeeper@868] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@73a8dfcc
2021-12-22 09:37:37,476 [myid:] - INFO  [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-12-22 09:37:37,492 [myid:] - INFO  [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes
2021-12-22 09:37:37,505 [myid:] - INFO  [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled=
--Client attempted to connect to localhost/127.0.0.1:2181
2021-12-22 09:37:37,518 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Welcome to ZooKeeper!
JLine support is enabled
--Client connection succeeded,The server starts initializing this new session.
2021-12-22 09:37:37,675 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:35332, server: localhost/127.0.0.1:2181
--Session initialization completed successfully. conversation ID(sessionid)Is 0 x20000103b190002
2021-12-22 09:37:37,710 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x20000103b190002, negotiated timeout = 30000

WATCHER::

----The server sends a message to the client SyncConnected event.
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]

Simple use session

--List root(root)All under znode. Now out/zookeeper outside, znode There are no other trees znode
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
--Create a file named/workers of znode
[zk: localhost:2181(CONNECTED) 1] create /workers ""
Created /workers
--see/workers
[zk: localhost:2181(CONNECTED) 2] ls /
[workers, zookeeper]
[zk: localhost:2181(CONNECTED) 3] ls /workers 
[]
--delete znode
[zk: localhost:2181(CONNECTED) 4] delete /workers 
[zk: localhost:2181(CONNECTED) 5] ls /
[zookeeper]
--Exit session
[zk: localhost:2181(CONNECTED) 6] quit

WATCHER::

WatchedEvent state:Closed type:None path:null
2021-12-22 09:56:02,382 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x20000103b190002 closed
2021-12-22 09:56:02,390 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x20000103b190002

4. Implementation of an example of master-slave mode

This example is for teaching purposes only and we do not recommend using zkcli SH build the system.

The master-slave model includes three roles:
- Master node: the master node is responsible for monitoring new slave nodes and tasks, and assigning tasks to available slave nodes.
- Worker (slave node): the slave node registers itself through the system to ensure that the master node sees that they can perform tasks, and then starts monitoring new tasks.
- Client: the client creates a new task and waits for a response from the system.

4.1. Master node role

Because only one process can become a master, a process must lock / Master in ZooKeeper to become a master. To do this, the process creates a temporary znode named / Master:
- A temporary node is automatically deleted when the session expires or closes.

--Open first session( zkCli.sh)，Session as master

--Create a file named/master of znode，In order to obtain management rights. use-e Flag to represent the created znode It is temporary.  --Adding host information is not necessary. It is only to show that we can add data when needed.
[zk: localhost:2181(CONNECTED) 0] create -e /master "master1.example.com:2223"
Created /master
--list ZooKeeper The root of a tree.
[zk: localhost:2181(CONNECTED) 1] ls /
[master, zookeeper]
--obtain/master Data.
[zk: localhost:2181(CONNECTED) 2] get /master
master1.example.com:2223

If other processes want to be master, try to create a / master node. Let's see what happens:
- ZooKeeper tells us that / Master already exists. In this way, the second process will know that there is already a master.

--Open second session( zkCli.sh)，Sessions as slave nodes
[zk: localhost:2181(CONNECTED) 0] create -e /master "master2.example.com:2223"
Node already exists: /master

The master node may crash, while the slave node may need to take over the role of the master node. To detect this, the slave node needs to set a monitoring point on / Master:
- stat command can get the properties of znode nodes and set monitoring points on existing znode nodes- The w option means to add a monitoring point.

--Second session (currently slave session)
[zk: localhost:2181(CONNECTED) 1] stat -w /master
cZxid = 0x100000028
ctime = Wed Dec 22 23:06:11 CST 2021
mZxid = 0x100000028
mtime = Wed Dec 22 23:06:11 CST 2021
pZxid = 0x100000028
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x100004099d30008
dataLength = 24
numChildren = 0

When the master node crashes, we observe the following conditions:
- The NodeDeleted event indicates that the primary node's session has been closed or expired.

--Close the first session and observe the second session
[zk: localhost:2181(CONNECTED) 2] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeDeleted path:/master

The master node no longer exists. Now the slave node can become the master node by creating / master.

--Second session (will be the master session)
[zk: localhost:2181(CONNECTED) 2] create -e /master "master2.example.com:2223"
Created /master

4.2. Slave node, task and task allocation

Before discussing the steps taken from the node and client, create three important parent znode: / workers, / tasks, and / assign.
These three new znodes are persistent znodes and do not contain any data. In this example, using these znodes can tell us which slave nodes are available, tell us when there are tasks to be assigned, and assign tasks to the slave nodes.

--Open a session to create a persistent znode，Close the session after creation
[zk: localhost:2181(CONNECTED) 0] create /workers ""
Created /workers
[zk: localhost:2181(CONNECTED) 1] create /tasks ""
Created /tesks
[zk: localhost:2181(CONNECTED) 2] create /assign ""
Created /assign

In a real application, these three znodes need to be created by the main process before starting to allocate tasks, or by a boot process. No matter how they are created, once they exist, the master node needs to monitor the changes of the child znodes of / workers and / tasks.
- The ls command uses the - w option to set monitoring points on the corresponding znode.

--Second session (currently the master session)

--monitor/workers Change of
[zk: localhost:2181(CONNECTED) 3] ls -w /workers 
[]
--monitor/tesks Change of
[zk: localhost:2181(CONNECTED) 4] ls -w /tasks
[]

4.3. Slave node role

The first step of the slave node is to inform the master node that it can perform tasks. It represents it by creating a temporary znode under / workers. The slave node uses the host name to identify itself.

--Reopen the first session( zkCli.sh)，Sessions as slave nodes
[zk: localhost:2181(CONNECTED) 0] create -e /workers/worker1.example.com "worker1.example.com:2223"
Created /workers/worker1.example.com

The master node has previously monitored the changes of the child nodes of / workers. Once the slave node creates a znode under / workers, the master node will observe the following notification information.

--Second session (currently the master session)
[zk: localhost:2181(CONNECTED) 5] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/workers

Next, the slave node needs to create a parent znode (/ assigning / Worker1. Example. Com) to receive the assigned tasks. The - w option of the ls command is used to monitor the changes of this node in order to wait for a new task.

--First session (currently slave session)
[zk: localhost:2181(CONNECTED) 1] create /assign/worker1.example.com ""
Created /assign/worker1.example.com
[zk: localhost:2181(CONNECTED) 2] ls -w /assign/worker1.example.com
[]

The slave node is now ready to receive the assigned tasks

4.4 client role

The client adds tasks to the system. In this case, it doesn't matter what the task is. Here, we assume that the client requires the master-slave system to run the cmd command. To add tasks to the system, the client needs to do the following:

--Open a third session( zkCli.sh)，Use as client session
[zk: localhost:2181(CONNECTED) 0] create -s /tasks/task- "cmd"
Created /tasks/task-0000000000

We need to add znode in the order of adding tasks, which is essentially a queue. The client must now wait for the task to complete. After the slave node executing the task completes the task, a child znode will be created under "/ tasks/task -" to represent the task status. The client determines whether the task is completed by checking whether the child znode is created. Therefore, the client needs to monitor the creation event of "/ tasks/task -".

--Third session (currently client session)
[zk: localhost:2181(CONNECTED) 1] ls -w /tasks/task-0000000000
[]

The master node has previously monitored the child znode of / tasks. Once the client creates a znode under / tasks, the master node will observe the following notification information.

--Second session (currently the master session)
[zk: localhost:2181(CONNECTED) 5] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/tasks

After that, the master node will check the new task, obtain the list of available slave nodes, and assign the task to the available slave nodes.

--Second session (currently the master session)
[zk: localhost:2181(CONNECTED) 5] ls /tasks 
[task-0000000000]
[zk: localhost:2181(CONNECTED) 6] ls /workers 
[worker1.example.com]
[zk: localhost:2181(CONNECTED) 7] create /assign/worker1.example.com/task-0000000000 ""
Created /assign/worker1.example.com/task-0000000000

The slave node receives the notification of task assignment (the slave node has previously monitored / assign/worker1.example.com):

--First session (currently slave session)
[zk: localhost:2181(CONNECTED) 3] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/assign/worker1.example.com

Check the new task from the node and confirm whether the task is assigned to yourself:

--First session (currently slave session)
[zk: localhost:2181(CONNECTED) 3] ls /assign/worker1.example.com
[task-0000000000]

Once the task is executed from the node, it will add a znode representing the status in "/ tasks/task -":

--First session (currently slave session)
[zk: localhost:2181(CONNECTED) 4] create /tasks/task-0000000000/status "done"
Created /tasks/task-0000000000/status

The client receives the notification of task execution completion and checks the execution results:

--Third session (currently client session)
[zk: localhost:2181(CONNECTED) 2] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/tasks/task-0000000000

[zk: localhost:2181(CONNECTED) 2] get /tasks/task-0000000000
cmd
[zk: localhost:2181(CONNECTED) 3] get /tasks/task-0000000000/status 
done

The client checks the contents of the status znode and confirms the execution result of the task. In this case, it has been executed successfully and the result is "done". Of course, the task can also be very complex, even involving another distributed system. In the end, no matter what kind of task it is, the mechanism of executing the task and transmitting the results through Zookeeper are essentially the same.

4.5. Three sessions

4.5.3. First session

--First session (currently the master session)
[zk: localhost:2181(CONNECTED) 0] create -e /master "master1.example.com:2223"
Created /master
[zk: localhost:2181(CONNECTED) 1] ls /
[master, zookeeper]
[zk: localhost:2181(CONNECTED) 2] get /master
master1.example.com:2223

--First session (currently slave session)
[zk: localhost:2181(CONNECTED) 0] create -e /workers/worker1.example.com "worker1.example.com:2223"
Created /workers/worker1.example.com
[zk: localhost:2181(CONNECTED) 1] create /assign/worker1.example.com ""
Created /assign/worker1.example.com
[zk: localhost:2181(CONNECTED) 2] ls -w /assign/worker1.example.com
[]
[zk: localhost:2181(CONNECTED) 3] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/assign/worker1.example.com

[zk: localhost:2181(CONNECTED) 3] ls /assign/worker1.example.com
[task-0000000000]
[zk: localhost:2181(CONNECTED) 4] create /tasks/task-0000000000/status "done"
Created /tasks/task-0000000000/status

4.5.2. Second session

--Second session (currently slave session)
[zk: localhost:2181(CONNECTED) 0] create -e /master "master2.example.com:2223"
Node already exists: /master
[zk: localhost:2181(CONNECTED) 1] stat -w /master
cZxid = 0x100000004
ctime = Thu Dec 23 10:51:28 CST 2021
mZxid = 0x100000004
mtime = Thu Dec 23 10:51:28 CST 2021
pZxid = 0x100000004
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x1000011abdf0000
dataLength = 24
numChildren = 0

--Second session (currently the master session)
[zk: localhost:2181(CONNECTED) 2] create -e /master "master2.example.com:2223"
Created /master
[zk: localhost:2181(CONNECTED) 3] ls -w /workers 
[]
[zk: localhost:2181(CONNECTED) 4] ls -w /tasks
[]
[zk: localhost:2181(CONNECTED) 5] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/workers

[zk: localhost:2181(CONNECTED) 5] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/tasks

[zk: localhost:2181(CONNECTED) 5] ls /tasks 
[task-0000000000]
[zk: localhost:2181(CONNECTED) 6] ls /workers 
[worker1.example.com]
[zk: localhost:2181(CONNECTED) 7] create /assign/worker1.example.com/task-0000000000 ""
Created /assign/worker1.example.com/task-0000000000

4.5.3 third session

--Third session (currently client session)
[zk: localhost:2181(CONNECTED) 0] create -s /tasks/task- "cmd"
Created /tasks/task-0000000000
[zk: localhost:2181(CONNECTED) 1] ls -w /tasks/task-0000000000
[]
[zk: localhost:2181(CONNECTED) 2] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/tasks/task-0000000000

[zk: localhost:2181(CONNECTED) 2] get /tasks/task-0000000000
cmd
[zk: localhost:2181(CONNECTED) 3] get /tasks/task-0000000000/status 
done

3. Start using ZooKeeper

3.2. The first ZooKeeper session

Use zkServer and zkCi tools under bin / directory in ZooKeeper for debugging and management.

Use the client to establish a session

]# zkCli.sh    --
...
--The client starts the program to establish a session.
2021-12-22 09:37:37,464 [myid:] - INFO  [main:ZooKeeper@868] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@73a8dfcc
2021-12-22 09:37:37,476 [myid:] - INFO  [main:X509Util@79] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-12-22 09:37:37,492 [myid:] - INFO  [main:ClientCnxnSocket@237] - jute.maxbuffer value is 4194304 Bytes
2021-12-22 09:37:37,505 [myid:] - INFO  [main:ClientCnxn@1653] - zookeeper.request.timeout value is 0. feature enabled=
--Client attempted to connect to localhost/127.0.0.1:2181
2021-12-22 09:37:37,518 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1112] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
Welcome to ZooKeeper!
JLine support is enabled
--Client connection succeeded,The server starts initializing this new session.
2021-12-22 09:37:37,675 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@959] - Socket connection established, initiating session, client: /127.0.0.1:35332, server: localhost/127.0.0.1:2181
--Session initialization completed successfully. conversation ID(sessionid)Is 0 x20000103b190002
2021-12-22 09:37:37,710 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1394] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x20000103b190002, negotiated timeout = 30000

WATCHER::

----The server sends a message to the client SyncConnected event.
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]

Simple use session

--List root(root)All under znode. Now out/zookeeper outside, znode There are no other trees znode
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
--Create a file named/workers of znode
[zk: localhost:2181(CONNECTED) 1] create /workers ""
Created /workers
--see/workers
[zk: localhost:2181(CONNECTED) 2] ls /
[workers, zookeeper]
[zk: localhost:2181(CONNECTED) 3] ls /workers 
[]
--delete znode
[zk: localhost:2181(CONNECTED) 4] delete /workers 
[zk: localhost:2181(CONNECTED) 5] ls /
[zookeeper]
--Exit session
[zk: localhost:2181(CONNECTED) 6] quit

WATCHER::

WatchedEvent state:Closed type:None path:null
2021-12-22 09:56:02,382 [myid:] - INFO  [main:ZooKeeper@1422] - Session: 0x20000103b190002 closed
2021-12-22 09:56:02,390 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@524] - EventThread shut down for session: 0x20000103b190002

Keywords: Zookeeper

Added by kaisellgren on Sun, 23 Jan 2022 01:57:24 +0200

Programming VIP

zookeeper05 - using ZooKeeper

3. Start using ZooKeeper

3.2. The first ZooKeeper session

4. Implementation of an example of master-slave mode

4.1. Master node role

4.2. Slave node, task and task allocation

4.3. Slave node role

4.4 client role

4.5. Three sessions

4.5.3. First session

4.5.2. Second session

4.5.3 third session

3. Start using ZooKeeper

3.2. The first ZooKeeper session

Popular Keywords