Apache ZooKeeper: Distributed Coordination Service
GitHub Repo
Apache-2.0
November 21, 2026 at 01:54 PM
0 views

Apache ZooKeeper: Distributed Coordination Service

@apacheProject Author

What Apache ZooKeeper is

Apache ZooKeeper is a coordination service for distributed systems. Its overview document describes it as "a distributed, open-source coordination service for distributed applications" that "exposes a simple set of primitives that distributed applications can build upon to implement higher level services for synchronization, configuration maintenance, and groups and naming." The motivation, in the project's own words, is that coordination services "are notoriously hard to get right" and prone to race conditions and deadlock, so applications should not have to build them from scratch.

In practice ZooKeeper is the small, strongly ordered store that sits underneath larger systems: a place to keep cluster membership, leader locks, configuration flags and service locations. It is not a general database. The overview says the data at each node "is usually small, in the byte to kilobyte range", and the administrator's guide calls the roughly 1 MB per-request limit "really a sanity check". Clients get a file-system-like tree, change notifications and ephemeral entries tied to a session, and build locks, barriers, queues and leader election on top.

The project lives at apache/zookeeper on GitHub, a repository created on 27 March 2009. It is written in Java, licensed under Apache-2.0, and the NOTICE.txt file reads "Copyright 2009-2026 The Apache Software Foundation". The current stable line is 3.9; this page tests 3.9.7, the newest version on the Apache download mirror when checked.

How it works

Data model: znodes, versions, ephemerals and sequences

The namespace is a tree of paths separated by slashes. Each node, a znode, can hold data and children at the same time; the overview compares it to "a file-system that allows a file to also be a directory". Every znode carries a stat structure with version numbers for data, children and ACL changes, plus creation and modification transaction IDs (czxid, mzxid). Reads return the whole value, writes replace it, and a write can be made conditional on an expected version.

Three node flavours matter most. Ephemeral znodes exist only while the session that created them is alive. Sequential znodes get a counter appended to the name; the programmer's guide says the counter is formatted as %010d, is "unique to the parent znode", and is a signed 32-bit integer that overflows past 2147483647. Container and TTL nodes, added in 3.5.x, are cleaned up by the server, but TTL nodes "must be enabled via System property as they are disabled by default".

Watches and sessions

A client can set a watch while reading. The classic watch "will be triggered and removed when the znode changes", so it fires once. Since 3.6.0 there are also persistent, recursive watches that are not removed when triggered and cover a whole subtree; the 3.9.7 CLI exposes them as addWatch. Sessions are kept alive by heartbeats. The guide says the server bounds the requested session timeout between 2 and 20 times tickTime, which with the default 2,000 ms tick is 4 to 40 seconds; the server logged exactly that range on startup.

Ensemble, leader and ZAB

Production deployments run as a replicated ensemble. Every server keeps the full tree in memory and persists a transaction log plus periodic snapshots. "As long as a majority of the servers are available, the ZooKeeper service will be available." Clients connect to any one server. Reads are answered from that server's local replica. Writes are forwarded to a single elected leader, which turns them into transactions that followers acknowledge through ZooKeeper's own atomic broadcast protocol. The ClickHouse Keeper documentation names this protocol ZooKeeper Atomic Broadcast (ZAB) and points out the consequence: reads are not linearizable by default, because each node serves them locally. ZooKeeper's guarantees section lists sequential consistency, atomicity, a single system image, reliability and timeliness; a client that needs the latest value can call sync before reading.

Each server in a replicated config needs two extra ports: one for followers to talk to the leader (2888 in the docs' examples) and one for leader election (3888). A myid file in the data directory tells each server which server.N line is its own. Optional observers receive the replicated log without voting, and dynamic reconfiguration can change membership at runtime, but the administrator's guide says reconfigEnabled defaults to false.

Key features in 3.9.7

  • Hierarchical, versioned key space with conditional updates, ACLs per znode and quotas on node count or bytes.
  • Coordination primitives: ephemeral, sequential, container and (opt-in) TTL nodes; one-shot and persistent recursive watches. The 3.9.7 release notes add limits on batched multi-read requests (zookeeper.multiRead.maxOps, default 1000, and zookeeper.multiRead.maxResponseBytes, default 64 MB).
  • Recipes documented in the repo: the recipes guide covers barriers and double barriers, queues and priority queues, locks (including shared and revocable shared locks), two-phase commit and leader election.
  • Client bindings: the getting-started guide lists Java and C bindings as "functionally equivalent", with single- and multi-threaded C variants. Third-party clients such as kazoo (Python) speak the same wire protocol.
  • Operations tooling: zkServer.sh, zkCli.sh (28 commands in 3.9.7), zkCleanup.sh for retention, and snapshot and transaction-log toolkits for offline inspection.
  • Monitoring: four-letter-word commands over the client port, an embedded Jetty AdminServer with JSON endpoints, JMX, and an optional Prometheus metrics provider (port 7000 by default) with a Grafana dashboard template referenced in the monitoring guide.
  • Security options: TLS for client and quorum traffic, SASL/Kerberos, digest and X.509 authentication. The 3.9.6 and 3.9.7 release notes include FIPS-mode fixes, TLS cipher changes (JDK defaults since 3.9.6) and dependency upgrades for Netty and Jackson CVEs.
  • High availability features: observers, read-only mode for partitioned servers (off by default), and dynamic reconfiguration (off by default).

Getting started

The project publishes source and binary tarballs through the Apache mirrors; the README says apache-zookeeper-[version]-bin.tar.gz "Contains all the jar files required to run ZooKeeper", and that Maven Central artifacts are deployed after each release vote. After unpacking, the getting-started guide asks for a config file at conf/zoo.cfg:

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181

Then the server starts with:

bin/zkServer.sh start

And the CLI connects with:

$ bin/zkCli.sh -server 127.0.0.1:2181

The guide walks through a first znode:

[zkshell: 9] create /zk_test my_data
Created /zk_test

For a replicated ensemble, each server gets the same config with the member list, plus a myid file in its data directory:

tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

The guide notes that for several servers on one machine, each needs a distinct client port and distinct quorum and election ports (for example 2888:3888, 2889:3889, 2890:3890). Four-letter words other than srvr must be allowed explicitly, for example:

4lw.commands.whitelist=stat, ruok, conf, isro

After which the administrator's guide shows a health check like:

$ echo ruok | nc 127.0.0.1 5111
imok

The same guide recommends binding the AdminServer to localhost, or turning it off if unused:

admin.serverAddress=127.0.0.1
admin.enableServer=false

Hands-on: Apache ZooKeeper 3.9.7 on a Linux VM

Everything in this section was run on 21 November 2026 in a cloud VM: Ubuntu 24.04.5 LTS, Linux 6.18, x86_64, 2 vCPUs, about 7.8 GiB RAM, no swap, no Docker, and no internet access other than PyPI and raw GitHub files. Java was OpenJDK 21.0.13. The input was apache-zookeeper-3.9.7-bin.tar.gz from dlcdn.apache.org, downloaded on another machine and checked against the official .sha512 file. Clients were the bundled zkCli.sh and kazoo 2.11.0, a third-party Python client installed from PyPI. All servers ran on one machine over loopback, so the timings are only indicative and say nothing about cross-host latency or real disks.

The distribution

CheckResult
Archive22,946,610 bytes, 1,725 entries; 53 MB unpacked (lib/ 21 MB, docs/ 33 MB, mostly Javadoc)
Server jarszookeeper-3.9.7.jar (1,373,194 bytes), zookeeper-jute-3.9.7.jar, zookeeper-prometheus-metrics-3.9.7.jar
Bundled dependenciesJetty 9.4.58, Netty 4.1.138, logback 1.3.15, slf4j-api 2.0.13, Jackson 2.22.3, JLine 3.30.14, snappy-java 1.1.10.5, Prometheus simpleclient 0.9.0; netty-tcnative native jars for five platforms add 6.1 MB
Version string3.9.7-0cc8919035e246f6a6e3ebb4ef1df15c4243bce7, built on 2026-10-22 14:12 UTC
Default JVM flags from zkEnv.sh-Xmx1000m for the server, -Xmx256m for the CLI; JMX enabled; -XX:OnOutOfMemoryError=kill
conf/zoo_sample.cfgtickTime=2000, initLimit=10, syncLimit=5, dataDir=/tmp/zookeeper (with a comment not to use /tmp), autopurge lines commented out

Standalone start-up, memory and defaults

With the three-line config from the guide, zkServer.sh start returned in 1.06 s. Measured from launching start-foreground to the first successful srvr reply, four starts took 0.82, 0.94, 0.96 and 1.46 s. The log showed about 0.37 s from reading the config to binding port 2181. Resident memory five seconds after start was 99,696 to 108,800 KiB (about 100 MB); after the throughput runs below it had grown to 230,192 KiB with the default 1,000 MB heap ceiling.

Observed defaultValue
Session timeout boundsLog: "minSessionTimeout 4000 ms maxSessionTimeout 40000 ms"
Client port binding0.0.0.0:2181
AdminServer"Started AdminServer on address 0.0.0.0, port 8080 and command URL /commands"
AutopurgesnapRetainCount 3, purgeInterval 0, "Purge task is not scheduled."
Transaction log filePreallocated to 67,108,880 bytes (64 MiB) as soon as the first write arrived
Snapshot compressionzookeeper.snapshot.compression.method = CHECKED

Four-letter words versus the AdminServer

The four-letter-word whitelist behaved as documented: srvr answered, while ruok, stat and mntr on port 2181 each returned "… is not executed because it is not in the whitelist." zkServer.sh status still worked because it uses srvr. The AdminServer on port 8080, however, answered the same commands over HTTP with no authentication: /commands/ruok, /commands/stat, /commands/mntr, /commands/environment and /commands/system_properties all returned 200, the last two exposing the Java version and the OS user (root here). The index listed 28 commands. Only snapshot and restore refused, with HTTP 401 and "Auth info is missing for the command". In practice, the 4lw whitelist does not restrict access while the AdminServer is running with its defaults; use the admin.* settings shown above to lock it down. Setting admin.enableServer=false removed the listener completely (connection refused on 8080).

zkCli and znode behaviour

ActionResult
One zkCli.sh command per invocationAbout 1.05 s each, almost all JVM start-up
create -e /app/eph, then create -s /app/seq- twiceseq-0000000001 and seq-0000000002: numbering started at 1, because the parent's counter had already moved when the ephemeral child was created
Ephemeral znode after a one-shot zkCli.sh command exitsStill listed by other clients for 31.4 s, until the log showed "Expiring session …, timeout of 30000ms exceeded"
Ephemeral znode after kazoo stop()Gone 2 ms later (session closed explicitly)
create -t 5000 /app/ttl with default configKeeperErrorCode = Unimplemented for /app/ttl, as the programmer's guide warns
One-shot data watch, then two set callsFired once (CHANGED, 7.7 ms after the write), not again on the second write
zkCli.sh help28 commands, including addWatch, version and whoami

The 1 MB limit

The administrator's guide gives the default jute.maxbuffer as 1,048,575 bytes, "just under 1M", and says it is the "maximum size of the data that can be stored in a znode". In practice the limit applies to the whole request packet. With an 11-character path, a binary search found the largest value that could be created was 1,048,517 bytes; 1,048,518 bytes failed. The failure is not a clean error. The server logged java.io.IOException: Len error ("advertised length of 1048576 … greater than jute.maxbuffer=1048575") and dropped the connection, so the client saw ConnectionLoss and reconnected. Code that retries on connection loss will therefore retry an oversized write that can never succeed. A value just under the limit took 19 ms to create and 3.7 ms to read back.

Throughput with a small Python client

kazoo ran a single client: 2,000 synchronous creates, sets and gets of 100-byte values, then 20,000 pipelined asynchronous sets and gets. Two standalone runs and three ensemble runs (one per server) give an idea of the spread. With one Python process on a 2-vCPU machine, the client is likely the bottleneck for the asynchronous numbers.

TargetSync createSync set (p50)Sync get (p50)Async setAsync get
Standalone, run 1563 ops/s1,184 ops/s (0.61 ms)3,775 ops/s (0.21 ms)4,373 ops/s6,040 ops/s
Standalone, run 21,128 ops/s906 ops/s (0.52 ms)5,445 ops/s (0.18 ms)5,908 ops/s4,774 ops/s
Ensemble, via follower 1325 ops/s484 ops/s (1.24 ms)2,656 ops/s (0.25 ms)2,685 ops/s4,570 ops/s
Ensemble, via leader527 ops/s671 ops/s (0.98 ms)2,290 ops/s (0.21 ms)2,488 ops/s4,599 ops/s
Ensemble, via follower 3545 ops/s623 ops/s (1.10 ms)1,422 ops/s (0.32 ms)3,108 ops/s4,503 ops/s

The pattern matches the design: reads cost about the same anywhere, while writes in the three-node ensemble roughly halve throughput and double median latency compared with standalone, even over loopback, because every write needs a quorum acknowledgement and a log append on each server. Synchronous p99 latencies for writes ranged from 5.7 to 15.7 ms.

Three-node ensemble, leader failover and quorum loss

Three servers ran on one host with client ports 2181 to 2183, quorum and election ports 2888:3888, 2889:3889 and 2890:3890, AdminServer ports 8081 to 8083 and 4lw.commands.whitelist=*. Started about a second apart, server 2 became leader ("LEADING - LEADER ELECTION TOOK - 297 MS"); server 1, which started first and had to wait for peers, logged 1,938 ms. Each idle server used about 102 MB RSS. mntr on the leader reported zk_synced_followers 2 and zk_quorum_size 3.

For failover, a kazoo client with all three addresses wrote in a tight loop while the leader was killed with kill -9.

TrialNew leader visible via srvrLongest gap between successful writesClient errors
1741 msnot recorded1 ConnectionLoss
2431 ms960 ms1 ConnectionLoss
3467 ms980 ms1 ConnectionLoss
4446 ms650 ms1 ConnectionLoss

Election itself was logged at 238 to 316 ms in the trials whose logs survived (zkServer.sh start overwrites the .out log on restart). The overview's figure of "less than 200ms to elect a new leader" comes from a 7-machine benchmark of release 3.2. From the application's side the write pause was 0.65 to 1 s, plus one connection-loss exception to handle.

Killing two of the three servers left the survivor answering srvr with "This ZooKeeper instance is not currently serving requests". Both a write and a read through it timed out after 15 s, with the client in the SUSPENDED state. Reads stop too unless read-only mode is enabled. After one server was restarted, the first write succeeded 3.61 s after the start command, JVM start-up included.

Snapshots, transaction logs and cleanup

A standalone server with snapCount=1000 received 500 creates and 10,000 sets. It produced 14 snapshots of 91,993 bytes and 14 transaction logs. Each log reported 67,108,880 bytes, so du --apparent-size showed 898 MB, while actual disk use was 3.1 MB because the preallocated files are sparse on this file system. Monitoring that counts apparent sizes will overstate usage. zkCleanup.sh -n 3 kept three snapshots and four logs. -n 2 was rejected, because the count "should be greater than or equal to 3". Autopurge was not timed, since its interval is set in whole hours. After the benchmarks, ensemble member 1 held a 379,631-byte snapshot and two 64 MiB logs (15 MB on disk), plus acceptedEpoch and currentEpoch files.

Enabling the Prometheus provider with two lines in zoo.cfg started a second Jetty listener on 0.0.0.0:7000 that served 764 lines and 180 metric families, including znode_count and JVM heap use.

Not tested

TLS, SASL and Kerberos; dynamic reconfiguration; observers; read-only mode; rolling upgrades; the C client; Apache Curator (Maven Central was unreachable); real multi-host networking and disk fsync latency; and the 3.10.0 candidate. No container images were tried because the VM has no Docker.

Use cases

  • Leader election and locks for a pool of workers, using ephemeral sequential znodes as the recipes guide describes, so a crashed holder releases the lock when its session expires.
  • Service discovery and group membership: each instance registers an ephemeral znode, and consumers watch the parent for changes. The programmer's guide mentions getEphemerals() for this pattern.
  • Shared configuration and feature flags that many processes read often and few write, the "read-dominant" workload the overview says ZooKeeper suits best, at ratios "around 10:1".
  • Coordination for existing systems that still use a ZooKeeper-compatible store, for example ClickHouse replication, where ClickHouse Keeper is the documented drop-in alternative.
  • Development and integration tests using a standalone server, which starts in about a second and needs around 100 MB of RAM.

How Apache ZooKeeper compares

Descriptions come from each project's own documentation read on 21 November 2026; none of the alternatives were run here.

ProjectConsensus and languageClient interfacePositioning in its docsLicence
Apache ZooKeeper 3.9.7Own atomic broadcast protocol (ZAB); JavaOwn binary protocol; Java and C bindings, third-party clientsGeneral coordination service: znodes, watches, ephemerals, recipesApache-2.0
etcdRaft; GogRPC API, etcdctl"distributed reliable key-value store for the most critical data of a distributed system"; often used with Kubernetes; README claims "benchmarked 10,000 writes/sec"Apache-2.0
ClickHouse Keeper (part of ClickHouse)Raft via NuRaft; C++ZooKeeper-compatible client protocol, so "any standard ZooKeeper client can be used"Coordination for ClickHouse replication and distributed DDL; can run standalone or inside the serverApache-2.0 (ClickHouse)
KRaft in Apache KafkaKafka controller quorumInternal to KafkaKafka's metadata quorum; the docs say ZooKeeper-to-KRaft migration needs a bridge release, "The last bridge release is Kafka 3.9"Apache-2.0

etcd is the most direct general-purpose alternative: a replicated key-value store with Raft consensus, a gRPC API and automatic TLS, per its README. Its README says it is frequently paired with Kubernetes. ZooKeeper's distinguishing features are its tree model, ephemeral and sequential znodes, watches and the recipes in its own docs. Porting between the two means mapping each primitive to the other's API, so check every one you rely on before switching.

ClickHouse Keeper is explicitly a ZooKeeper replacement for ClickHouse. Its docs say it gives "the same guarantees as ZooKeeper: linearizable writes and non-linearizable reads" by default, with an option for quorum reads. The same docs warn that snapshots, logs and the inter-server protocol are incompatible, so a mixed cluster "is impossible" and migration requires stopping ZooKeeper and converting data with clickhouse-keeper-converter.

Kafka is not an alternative for general coordination, but it matters for ZooKeeper's install base. Current Kafka documentation describes KRaft controllers and treats ZooKeeper only as a migration source, so new Kafka clusters do not need a ZooKeeper ensemble. For metrics on any of these, the ZooKeeper monitoring guide documents scraping with Prometheus and importing a Grafana dashboard.

Limits, trade-offs and when not to use it

  • Not a data store. Values are capped at roughly 1 MB per request (1,048,517 bytes in the test), the whole tree lives in memory on every server, and the guide warns that large znodes cause "unwarranted latency spikes" and unstable quorums. Put payloads elsewhere and keep pointers in ZooKeeper.
  • Writes do not scale with servers. Each write goes through the leader and a quorum; the test showed ensemble writes at about half of standalone throughput even over loopback. Adding voters increases fault tolerance, not write capacity. Observers help with read fan-out only.
  • Reads can be stale. A follower serves reads from its local replica. Call sync first when a read must reflect the latest committed write.
  • Quorum loss stops everything by default. With two of three servers down, reads and writes both stalled. Two-server ensembles are worse than one, as the getting-started guide explains; use three or five.
  • Insecure defaults on the HTTP side. The AdminServer listens on 0.0.0.0:8080 without TLS or authentication and exposes commands that the 4lw whitelist blocks on the client port. Port 8080 also clashes with many other services. The Prometheus provider adds another listener on all interfaces.
  • Operators must manage disk. Autopurge is off by default, transaction logs are preallocated in 64 MiB files, and the docs recommend a dedicated log device for production. Set autopurge.purgeInterval or schedule zkCleanup.sh.
  • Sessions shape behaviour. Ephemeral znodes from a client that exits without closing its session linger for the full session timeout (31 s observed with zkCli.sh). Oversized writes surface as ConnectionLoss, not as a size error.
  • JVM footprint. About 100 MB RSS idle per server and a 1,000 MB default heap ceiling. That is small for a JVM service but larger than single-binary alternatives.
  • Documentation drift observed. The 3.9.7 administrator's guide lists Java "1.8 or greater (JDK 8 LTS, JDK 11 LTS, JDK 12 - Java 9 and 10 are not supported)", yet 3.9.7 ran on JDK 21 without problems. Its PurgeTxnLog example names slf4j-api-1.7.30.jar and logback-classic-1.2.10.jar, while the 3.9.7 tarball ships slf4j-api 2.0.13 and logback 1.3.15. The getting-started guide's help listing omits addWatch, version and whoami. The 3.9.6 release notes list ZOOKEEPER-5050, "Disable AdminServer and enhance documentation to highlight security considerations", but the 3.9.7 guide and the test both show the AdminServer enabled by default. The overview's performance figures date from release 3.2.
  • Upcoming Java baseline change. The 3.9.7 build targets Java 8 (maven.compiler.release 8). The pom.xml at the release-3.10.0-0 tag sets version 3.10.0 with maven.compiler.release 17. That tag looks like a release candidate: 3.10.0 was not on the download mirror on 21 November 2026, so plan for a Java 17 minimum once it ships, but do not treat it as released.

Release lines and documentation sources

The Apache download mirror listed three lines on 21 November 2026: 3.9.7, 3.8.8 and 3.7.2. The 3.9.7 release notes cover client thread shutdown fixes after authentication failures, stream and resource leak fixes in the transaction log code, iterative snapshot serialization, the new multi-read limits, and upgrades of Netty to 4.1.138 and Jackson to 2.22.3 for CVE fixes. The 3.9.6 notes add TLS and FIPS fixes, AdminServer TLS settings, removal of hard-coded TLS cipher lists, and a fix for Prometheus provider passwords being logged in clear text.

On master, the root pom.xml declares version 3.11.0-SNAPSHOT and replaces the zookeeper-docs module with a zookeeper-website module; the markdown paths used by the 3.9.7 docs return 404 there. Use the documentation from the release tag that matches the deployed version, which is what this page does. The project also points to its website, zookeeper.apache.org, and a Confluence wiki for contribution guidelines.

Project activity

Checked 21 November 2026: latest release 3.9.7 (tag release-3.9.7, commit dated 22 October 2026; the binary reports a build time of 22 October 2026), licence Apache-2.0, per the Apache download mirror, the release tag and the repository's licence file. A release-3.10.0-0 tag (commit dated 10 November 2026) exists but 3.10.0 was not yet on the mirror. The apache/zookeeper repository had about 12,800 stars (12,833 at the time of writing) and 7,330 forks. It is written in Java, was created on 27 March 2009, develops on master (last pushed 20 November 2026) and is not archived. Documentation and downloads are at zookeeper.apache.org.

Enjoying this project?

Discover more amazing open-source projects on TechLogHub. We curate the best developer tools and projects.

Project
apache-zookeeper
Created
November 21
Last Updated
November 21, 2026 at 01:54 PM

Maintain Apache ZooKeeper: Distributed Coordination Service? Add the badge to your README

Show that the project is featured on TechLogHub and send readers to this write-up.

Featured on TechLogHub
[![Featured on TechLogHub](https://techloghub.com/badge-neutral.svg)](https://techloghub.com/open-source/apache-zookeeper?utm_source=techloghub_badge&utm_medium=referral&utm_campaign=readme)

Find more projects like this

One email a week: new and trending developer tools, fresh comparisons, and what shipped. Unsubscribe in one click.