System Design

Distributed Job Scheduler with Leader Election

System Design diagram · made in Skemio

Distributed job scheduler diagram showing consensus leader election with ZooKeeper, active leader dispatching tasks, and worker pools consuming queues.
Distributed job scheduler diagram showing consensus leader election with ZooKeeper, active leader dispatching tasks, and worker pools consuming queues.
Open it in SkemioMake your own

What it shows

  • Consensus engines use Raft or ZooKeeper leases to elect a single leader.
  • The active leader scans upcoming tasks and pushes them to worker queues.
  • Worker pools consume partitioned queues with heartbeats to prevent locks.

The takeaway

Elect a single coordinator for scheduling while worker pools execute jobs independently at scale.

More system design diagrams