System Design Series: A Step-by-Step Breakdown of Temporal’s Internal Architecture

1️⃣ Foundations: Workflows, Activities, and Tasks

• Workflow: Deterministic orchestration of business logic (e.g., booking flow, video processing).

• Activity: Executes actual work (API calls, file processing, sending emails).

• Task: A unit of work Temporal assigns to a worker — either a workflow task or an activity task.

2️⃣ Level 0 – Core Temporal Services

• Frontend Service: Entry point for API/gRPC calls; routes requests to other services.

• History Service: Owns workflow runs, tracks event history, manages mutable state, persists state transitions.

• Matching Service: Routes workflow/activity tasks to workers via task queues.

• Worker Service: Runs internal system workflows (archival, deletions, DLQ handling).

• Workers (user-defined): Poll task queues and execute workflows/activities.

3️⃣ Level 1 – Scaling with Shards & Task Queue Partitions

• History Service Shards: Fixed number (e.g., 512/1024), each owning a subset of workflow runs. Enables horizontal scaling.

• Transfer Queue: Inside each shard, holds tasks to be sent to Matching Service.

• Matching Service Partitions: Task queues split into partitions for scalability; transparent to developers.

4️⃣ Level 2 – Additional Queues & Worker Polling

• Extra Queues in History Service:

  • Timer Queue: For timeouts, delays, retries, cron.

  • Replication Queue: For multi-cluster replication.

  • Visibility Queue: Updates searchable workflow records.

  • Archival Queue: Triggers archival of completed workflows.

• Worker Pollers: Multiple threads per worker for workflow and activity tasks; tuning is important to avoid over-polling.

• Task Queue Tree Structure: Optimizes forwarding when partitions are empty or overloaded.

🏗 Key Architectural Insights

• Temporal separates orchestration from execution — it never runs your workflow code itself, only schedules it.

• Sharding and partitioning are the backbone of Temporal’s scalability.

• Deterministic workflows + durable event history = fault tolerance and replayability.

• Multiple internal queues handle different system responsibilities, ensuring modularity.

🎯 Takeaway

Temporal’s architecture is layered and modular, designed for massive scale, fault tolerance, and developer productivity. Understanding its internals helps in tuning performance, scaling workers, and designing resilient workflows.