Atlassian Engineering Unveils the Architecture and Scaling Strategy Behind Its Global Webhooks-Processor Infrastructure.

Atlassian’s ecosystem of products, including Jira, Confluence, and Bitbucket, generates a continuous stream of digital events. These range from the creation of a Jira issue and the merging of a Bitbucket pull request to the publishing of a Confluence page or the recording of an administrative action in an audit log. At the heart of this activity is Forge, Atlassian’s extensibility platform, which serves as the primary consumer of these events. To ensure these events reach Forge apps, repository hooks, and audit-log endpoints reliably and securely at an enterprise scale, Atlassian utilizes a dedicated service known as the webhooks-processor.

Described internally as Atlassian’s "events rail," this multi-tenant infrastructure is designed to evaluate, match, and deliver events from various products to the specific apps and systems that subscribe to them. The "rail" is engineered for modularity, allowing the addition of new event sources or destinations without requiring a fundamental rebuild of the underlying platform. As Atlassian’s cloud footprint expands, the capabilities and complexity of this rail have grown significantly to meet the demands of modern software development and collaboration.

The challenge of webhook delivery at Atlassian’s scale is more complex than a simple sequence of receiving an event and posting it to a URL. The system must manage massive volume while maintaining strict security and reliability. Three primary constraints shape the design of the webhooks-processor: the necessity of multi-tenant fairness, the requirement for high throughput, and the need for a contained blast radius to ensure that a failure in one area does not destabilize the entire system.

Architecturally, the webhooks-processor occupies a critical position between three distinct environments: the event producers (Atlassian products), the subscription registries (which determine who receives what), and the event recipients (Forge apps and external URLs). Atlassian products do not communicate with the webhooks-processor directly; instead, they publish to an internal event substrate. This substrate validates each event against a registered schema and persists it durably. Subscriptions are declared against this substrate, and a distribution layer fans events out to the service’s internal Events Queue. This ensures that all inbound work is standardized, featuring typed events with strong schemas and clear attribution to the originating site and tenant.

The processing pipeline is divided into two primary stages: Event Matching and Event Delivery. When an event arrives, the matching component first identifies which subscriptions are active for that specific event and then determines the appropriate destination for delivery. Matched events are then placed onto a Webhooks Queue, while unmatched events are discarded immediately. This "fast-fail" approach for unmatched events is intentional and monitored via metrics to understand the efficiency of the inbound stream.

The delivery component is responsible for pulling messages from the Webhooks Queue and dispatching them. For Forge apps, this involves routing the event into the Forge runtime environment. For other recipients, such as repository hooks, it involves an outbound HTTPS POST to a customer-controlled URL. Reliability is maintained through a series of integrated primitives, including rate limiting, retries with exponential backoff, and a dead-letter queue for messages that remain undeliverable after multiple attempts.

To prevent the service from becoming a complex web of conditional logic for different event types, Atlassian built the webhooks-processor around a per-surface configuration model. The core pipeline exposes well-defined interfaces for matching, delivery, and retry policies. Each specific event type—whether it be a Jira issue update or a Bitbucket repository event—supplies its own implementation of these interfaces. This allows the core system to remain agnostic of the specific details of any single event surface, enabling the platform to scale and accommodate new products with minimal friction.

A defining feature of the architecture is the separation of the pipeline into two stages with independent queues. This design choice is driven by the different scaling characteristics of matching and delivery. Event Matching is primarily CPU-bound. It involves registry lookups and the evaluation of complex expressions against subscriber criteria. A single matching pod can sustain tens of thousands of evaluations per second, provided its subscription caches are warm. In contrast, Event Delivery is network-bound. It must manage connections to external endpoints with varying latencies. A delivery pod’s performance is defined by its ability to handle hundreds of concurrent outbound HTTP exchanges without suffering from head-of-line blocking. By splitting these stages, Atlassian can scale each component based on the specific resource it consumes most.

Multi-tenant fairness is another cornerstone of the system’s design. Rather than relying on a single centralized mechanism, Atlassian employs a layered approach consisting of four distinct safeguards: a negative-lookup cache, queue isolation, per-tenant rate limiting, and per-recipient concurrency limiting.

The negative-lookup cache addresses the reality that most events have no subscribers. Because evaluation runs on every event while delivery only runs on matches, the evaluation path is extremely high-volume. The negative-lookup cache stores "no subscription" results for a short time, significantly reducing the workload on subscription registries for high-volume event types. This allows the system to absorb bursts of activity without wasting resources on unnecessary lookups.

Queue isolation is used to prevent unrelated workloads from interfering with one another. Each pipeline—such as those for product events or audit logs—operates on its own physically separate queue and worker fleet. This ensures that a traffic spike in one area cannot slow down processing in another. Per-tenant rate limiting further protects the system by capping the volume of outbound deliveries for any single app or tenant. This is enforced by a cluster-wide service to maintain consistency across all pods. Finally, per-recipient concurrency limiting prevents slow-responding endpoints from tying up the platform’s delivery capacity. If a recipient takes several seconds to respond, the system caps the number of in-flight deliveries to that specific URL, deferring further messages until capacity is freed.

Reliability is further reinforced through the use of stable identifiers and idempotency-friendly delivery. Every webhook carries a stable ID, allowing recipients to deduplicate events if they receive the same message twice during a retry cycle. The system also employs back-pressure on unhealthy paths; if a downstream dependency or recipient shows signs of struggle, the rail reduces the rate of work sent in that direction, recovering gradually only when signals improve. Circuit breakers are also utilized to ensure that if an internal upstream dependency stalls, the system fails fast rather than locking up worker threads.

The scale of the rail is currently experiencing compounding growth of approximately 18% per month across all production regions. This rapid expansion is driven by two major trends: the rise of AI agents and large-scale customer migrations to the cloud. AI agents, acting as first-class users within Atlassian products, generate machine-paced bursts of traffic that are often an order of magnitude denser than human activity. Simultaneously, large enterprises migrating from self-hosted Data Center environments to the Cloud bring massive volumes of event traffic onto the rail, often in significant step-changes on migration days.

The architectural decisions detailed by Atlassian—the two-stage split, the layered fairness model, and the per-surface configuration—are designed to ensure the platform can handle the next order of magnitude in volume without requiring a rebuild. By building a single, well-engineered events rail that supports all event types, Atlassian ensures that new features and products inherit high-standard delivery discipline, fairness, and observability from their first day of operation. This disciplined approach to engineering extensibility platforms allows the webhooks-processor to remain a reliable foundation for the thousands of apps and millions of users within the Atlassian ecosystem.