Historically, Web servers like those offered by Apache and Microsoft have been the main solutions for delivering content over the Web. However, these servers are not designed to handle long-lived HTTP responses, but are designed to serve requests as quickly as possible, complete the response, and move on to the next request. While a long-lived connection may not be a problem in itself, traditional Web servers will not scale when serving long-lived connections to large audiences, creating demand for reinvention of the traditional Web server.

Traditional threading models use a synchronous, blocking thread model, in which each connection served requires a dedicated thread. This results in rapid degradation of server performance as the number of concurrent users increases.

To vertically scale, servers must have the ability to process asynchronous requests. Asynchronous request processing (ARP) allows servers to handle higher volumes of concurrent sessions – a characteristic of both AJAX and Comet applications – with the least number of threads possible. Twisted Python and Java Native IO (NIO) are two examples of technologies that support ARP. Non-blocking technologies such as ARP are able to release resources while waiting for another event, allowing a single thread to serve multiple connections. Therefore, the number of clients served is no longer directly proportional to the number of threads a machine can handle. Comet servers like Enterprise Comet provide vertical scalability by supporting asynchronous requests.

Horizontal Scalability
Few Comet servers provide a means beyond long-polling and ARP to cope with high volumes of concurrent users. Neither approach used in isolation or combination completely addresses the substantive hardware requirements necessary to serve many thousands of concurrent users.

Clustering schemes are also employed by application servers to meet high demands for resources. Clustering allows clients to be served by a pool of servers running on separate machines, thus distributing resource allocation. Comet clusters generally fall into two categories – embedded and distributed – that are primarily a result of the server’s integration point with the message origin, i.e., the data source.

In an embedded architecture – a Comet server running in an application server – clients are pinned directly to a specific application server instance in the cluster, and shared data is forced to be replicated among all the nodes in the cluster. This, in turn, requires high volumes of network traffic overhead.

In addition, Comet integration at the application server level lacks the benefit of separating the servicing of long-lived connections from the execution of time-sensitive application logic. Sharing a single resource to serve both of these concerns can introduce a point of contention for applications that need to respond to tens of thousands of requests within sub-second intervals.

To address this issue, some Comet implementations sit alongside an application server in a so-called distributed architecture. These implementations typically rely on traditional HTTP load balancing with Comet session affinity for clustering. This form of clustering works well to offset connection loads, but still requires a significant amount of state management in terms of message queues and client subscriptions.

Enterprise Comet enables subscription groups to be partitioned at the application server independent of long-lived connections. This leads to significant scalability benefits, especially when the frequency of publishing messages far exceeds the frequency of subscribing to groups.

Enterprise Comet does not store message queues within the confines of the application server, but instead distributes message queues across a delivery network. Each node on the network maintains a queue of messages for each browser connected to that particular node. This distribution balances the resources for state management across a larger set of resources, enabling Enterprise Comet to serve more long-lived connections and allowing it to rely on streaming as a primary method of message delivery.

The careful distribution of message queues also enables better fault tolerance, freeing up more nodes to serve orphaned clients in the case of a node failure.

Message Delivery
Volatile data can easily cause message storms, and with thousands of concurrent connections, CPU and bandwidth utilization will suffer.

In most Comet implementations every message sent has to be copied across multiple connections, possible thousands of connections. Just imagine sending one message to 100,000 users, and repeat that every 50ms!

Reliable Message Delivery
Enterprise Comet extends enterprise-messaging services to the browsers. As such, Enterprise Comet must ensure the reliable delivery of messages to clients no matter their location. Therefore, the server provides message sequencing to Comet, allowing clients to maintain a record of the last message delivered and enabling the client to re-query for messages that may have been lost in transit.

Since message queues are distributed and replicated across the Enterprise Comet network, buffers are highly fault tolerant and primed for resends in the event of a failure.

Fair Message Delivery
Another feature unique to Enterprise Comet is the introduction of fair messaging. Fair messaging enables administrators to create policies that govern the timeliness of message delivery for all clients connected to an application. All connected clients send information about the quality of their connection to the Enterprise Comet node they are served by. The aggregation of this information is used by Enterprise Comet to interpret the time to deliver a message to any client in the network. This information can then be used by administrators to govern access requirements to an application by bandwidth or time the delivery of information to a diverse set of clients connected at various rates.

Group Message Delivery
Few Comet servers provide a scalable means to broadcast information to large groups of users, e.g., channels of subscribers. This is largely due to the servers’ need to send multiple copies of a single message rather than sending a single message that is routed to multiple destinations, which in turn avoids otherwise high CPU loads.

Enterprise Comet introduces the concept of multi-way routing over HTTP to overcome redundant message sends. This strategy for message delivery enables the server to send a message over a single connection to multiple destinations without sending copies of the message over multiple connections. Enterprise Comet does this by fanning out message delivery across a network of Enterprise Comet nodes, where each point in the delivery network serves a subset of the connections served by an application.

The benefit of distributing message delivery this way extends beyond eliminating message copying to facilitating greater distribution of connections across machines. This, in turn, allows more finely tuned load balancing, resulting in better overall application performance and the ability to serve tens of thousands of clients.

Global Message Delivery
Group Messaging and multi-way routing are not only limited to the internal networks of the enterprise, but also include the delivery of messaging services to customers across the globe. Enterprise Comet’s ability to fan out messages efficiently to clients connected across its delivery network allows high-performance messaging to any client connected to the network’s edge – a place once limited to the static caching of content can now dynamically stream data with the accuracy of real time (see Figure 1).

Summary
Comet has laid the foundation for the future of Web applications and dethroned the stateless “click-and-wait” Web dinosaurs. By using Comet, developers can now build event-driven solutions deployed for the Web without needing plug-ins or downloads.