Load Balancing and Failover in the JBoss Application Server

Sacha Labourey
JBoss Group

JBoss is an open-source Java application server based on the Java 2 Platform, Enterprise Edition (J2EE). J2EE is a set of specifications that aim to define a distributed enterpise computing environment for Java. J2EE features dynamic Web pages (Java servlets and Java Server Pages, or JSPs), business components (Enterprise Java Beans, or EJBs), asynchronous communication (Java Messaging Service, or JMS), naming and directory services (Java Naming and Directory Interface, or JNDI), database access (Java Database Connectivity API, or JDBC), connection to sub-systems (Java Connector Architecture, or JCA), security, and so forth.

At the same time, J2EE leaves some capabilities required by a typical enterprise-ready environment undefined. Clustering of J2EE servers is one such example: vendors can provide a clustered implementation in any way, as long as that implementation complies with the J2EE specification.

Architecting a clustered J2EE implementation involves several concerns: cluster administration, load-balancing and failover, state replication, and so on. All these issues must be considered in light of both synchronous and asynchronous services. This article focuses on the JBoss J2EE server's load-balancing and failover features supporting Java clients and EJBs.

The JBoss Open Source project has been founded in 1999 by Marc Fleury. Currently, it has about 100 active developers, with 30 core developers, and more than 350,000 downloads per month. The current production-level release is version 3.2, and the upcoming 4.0 release is under development (release candidate versions are available for download). Clustering has been available in JBoss since release 3.0, with clustering support for all EJB types, JNDI, and Web sessions. JBoss 4.0 will have support for clustered JMS as well.

Dramatis Personae

Figure 1 depicts the JBoss distributed J2EE environment in which a client invokes a remote EJB component. To enable this mode of communication, such Java clients must first acquire a proxy object to the remote EJB, often from a naming and directory service. The actual implementation of a remote proxy is left to a J2EE vendor. In JBoss, a proxy represents an elaborate piece of code.

Figure 1: JBoss remote EJB invocation architecture

From the outside, a proxy (1) looks like the remote object: It implements the same interface, and forwards the invocations it receives to its server-side counterpart (2). That remote interface is termed the business interface. When the proxy's interface is invoked (3), that invocation is translated from a typed call against the business interface ("application-level programming") to an untyped call ("system-level programming"). The untyped call is no longer typed against the business interface, but against a generic system-level interface (5). To illustrate, consider the following application-level invocation:

That code would be translated to the following system-level invocation code (5):

Where the invocation parameter is an instance of the class Invocation, from which here is snippet:

public class Invocation {

   Object[] args; // arguments passed to the method
   Method method; // method being called

   Map payload; // arbitrary payload that can
                // be attached to the invocation;
	....
}

The detyped invocation will now traverse through a set of client-side interceptors (6 and 7). Each interceptor sequentially receives the invocation and has the ability to:

1. Analyze the content of the invocation and take any action, and/or
2. Add some arbitrary information inside the invocation object itself, based on the payload of the Invocation object, and/or
3. Read or extract any information placed in the payload of the invocation, and/or
4. Forward the request to the next interceptor in the chain, or
5. Shortcut the invocation path and directly return a value or an exception to the
   caller

The last interceptor in the chain will perform the actual invocation against the remote server using a transport-specific invoker (8). If the client is in the same Java virtual machine (JVM) as the target object, the interceptor will detect this situation and simply bypass the invoker to make a significantly faster direct Java call against the target object.

One interesting facet of this communication pattern is that the proxy remains transport agnostic: only the transport-invoker knows about the transport and protocol used to make the invocation travel through the network. Currently, JBoss provides RMI/JRMP, IIOP, HTTP, SOAP and optimized invokers.

On the return path, any interceptor can modify the result of the invocation: change the returned value, throw an exception, modify the type of an exception already thrown on the server, etc.

Clustering Extensions: Le Client Est Roi!

When designing JBoss clustering, we had to decide where the load-balancing/failover activity would occur (see Figure 2):

1. On one of the nodes, or
2. On an intermediary load-balancer, or
3. In the client application itself (inside the proxy).

Figure 2: Design alternatives for remote invocation failover

While each solution offers advantages and disadvantages, the last option's benefits outweigh the advantages of the others:

1. It removes any single point of failure, as there is no intermediary load-balancer or server involved in the communication path
2. The load-balancing activity can only die when the client application dies. That scarcely presents a problem as no one remains to complain about it
3. The performance cost of the load-balancing activity is minimal, or at least fully distributed: As the load-balancing activity takes place on the client, no server or intermediary load-balancer can become a bottleneck. The client pays the full price.

Consequently, we have chosen the last design for JBoss clustering. The client-side mechanism described in the first section of this article served as a basis for JBoss' load-balancing and failover features. We had to extend that architecture in three areas:

1. We needed to add the load-balancing and failover logic
2. We wanted the decision of which target node to use in an invocation to be configurable We wanted the proxy to have as correct a view of the cluster topology as possible so that it can make the best load-balancing/failover decision.

As a side note, a design that locates the load-balancing/failover activity on the client side is not new: CORBA's GIOP protocol, for example, has a multi-profile mechanism that populates object references (IOR) with alternate destinations in case a problem occurs with the main target. However, we will see that our solution provides a more flexible approach, thanks mostly to the generalized usage of Java on both the client and server sides.

Pluggable Load Balancing Logic

The load-balancing and failover logic is located in the last interceptor of the client-side proxy (see Figure 3, item (1)). However, instead of a single client-side invoker, the proxy now contains one invoker for each possible target node (2).

Figure 3: Load-balancing logic

The interceptor bases its routing decision on a pluggable load-balancing policy (see Figure 4):

Figure 4: Pluggable load-balancing policy

The last interceptor asks the load-balancing policy (1) to elect a target node. If the invocation succeeds (i.e. there are no system-level exceptions), the result of the invocation is simply sent back to the caller of the proxy. If the invocation fails, the interceptor will check if the exception is transparently recoverable.

One example of a transparently recoverable exception occurs when the target node becomes unreachable. The interceptor transmits this information to the load-balancing policy and asks it to elect a new target. The interceptor knows it is allowed to silently failover to another node. Because the target node has never been reached, no state could have been modified on the server. There is no risk of duplicate work, or incomplete work, that needs to be cleaned up by the client application.

On the other hand, if the interceptor receives a different type of exception, such as a database exception originating from the server, it cannot silently failover to another node. In that case, it will forward this exception to the caller and let the client application make its own business decision on how to solve the issue. The interceptor cannot generically solve this kind of situations.

While this behavior is highly flexible, as with any flexible solution, its configuration could be complex. Fortunately, that is not the case with JBoss: All configuration takes place of the server, and nothing needs to be configured on the clients.

JBoss's cluster configuration follows the J2EE pattern of XML deployment descriptors. Here is a typical JBoss-specific deployment descriptor for a single EJB:

<jboss>
<enterprise-beans>
  <session>
    <ejb-name>MySessionBean</ejb-name>
    <clustered>True</clustered>
    <cluster-config>
      <partition-name>DefaultPartition</partition-name>
      <home-load-balance-policy >
         org.jboss.ha.framework.interfaces.RoundRobin
      </home-load-balance-policy>
      <bean-load-balance-policy>
         org.jboss.ha.framework.interfaces.FirstAvailable
      </bean-load-balance-policy>
    </cluster-config>
  </session>
</enterprise-beans>
</jboss>

This descriptor uses all possible XML configuration tags. If the defaults are acceptable, the cluster-config tag is
optional.

Java's class-loader mechanism makes it possible to not have some of the classes available on the client-side (such as the load-balancing policy or one of the interceptors): Upon downloading the proxy on the client-side, the client would download the needed class bytecode from the JBoss server as well, in a lazy fashion.

Refreshing Cluster Topology

The JBoss clustering implementation allows for dynamic startup and shutdown of cluster nodes. There is no need to pre-define a static cluster composition. While that allows flexibility, it also means that if a proxy is downloaded to a client application, it may contain stale information about the current members of the cluster (see Figure 5).

Figure 5: Stale proxies

To avoid that problem, each proxy in JBoss has the opportunity to refresh its knowledge of the cluster members each time an invocation is performed against one of the nodes (see Figure 6):

Figure 6: Refreshing JBoss proxies

When a client makes a request against the cluster, the proxy wraps that invocation, and adds to it a view ID number that represents its current knowledge of the cluster composition (see Figure 6, item (1)). When the invocation reaches the server, the server-side of the invoker checks if the view ID, as seen by the client, matches the current view ID computed by the cluster (2). The view ID is simply a sum of all cluster members' name hash values. The intent was:

1. To have an ID that is easy to generate,
2. To have an ID that does not depend on member ordering, and
3. To be able to compute an ID which takes in account only a subset of the nodes in the cluster, for example only nodes that have a specific application running on them.

If they match, the invoker simply propagates the invocation to the target EJB and returns the answer to the client.

If they don't match, the invoker propagates the invocation to the target EJB and wraps the answer in a specific envelope. The invoker then populates that envelope with the new view-ID, as well as the new list of target nodes that now comprise the cluster (3). Once the answer returns to the client, the client-side invoker refreshes its knowledge of the cluster based on the information contained in the envelope, and forwards the invocation result to the user's code.

Performance

From a load-balancing/failover point of view, the performance cost of such an implementation is marginal. The decision process is not computationally intensive, and takes place exclusively on the client-side.

As this discussion focuses on the load-balancing/failover design of JBoss, it eludes the replication activity that could take place between the cluster nodes. Such replication might be important for components that maintain a distinct state between client invocations ("stateful components"). The presence of such components will impact performance, depending on the content of the sessions and the frequency at which that content changes. However, many applications rely on stateless services in which no replication is involved.

The Future

This article introduced the load-balancing and failover mechanism used in JBoss for EJB components that form part of the J2EE specification. JBoss 4.0, currently available in Developer Release, and its AOP1 (Aspect Oriented Programming) features aim to provide the developer with additional enterprise-level features, not only for EJB components, but for any object instance, even if source code for those objects is not available.

For example, suppose you have a POJO class (Plain Old Java Object) for which you lack the source code, and whose instances provide some kind of stateless service. The following lines of Java code bind a clustered proxy for an instance of POJO in a naming service:

POJO pojo = new POJO("hello");
POJO proxy = (POJO)ClusteredRemoting.registerClusteredObject(
          "objName",          // identity of this object in the cluster
           pojo,              // object for which to generate a proxy
           "DefaultPartition",// name of the cluster
           new RoundRobin(),  // load-balance policy
          "socket://localhost:5150"); // transport to be used by the
                                      // proxy

new InitialContext().bind ("myObject", proxy); // bind proxy in naming
                                               // service

Once these steps are performed on the server, any remote client application can obtain a clustered proxy to POJO. That proxy will allow a client to make remote invocations against the POJO instance, and failover to another node if a problem occurs.

The current JBoss 4.0 Developer Release already provides an interesting set of Aspects: clustering, remoting, transactional objects (ACID), role-based security, replicated objects (distributed cache), transactional locking (expanded Java synchronized) and persistence.

Conclusions

JBoss provides a very flexible and efficient clustering environment for J2EE applications. It supports transparent failover with marginal performance costs, and dynamic configuration of a cluster. Furthermore, thanks to the forthcoming 4.0 release, JBoss brings this value to any standard Java application, giving it access to advanced middleware and application server features.

Resource:

JBoss

J2EE Specification

JBoss Clustering project

JBoss Aspect-Oriented Framework

Sacha Labourey, JBoss Optimizations 101

Bill Burke and Sacha Labourey, Clustering with JBoss 3.0

Cogito Informatique