Load balancing with sticky sessions (httpd / Tomcat)

Archive - Originally posted on "The Horse's Mouth" - 2009-10-29 07:14:53 - Graham Ellis
Subject: Load balancing with sticky sessions, using Apache http server (version 2.2.14 in my example) with mod_proxy_balancer as the front end balancer and Apache Tomcat 6.0.20 as the application engine. [[Tip should also work for other recent 2.2.x and 6.0.x versions]]

Background

When you have too many big requests for one Tomcat server to look after the compute for all of them, you can set up an Apache http server ("Apache httpd") to act as a switchboard for all incoming requests, farming them on to a whole series of Tomcats to do the real work. It's rather like having a receptionist at the hotel - we only have one on duty, but he / she has plenty of capacity to check the customers in, and deal with any follow up inquiries during their stay. The receptionist leaves them to get on in their rooms with watching the large TVs, having a relaxing shower, working, sleeping ... and wouldn't dream of actually doing those other things with them, nor would he/she have the time to do so.

When a new customer checks in, a 'random' available room can be allocated from the appropriate pool. But when follow up requests are made by a customer, they have to be applied to the right room. Jo Smith in room 210 will love a knock on his door at 05:00 to tell him that the taxi for Melinda and Robert Brown in room 218 has arrived! And so it is with httpd - initial requests can be passed to any appropriate system, but then follow up requests which are part of the same sequence must be forwarded to the same machine (((Possible exception - clustering - see following article))).

Using mod_proxy_balancer without maintaining state

Here's an example where requests to the url
http://www.melkshamhotel.co.uk/manor/
will be passed on to a balancer called "robin", which will share out the traffic between two machines on its local subnet, running the web application there that's called "latmjdemo". If neither of these machines has a running Tomcat on it, then a third machine has been set up as a hot standby to take the traffic.

ProxyPass /manor/ balancer://robin/

# Two servers, sharing traffic (2 to 1 ratio)

# Third server is only used when neither of the others is available

<Proxy balancer://robin>

BalancerMember http://192.168.200.218:8880/latmjdemo loadfactor=10

BalancerMember http://192.168.200.210:8880/latmjdemo loadfactor=5

BalancerMember http://192.168.200.219:8880/latmjdemo status=+H

</Proxy>

That code works well, except that "state is not maintained". In hotel terms, our receptionist will provide it to a random room when asked for a follow up service. The scheme works well for 'single shot' requests, but fails where there's a whole series of pages involved such as in an online ordering system, where cookies will be involve. We need to tell our receptionist that a request isn't a new one - it's for the folks in room 218!

Maintaining state with mod_proxy_balancer

Your Java Servlet application that's running on Tomcat will almost certainly be using the built in session handling capacities within the code - we have examples here and here. Such applications will serve a cookie - in hotel terms, a room card - which the customer presents to the receptionist at each subsequent service request.

Then:

a) Each individual Tomcat needs to have a unique "jvmroute" set - there will be a line in the server.xml file that needs to be configure along the lines of:

<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm6">

b) The mod_proxy_balancer needs to be told which cookie is the one that's used for the sticky session, and which jvmroute has been used on which server in the balanced group. Here's a configuration sample:

ProxyPass /manor/ balancer://rednose/ stickysession=JSESSIONID|jsessionid

# Ensure that cookies get rewritten on their way back

ProxyPassReverseCookiePath /latmjdemo /manor

<Proxy balancer://rednose>

BalancerMember http://192.168.200.218:8880/latmjdemo/servlet loadfactor=10 route=jvm6

BalancerMember http://192.168.200.210:8880/latmjdemo/servlet loadfactor=10 route=jvm7

BalancerMember http://192.168.200.219:8880/latmjdemo/servlet loadfactor=10 route=jvm8

</Proxy>

These are working, tested examples. The Apache httpd mod_proxy_balancer manual is somewhat unclear with regard to the need for the route to be set (at both Tomcat and in the balancer) and indeed it could be (mis)read to imply that it isn't necessary to set the route - as id it has some large table of all the sessionids so that it knows which to forward to where. If you try to run balanced sticky sessions without the route, you'll fail - you'll be passed to what appears to be a random Tomcat each time, even if you have "stickysession" set in your ProxyPass.

Other Notes

1. With PHP, your session cookie does NOT by default include a "jvmroute" and you'll need to find a mechanism to add one. Mark Round has blogged about this here - a story that's beyond this Java based post, but we can help delegates work this out on our LAMP deployment course or on a one on one day if you already know the rest of the LAMP stuff!

2. You should ensure that you have code in your proxy configuration that prevents you being used as an open proxy ...

3. Under normal circumstances, sticky sessions are very hard indeed to test as you're looking to get virtually identical resources from each machine in the group. We have a test servlet here which reports on the server being hit by name, to help in debugging / tracking your session. (This same example will be used in my follow up article on clustering)

4. The balancer manager (part of mod_proxy_balancer) is a further useful monitoring tool. Turn it on as follows in the httpd configuration:

<Location /watch>

        SetHandler balancer-manager

        Order Deny,Allow

        Deny from all

        Allow from 192.168.200.

<Location>

5. There is a file containing various proxy configuration examples here, which we pulled in to our main httpd.conf via an include. You'll need to choose the bits you want - there are various conflicting proxies in there that will default to issuing warnings.

Main Content

Load balancing with sticky sessions (httpd / Tomcat)