Distributing the server load - yet ensuring that each user return to the same system (Apache httpd and Tomcat)
Archive - Originally posted on "The Horse's Mouth" - 2011-05-18 22:56:54 - Graham EllisHave you ever tried to sort out a complex situation with a supplier - perhaps something's gone wrong with your electic and gas bills (as happened with us a couple of months ago, when our gas payment got credited to our electic account with the same company). You get on the phone, and the initial person you speak to is in a "traige" roll - filtering calls, dealing with trivial ones, and passing the rest on to a whole team that's behind him/her dealing with the longer cases.
Such issues are rarely solved on a first call - we had to find our meters, look for meter numbers on them as well as readings, and phone back. If triage person puts you through to someone different, you're left explaining the whole thing from scratch again, but if you're able to be put through to the same person again, then they will (you hope) remember the initial details and you can pick up where you left off.
Similar issues can apply when you're using a web server for a major application. A server such as the Apache httpd Server may be used as the front end / triage system, making calls to other servers such as Apache Tomcat sitting behind it to perform more intensive operations. A user of a system such as a flight booking system will make a series of calls to the front end server, and if there's more than one server behind it will want to be routed to the same back server each time so that he can carry on from where he left off. The running of multiple servers behind the front end, with traffic distributed around them, is known as load balancing, and the passing of second and subsequent calls to the same system is known as haveing a sticky session.
On our Deploying Apache httpd and Apache Tomcat course (which I ran as a single-company course this Monday and Tuesday), we set up both servers and - where it's relevant to the customer - we set up load balanced servers with sticky sessions.
Although we're balancing the load between a whole series of Tomcats, most of the setup is done in Apache httpd, as - by the time a request gets to Tomcat - it's already been routed (the load balancer supplied in earlier versions of Tomcat that 6.0 was a Syren, tempting unway administrators off course and onto the rocks!). On Tuesday's course, I chose to add a single line to my httpd.conf file to include an extra set of configurations from a separate proxy forwarder file:
Include conf/extra/httpd-forwarder.conf(Complete httpd.conf example file, with other changes made on the course, [here])
The forwarder file itself is also on our web site - it's [here]. After a few headers which start proxying from a clean base, and allow it from any remote host, I defined a balancer group to include 2 machines:
<Proxy balancer://unicycle>
BalancerMember http://192.168.200.100:5080/latmjdemo loadfactor=5 route=jvm100
BalancerMember http://192.168.200.107:5080/latmjdemo loadfactor=10 route=jvm107
</Proxy>And I then requested that any requests for application "wide" get passed on to the balancer:
ProxyPass /wide/ balancer://unicycle/ stickysession=JSESSIONID|jsessionid
ProxyPassReverse /wide/ balancer://unicycle/
ProxyPassReverseCookiePath /latmjdemo /wideLet's look at those in more detail:
• I'm asking for request from new clients to be passed on to servers at 192.168.200.100 and 192.168.200.107 in a ration of 1 to 2 (perhaps .107 has a lot of other things loaded on it)
• If there's a cookie called JSESSIONID or jsessionid in the request, it's to be examined to see whether it points forward to a machine called "jvm100" or "jvm107" and, if it does, that's where the forwarding is done to, overriding the selection that would be made for a new connection
• Replies from jvm100 and jvm107 are rewritten (by the ProxyPassReverse directive) to include the name of the URL that the user called up in the header to the response so that following links will be in the right context
• Cookies are also rewritten in reverse, so that JSESSIONID (and all other cookies) truely get sent back up to the server with subsequent requests.
We are currently at httpd release 2.2.17; note that there were 'issues' with using both the balancer and the reverse pass directives within the same proxy request until about release 2.2.11, so you really should be quite up to date if you're going to use this very powerful capabilty.
As well as changing the configuration of Apache httpd, you should make a small change to each Tomcat server.xml file - adding a jvmroute into then engine so that each Tomcat includes its identity in the JSESSIONID cookie. Here's the syntax:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm100">and
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm107">on my second machine. Complete sample file [here].
The Application should be installed "identically" on each of the Tomcat servers to ensure a consistent service for all users, irrespective of which machine they happen to end up connect to on their first contact. But for testing and demonstration purposes, I like to change the background colour of the response page on each of the server so that it's very clear as each of my delegates visits the site / runs the application that they're initially attached to one (or other) machine - differing between them - buton subsequent contacts they're firmly going to the same server each time. There's a complete Java test appliaction, using sessions, [here].
Notes
1. mod_jk may be used in place of mod_proxy for balancing. mod_proxy is prevelant in new installations these days, but we do have separate notes available for delegates who will be looking after systems with the older connectors.
2. Clustering is different to load balancing. With clustering, each request is forwarded to any of the back end systems, and those back end systems constantly pass information back and forth between themselves so that the are all aware of the context (state) of any request that comes their way. Clustering is much more expensive in terms of processor and network bandwidth, but also more robust in highly sensitive applications where it's critical that a session isn't lost if a one of the back servers becomes unavailable for any reason.