Understanding HTTP plug-in failover in a clustered environment

Problem (Abstract) After setting up the HTTP plug-in for load balancing in a clustered IBM ® WebSphere ® environment, the HTTP plug-in is not performing failover in a timely manner or at all when a cluster member becomes unavailable. Cause In most cases, the preceding behavior is observed because of a misunderstanding of how HTTP plug-in failover works or might be due to an improper configuration. Also, the type of Web server (multi-threaded versus single threaded) being used can affect this behavior. Resolving the problem The following document is designed to assist you in understanding how HTTP plug-in failover works, along with providing you some helpful tuning parameters and suggestions to better maximize the ability of the HTTP plug-in to failover effectively and in a timely manner . Note: The following information is written specifically for the IBM HTTP Server, however, this information in general is applicable to other Web servers which currently support the HTTP plug-in (for example: IIS, SunOne, Domino ®, and so on) . Failover Background In clustered IBM WebSphere Application Server environments, the HTTP plug-in has the ability to provide failover in the event the HTTP plug-in is no longer able to send requests to a particular cluster member. By default, there are several conditions under which the HTTP plug-in will mark a particular cluster member down and failover client requests to another cluster member that is still able to receive connections. They are listed as follows: The HTTP plug-in is unable to establish a connection to a cluster member's Application Server transport. The HTTP plug-in detects a newly connected socket that was prematurely closed by a cluster member during an active read or write. There are several configurable settings in the plugin-cfg.xml that can be tuned to affect how quickly the HTTP plug-in will mark a cluster member down and failover to another cluster member. ConnectTimeout The ConnectTimeout attribute of a Server element enables the HTTP plug-in to perform non-blocking connections with a backend cluster member. Non-blocking connections are beneficial when the HTTP plug-in is unable to contact the destination to determine if the port is available or unavailable for a particular cluster member. <server cloneid = "10k66djk2" connecttimeout = "10" extendedhandshake = "false" loadbalanceweight = "1000" maxconnections = "0" name = "Server1_WebSphere_Appserver" waitforcontinue = "false"> <transport hostname="server1.domain.com" port="9091" protocol="http"> </ transport> </ server> If no ConnectTimeout value is specified, the HTTP plug-in performs a blocking connect in which the HTTP plug-in sits until an operating system TCP timeout occurs (as long as 2 minutes depending on the platform) and allows the HTTP plug-in to mark the cluster member unavailable . A value of 0 causes the HTTP plug-in to perform a blocking connect. A value greater than 0 specifies the number of seconds you want the HTTP plug-in to wait for a successful connection. If a connection does not occur after that time interval, the HTTP plug-in marks the cluster member unavailable and fails over to one of the other cluster members defined in the cluster. Caution: In an environment with busy workload or a slow network connection, setting this value too low could make the HTTP plug-in mark a cluster member down falsely. Therefore, caution should be used whenever choosing a value for ConnectTimeout. ServerIOTimeout The ServerIOTimeout attribute of a server element enables the HTTP plug-in to set a time out value, in seconds, for sending requests to and reading responses from a cluster member. If a value is not set for the ServerIOTimeout attribute, the HTTP plug-in, by default, uses blocked I / O to write request to and read responses from the cluster member until the TCP connection times out. For example, if you specify: <server cloneid = "10k66djk2" serveriotimeout = "120" connecttimeout = "10" extendedhandshake = "false" loadbalanceweight = "1000" maxconnections = "0" name = "Server1_WebSphere_Appserver" waitforcontinue = "false "> <transport hostname="server1.domain.com" port="9091" protocol="http"> </ transport> </ server> In this case, if a cluster member stops responding to requests, the HTTP plug-in waits 120 seconds (2 minutes) before timing out the TCP connection. Setting the ServerIOTimeout attribute to a reasonable value enables the HTTP plug-in to time out the connection sooner, and transfer requests to another cluster member when possible. When selecting a value for this attribute, remember that sometimes it might take a couple of minutes for a cluster member to process a request. Setting the value of the ServerIOTimeout attribute too low could cause the HTTP plug-in to send a false server error response to the client. The ServerIOTimeout is ideal for situations where Keep-Alive connections exist between the WebSphere Application Server and HTTP plug-in, and the Application Server machine is abruptly disconnected from the network. For example, without ServerIOTimeout, the HTTP plug-in would take a long time to detect that the connection was closed abruptly on the WebSphere Application Server machine. This is illustrated as follows: When an application host machine is shut down abruptly, the Keep-Alive connections between HTTP plug-in and Application Server might not get closed completely. As a result, when the HTTP plug-in needs to route a request to the host machine, the HTTP plug-in would use an existing Keep-Alive connection if there was one in the pool. When plug-in sends the request over such a connection, since the host machine had been taken down abruptly, the HTTP plug-in machine does not receive any TCP packets to close the connection. The HTTP plug-in request writing would not return a failure until the connection timed out at the TCP level. The HTTP Plug-in would then try to contact to the same application server by establishing a new connection. The connect () call would then fail after the TCP timeout. As a result, it could take a considerable amount of time depending on the operating system TCP timeout setting for the HTTP plug-in to detect the application server status and mark it down before failing over to another application server. If there were many requests sent to the server during this time, this fact would apply to every request. Note: To avoid the preceding behavior, ServerIOTimeout attribute was introduced with APAR PQ96015 and included in WebSphere Application Server V5.0.2.10 and 5.1.1.4. Caution: When both ConnecTimeout and ServerIOTimeout are specified, it could take as long as (ConnecTimeout + ServerIOTimeout) for the HTTP plug-in to detect and mark a server down. RetryInterval An integer specifying the length of time that should elapse from the time that a server is marked down to the time that the HTTP plug-in will retry a connection. The default is 60 seconds. This setting is specified in the ServerCluster element. An example of this in the plugin-cfg.xml file is as follows: <servercluster cloneseparatorchange = "false" loadbalance = "Round Robin" name = " Server_WebSphere_Cluster "postsizelimit =" 10000000 "removespecialheaders =" true "retryinterval =" 120 "> This would mean that if a cluster member were marked as down, the HTTP plug-in would not retry it for 120 seconds. There is no way to recommend one specific value; the value chosen depends on your environment. For example, if you have numerous cluster members, and one cluster member being unavailable does not affect the performance of your application, then you can safely set the value to a very high number. Alternatively, if your optimum load has been calculated assuming all cluster members to be available or if you do not have very many, then you will want your cluster members to be retried more often to maintain the load. Also, take into consideration the time it takes to restart your server. If a server takes a long time to boot up and load applications, then you will need a longer retry interval. PrimaryServers versus BackupServers The HTTP plug-in can be configured for true failover by using PrimaryServers and BackupServers Elements in the plugin-cfg.xml configuration file. In the following example, the plug-in will load balance between both servers, Server1_WebSphere_Appserver and Server2_WebSphere_Appserver defined in PrimaryServers element only. However, in the event that bothServer1_WebSphere_Appserver and Server1_WebSphere_Appserver become unavailable and marked down, the HTTP plug-in will then failover and start sending requests to Server3_WebSphere_Appserver defined in the BackupServers Element. <servercluster cloneseparatorchange = "false" loadbalance = "Round Robin" name = "Server_WebSphere_Cluster" postsizelimit = "10000000" removespecialheaders = "true" retryinterval = " 120 "> <server cloneid="10k66djk2" serveriotimeout="120" connecttimeout="10" extendedhandshake="false" loadbalanceweight="1000" maxconnections="0" name="Server1_WebSphere_Appserver" waitforcontinue="false"> <transport hostname = "server1.domain.com" port = "9091" protocol = "http"> </ transport> </ server> <server cloneid = "10k67eta9" serveriotimeout = "120" connecttimeout = "10" extendedhandshake = "false" loadbalanceweight = "999" maxconnections = "0" name = "Server2_WebSphere_Appserver" waitforcontinue = "false"> <transport hostname="server2.domain.com" port="9091" protocol="http"> </ transport> </ server> < server cloneid = "10k68xtw10" serveriotimeout = "120" connecttimeout = "10" extendedhandshake = "false" loadbalanceweight = "998" maxconnections = "0" name = "Server3_WebSphere_Appserver" waitforcontinue = "false"> <transport hostname = "server3.domain. Com "port =" 9091 "protocol =" http "> </ transport> </ server> <primaryservers> <server name="Server1_WebSphere_Appserver"> </ server> <server name="Server2_WebSphere_Appserver"> </ server> < / primaryservers> <backupservers> <server name="Server3_WebSphere_Appserver"> </ server> </ backupservers> </ servercluster> </ servercluster>
  • del.icio.us
  • StumbleUpon
  • Digg
  • TwitThis
  • Mixx
  • Technorati
  • Facebook
  • NewsVine
  • Reddit
  • Google
  • LinkedIn
  • YahooMyWeb

Related Posts of Understanding HTTP plug-in failover in a clustered environment

  • Rails source code analysis (1): RailsFCGIHandler

    In accordance with the sequence starting from the beginning CGI Ruby CGI Doc: The Common Gateway Interface ( CGI ) Is a simple protocol for passing an HTTP request from a web server to a standalone program, and returning the output to the web browser ...

  • The fastest web server cherokee, to support the rails

    Online to see this web server, reportedly faster than lighttpd, go read it under the site and found a lot of popular support for the framework, including the rails, are interested can try http://www.cherokee-project.com/doc/cookbook_ror.html

  • IBM Ajax Tutorial Series

    Links: http://ibm.csdn.net/ISN_J.aspx?action=JMP&pointid=1550 Part 10: Using JSON for data transmission In the asynchronous application to send and receive information, you can choose to plain text and XML as data format. Ajax grasp of this issue ...

  • Using Rails Captcha plug-in, easy implementation Verification Code

    Simple Captcha Plugin Can help us easily in the Rails implementation of Verification Code function. In addition, he has the option to provide sufficient to meet the U.S. requirements for certification, the use of easy. Supported picture and digital authen

  • Hibernate Inteceptor

    The end of the project stage, the client suddenly put forward a very troublesome but normal demand, the system records all changes must be carried out. Formats such as: 2004.1.1 12:30 Ikuya wind orders Sales Order Date 2004.1.2-> 2004.1.3 The firs ...

  • jboss ejb3 Message Driven Bean

    Super Medium ejb hate. . . . . . . . . . . . . . . . . . . ================================================ To configure a Message Driven Bean in a different application server parameters are not the same. Currently only passed the test jboss. Message Dri

  • FLEX: integrating Spring + Hibernate

    Before a friend also wanted to study development of FLEX. Asked me to help him to be a small sample. Spent a weekend time, to integrate a sampleproject. Client: FLEX Server: Spring2.5 + Hibernate3.2 + Hibernate-annotations3.3.1 + MySQL5 FDS: BlazeDS3 IDE:

  • Ruby on Rails Routing - Simple Examples

    This article contains a list of ruby on rails routing examples. If you find you have any questions please leave a comment. Routes are processed from the top of routes.rb down. If a route is matched it will stop processing the routes.rb file and use that r

  • myeclipse plugin

    myeclipsePlug-ins? 1.tomcatPlugin(Start tomcat ):http ://www.sysdeo.com/eclipse/tomcatPlugin.html,2.xVersions of eclipse 3 version 2 .1Version doesn't work. 2.Lomboz(Development of jsp program ,jspDynamic prompt, debugging ):http://forge.objectweb.org/pro

blog comments powered by Disqus
Recent
Recent Entries
Tag Cloud
Random Entries