JBOSS Cluster


1. Preface


In 2006, Jboss Was Redhat Acquired. This is a direct result of structural adjustment Jboss products and Jboss AS charges before the document be changed free of charge. jijian91 The cluster is based on the initial test of the resultant Jboss 4.0.5 cluster configuration files .

However, the official document is not reliable, some of the key configuration in the ambiguity, and the application of a mixture of a lot in the last version of Jboss is now disposed of the contents of the cluster configuration. This has led to this test jboss cluster configuration is largely dependent on speculation and repeated attempts to complete. The reason for this is because the characteristics of Jboss cluster is still in development, Jboss developers still constantly modify, improve, increase the characteristics of a great deal of changes between versions, the updated document difficult to ensure the speed and quality. Therefore, jijian91 Remind you to look for the official version of the document can only be a reference, should not believe everything. Absolutely sure.

2. Cluster knowledge

Cluster concept


Cluster (Cluster) is a set of a collection of computer nodes, which as a whole to provide users with a set of network resources. An ideal cluster is transparent to the user. Import users from a single visit to the cluster resources, has never aware of the node cluster. In their view, the cluster is a system, rather than a number of computer systems. Clusters should also be free to increase support and reduce the node cluster system, which will not affect the same to the user's visit.

Cluster Classification


Traditionally, the cluster is divided into high-availability (High Availability, referred to as HA) and high-performance computing clusters (High Perfermance Computing, referred to as HPC) cluster the two types.

HA cluster goal is to improve system availability (availability), that is, reliability (reliability) and maintainability (maintainability). Please do not use clusters of the (availability) with the UE and the availability of interactive design (Usability) confusion.

HA Cluster core is to prevent a single point of failure, which is generally achieved through the failure of the transfer, that is, failure in one node to another node to take over after the service. User status is not lost. HA clusters, the other major features include load balancing, session synchronization. We use the SQL Server database, Hot Standby and Oracle are among the HA of the RAC cluster.

HPC clusters using parallel computing technology to provide ultra-large-scale computing and storage capacity, the majority of HPC supercomputers are clusters. This is not our concern to the cluster.

Jboss cluster structure


HA Cluster is the cluster Jboss.

Jboss cluster there are two kinds of structure. First, the interceptor client (Client-side interceptor) structure, First, load balancers (Load balancer) structure.

Client-side interceptor architecture used for C / S structure, load-balancing architecture for B / S structure. This article only describes the architecture of the load balancing cluster Jboss.

And installation of JBOSS Cluster

Load balancing by load balancing architecture and composition of n-node cluster. Each node is an instance of Jboss server. Global Load Balancing is the only front-end, all user requests are sent to the load balancers, which transmit to each node. When the load balancing device was found after a node failure, the request will be forwarded to another node, thus ensuring continuity of services. Load Balancer is responsible for scheduling weighted static load balancing. In short, the Load Balancer determines the health of the global health cluster, load balancing device failure will lead to failure of all clusters. This is the front-end structure of the main potential problems in the cluster.

Load Balancing Jboss cluster architecture is the actual Tomcat's HTTP cluster achieved. Jboss has its own load balancing, but ineffective, there is no official document that almost no one used. Under normal circumstances, are jijian91 Like the use of apache + mod_jk as a load equalizer. Described below are based on this architecture.

mod_jk is a plug-apache, apache and tomcat is responsible for communications between the jboss cluster (tomcat cluster) key.

Jboss version of choice


At present, Jboss main series in three versions 3,4,5.

Jboss 5 At present, only two beta version, it still takes time and practical.

Jboss 3 is the final version in March 2006 update 3.2.8.SP1. Jboss 4 With the increasing maturity and development of Jboss 5, has stopped more than 1 year to update the Jboss 3 gradually fade out the stage of history.

Jboss 4 followed by the latest version of 4.2.1.GA, 4.2.0.GA and 4.0.5.GA. However, the official website of the document provided by Jboss cluster only updated to the 4.0.5 GA, so I chose the realization of Jboss 4.0.5 GA cluster Jboss.

The Jboss version is not the same as to install and configure. Jboss 3 and not only the Jboss 4 configuration file is completely different from the small versions have subtle differences. In the cluster, Jboss, apache, mod_jk exist between the specific version in order to mismatch. Therefore, in order to avoid problems, please in strict accordance with the following versions of installed software:

JDK 1.5.0.12

Jboss 4.0.5 GA

apache 2.2.4

apache mod_jk 1.2.23

Note, mod_jk have two versions 1.x and 2.x series. mod_jk 2.x has stopped development, can not be used. Many people intuitively think that mod_jk 2.x certainly better than mod_jk 1.x, resulting in a detour away.

Example of an overview of cluster configuration 3.Jboss


Below, jijian91 Examples will be described in Jboss cluster configuration.

The example contains three nodes Jboss. Each node to receive a passive load balancers forward the request. Each node there is no horizontal links.



4. JBoss clustering load balancing configuration

Steps


Install apache, and then configure the mod_jk module.

Install apache


Download apache-source packet Uploaded to the server.

Unlock code package
tar xfvz httpd-2.2.4.tar.gz

Compiler
. / configure-prefix = / usr/local/apache2-enable-module = so-enable-module = setenvif-enable-module = rewrite-enable-rewrite = shared-enable-proxy = shared-with-mpm = prefork-enable -so-enable-auth-anon-enable-file-cache = shared-enable-cache = shared-enable-disk-cache = shared-enable-mem-cache = shared

make clean
make
make install

Modified configuration. This case, jijian91 The use of the monitor port is 8080, please modify according to the actual situation.
vi / usr/local/apache2/conf/httpd.conf
Listen 80 will be changed to Listen 8080
User daemon will be replaced by Group daemon and User apache and Group apache
# ServerName before delete the line ServerName 127.0.0.1:8888 changed

Add users and groups
groupadd apache
useradd apache-g apache

apache mod_jk configuration


Download mod_jk Its changed its name to mod_jk.so, copy to / usr/local/apache2/modules under.

Incidentally, to find a mod_jk module costs jijian91 Great deal of energy, and finally in http://www.apache.org/dist/tomcat/tomcat-connectors/jk/binaries/linux/ Directory to find. I'm using mod_jk 1.2.23, see this article when, mod_jk should be a new version. Mod_jk downward compatible but has not fared any better, the best use of mod_jk 1.2.23 to debug, and then try for success with the new version of mod_jk.

chmod + x / usr/local/apache2/modules/mod_jk.so

In / usr/local/apache2/conf/httpd.conf added to the end:
Include conf / mod_jk.conf

Create an empty file / usr/local/apache2/conf/uriworkermap.properties

vi / usr/local/apache2/conf/mod_jk.conf, enter the following:
# Load mod_jk module
# Specify the filename of the mod_jk lib
LoadModule jk_module modules / mod_jk.so
# Where to find workers.properties
JkWorkersFile conf / workers.properties
# Where to put jk logs
JkLogFile logs / mod_jk.log
# Set the jk log level [debug / error / info]
JkLogLevel info
# Select the log format
JkLogStampFormat "[% a% b% d% H:% M:% S% Y]"
# JkOptions indicates to send SSK KEY SIZE
JkOptions + ForwardKeySize + ForwardURICompat-ForwardDirectories
# JkRequestLogFormat
JkRequestLogFormat "% w% V% T"
# Mount your applications
JkMount / application / * loadbalancer
# You can use external file for mount points.
# It will be checked for updates each 60 seconds.
# The format of the file is: / url = worker
# / Examples / *= loadbalancer
JkMountFile conf / uriworkermap.properties
# Add shared memory.
# This directive is present with 1.2.10 and
# Later versions of mod_jk, and is needed for
# For load balancing to work properly
JkShmFile logs / jk.shm
# Add jkstatus for managing runtime data

JkMount status
Order deny, allow
Deny from all
Allow from 127.0.0.1

vi / usr/local/apache2/conf/workers.properties, add the following:
# Define list of workers that will be used
# For mapping requests
worker.list = loadbalancer, status
# Define Node1
# Modify the host as your host IP or DNS name.
worker.node1.port = 8009
worker.node1.host = 192.168.130.95
worker.node1.type = ajp13
worker.node1.lbfactor = 1
# Define Node2
# Modify the host as your host IP or DNS name.
worker.node2.port = 8009
worker.node2.host = 192.168.130.99
worker.node2.type = ajp13
worker.node2.lbfactor = 0
# Define Node3
# Modify the host as your host IP or DNS name.
worker.node3.port = 8009
worker.node3.host = 192.168.130.112
worker.node3.type = ajp13
worker.node3.lbfactor = 1
# Load-balancing behaviour
worker.loadbalancer.type = lb
worker.loadbalancer.balance_workers = node1, node2, node3
worker.loadbalancer.sticky_session = 1
# worker.list = loadbalancer
# Status worker for managing load balancer
worker.status.type = status

Description:
worker.node1.host, worker.node2.host and jboss cluster worker.node3.host to the machine into the actual IP.
If there are more nodes, the node order of the definition of more paragraphs, and list them all after worker.loadbalancer.balance_workers.
lbfactor is the weight of the load distribution, the greater the value the more the load distribution.
More detailed configuration parameters tomcat configuration

Automatically configure apache to start


ln-s / usr/local/apache2/apachectl / etc / init.d / apache
chmod 777 / etc / init.d / apache
ln-s / etc / init.d / apache / etc/rc3.d/S80apache
ln-s / etc / init.d / apache / etc/rc3.d/K20apache
ln-s / etc / init.d / apache / etc/rc4.d/S80apache
ln-s / etc / init.d / apache / etc/rc4.d/K20apache
ln-s / etc / init.d / apache / etc/rc5.d/S80apache
ln-s / etc / init.d / apache / etc/rc5.d/K20apache

5.Jboss node cluster configuration

Steps


Each node in the cluster, the first installation of JDK, and then install the configured Jboss. Do not worry, followed by jijian91 Gradually.

Install JDK


Download in http://sun.com JDK installation package Uploaded to the server.

chmod + x jdk-1_5_0_12-linux-i586-rpm.bin
./jdk-1_5_0_12-linux-i586-rpm.bin
According to the installation prompts.

Java path set.
vi / etc / profile, in the end add the following:
JAVA_HOME = / usr/java/jdk1.5.0_12
CLASSPATH =.: $ JAVA_HOME / lib / dt.jar: $ JAVA_HOME / lib / tools.jar
PATH =.: $ JAVA_HOME / bin: $ PATH
export JAVA_HOME CLASSPATH PATH

JDK instead of using pre-installed Redhat Linux AS4 the gcj:

cd / usr / bin
ln-s-f / usr/java/jdk1.5.0_12/bin/java
ln-s-f / usr/java/jdk1.5.0_12/bin/javac

The implementation of java-version, the following information is displayed that the correct installation of jdk:
java version "1.5.0_12"
Java (TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04)
Java HotSpot (TM) Client VM (build 1.5.0_12-b04, mixed mode, sharing)

Install Jboss


Download jboss-4.0.5.GA And jems-installer , And upload to the server.

Complete the installation of Jboss 4.0.3 sp1 to 4.0.5.GA the version must be used jems-installer and JDK 1.5.

Jboss can use graphical and command-line install. Graphical installation convenience, but asked the server to install X windows, and on the server to support the operation or use of the x windows remote terminal ( jijian91 Recommended by xmanager).

Graphical


X windows in the terminal window, the implementation of java-jar jems-installer-1.2.0.GA.jar, there Jboss installation graphical interface.

Select "Chinese."
The next step.
Consent agreement.
Installation path is set to / usr / local / jboss, the next step.
Select "all", the next step.
Select "advance", the Name by default changed to all. The rest of the next step directly.
Enter the Admin password, the next step.
Waiting for the end of the installation to complete.

Command-line mode


java-jar jems-installer-1.2.0.GA.jar-installGroup all installpath = / usr / local / jboss
mv / usr / local / jboss / server / default / usr / local / jboss / server / all

Configure Jboss


Modified configuration. This case, the provision of services is the monitor port 8888, please modify according to the actual situation.
vi / usr/local/jboss/server/all/deploy/jbossweb-tomcat55.sar/server.xml
Find find

vi / usr/local/jboss/server/all/deploy/jbossweb-tomcat55.sar/META-INF/jboss-service.xml
Find UseJK, replaced by true.

Jboss automated startup configuration


cp / usr / local / jboss / jboss_init_redhat.sh / etc / init.d / jboss
chmod 777 / etc / init.d / jboss
ln-s / etc / init.d / jboss / etc/rc3.d/S80jboss
ln-s / etc / init.d / jboss / etc/rc3.d/K20jboss
ln-s / etc / init.d / jboss / etc/rc4.d/S80jboss
ln-s / etc / init.d / jboss / etc/rc4.d/K20jboss
ln-s / etc / init.d / jboss / etc/rc5.d/S80jboss
ln-s / etc / init.d / jboss / etc/rc5.d/K20jboss

vi / etc / init.d / jboss
JBOSS_HOME line will be changed to JBOSS_HOME = $ (JBOSS_HOME: - "/ usr / local / jboss")
JBOSS_USER line will be replaced by JBOSS_USER = $ (JBOSS_USER: - "root")
JAVAPTH line will be replaced by JAVAPTH = $ (JAVAPTH: - "/ usr/java/jdk1.5.0_12")
JBOSS_CONF line will be replaced by JBOSS_CONF = $ (JBOSS_CONF: - "all")
JBOSSSH line will be replaced by JBOSSSH = $ (JBOSSSH: - "$ JBOSS_HOME / bin / run.sh-c $ JBOSS_CONF")
Will be eval $ JBOSS_CMD_START> $ (JBOSS_CONSOLE) 2> & 1 & replace eval $ JBOSS_CMD_START> $ (JBOSS_CONSOLE) &
Will be $ SUBIT "$ JBOSS_CMD_START> $ (JBOSS_CONSOLE) 2> & 1 &" changed to $ SUBIT "$ JBOSS_CMD_START> $ (JBOSS_CONSOLE) &"

6. Jboss cluster start

Steps


Load Balancer to start apache, then restart the jboss node. The jboss node does not require the boot sequence.

Restart apache load balancing device



In / usr/local/apache2/bin implementation directory. / Httpd-k start

In / usr/local/apache2/logs/mod_jk.log see mod_jk initialization apache log:
[Wed Aug 01 10:36:11 2007] [6636:63168] [info] init_jk:: mod_jk.c (2743): mod_jk/1.2.23 initialized
[Wed Aug 01 10:36:11 2007] [6637:63168] [info] init_jk:: mod_jk.c (2743): mod_jk/1.2.23 initialized
If an error message back, because the nodes in the cluster have not yet started, the time being ignored.

8080 using netstat to check the existence of the listening port, apache startup success.

Jboss start node


All machines in the cluster / usr / local / jboss / bin, respectively, the Executive / run.sh-c all 1> 1 2> 2 &

Port 8888 using netstat to check the existence of monitoring.
Browser to visit each node of the 8888 port, able to see the status of jboss.
Jboss startup success.

Cluster lifetime


apache can automatically discover new nodes Jboss start. When the first node after Jboss, Jboss cluster is automatically created. Followed by the other nodes will be added to the cluster. Now visit http://192.168.8.246:8888, be able to see the status of jboss. That the request has been forwarded to the Jboss nodes. Start a successful cluster.

apache cluster to monitor the status of each node automatically removed from the cluster nodes can not access, thus ensuring that have been in response to each request. When all cluster nodes to stop working, the cluster extinction.

Clusters of defects and adhesion session


As of now, Jboss cluster worked perfectly, apart from the session.

Jboss current cluster configuration, each node is passive independent of each other, which is equivalent to an increase of node-state detection DNS poll, it is also independent of each other session. When two of the same user request was forwarded to the Jboss different nodes, the false information session will be "lost" phenomenon.

To this end, Jboss adhesion to session (sticky session) concept. After the session the use of adhesive, the request from the same IP will be sent to the same node Jboss, thus ensuring the continuity of the use of session. If the application does not use the session, you can use the non-adhesion of jboss cluster session. In this way, the load distribution more reasonable. ( jijian91 Words: more simple. )

Set up load balancing method is editor's / usr/local/apache2/conf/workers.properties the worker.loadbalancer.sticky_session. Set to 1, the use of adhesive session, set to 0 not to use adhesive session.

The use of adhesive session there is a risk, when a node failure, the node of the session will be lost. Order to solve this problem, it is necessary to use jboss session replication .

7.Jboss cluster session replication

Principle


jboss session replication is a jboss session to achieve synchronization. Jboss principle in the establishment of horizontal linkages between nodes, each node will change the node synchronization session to all other nodes.

jboss the HTTP cluster session replication and with each other, the two systems are independent of each other. session replication is horizontal linkages between nodes, HTTP Load Balancer cluster is the vertical associated with the node.

Jboss configuration node



vi / usr/local/jboss/server/all/deploy/tc5-cluster.sar/META-INF/jboss-service.xml

<config> <udp,将> <config> To find </ config> all commented out.
Copy jboss session are UDP and TCP in two ways. UDP Multicast adopted, but more problems, so jijian91 Means the use of TCP.

<config> <tcp,将> <config> To find </ config> the entry into force. And the part the following modifications:

  • Up_thread all down_thread and the false are changed to true.
  • In <tcp bind_addr = "fill in the machine after the IP, such as <TCP bind_addr=">
  • In <tcpping initial_hosts = "fill in the machine and after all the other Jboss cluster nodes IP [7810], for example, <tcpping initial_hosts =" 192.168.130.95 [7810], 192.168.130.99 [7810], 192.168.130.112 [7810] "> </ tcpping>

Other parameters include:
ClusterName is the cluster name, for example, partition1.
In the same LAN, you can have more than one cluster jboss, under the cluster name to distinguish between them. Therefore, each node in the cluster configuration cluster name must be consistent, and the machine IP is not special requirements, as long as they are connected to each other. Theory on a single machine can be installed on more than Jboss instance, belong to different sub-clusters. However, this will greatly increase the complexity of the configuration is not good. jijian91 Does not recommend to their own serious trouble.

IsolationLevel is the isolation level.
Optional values include: SERIALIZABLE, REPEATABLE_READ, READ_COMMITTED, READ_UNCOMMITTED, and NONE. The isolation of the isolation level and database level have the same meaning, for most applications is usually WEB set to REPEATABLE_READ.

CacheMode is cached mode.
As the session replication is achieved through the cache, so this is actually a copy mode. Optional values include: REPL_SYNC and REPL_ASYNC, determine the change should be synchronous or asynchronous replication. Default value is REPL_ASYNC. Using synchronous replication, to ensure that the request to change prior to the completion of dissemination, session synchronization does not lag, but inefficient.

Configuration Application


Web.xml in the application of the increase in <web-app> paragraph <distributable />.

In jboss-web.xml to add the following:

<Jboss-web>
<Replication-config>
<Replication-trigger> SET_AND_NON_PRIMITIVE_GET </ replication-trigger>
<Replication-granularity> SESSION </ replication-granularity>
<Replication-field-batch-mode>

This jboss4 the end of the cluster configuration.