Sunday, June 1, 2014

Active/Passive and Active/Active ESB workers with HAproxy

I’m not going to mention about what is the HAproxy ..etc. If you are new to it, please go around Google and come back :).

Environment

 OS : Linux , JDK:  1.6.0_32,  WSO2 ESB 4.8.1 , HAproxy 1.4

1. First install HAproxy.

sudo apt-get install haproxy

2. Enable the HAproxy.

Open the file below and replace the default value 0 to 1.

sudo  vi /etc/default/haproxy

ENABLED=1

3. Open the file below and add the proxy configuration. (You can replace default configuration from the following sample config).

sudo vi /etc/haproxy/haproxy.cfg

global
        maxconn 4096
        user haproxy
        group haproxy
        daemon
        quiet
        nbproc 2
defaults
        log global
        mode http
        option httplog
        option dontlognull
        retries 3
        option redispatch
        maxconn 2000
        contimeout 5000
        clitimeout 50000
        srvtimeout 50000
        stats enable
        stats auth admin:admin
listen  ESB-Cluster *:90
        mode http
        monitor-uri /haproxy-test
        balance roundrobin
        option httpchk HEAD / HTTP/1.0
        option forwardfor
        cookie LSW_WEB insert nocache
        option httpclose
        server esb-01 xxx.xxx.xxx.xxx:8281 cookie s1 check
        server esb-02 xxx.xxx.xxx.xxx:8282 cookie s1 backup check
        redispatch
        stats admin if TRUE 

If you like to know about those properties and what it is does, please go through the official documentation .


4. Some important properties of above HAproxy configuration.

stats enable

Once you enable the stats, you can see the stats on the HTML page.

stats auth admin:admin

The stats page is secured using the basic auth.

server esb-01 xxx.xxx.xxx.xxx:8281 cookie s1 check
server esb-02 xxx.xxx.xxx.xxx:8282 cookie s1 backup check

I’m going to use two ESB worker nodes, esb-01,a and esb-02. I have set the cookie to s1 and s2.  
The “check” means, the proxy doing the health check to servers. The “backup” means , I’m going to use the esb-02 node as the passive worker node.

balance roundrobin

Load balance algorithm is roundrobin. (You can find the other algorithms in the documentation)

If you need more info, please go through the official documentation.

3. Start the HAproxy.

sudo service haproxy start {stop, restart, status}


4. Create two nodes of ESB worker cluster.

According to above configurations esb-01 running on offSet 1 and  and esb-02 running on port offSet 2. If you are new to WSO2 ESB and clustering, please go through the documentations[1]


5. Copy the following sample proxy configuration (EchoProxy.xml)to the proxy-services directory of worker ESB nodes. (Please change the address uri of the echo service according to your environment).

<ESB_HOME>/repository/deployment/server/synapse-configs/default/proxy-services

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="EchoProxy"
       transports="https http"
       startOnLoad="true"
       trace="disable">
   <description/>
   <target>
      <endpoint>
         <address uri="http://ajith-t530:8290/services/echo?wsdl"/>
      </endpoint>
      <inSequence>
         <log>
            <property name="printSend" value="******************************"/>
         </log>
      </inSequence>
      <outSequence>
         <send/>
         <log>
            <property name="printResponse" value="-------------------------------"/>
         </log>
      </outSequence>
   </target>
</proxy>


6) Start another fresh  ESB with port offSet 10, that ESB will use the service(echo) hosting endpoint.

7. Start  all the servers. Now if you refresh HAproxy stats page, you can see the colors are changed according to the server status.

Green - Active ESB worker (esb-01)
Blue    - Passive ESB worker(esb -02)

Now you should see the following info logs in the ESB nodes (worker) due to health check of the
HAproxy.



8. Create a SOAPUI project for the proxy service and send a requests through the HAproxy.

http://xxx.xx.com:90/services/EchoProxy

Now you should observe that the HAproxy send the request only to the  esb-01. Because in above HAproxy config, we have mentioned that the worker esb-02 as the  backup instance.



9. Now create new load test  (in SoapUI project ) using that Request1 and start the load test. So now you should see still the esb-01 is responding, then shut down the esb-01 while load is sending to HAproxy. When esb-01 is shutting down HAproxy will dispatch the request to the backup esb-02 and it will start  responding  to the requests.


10. Start the esb-01 again, Now you should see that HAproxy dispatch all the request back to esb-01 and esb-02 stop responding.


Active/Active worker ESB.

Now if you want to achieve the active/active setup, remove the “backup” property from the esb-02 configuration of the HAproxy settings (/etc/haproxy/haproxy.cfg) and restart the HAproxy.(No need to restart the ESB nodes)

sudo service haproxy restart

11.  Start the load test again, Now you should see both workers (active/active) are responding and HAproxy sending the request to the workers in roundrobin manner.


In my next post i’m planning to discuss about the SSL configurations with HAproxy.
Enjoy !! :)