Friday, August 21, 2015

Advance Mediation with WSO2 API Manager(1.9) + WSO2 ESB -3

In this use case WSO2 API Manager act as the gateway, and WSO2 ESB used to execute the mediation logic.
1. This service (http://www.webservicex.com/globalweather.asmx) provides two operations.
  • GetCitiesByCountry
                   Get all major cities by country name(full / part).
  • GetWeather
                 Get weather report for all major cities around the world.

2. Create an API in ESB to invoke the above operations  using following REST operations (ESB start with port offset as 1).
http://localhost:8281/weatheresb/GetCitiesByCountry?CountryName=Romania
http://localhost:8281/weatheresb/GetCitiesByCountry?CountryName=Romania&CityName=Timisoara

Save this file as WeatherAPIESB.xml in <ESB_Home>\repository\deployment\server\synapse-configs\default\api)

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse"
     name="WeatherAPIESB"
     context="/weatheresb">
   <resource methods="GET" url-mapping="/GetCitiesByCountry">
      <inSequence>
         <payloadFactory media-type="xml">
            <format>
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                                 xmlns:web="http://www.webserviceX.NET">
                  <soapenv:Header/>
                  <soapenv:Body>
                     <web:GetCitiesByCountry><web:CountryName>$1</web:CountryName>
                     </web:GetCitiesByCountry>
                  </soapenv:Body>
               </soapenv:Envelope>
            </format>
            <args>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CountryName')"/>
            </args>
         </payloadFactory>
         <send>
            <endpoint>
               <address uri="http://www.webservicex.com/globalweather.asmx" format="soap12"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <property name="messageType" value="application/json" scope="axis2"/>
         <send/>
      </outSequence>
   </resource>
   <resource methods="GET" url-mapping="/GetWeather">
      <inSequence>
         <payloadFactory media-type="xml">
            <format>
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                                 xmlns:web="http://www.webserviceX.NET">
                  <soapenv:Header/>
                  <soapenv:Body>
                     <web:GetWeather><web:CityName>$1</web:CityName>
                        <web:CountryName>$2</web:CountryName>
                     </web:GetWeather>
                  </soapenv:Body>
               </soapenv:Envelope>
            </format>
            <args>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CityName')"/>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CountryName')"/>
            </args>
         </payloadFactory>
         <send>
            <endpoint>
               <address uri="http://www.webservicex.com/globalweather.asmx" format="soap12"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <property name="messageType" value="application/json" scope="axis2"/>
         <send/>
      </outSequence>
   </resource>
</api>
3.Create an API in API Manager considering the REST API exposed through the ESB (http://localhost:8281/weatheresb).

4.  Design API with  two GET resource.


5. Go to the implement wizard and define HTTP endpoint (http://localhost:8281/weatheresb).

6. Go the manage wizard , select the tier then "save & publish".


7.  Login to the store, subscribe to an application, then  generate token.



8.  You can use the SOAPUI to invoke this API.



Related posts:

i) http://www.vitharana.org/2015/01/soap-web-service-as-rest-api-using-wso2.html

ii) http://www.vitharana.org/2015/08/soap-webservice-as-rest-api-wso2-api.html

Thursday, August 20, 2015

SOAP webservice as a REST API - WSO2 API Manager (1.9) -2

1. I'm going to use the weather service which is available in public. (http://www.webservicex.com/globalweather.asmx)

2. This SOAP service has a operation called "GetCitiesByCountry". You can get the following results when It access directly from the SOAPUI.

3. Now I'm going to create API in WSO2 API Manager to expose this service operation as a REST operation.

Eg: Expose the  GetCitiesByCountry as GET method as bellow.
https://localhost:8243/weather/v1.0.0/GetCitiesByCountry?country=Romania
4. We want to write a mediation extension to read the "country" from request and construct the SOAP payload which is  expect by back end service (I'm going to save this file as JSONtoSOAP.xml). This file shroud upload to /_system/governance/apimgt/customsequences/in in registry.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="admin--weather:vv1.0.0--In">  
 <property name="country" expression="$url:country"></property>
   <payloadFactory media-type="xml">  
            <format>  
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"  
                                 xmlns:web="http://www.webserviceX.NET">  
                  <soapenv:Header/>  
                  <soapenv:Body>  
                     <web:GetCitiesByCountry>  
                     <web:CountryName>$1</web:CountryName>  
                     </web:GetCitiesByCountry>  
                  </soapenv:Body>  
               </soapenv:Envelope>  
            </format>  
            <args>  
               <arg xmlns:m0="http://services.samples/xsd" evaluator="xml" 
                                     expression="get-property('country')"></arg>  
            </args>  
         </payloadFactory>          
</sequence>


5. We want another mediation extension file to convert the SOAP response to JSON.(I'm going to save this file as SOAPtoJSON.xml). This file should upload to /_system/governance/apimgt/customsequences/out in registry.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="admin--weather:vv1.0.0--Out">
   <property name="messageType" value="application/json" scope="axis2"/>
</sequence>

6.  Create an API as bellow. (Make sure to add new query parameter called country for GetCitiesByCountry resource)




7. Go to the implement wizard  and select the end point type as "Address Endpoint". Then define the endpoint as http://www.webservicex.com/globalweather.asmx


8.  Click on the "Advance Options" button and select the "Format" as SOAP 1.2.


9. Go to the manage wizard and select the custom mediation form the "Message Mediation Policies".


10. Click on "save & publish" button to publish this API to store.

11. Login to the store ,  subscribe that API  to an application and generate a token.

12. Use the following Curl command to invoke the  GetCitiesByCountry operation.

curl  -k -H "Authorization :Bearer 99215d98fa10ce6f2d83cd11ba9828" -H "Content-Type:application/json"   https://localhost:8243/weather/v1.0.0/GetCitiesByCountry?country=Romania

13. You can also use the SOAPUI to invoke that API. (Make sure to define the  Authorization header)


Related posts:

i) http://www.vitharana.org/2015/01/soap-web-service-as-rest-api-using-wso2.html

ii) http://www.vitharana.org/2015/08/advance-mediation-with-wso2-api-manager.html

Sunday, August 16, 2015

[WSO2 AM] API Manager distributed setup with Apache proxy

In this post I'm going to explain about how to setup API Manager distributed setup with Apache proxy. (The configuration details include for  both Windows and Linux environments)


  • The proxy1 used to accept the API request and distribute among the gaeway nodes.
  • The proxy2 used to distribute the key validation and authentication request among the manager nodes.
  • The proxy3 used to publish API to gateway nodes.
  • SVN/Rsync used to synchronize the API artifacts xml across the gateway nodes.
  • The registry(config and governance), user management and API manager database shared across all the nodes. 
Required applications:

1. Java 1.7.X
2 Apache2
3. Open SSL
4. MySQL

 1.0 Install and configure the Apache2 on Windows

i) You can download and install XAMP (https://www.apachefriends.org/index.html), that provides a control panel to manage the Apache server on Windows.

 Open the httpd.conf file (C:\xampp\apache\conf) and enable the following modules.
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
 ii) In Linux environment.

Install Apache2 using the following command.
$sudo apt-get install apache2
To enable the modules.
$sudo a2enmod proxy_http
$sudo a2enmod ssl
$sudo a2enmod proxy_balancer
$sudo a2enmod lbmethod_byrequests
$sudo a2enmod slotmem_shm
$sudo a2enmod proxy_wstunnel 
2.0 Create virtual host configuration for proxy-1.

i) In  windows,  virtual host file (eg: gw.wso2am.conf) should be saved in C:\xampp\apache\conf\extra directory.  (in Linux /etc/apache2/site-available).
<VirtualHost gw.wso2am:443>

 ServerAdmin abc@wso2.com
 ServerName  gw.wso2am
 ServerAlias gw.wso2am
 ProxyRequests Off
 SSLEngine On
 SSLProxyEngine On
 #SSLProxyVerify none
 SSLProxyCheckPeerCN off
 SSLProxyCheckPeerName off
 #SSLProxyCheckPeerExpire off

 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://gw.wso2am>
        BalancerMember https://gw1.wso2am:8243 route=gwNode1 loadfactor=1
        BalancerMember https://gw2.wso2am:8244 route=gwNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://gw.wso2am/
ProxyPassReverse / balancer://gw.wso2am/

</VirtualHost>
ii) In windows,  include this virtual host configuration in httpd.conf file. (Find the # Virtual hosts configuration section)
Include conf/extra/gw.wso2am.conf
iii) In linux you can use the following command.
sudo a2ensite gw.wso2am.conf
3.0 Create certificate file and key for the proxy-1.

i) You can download the open ssl for windows (https://code.google.com/p/openssl-for-windows/) . Exact the zip file and set the path variable to bin directory.

Generate key:
openssl genrsa -out ca.key 1024 (windows)
sudo openssl genrsa -out ca.key 1024 (linux)
Generate CSR (Certificate Signing Request ).
  • Here you need to enter CSR details and make sure to enter the gw.wso2am as the Common Name (CN).
openssl req -new -key ca.key -out ca.csr (windows)
sudo openssl req -new -key ca.key -out ca.csr (linux)
Generate self sign certificate.
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt (Windows)
sudo openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt (linux)


  • The ca.crt and ca.key has configured as SSLCertificateFile  and SSLCertificateKeyFile in above virtual host configuration.
  •  If the CN (Common Name) of the back end server certificate is different from the virtual host name, then you need to disable the SSLProxyCheckPeerCN and SSLProxyCheckPeerName in the virtual host configuration.
4.0 Create new key store for gateway nodes.

i) Generate new key store.
  •  Make sure to enter the gw.wso2am as the first and last name , that value will be set as common name (CN).
  •  Enter the key store password as wso2carbon , then you don't need to change the default key store configurations in API Manager.
keytool -genkey -keyalg RSA -alias wso2carbon -keystore wso2carbon.jks -storepass wso2carbon -validity 360 -keysize 2048

ii) Export the public  key.
keytool -export -alias wso2carbon -keystore wso2carbon.jks -storepass wso2carbon -file wso2carbon.pem



iii) Copy this new  wso2carbon.jks and wso2carbon.pem files to repository\resources\security directory of the gateway nodes.

iv) Go to the repository\resources\security directory from command line and execute the following command to remove the default wso2carbon certificate from the client-truststore.jks.
keytool -delete -alias wso2carbon -keystore client-truststore.jks
v) Import new certificate.
keytool -import -alias wso2carbon -file wso2carbon.pem -keystore client-truststore.jks -storepass wso2carbon
5.0 Configure the gateway nodes.

i) Open the carbon.xml file (repository\conf) and define the HostName and MgtHostName.
<HostName>gw1.wso2am</HostName>
<MgtHostName>mgt1.gw.wso2am</MgtHostName> 
ii) Open the catalina-server.xml file (repository\conf\tomcat) and add the following two properties in http connector configuration.
proxyPort="80"
proxyName="gw.wso2am"

iii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in https connector configuration.
proxyPort="443"
proxyName="gw.wso2am"

iv) Open the master-datasources.xml and configure new data source for mount registry.
<datasource>       
    <name>WSO2_CARBON_DB_cluster_db</name>
    <description>The datasource used for mount registry </description>
    <jndiConfig>
        <name>jdbc/WSO2CarbonDB_cluster_db</name>
    </jndiConfig>
    <definition type="RDBMS">
    <configuration>
        <url>jdbc:mysql://localhost:3306/cluster_db</url>
        <username>root</username>
        <password></password>
        <driverClassName>com.mysql.jdbc.Driver</driverClassName>
        <maxActive>50</maxActive>
        <maxWait>60000</maxWait>
        <testOnBorrow>true</testOnBorrow>
        <validationQuery>SELECT 1</validationQuery>
        <validationInterval>30000</validationInterval>
    </configuration>
</definition>
</datasource>
v) Open the master-datasources.xml and update the WSO2AM_DB  data source.
<datasource>
            <name>WSO2AM_DB</name>
            <description>The datasource used for API Manager database</description>
            <jndiConfig>
                <name>jdbc/WSO2AM_DB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://localhost:3306/cluster_db</url>
                    <username>root</username>
                    <password></password>
                    <defaultAutoCommit>false</defaultAutoCommit>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>
vi) Open the registry.xml and add the following configuration for registry mounting.
<dbConfig name="wso2registry_mount">
        <dataSource>jdbc/WSO2CarbonDB_cluster_db</dataSource>
    </dbConfig>
    <remoteInstance url="https://localhost:9443/registry">
        <id>instanceid</id>
        <dbConfig>wso2registry_mount</dbConfig>
        <readOnly>false</readOnly>
        <enableCache>true</enableCache>
        <registryRoot>/</registryRoot>
        <cacheId>root@jdbc:mysql://localhost:3306/cluster_db</cacheId>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>instanceid</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>
       <mount path="/_system/governance" overwrite="true">
        <instanceId>instanceid</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount> 
vii) Open the user-mgt.xml and  update the dataSource property as bellow.
<Property name="dataSource">jdbc/WSO2CarbonDB_cluster_db</Property>
6.0 Add the virtual host configuration for proxy-2.

i) In windows,  this file (mgt.wso2am.conf)should be save in C:\xampp\apache\conf\extra directory (in Linux /etc/apache2/site-available)
<VirtualHost mgt.wso2am:443>


 ServerAdmin abc@wso2.com
 ServerName  mgt.wso2am
 ServerAlias mgt.wso2am

 ProxyRequests Off

 SSLEngine On
 SSLProxyEngine On

 #SSLProxyVerify none
 SSLProxyCheckPeerName off
 SSLProxyCheckPeerCN off
 #SSLProxyCheckPeerExpire off


 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://mgt.wso2am>
        BalancerMember https://mgt1.wso2am:9445 route=amNode1 loadfactor=1
        BalancerMember https://mgt2.wso2am:9446 route=amNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://mgt.wso2am/
ProxyPassReverse / balancer://mgt.wso2am/

</VirtualHost>

ii) In windows, include this virtual host configuration in httpd.conf file. (Find the # Virtual hosts configuration section and include )
Include conf/extra/mgt.wso2am.conf
iii) In linux you can use the following command.
sudo a2ensite mgt.wso2am.conf
iv) Create new key and crt for the proxy-2 like in steps 3.0.
  • Make sure to enter the mgt.wso2am as the Common Name (CN). 
  • The ca.crt and ca.key has configured as SSLCertificateFile  and SSLCertificateKeyFile in above virtual host configuration.
v) Generate new key store for manager nodes like in steps 4.0.
  • Make sure to enter the mgt.am.wso2.com as the first and last name , that value will be set as common name (CN).
  •  Enter the key store password as wso2carbon , then you don't need to change the default key store configurations.
 7.0 Configure the manager nodes.

i) Open the carbon.xml file (repository\conf) and define the HostName and MgtHostName in manager-1 and manager-2.
<HostName>am1.wso2am</HostName>
<MgtHostName>mgt1.wso2am</MgtHostName>
<HostName>am2.wso2am</HostName>
<MgtHostName>mgt2.wso2am</MgtHostName> 
ii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in http connector configuration.
proxyPort="80"
proxyName="mgt.wso2am"
iii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in https connector configuration.
proxyPort="443"
proxyName="mgt.wso2am"
 iv) Repeat the  same configuration steps mentioned  in 5. iv) , 5. v) , 5. vi) , 5. vii) for manager nodes

8.0 Add virtual host configuration for gateway management requests(publish API). (proxy-3).

<VirtualHost mgt.gw.wso2am:443>


 ServerAdmin abc@wso2.com
 ServerName  mgt.gw.wso2am
 ServerAlias mgt.gw.wso2am

 ProxyRequests Off

 SSLEngine On
 SSLProxyEngine On

 #SSLProxyVerify none
 SSLProxyCheckPeerName off
 SSLProxyCheckPeerCN off
 #SSLProxyCheckPeerExpire off


 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://mgt.gw.wso2am>
        BalancerMember https://mgt1.gw.wso2am:9443 route=gwNode1 loadfactor=1
        BalancerMember https://mgt2.gw.wso2am:9444 route=gwNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://mgt.gw.wso2am/
ProxyPassReverse / balancer://mgt.gw.wso2am/

</VirtualHost>
i) In windows,  include this virtual host configuration (mgt.gw.wso2am.conf) in httpd.conf file. (Find the # Virtual hosts configuration section and include )
Include conf/extra/mgt.gw.wso2am.conf
ii) In linux you can use the following command.
sudo a2ensite mgt.gw.wso2am.conf
9. Create database .
  • Here I'm going to create one database for shared registry, user manager and API manager database. In production setup that should configure as separate databases.
i) Login to the MySQL server and execute the following command. (I'm using MySQL 5.6.26)
CREATE DATABASE cluster_db CHARACTER SET latin1;  
ii) Download the  connector jar ("mysql-connector-java-5.1.29.jar") and copy to the repository\components\lib directory of all the gateway and manager nodes.

10. Configure api-manager.xml

i) Open the api-manager.xml file of the gateway nodes and configure the <AuthManager> configuration section. (ServerURL should point to proxy-2).
<AuthManager>
        <!--
            Server URL of the Authentication service
        -->
        <ServerURL>https://mgt.wso2am/services/</ServerURL>
        <!--
            Admin username for the Authentication manager.
        -->
        <Username>${admin.username}</Username>
        <!--
            Admin password for the Authentication manager.
        -->
        <Password>${admin.password}</Password>
    </AuthManager>
ii) Open the api-manager.xml file of the gateway node and configure the <APIKeyValidator> section. (the ServerURL should point to proxy2).
        <ServerURL>https://mgt.wso2am/services/</ServerURL>

        <!--
            Admin username for API key manager.
        -->
        <Username>${admin.username}</Username>

        <!--
            Admin password for API key manager.
        -->
        <Password>${admin.password}</Password>
iii) Open the api-manager.xml file of the manager nodes and configure the <APIGateway> configuration.(ServerURL should point to proxy-3)
<APIGateway>
    <!-- The environments to which an API will be published -->
    <Environments>
        <!-- Environments can be of different types. Allowed values are 'hybrid', 'production' and 'sandbox'.
             An API deployed on a 'production' type gateway will only support production keys
             An API deployed on a 'sandbox' type gateway will only support sandbox keys
             An API deployed on a 'hybrid' type gateway will support both production and sandbox keys -->
                <Environment type="hybrid" api-console="true">
                        <Name>Production and Sandbox</Name>
                    <Description> Description of environment</Description>
            <!--
                        Server URL of the API gateway.
                -->
                        <ServerURL>https://mgt.gw.wso2am/services/</ServerURL>
            <!--
                        Admin username for the API gateway.
                -->
                        <Username>${admin.username}</Username>
            <!--
                        Admin password for the API gateway.
                -->
                        <Password>${admin.password}</Password>
            <!--
                        Endpoint URLs for the APIs hosted in this API gateway.
                -->
                        <GatewayEndpoint>http://gw.wso2am,https://gw.wso2am</GatewayEndpoint>
                </Environment>
        </Environments>
</APIGateway>

11.  Insall the proxy certificates to the client-truststore.jks file in gateway and manager nodes.

12. Configure gateway cluster.

i) Open the axis2.xml (repository\conf\axis2) file and enable clsutering.
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
ii) Change membershipScheme as wka.
<parameter name="membershipScheme">wka</parameter>
iii) Add a name for cluster domain. (eg: wso2.gw.domain)
<parameter name="domain">wso2.gw.domain</parameter>
iv) Add localMemberHost name as server IP (eg for gateway-1: *.*.*.1 and for gateway-2: *.*.*.2 ).
<parameter name="localMemberHost">*.*.*.1</parameter>
v) Add localMemberPort.   (eg: use 4100 for gateway-1 and 4200 for gateway-2).
<parameter name="localMemberPort">4100</parameter>
vi ) Add members.
for gateway-1:
 <members>
          <member>
             <hostName>*.*.*.2</hostName>
             <port>4200</port>
          </member>
  </members>
for gateway-2:
 <members>
        <member>
             <hostName>*.*.*.1</hostName>
             <port>4100</port>
        </member>
  </members>

13. Start servers.

i) Go to the bin directory and execute the startup script (wso2server.sh for Linux and wso2server.bat for windows) as bellow. (If you have alreay started the server ,then delete the repository\database directory before start the serevr - This is to create the mount configuration in local registry).
Eg: for Linux
Gateway-1/bin >  sh wso2server.sh -Dsetup -DjvmRoute=gwNode1 -Dprofile=gateway-worker
Gateway-2/bin >  sh wso2server.sh -Dsetup -DjvmRoute=gwNode2 -Dprofile=gateway-worker
Manager-1/bin >  sh wso2server.sh -Dsetup -DjvmRoute=amNode1
Manager-2/bin >  sh wso2server.sh -Dsetup -DjvmRoute=amNode1
  • The "jvmRoute" Identifier which must be used in load balancing scenarios to enable session affinity.
  • The value for the "jvmRoute" is configured in virtual host configuration.
  • The -Dsetup parameter is to populate the tables for registry, user manager and api manager in cluster_db.
  • The -Dprofile=gateway-worker is to start the gateway nodes from gateway worker profile.
14. Add the hostnames to the etc host file. (in Windows C:\Windows\System32\drivers\etc\hosts)

15. Login to the publisher (https://mgt.wso2am/publisher) and deploy the default sample.

16. Login to the store (https://mgt.wso2am/store) , then subscribe to API and generate token.

17. You can use the SoapUI REST project for https://gw.wso2am//weatherapi/1.0.0?q=colombo&mode=xml

i) Add Authorization header and invoke the API.

Friday, August 14, 2015

Rsync to synchronize the artifacts - WSO2 products.

We have to configure the artifact synchronization mechanism to  synchronize the artifacts across the cluster nodes . The SVN based synchronization  is the most recommend way to do it .

The following use case is  just my local testing, try it ..!!  if you also interest. :)

In WSO2 products,  artifacts has stored in the /repository/deployment/server directory. In Linux base system,  we can use the Rsync (https://rsync.samba.org/) to synchronize artifacts across  the cluster nodes.

The syncd (https://github.com/drunomics/syncd) is simple bash script which is based on the rsync  and notify-tools utility. That provides very easy way to run the rsync in daemon mode.
apt-get install inotify-tools rsync
1. Go to the download directory and execute the following command to create symbolic link.
sudo ln -s $PWD/syncd /usr/local/bin/syncd
2. Copy the syncd.conf file to manager-1/repository/deployment/ directory.

3. Open that syncd.conf file and configure the following property.
WATCH_DIR=$SYNCD_CONFIG_DIR/server
SSH_USER=ajith
SSH_HOST=127.0.0.1
REMOTE_TARGET_DIR="/home/ajith/cluster/worker-1/repository/deployment/server"
5. Generate key for remote authentication to establish connection without prompting for the  password.

i)  Execute the following command and just enter for all the prompts.
ssh-keygen
ii) cd /home/<username>/.ssh 

iii) Open the id_rsa.pub   file , copy the key and save the file.
 vi id_rsa.pub  
iv) Execute the following command in same directory (.ssh) and past the key. [Ctrl+X then Y to save]
nano authorized_keys 
6. Go back to manager-1/repository/deployment/ and execute the following command to start the daemon task.
syncd start [stop|restart].

7. Now when you are  deploying the artifacts in manager node , that will be synchronize to the worker node.

8. You can extend the syncd script to synchronize the multiple target locations. :)

Saturday, August 8, 2015

Limit the size of the wso2carbon.log file

We can limit the size of the wso2carbon log file as bellow.

1. Open the log4j.properties file under <server_home>/repository/conf directory.

2. Find this log appender "log4j.appender.CARBON_LOGFILE=org.wso2.carbon.logging.appenders.CarbonDailyRollingFileAppender"
3. Change it as bellow.
log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
4. Add following two property under the RollingFileAppender.
log4j.appender.CARBON_LOGFILE.MaxFileSize=10MB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=20
Finally it should look like bellow:

# CARBON_LOGFILE is set to be a DailyRollingFileAppender using a PatternLayout.
log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
# Log file will be overridden by the configuration setting in the DB
# This path should be relative to WSO2 Carbon Home
log4j.appender.CARBON_LOGFILE.File=${carbon.home}/repository/logs/${instance.log}/wso2carbon${instance.log}.log
log4j.appender.CARBON_LOGFILE.Append=true
log4j.appender.CARBON_LOGFILE.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
# ConversionPattern will be overridden by the configuration setting in the DB
log4j.appender.CARBON_LOGFILE.layout.ConversionPattern=TID: [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n
log4j.appender.CARBON_LOGFILE.layout.TenantPattern=%U%@%D [%T] [%S]
log4j.appender.CARBON_LOGFILE.threshold=DEBUG
log4j.appender.CARBON_LOGFILE.MaxFileSize=10MB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=20