Monday, December 14, 2015

[JMeter] Invoke API chain using JMeter

I have created two ESB APIs. You can download these two files and copy to wso2esb-4.9.0/repository/deployment/server/synapse-configs/default/api directory.

1. First API (API-1) will  return the TV channels (tvstations.xml).

GET http://localhost:8280/channels

{
  "details":{
    "totalChannelCount":4,
    "channelDetails":{
      "channelDetail":[
        {
          "name":"TV-01",
          "owner":"ajith",
          "location":"Colombo 05"
        },
        {
          "name":"TV-02",
          "owner":"vajira",
          "location":"Colombo 05"
        },
        {
          "name":"TV-03",
          "owner":"ranjith",
          "location":"Colombo 05"
        },
        {
          "name":"TV-04",
          "owner":"lakmali",
          "location":"Colombo 05"
        }
      ]
    }
  }
}


2. The second API (API-2) will return the programs available in each channel (tvprograms.xml).

GET http://localhost:8280/programs?channel=TV-01

{
  "details": {
    "totalPrograms": 4,
    "channelName": "TV-01",
    "programsDetails": {
      "program": [
        {
          "name": "Morning Show",
          "presenter": "ajith",
          "duration": "1h"        },
        {
          "name": "News",
          "presenter": "vajira",
          "duration": "2h"        },
        {
          "name": "Lunch Time Music",
          "presenter": "nishatha",
          "duration": "3h"        },
        {
          "name": "movie",
          "presenter": "lakmali",
          "duration": "4h"        }
      ]
    }
  }
}
Now I'm going to create a JMeter test plan. First invoke  API-1 to get  TV channels, then iterate through each channel to get TV programs invoking  API-2.

Steps:

1. Download and install the JMeter plugins from http://jmeter-plugins.org/downloads/all/

(Download the "Extras with Libs Set")

2. This plugin provides "Json Xpath extractor" feature, which can be used to filter the TV channel names  from first API response.

3. Create a new "Thread Group".

"Test Plan" ----> "Add" ---> "Threads (users)" -----> "Thread Group"

4. Add "HTTP Header Manager", this is  useful to set the HTTP headers which is required to invoke the API.

 "Thread Group" ---> "Add" ---->  "Config Element" --->  "HTTP Header Manager"

5.  Add HTTP Request, then define a host , protocol, path ..etc.

"Thread Group ----> "Add"  ---> "Sampler" -----> HTTP Request"


5. Add JSON path extractor to filter the channel names from the first response.

"HTTP Request" ---> "Add" ---> "Post Processors" ---> "Json path extractor"

variable name : channelNames
Json path expression: $.details.channelDetails.channelDetail[*].name


6.  Add another JSON path extractor to assign channel count to a variable.

variable name : channelCount
Json path expression:$.details.totalChannelCount


7.  Add "Loop Controller" -  set the Loop Count as ${channelCount}

"Thread Group" ---> "Add" ---> "Logic Controller" ---> "Loop Controller"


8. Add counter to construct the for loop.

"Loop Controller" ---> "Add" ----> "Config Element" ----> "Counter"

Start : 0
Increment : 1
Maximum : ${channelCount}
Reference Name : loopCounter


9.  Add "BeanShell Processor" to set channel name as a variable.

"Loop Controller" --> "Add" ---> "Post Processors" ---> "BeanShell Processor"



// channelNames is the Json array as string.

String tvChannels = vars.get("channelNames");

// Create channle list as array

tvChannels = tvChannels.replace("[","").replace("]","").replace("\"","");

String[] channelArray = tvChannels.split(",");

// "loopCounter" is reference variable used in Counter

idx = Integer.parseInt(vars.get("loopCounter"));

// Set the chanle name to a variable

vars.put("channelName",channelArray[idx]);

10. Add the second API as HTTP Request.

The channelName  is variable defined in the BeanShell processor.
Path : /programs?channel=${channelName}

11. Add a listener to monitor the results.

Eg:

"HTTP Request"  ---> "Add" ---> "Listeners" ---> "Summary Report"


11. You can download the .jmx file  here and import to JMeter.

Friday, August 21, 2015

Advance Mediation with WSO2 API Manager(1.9) + WSO2 ESB -3

In this use case WSO2 API Manager act as the gateway, and WSO2 ESB used to execute the mediation logic.
1. This service (http://www.webservicex.com/globalweather.asmx) provides two operations.
  • GetCitiesByCountry
                   Get all major cities by country name(full / part).
  • GetWeather
                 Get weather report for all major cities around the world.

2. Create an API in ESB to invoke the above operations  using following REST operations (ESB start with port offset as 1).
http://localhost:8281/weatheresb/GetCitiesByCountry?CountryName=Romania
http://localhost:8281/weatheresb/GetCitiesByCountry?CountryName=Romania&CityName=Timisoara

Save this file as WeatherAPIESB.xml in <ESB_Home>\repository\deployment\server\synapse-configs\default\api)

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse"
     name="WeatherAPIESB"
     context="/weatheresb">
   <resource methods="GET" url-mapping="/GetCitiesByCountry">
      <inSequence>
         <payloadFactory media-type="xml">
            <format>
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                                 xmlns:web="http://www.webserviceX.NET">
                  <soapenv:Header/>
                  <soapenv:Body>
                     <web:GetCitiesByCountry><web:CountryName>$1</web:CountryName>
                     </web:GetCitiesByCountry>
                  </soapenv:Body>
               </soapenv:Envelope>
            </format>
            <args>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CountryName')"/>
            </args>
         </payloadFactory>
         <send>
            <endpoint>
               <address uri="http://www.webservicex.com/globalweather.asmx" format="soap12"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <property name="messageType" value="application/json" scope="axis2"/>
         <send/>
      </outSequence>
   </resource>
   <resource methods="GET" url-mapping="/GetWeather">
      <inSequence>
         <payloadFactory media-type="xml">
            <format>
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                                 xmlns:web="http://www.webserviceX.NET">
                  <soapenv:Header/>
                  <soapenv:Body>
                     <web:GetWeather><web:CityName>$1</web:CityName>
                        <web:CountryName>$2</web:CountryName>
                     </web:GetWeather>
                  </soapenv:Body>
               </soapenv:Envelope>
            </format>
            <args>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CityName')"/>
               <arg xmlns:m0="http://services.samples/xsd"
                    evaluator="xml"
                    expression="get-property('query.param.CountryName')"/>
            </args>
         </payloadFactory>
         <send>
            <endpoint>
               <address uri="http://www.webservicex.com/globalweather.asmx" format="soap12"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <property name="messageType" value="application/json" scope="axis2"/>
         <send/>
      </outSequence>
   </resource>
</api>
3.Create an API in API Manager considering the REST API exposed through the ESB (http://localhost:8281/weatheresb).

4.  Design API with  two GET resource.


5. Go to the implement wizard and define HTTP endpoint (http://localhost:8281/weatheresb).

6. Go the manage wizard , select the tier then "save & publish".


7.  Login to the store, subscribe to an application, then  generate token.



8.  You can use the SOAPUI to invoke this API.



Related posts:

i) http://www.vitharana.org/2015/01/soap-web-service-as-rest-api-using-wso2.html

ii) http://www.vitharana.org/2015/08/soap-webservice-as-rest-api-wso2-api.html

Thursday, August 20, 2015

SOAP webservice as a REST API - WSO2 API Manager (1.9) -2

1. I'm going to use the weather service which is available in public. (http://www.webservicex.com/globalweather.asmx)

2. This SOAP service has a operation called "GetCitiesByCountry". You can get the following results when It access directly from the SOAPUI.

3. Now I'm going to create API in WSO2 API Manager to expose this service operation as a REST operation.

Eg: Expose the  GetCitiesByCountry as GET method as bellow.
https://localhost:8243/weather/v1.0.0/GetCitiesByCountry?country=Romania
4. We want to write a mediation extension to read the "country" from request and construct the SOAP payload which is  expect by back end service (I'm going to save this file as JSONtoSOAP.xml). This file shroud upload to /_system/governance/apimgt/customsequences/in in registry.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="admin--weather:vv1.0.0--In">  
 <property name="country" expression="$url:country"></property>
   <payloadFactory media-type="xml">  
            <format>  
               <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"  
                                 xmlns:web="http://www.webserviceX.NET">  
                  <soapenv:Header/>  
                  <soapenv:Body>  
                     <web:GetCitiesByCountry>  
                     <web:CountryName>$1</web:CountryName>  
                     </web:GetCitiesByCountry>  
                  </soapenv:Body>  
               </soapenv:Envelope>  
            </format>  
            <args>  
               <arg xmlns:m0="http://services.samples/xsd" evaluator="xml" 
                                     expression="get-property('country')"></arg>  
            </args>  
         </payloadFactory>          
</sequence>


5. We want another mediation extension file to convert the SOAP response to JSON.(I'm going to save this file as SOAPtoJSON.xml). This file should upload to /_system/governance/apimgt/customsequences/out in registry.
<sequence xmlns="http://ws.apache.org/ns/synapse" name="admin--weather:vv1.0.0--Out">
   <property name="messageType" value="application/json" scope="axis2"/>
</sequence>

6.  Create an API as bellow. (Make sure to add new query parameter called country for GetCitiesByCountry resource)




7. Go to the implement wizard  and select the end point type as "Address Endpoint". Then define the endpoint as http://www.webservicex.com/globalweather.asmx


8.  Click on the "Advance Options" button and select the "Format" as SOAP 1.2.


9. Go to the manage wizard and select the custom mediation form the "Message Mediation Policies".


10. Click on "save & publish" button to publish this API to store.

11. Login to the store ,  subscribe that API  to an application and generate a token.

12. Use the following Curl command to invoke the  GetCitiesByCountry operation.

curl  -k -H "Authorization :Bearer 99215d98fa10ce6f2d83cd11ba9828" -H "Content-Type:application/json"   https://localhost:8243/weather/v1.0.0/GetCitiesByCountry?country=Romania

13. You can also use the SOAPUI to invoke that API. (Make sure to define the  Authorization header)


Related posts:

i) http://www.vitharana.org/2015/01/soap-web-service-as-rest-api-using-wso2.html

ii) http://www.vitharana.org/2015/08/advance-mediation-with-wso2-api-manager.html

Sunday, August 16, 2015

[WSO2 AM] API Manager distributed setup with Apache proxy

In this post I'm going to explain about how to setup API Manager distributed setup with Apache proxy. (The configuration details include for  both Windows and Linux environments)


  • The proxy1 used to accept the API request and distribute among the gaeway nodes.
  • The proxy2 used to distribute the key validation and authentication request among the manager nodes.
  • The proxy3 used to publish API to gateway nodes.
  • SVN/Rsync used to synchronize the API artifacts xml across the gateway nodes.
  • The registry(config and governance), user management and API manager database shared across all the nodes. 
Required applications:

1. Java 1.7.X
2 Apache2
3. Open SSL
4. MySQL

 1.0 Install and configure the Apache2 on Windows

i) You can download and install XAMP (https://www.apachefriends.org/index.html), that provides a control panel to manage the Apache server on Windows.

 Open the httpd.conf file (C:\xampp\apache\conf) and enable the following modules.
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
 ii) In Linux environment.

Install Apache2 using the following command.
$sudo apt-get install apache2
To enable the modules.
$sudo a2enmod proxy_http
$sudo a2enmod ssl
$sudo a2enmod proxy_balancer
$sudo a2enmod lbmethod_byrequests
$sudo a2enmod slotmem_shm
$sudo a2enmod proxy_wstunnel 
2.0 Create virtual host configuration for proxy-1.

i) In  windows,  virtual host file (eg: gw.wso2am.conf) should be saved in C:\xampp\apache\conf\extra directory.  (in Linux /etc/apache2/site-available).
<VirtualHost gw.wso2am:443>

 ServerAdmin abc@wso2.com
 ServerName  gw.wso2am
 ServerAlias gw.wso2am
 ProxyRequests Off
 SSLEngine On
 SSLProxyEngine On
 #SSLProxyVerify none
 SSLProxyCheckPeerCN off
 SSLProxyCheckPeerName off
 #SSLProxyCheckPeerExpire off

 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://gw.wso2am>
        BalancerMember https://gw1.wso2am:8243 route=gwNode1 loadfactor=1
        BalancerMember https://gw2.wso2am:8244 route=gwNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://gw.wso2am/
ProxyPassReverse / balancer://gw.wso2am/

</VirtualHost>
ii) In windows,  include this virtual host configuration in httpd.conf file. (Find the # Virtual hosts configuration section)
Include conf/extra/gw.wso2am.conf
iii) In linux you can use the following command.
sudo a2ensite gw.wso2am.conf
3.0 Create certificate file and key for the proxy-1.

i) You can download the open ssl for windows (https://code.google.com/p/openssl-for-windows/) . Exact the zip file and set the path variable to bin directory.

Generate key:
openssl genrsa -out ca.key 1024 (windows)
sudo openssl genrsa -out ca.key 1024 (linux)
Generate CSR (Certificate Signing Request ).
  • Here you need to enter CSR details and make sure to enter the gw.wso2am as the Common Name (CN).
openssl req -new -key ca.key -out ca.csr (windows)
sudo openssl req -new -key ca.key -out ca.csr (linux)
Generate self sign certificate.
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt (Windows)
sudo openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt (linux)


  • The ca.crt and ca.key has configured as SSLCertificateFile  and SSLCertificateKeyFile in above virtual host configuration.
  •  If the CN (Common Name) of the back end server certificate is different from the virtual host name, then you need to disable the SSLProxyCheckPeerCN and SSLProxyCheckPeerName in the virtual host configuration.
4.0 Create new key store for gateway nodes.

i) Generate new key store.
  •  Make sure to enter the gw.wso2am as the first and last name , that value will be set as common name (CN).
  •  Enter the key store password as wso2carbon , then you don't need to change the default key store configurations in API Manager.
keytool -genkey -keyalg RSA -alias wso2carbon -keystore wso2carbon.jks -storepass wso2carbon -validity 360 -keysize 2048

ii) Export the public  key.
keytool -export -alias wso2carbon -keystore wso2carbon.jks -storepass wso2carbon -file wso2carbon.pem



iii) Copy this new  wso2carbon.jks and wso2carbon.pem files to repository\resources\security directory of the gateway nodes.

iv) Go to the repository\resources\security directory from command line and execute the following command to remove the default wso2carbon certificate from the client-truststore.jks.
keytool -delete -alias wso2carbon -keystore client-truststore.jks
v) Import new certificate.
keytool -import -alias wso2carbon -file wso2carbon.pem -keystore client-truststore.jks -storepass wso2carbon
5.0 Configure the gateway nodes.

i) Open the carbon.xml file (repository\conf) and define the HostName and MgtHostName.
<HostName>gw1.wso2am</HostName>
<MgtHostName>mgt1.gw.wso2am</MgtHostName> 
ii) Open the catalina-server.xml file (repository\conf\tomcat) and add the following two properties in http connector configuration.
proxyPort="80"
proxyName="gw.wso2am"

iii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in https connector configuration.
proxyPort="443"
proxyName="gw.wso2am"

iv) Open the master-datasources.xml and configure new data source for mount registry.
<datasource>       
    <name>WSO2_CARBON_DB_cluster_db</name>
    <description>The datasource used for mount registry </description>
    <jndiConfig>
        <name>jdbc/WSO2CarbonDB_cluster_db</name>
    </jndiConfig>
    <definition type="RDBMS">
    <configuration>
        <url>jdbc:mysql://localhost:3306/cluster_db</url>
        <username>root</username>
        <password></password>
        <driverClassName>com.mysql.jdbc.Driver</driverClassName>
        <maxActive>50</maxActive>
        <maxWait>60000</maxWait>
        <testOnBorrow>true</testOnBorrow>
        <validationQuery>SELECT 1</validationQuery>
        <validationInterval>30000</validationInterval>
    </configuration>
</definition>
</datasource>
v) Open the master-datasources.xml and update the WSO2AM_DB  data source.
<datasource>
            <name>WSO2AM_DB</name>
            <description>The datasource used for API Manager database</description>
            <jndiConfig>
                <name>jdbc/WSO2AM_DB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:mysql://localhost:3306/cluster_db</url>
                    <username>root</username>
                    <password></password>
                    <defaultAutoCommit>false</defaultAutoCommit>
                    <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                </configuration>
            </definition>
        </datasource>
vi) Open the registry.xml and add the following configuration for registry mounting.
<dbConfig name="wso2registry_mount">
        <dataSource>jdbc/WSO2CarbonDB_cluster_db</dataSource>
    </dbConfig>
    <remoteInstance url="https://localhost:9443/registry">
        <id>instanceid</id>
        <dbConfig>wso2registry_mount</dbConfig>
        <readOnly>false</readOnly>
        <enableCache>true</enableCache>
        <registryRoot>/</registryRoot>
        <cacheId>root@jdbc:mysql://localhost:3306/cluster_db</cacheId>
    </remoteInstance>

    <mount path="/_system/config" overwrite="true">
        <instanceId>instanceid</instanceId>
        <targetPath>/_system/nodes</targetPath>
    </mount>
       <mount path="/_system/governance" overwrite="true">
        <instanceId>instanceid</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount> 
vii) Open the user-mgt.xml and  update the dataSource property as bellow.
<Property name="dataSource">jdbc/WSO2CarbonDB_cluster_db</Property>
6.0 Add the virtual host configuration for proxy-2.

i) In windows,  this file (mgt.wso2am.conf)should be save in C:\xampp\apache\conf\extra directory (in Linux /etc/apache2/site-available)
<VirtualHost mgt.wso2am:443>


 ServerAdmin abc@wso2.com
 ServerName  mgt.wso2am
 ServerAlias mgt.wso2am

 ProxyRequests Off

 SSLEngine On
 SSLProxyEngine On

 #SSLProxyVerify none
 SSLProxyCheckPeerName off
 SSLProxyCheckPeerCN off
 #SSLProxyCheckPeerExpire off


 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://mgt.wso2am>
        BalancerMember https://mgt1.wso2am:9445 route=amNode1 loadfactor=1
        BalancerMember https://mgt2.wso2am:9446 route=amNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://mgt.wso2am/
ProxyPassReverse / balancer://mgt.wso2am/

</VirtualHost>

ii) In windows, include this virtual host configuration in httpd.conf file. (Find the # Virtual hosts configuration section and include )
Include conf/extra/mgt.wso2am.conf
iii) In linux you can use the following command.
sudo a2ensite mgt.wso2am.conf
iv) Create new key and crt for the proxy-2 like in steps 3.0.
  • Make sure to enter the mgt.wso2am as the Common Name (CN). 
  • The ca.crt and ca.key has configured as SSLCertificateFile  and SSLCertificateKeyFile in above virtual host configuration.
v) Generate new key store for manager nodes like in steps 4.0.
  • Make sure to enter the mgt.am.wso2.com as the first and last name , that value will be set as common name (CN).
  •  Enter the key store password as wso2carbon , then you don't need to change the default key store configurations.
 7.0 Configure the manager nodes.

i) Open the carbon.xml file (repository\conf) and define the HostName and MgtHostName in manager-1 and manager-2.
<HostName>am1.wso2am</HostName>
<MgtHostName>mgt1.wso2am</MgtHostName>
<HostName>am2.wso2am</HostName>
<MgtHostName>mgt2.wso2am</MgtHostName> 
ii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in http connector configuration.
proxyPort="80"
proxyName="mgt.wso2am"
iii) Open the catalina-server.xml file (repository\conf\tomcat) add add the following two properties in https connector configuration.
proxyPort="443"
proxyName="mgt.wso2am"
 iv) Repeat the  same configuration steps mentioned  in 5. iv) , 5. v) , 5. vi) , 5. vii) for manager nodes

8.0 Add virtual host configuration for gateway management requests(publish API). (proxy-3).

<VirtualHost mgt.gw.wso2am:443>


 ServerAdmin abc@wso2.com
 ServerName  mgt.gw.wso2am
 ServerAlias mgt.gw.wso2am

 ProxyRequests Off

 SSLEngine On
 SSLProxyEngine On

 #SSLProxyVerify none
 SSLProxyCheckPeerName off
 SSLProxyCheckPeerCN off
 #SSLProxyCheckPeerExpire off


 SSLCertificateFile /home/ajith/ca.crt
 SSLCertificateKeyFile /home/ajith/ca.key

<Proxy balancer://mgt.gw.wso2am>
        BalancerMember https://mgt1.gw.wso2am:9443 route=gwNode1 loadfactor=1
        BalancerMember https://mgt2.gw.wso2am:9444 route=gwNode2 loadfactor=1
        ProxySet lbmethod=byrequests
        ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>

ProxyPass /  balancer://mgt.gw.wso2am/
ProxyPassReverse / balancer://mgt.gw.wso2am/

</VirtualHost>
i) In windows,  include this virtual host configuration (mgt.gw.wso2am.conf) in httpd.conf file. (Find the # Virtual hosts configuration section and include )
Include conf/extra/mgt.gw.wso2am.conf
ii) In linux you can use the following command.
sudo a2ensite mgt.gw.wso2am.conf
9. Create database .
  • Here I'm going to create one database for shared registry, user manager and API manager database. In production setup that should configure as separate databases.
i) Login to the MySQL server and execute the following command. (I'm using MySQL 5.6.26)
CREATE DATABASE cluster_db CHARACTER SET latin1;  
ii) Download the  connector jar ("mysql-connector-java-5.1.29.jar") and copy to the repository\components\lib directory of all the gateway and manager nodes.

10. Configure api-manager.xml

i) Open the api-manager.xml file of the gateway nodes and configure the <AuthManager> configuration section. (ServerURL should point to proxy-2).
<AuthManager>
        <!--
            Server URL of the Authentication service
        -->
        <ServerURL>https://mgt.wso2am/services/</ServerURL>
        <!--
            Admin username for the Authentication manager.
        -->
        <Username>${admin.username}</Username>
        <!--
            Admin password for the Authentication manager.
        -->
        <Password>${admin.password}</Password>
    </AuthManager>
ii) Open the api-manager.xml file of the gateway node and configure the <APIKeyValidator> section. (the ServerURL should point to proxy2).
        <ServerURL>https://mgt.wso2am/services/</ServerURL>

        <!--
            Admin username for API key manager.
        -->
        <Username>${admin.username}</Username>

        <!--
            Admin password for API key manager.
        -->
        <Password>${admin.password}</Password>
iii) Open the api-manager.xml file of the manager nodes and configure the <APIGateway> configuration.(ServerURL should point to proxy-3)
<APIGateway>
    <!-- The environments to which an API will be published -->
    <Environments>
        <!-- Environments can be of different types. Allowed values are 'hybrid', 'production' and 'sandbox'.
             An API deployed on a 'production' type gateway will only support production keys
             An API deployed on a 'sandbox' type gateway will only support sandbox keys
             An API deployed on a 'hybrid' type gateway will support both production and sandbox keys -->
                <Environment type="hybrid" api-console="true">
                        <Name>Production and Sandbox</Name>
                    <Description> Description of environment</Description>
            <!--
                        Server URL of the API gateway.
                -->
                        <ServerURL>https://mgt.gw.wso2am/services/</ServerURL>
            <!--
                        Admin username for the API gateway.
                -->
                        <Username>${admin.username}</Username>
            <!--
                        Admin password for the API gateway.
                -->
                        <Password>${admin.password}</Password>
            <!--
                        Endpoint URLs for the APIs hosted in this API gateway.
                -->
                        <GatewayEndpoint>http://gw.wso2am,https://gw.wso2am</GatewayEndpoint>
                </Environment>
        </Environments>
</APIGateway>

11.  Insall the proxy certificates to the client-truststore.jks file in gateway and manager nodes.

12. Configure gateway cluster.

i) Open the axis2.xml (repository\conf\axis2) file and enable clsutering.
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
ii) Change membershipScheme as wka.
<parameter name="membershipScheme">wka</parameter>
iii) Add a name for cluster domain. (eg: wso2.gw.domain)
<parameter name="domain">wso2.gw.domain</parameter>
iv) Add localMemberHost name as server IP (eg for gateway-1: *.*.*.1 and for gateway-2: *.*.*.2 ).
<parameter name="localMemberHost">*.*.*.1</parameter>
v) Add localMemberPort.   (eg: use 4100 for gateway-1 and 4200 for gateway-2).
<parameter name="localMemberPort">4100</parameter>
vi ) Add members.
for gateway-1:
 <members>
          <member>
             <hostName>*.*.*.2</hostName>
             <port>4200</port>
          </member>
  </members>
for gateway-2:
 <members>
        <member>
             <hostName>*.*.*.1</hostName>
             <port>4100</port>
        </member>
  </members>

13. Start servers.

i) Go to the bin directory and execute the startup script (wso2server.sh for Linux and wso2server.bat for windows) as bellow. (If you have alreay started the server ,then delete the repository\database directory before start the serevr - This is to create the mount configuration in local registry).
Eg: for Linux
Gateway-1/bin >  sh wso2server.sh -Dsetup -DjvmRoute=gwNode1 -Dprofile=gateway-worker
Gateway-2/bin >  sh wso2server.sh -Dsetup -DjvmRoute=gwNode2 -Dprofile=gateway-worker
Manager-1/bin >  sh wso2server.sh -Dsetup -DjvmRoute=amNode1
Manager-2/bin >  sh wso2server.sh -Dsetup -DjvmRoute=amNode1
  • The "jvmRoute" Identifier which must be used in load balancing scenarios to enable session affinity.
  • The value for the "jvmRoute" is configured in virtual host configuration.
  • The -Dsetup parameter is to populate the tables for registry, user manager and api manager in cluster_db.
  • The -Dprofile=gateway-worker is to start the gateway nodes from gateway worker profile.
14. Add the hostnames to the etc host file. (in Windows C:\Windows\System32\drivers\etc\hosts)

15. Login to the publisher (https://mgt.wso2am/publisher) and deploy the default sample.

16. Login to the store (https://mgt.wso2am/store) , then subscribe to API and generate token.

17. You can use the SoapUI REST project for https://gw.wso2am//weatherapi/1.0.0?q=colombo&mode=xml

i) Add Authorization header and invoke the API.

Friday, August 14, 2015

Rsync to synchronize the artifacts - WSO2 products.

We have to configure the artifact synchronization mechanism to  synchronize the artifacts across the cluster nodes . The SVN based synchronization  is the most recommend way to do it .

The following use case is  just my local testing, try it ..!!  if you also interest. :)

In WSO2 products,  artifacts has stored in the /repository/deployment/server directory. In Linux base system,  we can use the Rsync (https://rsync.samba.org/) to synchronize artifacts across  the cluster nodes.

The syncd (https://github.com/drunomics/syncd) is simple bash script which is based on the rsync  and notify-tools utility. That provides very easy way to run the rsync in daemon mode.
apt-get install inotify-tools rsync
1. Go to the download directory and execute the following command to create symbolic link.
sudo ln -s $PWD/syncd /usr/local/bin/syncd
2. Copy the syncd.conf file to manager-1/repository/deployment/ directory.

3. Open that syncd.conf file and configure the following property.
WATCH_DIR=$SYNCD_CONFIG_DIR/server
SSH_USER=ajith
SSH_HOST=127.0.0.1
REMOTE_TARGET_DIR="/home/ajith/cluster/worker-1/repository/deployment/server"
5. Generate key for remote authentication to establish connection without prompting for the  password.

i)  Execute the following command and just enter for all the prompts.
ssh-keygen
ii) cd /home/<username>/.ssh 

iii) Open the id_rsa.pub   file , copy the key and save the file.
 vi id_rsa.pub  
iv) Execute the following command in same directory (.ssh) and past the key. [Ctrl+X then Y to save]
nano authorized_keys 
6. Go back to manager-1/repository/deployment/ and execute the following command to start the daemon task.
syncd start [stop|restart].

7. Now when you are  deploying the artifacts in manager node , that will be synchronize to the worker node.

8. You can extend the syncd script to synchronize the multiple target locations. :)

Saturday, August 8, 2015

Limit the size of the wso2carbon.log file

We can limit the size of the wso2carbon log file as bellow.

1. Open the log4j.properties file under <server_home>/repository/conf directory.

2. Find this log appender "log4j.appender.CARBON_LOGFILE=org.wso2.carbon.logging.appenders.CarbonDailyRollingFileAppender"
3. Change it as bellow.
log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
4. Add following two property under the RollingFileAppender.
log4j.appender.CARBON_LOGFILE.MaxFileSize=10MB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=20
Finally it should look like bellow:

# CARBON_LOGFILE is set to be a DailyRollingFileAppender using a PatternLayout.
log4j.appender.CARBON_LOGFILE=org.apache.log4j.RollingFileAppender
# Log file will be overridden by the configuration setting in the DB
# This path should be relative to WSO2 Carbon Home
log4j.appender.CARBON_LOGFILE.File=${carbon.home}/repository/logs/${instance.log}/wso2carbon${instance.log}.log
log4j.appender.CARBON_LOGFILE.Append=true
log4j.appender.CARBON_LOGFILE.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
# ConversionPattern will be overridden by the configuration setting in the DB
log4j.appender.CARBON_LOGFILE.layout.ConversionPattern=TID: [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n
log4j.appender.CARBON_LOGFILE.layout.TenantPattern=%U%@%D [%T] [%S]
log4j.appender.CARBON_LOGFILE.threshold=DEBUG
log4j.appender.CARBON_LOGFILE.MaxFileSize=10MB
log4j.appender.CARBON_LOGFILE.MaxBackupIndex=20

Saturday, July 25, 2015

[WSO2 AM] Invoke file upload spring service using WSO2 API Manager(1.9).

This is one use case I tried recently. So I thought to post that as a blog post :).

1. Deploy sample spring service to upload a file.

i  Download the sample service from this website https://spring.io/guides/gs/uploading-files/  (download link https://github.com/spring-guides/gs-uploading-files/archive/master.zip)

The FileUploadController.java class like bellow.

package hello;

import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;

import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.multipart.MultipartFile;

@Controller
public class FileUploadController {
    
    @RequestMapping(value="/upload", method=RequestMethod.GET)
    public @ResponseBody String provideUploadInfo() {
        return "You can upload a file by posting to this same URL.";
    }
    
    @RequestMapping(value="/upload", method=RequestMethod.POST)
    public @ResponseBody String handleFileUpload(@RequestParam("name") String name, 
            @RequestParam("file") MultipartFile file){
        if (!file.isEmpty()) {
            try {
                byte[] bytes = file.getBytes();
                BufferedOutputStream stream = 
                        new BufferedOutputStream(new FileOutputStream(new File(name)));
                stream.write(bytes);
                stream.close();
                return "You successfully uploaded " + name + "!";
            } catch (Exception e) {
                return "You failed to upload " + name + " => " + e.getMessage();
            }
        } else {
            return "You failed to upload " + name + " because the file was empty.";
        }
    }

}
ii Unzip the file and go to the gs-uploading-files-master\complete directory from command window.

iii. Execute the following command to build the executable jar file.(You should have maven 3.x installed)

mvn  clean install

(The default pom file configured to build with Java 8 , if you are using  java 6 or 7 change the following section in the pom file gs-uploading-files-master\complete\pom.xml)
 <properties>
        <java .version="">1.7</java>
    </properties>


iv.  Go to the target directory and execute the following command to run our file upload service.

java -jar gs-uploading-files-0.1.0.jar


v. When you locate your browser to http://localhost:8080/ , you should be able to see the service UI to upload a file.

vi. You can send a POST request to http://localhost:8080/upload to upload file as well.

2. Create a API for this service using WSO2 API Manager.

i. Required fields to create an API.

API Name: FileUploadAPI
Context    : /fileupload
Version    : 1.0.0
URL pattern : /*
HTTP Method : POST
Production URL :  http://localhost:8080/upload
Tier Availability : Unlimited 
 
ii. After published API , log in to the store , subscribe to API and generate token.

iii) Generate SOAP UI project using the API endpoint https://localhost:8243/fileupload/1.0.0

  • Change the HTTP method to POST.
  • Add a query parameter "name". (because hanleFileUpload operation in FileUploadController.java expect that parameter).
  • Select the Media Type as "multipart/form-data" . (because we send file to the upload service as an attachment)
  • Add Authorization header.(because we invoke OAuth2 protected resource exposed by WSO2 API Manager)

iv) Add a file as attachment in SOAP UI.


v) Now when you send a request,  you should see the error message which similar to this.

{
   "timestamp": 1437841327984,
   "status": 400,
   "error": "Bad Request",
   "exception": "org.springframework.web.bind.MissingServletRequestParameterException",
   "message": "Required MultipartFile parameter 'file' is not present",
   "path": "/upload"
}



vi) To avoid this issue , you need to define the ContentID as "file", in the attachment window.


This ContentID value is depend on the name given for the RequestParam [eg: @RequestParam("file")] defined for the MultipartFile in method signature.
public @ResponseBody String handleFileUpload(@RequestParam("name") String name, 
            @RequestParam("file") MultipartFile file){


vii) Go to the place you execute the jar file, then you should see the uploaded file.


Friday, July 3, 2015

[WSO2 AM] Add "getRecentlyAddedAPIs" operation for Store API - WSO2 API Manager.

The WSO2 API Manager doesn't expose the "getRecentlyAddedAPIs" operation through the Store API (https://docs.wso2.com/display/AM190/Store+APIs). But you can do the following workaround to expose that operation.

1. Open the file <am_home>/repository/deployment/server/jaggeryapps/store/site/blocks/api/recently-added/ajax/list.jag and change it as bellow.

<%
include("/jagg/jagg.jag");
(function () {
    response.contentType = "application/json; charset=UTF-8";
    var mod, obj, tenant, result, limit,

            msg = require("/site/conf/ui-messages.jag"),
            action = request.getParameter("action");
    if (action == "getRecentlyAddedAPIs") {
        tenant = request.getParameter("tenant");
        limit = request.getParameter("limit");
        mod = jagg.module("api");
        result = mod.getRecentlyAddedAPIs(limit,tenant);
        if (result.error) {
            obj = {
                error:result.error,
                message:msg.error.authError(action)
            };
        } else {
            obj = {
                error:false,
                apis:result.apis
            }
        }
        print(obj);
    } else {
        print({
            error:true,
            message:msg.error.invalidAction(action)
        });
    }
}());
%>

2. Now you can invoke the getRecentlyAddedAPIs operation as bellow.

curl -b cookies 'http://localhost:9763/store/site/blocks/api/recently-added/ajax/list.jag?action=getRecentlyAddedAPIs&limit=10&tenant=carbon.super'

limit - Number of recently added APIs
tenant - Tenant domain.

Sunday, June 28, 2015

Add third party library to custom feature developed for WSO2 product.


This is a great article which explain how to write a custom feature for WSO2 product.(http://wso2.com/library/articles/2013/09/how-to-build-a-custom-stack-with-wso2-carbon/)

This blog post is going to explain how to add third party library to your custom feature.

1. You can create orbit bundle from that third part library.

Eg : This[i] is to make the Apache POI library as OSGI bundle.

[i] https://svn.wso2.org/repos/wso2/carbon/orbit/branches/4.2.0/poi/3.9.0.wso2v1/pom.xml

2. Build your orbit bundle. (use maven)

3. Then you need to add that dependency to student-manager/features/org.wso2.carbon.student.mgt.server.feature/pom.xml file.

Eg:
<dependency>
       <groupId>org.apache.poi.wso2</groupId>
       <artifactId>poi</artifactId>
       <version>3.9.0.wso2v1</version>
</dependency> 

4. Add new <bundleDef> entry inside the <bundles> element in student-manager/features/org.wso2.carbon.student.mgt.server.feature/pom.xml file.

The format of the <bundleDef> should be , <bundleDef>[groupId]:[artifactId]</bundleDef>

Eg:
<bundleDef>org.apache.poi.wso2:poi</bundleDef>

org.apache.poi.wso2 - groupId of above dependency
poi                           - artifactId of above dependency

5. Build the student-manager project again, now you should see that poi_3.9.0.wso2v1.jar file in student-manager/repository/target/p2-repo/plugins directory.

6. Finally when you install student-manager feature to WSO2 server (from that p2-repo), that third party library(poi_3.9.0.wso2v1.jar) will automatically install.

Wednesday, May 20, 2015

[WSO2 AM] Access token related issues

Create an API with following details.

Name      : StockquoteAPI
Context   : stockquote
Version   : 1.0.0
Endpoint : http://www.webservicex.net/stockquote.asmx
Resource : GetQuote
Query      : symbol




1. Invoke with invalid token.

Client side errors:

401 Unauthorized

<ams:fault>
 <ams:code>900901</ams:code>
 <ams:message>Invalid Credentials</ams:message>
 <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8</ams:description>
</ams:fault>
Backend error :

[2015-05-16 22:22:14,630] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


Solution: Double check the token.

2. Invoke  API with invalid token type.

Eg: Invoke  API with application token , But resource is allowed only for the application user tokens.

Client Errors:

401 Unauthorized

<ams:fault>
   <ams:code>900905</ams:code>
   <ams:message>Incorrect Access Token Type is provided</ams:message>
   <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8a</ams:description>
 </ams:fault>
Back end Error:

[2015-05-16 22:29:05,262] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8a
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Solution: Edit  API from publisher and go to the manage wizard. Then check for the authentication type.


3. Invoke non-existing API resource.

Client Errors:

403 Forbidden

<ams:fault>
 <ams:code>900906</ams:code>
 <ams:message>No matching resource found in the API for the given request</ams:message>
 <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8a</ams:description>
</ams:fault>

Back end Error:

[2015-05-16 22:40:00,506] ERROR - APIKeyValidator Could not find matching resource for /GetQuote1?symbol=ibm
[2015-05-16 22:40:00,507] ERROR - APIKeyValidator Could not find matching resource for request
[2015-05-16 22:40:00,508] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: lI2XVmmRJ9_B_rbh1rwV7Pg3Pp8a
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745 

Solution: Edit  API from publisher (Design wizard) and double check the availability of the resource names.


4. Token has generated without scope (scope as default), But API resource configured with scope.

Client Errors:

403 Forbidden

<ams:fault>
 <ams:code>900910</ams:code>
 <ams:message>The access token does not allow you to access the requested resource</ams:message>
 <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: 1e1b6aa805d4bfd89b6e36ac48345a</ams:description>
</ams:fault>
Back end Error:

[2015-05-16 23:08:57,103] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: 1e1b6aa805d4bfd89b6e36ac48345a
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


Solution: Generate new token with scope(s).
eg:
curl -k -d "grant_type=password&username=admin&password=admin&scope=stock" -H "Authorization: Basic THUwUVlFUUIxYVRKY3B6YTIxQnFxa0ZhU1I0YTo0ZE1FRUs3N1k4emZhSU56aVdGbTB1aFNBdjBh, Content-Type: application/x-www-form-urlencoded" https://localhost:8243/token


5. Invoke with expired token.

Client Errors

401 Unauthorized

<ams:fault>
 <ams:code>900903</ams:code>
 <ams:message>Access Token Expired</ams:message>
 <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: 8d438b49d9b24c752ce2b89c24bc198</ams:description>
</ams:fault>

Back end error:

[2015-05-17 13:30:50,155] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: 8d438b49d9b24c752ce2b89c24bc198
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Solution : You need to re-generate a token. If it is user token , you can use the refresh token to generate new token.

curl -k -d "grant_type=refresh_token&refresh_token=<retoken>&scope=PRODUCTION" -H "Authorization: Basic SVpzSWk2SERiQjVlOFZLZFpBblVpX2ZaM2Y4YTpHbTBiSjZvV1Y4ZkM1T1FMTGxDNmpzbEFDVzhh, Content-Type: application/x-www-form-urlencoded" https://localhost:8243/token

6. Token generated with 60 seconds life span , but API can invoke with that token after 60 seconds.


Reason:

The default token validation time is 3600 seconds that is configured in identity.xml file.
<AccessTokenDefaultValidityPeriod>3600</AccessTokenDefaultValidityPeriod>
But there is another configuraion called "TimestampSkew"
<TimestampSkew>300</TimestampSkew>
You can find the usage of that configuration here https://docs.wso2.com/display/AM180/Token+API#TokenAPI-Configuringthetokenexpirationtime

According to that description, token will be valid until the TimestampSkew eventhough the generated time less than the TimestampSkew.

7. User can generate access token, but API is not subscribed to that application.

Client Errors

401 Unauthorized

<ams:fault>
 <ams:code>900901</ams:code>
 <ams:message>Invalid Credentials</ams:message>
 <ams:description>Access failure for API: /stockquote, version: 1.0.0 with key: b31077463e7e7856762234c5d0b599</ams:description>
</ams:fault>
Back end Error

[2015-05-17 22:32:44,609] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /stockquote, version: 1.0.0 with key: b31077463e7e7856762234c5d0b599
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:168)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)  

Solution: Logged in to the store and subscribe API to application.




Monday, May 11, 2015

H2 database in WSO2 products.

H2 is open source and free to distribute. Therefore WSO2 product has selected H2 as the default embedded database. All WSO2 products use embedded H2  databases to store application data in default distribution.

Eg:  WSO2 Identity Server use WSO2CARBON_DB to store registry, user management and identity management related data (service providers, SSO configurations, tokens ..etc). WSO2 API Manager use WSO2AM_DB to store API related data.

You can browse those databases , please look at this post http://www.vitharana.org/2012/04/how-to-browse-h2-database-of-wso2.html

The H2 database can run in different modes including embedded mode, server mode and mixed mode. http://www.h2database.com/html/features.html#connection_modes

This embedded database can be corrupted due to various reasons including failed file locking at the concurrent access (http://www.h2database.com/html/features.html#database_file_locking), kill Java process to shut down the server, low disc space  ..etc.

WSO2 highly recommend to use standard production ready database like MySQL, Oracle, PostgreSQL ..etc for production/Dev/QA deployment rather using embedded H2 database(s).

The registry database has three logical partition called , local, config and governance. You can only use the default embedded H2 database only for the local registry.

In local registry we store mount configurations, if the local registry is corrupted we can simply delete default embedded H2 database and restart the server  with -Dsetup parameter. Then new database will be created and mount configuration will populate automatically.