Wednesday, April 29, 2015

Build single bundle checkout - WSO2

Sometime you need to checkout single bundle from WSO2 source repository and build. Lets say you need to check out this bundle and build.

svn co https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/identity/org.wso2.carbon.identity.application.authenticator.facebook/4.2.0

If you try to build using maven (version >= 3.0.5), it will end with following error.
[INFO] Scanning for projects...
Downloading: http://repo.maven.apache.org/maven2/org/wso2/carbon/identity/4.2.0/identity-4.2.0.pom
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.wso2.carbon:org.wso2.carbon.identity.application.authenticator.facebook:4.2.0 (/home/ajith/Desktop/4.2.0/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM: Could not find artifact org.wso2.carbon:identity:pom:4.2.0 in central (http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 18, column 10 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
 
How to fix.

1. You need to remove the <parent> element.
<!--parent>
        <groupId>org.wso2.carbon</groupId>
        <artifactId>identity</artifactId>
        <version>4.2.0</version>
        <relativePath>../../pom.xml</relativePath>
</parent-->
2. Add groupId element.
<groupId>org.wso2.carbon</groupId>
3. Define the version for all the dependencies that not available. Eg:
<dependency>
         <groupId>org.wso2.carbon</groupId>
         <artifactId>org.wso2.carbon.logging</artifactId>
         <version>4.2.0</version>
</dependency>

      <dependency>
         <groupId>org.wso2.carbon</groupId>
         <artifactId>org.wso2.carbon.ui</artifactId>
         <version>4.2.0</version>
</dependency>
4. Define the maven repositories to get the artifacts.
<repositories>
      <!-- WSO2 released artifact repository -->
      <repository>
         <id>wso2.releases</id>
         <name>WSO2 Releases Repository</name>
         <url>http://maven.wso2.org/nexus/content/repositories/releases/</url>
         <releases>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>ignore</checksumPolicy>
         </releases>
      </repository>
      <repository>
         <id>wso2-nexus</id>
         <name>WSO2 internal Repository</name>
         <url>http://maven.wso2.org/nexus/content/groups/wso2-public/</url>
         <releases>
            <enabled>true</enabled>
            <updatePolicy>daily</updatePolicy>
            <checksumPolicy>ignore</checksumPolicy>
         </releases>
      </repository>
</repositories> 

[WSO2 ESB] Java client to invoke secured(UsernameToken) proxy service

1. Download the latest version of WSO2 ESB and deploy  this proxy service.(You can copy this file to to wso2esb-4.8.1/repository/deployment/server/synapse-configs/default/proxy-services directory)

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="SimpleStockQuoteServiceProxy"
       transports="https"
       startOnLoad="true"
       trace="disable">
    <description/>
    <target>
        <inSequence>
            <send>
                <endpoint>
                    <address uri="https://localhost:9002/services/SimpleStockQuoteService"/>
                </endpoint>
            </send>
            <header xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
                    name="wsse:Security"
                    action="remove"/>
        </inSequence>
        <outSequence>
            <send/>
        </outSequence>
    </target>
</proxy> 


2. Start the ESB  server.

3. Go to the wso2esb-4.8.1/samples/axis2Server/src/SimpleStockQuoteService and execute the ant command . That will build the SimpleStockQuoteService.aar and copy in to deployment directory of the  simple axis2 server.

4. Go to the wso2esb-4.8.1/samples/axis2Server and start the simple axis2 server(sh axis2server.sh)

5. The service endpoint  of SimpleStockQuoteService will be https://localhost:9002/services/SimpleStockQuoteService .

6. Login to the ESB management console and upload this policy to  registry. (/_system/config/repository/components/org.wso2.carbon.security.mgt/policy)



7. Go to the service list and start to secure the SimpleStockQuoteServiceProxy.


8. Check the UsernameToken option and pick the policy.xml from registry.



9. Select the user roles that proxy service can invoke.


10.Execute the ant command inside wso2esb-4.8.1/bin and set the wso2esb-4.8.1/lib as the class path of the project.

11. Copy the policy.xml to ESB home directory that path used in in  StockQuoteSecureClient.

12. Update the carbon_home and proxyEndpoint in StockQuoteSecureClient class and execute the client .



Thursday, April 16, 2015

Audit log publsiher for WSO2 Identity Server + WSO2 Business Activity Monitor(BAM)

WSO2 Identity server create an audit logs for  user account activities (Add users/roles, Delete users/roles, assign users to roles, ..etc) . We can publish those logs to BAM and analyze .

This is a custom log appender which is written to publish audit logs to BAM.

How to run.
-------------------
1.  Download the source from here .

2. Open the build.xml file and change the value of product.home property and execute the ant command to build the jar.

3. Copy the org.wso2.carbon.auditlog.publisher-1.0.0.jar file to <IS_HOME>/repository/components/lib

4.  Open the log4j.properties file and add the following configuration. (Change BAM url, username,password according to BAM configurations.)
log4j.appender.AUDIT_LOGFILE1=org.wso2.carbon.logging.appender.AuditLogEventAppender
log4j.appender.AUDIT_LOGFILE1.File=${carbon.home}/repository/logs/audit.log
log4j.appender.AUDIT_LOGFILE1.Append=true
log4j.appender.AUDIT_LOGFILE1.layout=org.wso2.carbon.utils.logging.TenantAwarePatternLayout
log4j.appender.AUDIT_LOGFILE1.layout.ConversionPattern=[%d] %P%5p - %x %m %n
log4j.appender.AUDIT_LOGFILE1.layout.TenantPattern=%U%@%D [%T] [%S]
log4j.appender.AUDIT_LOGFILE1.threshold=INFO
log4j.appender.AUDIT_LOGFILE1.url=tcp://localhost:7611
log4j.appender.AUDIT_LOGFILE1.columnList=%T,%S,%A,%d,%H,%c,%p,%m,%I,%Stacktrace
log4j.appender.AUDIT_LOGFILE1.userName=admin
log4j.appender.AUDIT_LOGFILE1.password=admin
log4j.appender.AUDIT_LOGFILE1.processingLimit=1000
log4j.appender.AUDIT_LOGFILE1.maxTolerableConsecutiveFailure=20
log4j.appender.AUDIT_LOGFILE1.trustStorePassword=wso2carbon
log4j.appender.AUDIT_LOGFILE1.truststorePath=/repository/resources/security/wso2carbon.jks 
5. Add the AUDIT_LOGFILE1 name.
log4j.logger.AUDIT_LOG=INFO, AUDIT_LOGFILE, AUDIT_LOGFILE1
6. Start the BAM server first.

7. Start the IS server.

8. Create  users/roles.

9. Logged in to the BAM and browse the Cassandra using "Explore Cluster" feature. Now you should see the "audit_log_IS" under the "EVENT_KS"


10. Browse the rows to view the attributes of the published events.


11. Run the following hive query to summarize the audit logs.

CREATE EXTERNAL TABLE IF NOT EXISTS ACCOUNT_ACTIVITY
(key STRING, initiator STRING, action STRING, target STRING, result STRING, uuid STRING, logTime BIGINT) STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' WITH SERDEPROPERTIES (
"wso2.carbon.datasource.name" = "WSO2BAM_CASSANDRA_DATASOURCE",
"cassandra.cf.name" = "audit_log_IS" ,
"cassandra.columns.mapping" =
":key,payload_initiator, payload_action, payload_target, payload_result,payload_uuid, payload_logTime" );


CREATE EXTERNAL TABLE IF NOT EXISTS ACCOUNT_ACTIVITY_SUMMARY1(initiator STRING, action STRING, target STRING, result STRING,uuid STRING, logTime STRING) STORED BY
'org.wso2.carbon.hadoop.hive.jdbc.storage.JDBCStorageHandler' TBLPROPERTIES (
'wso2.carbon.datasource.name'='WSO2BAM_DATASOURCE',
'hive.jdbc.update.on.duplicate' = 'false' ,
'hive.jdbc.primary.key.fields' = 'uuid' ,
'hive.jdbc.table.create.query' = 'CREATE TABLE ACCOUNT_ACTIVITY_SUMMARY1_TBL (initiator VARCHAR(100), action VARCHAR(100), target VARCHAR(100),result VARCHAR(100),uuid VARCHAR(100), logTime VARCHAR(100))' );

insert overwrite table ACCOUNT_ACTIVITY_SUMMARY1 select initiator,action,target,result,uuid, from_unixtime(cast(logTime/1000 as BIGINT), 'yyyy-MM-dd HH:mm:ss') as logTime  from ACCOUNT_ACTIVITY;

 CREATE EXTERNAL TABLE IF NOT EXISTS ACCOUNT_ACTIVITY_SUMMARY2(initiator STRING, action STRING, result STRING,totalcount INT) STORED BY
 'org.wso2.carbon.hadoop.hive.jdbc.storage.JDBCStorageHandler' TBLPROPERTIES ( 
 'wso2.carbon.datasource.name'='WSO2BAM_DATASOURCE',
 'hive.jdbc.update.on.duplicate' = 'false' , 
 'hive.jdbc.primary.key.fields' = 'uuid' , 
 'hive.jdbc.table.create.query' = 'CREATE TABLE ACCOUNT_ACTIVITY_SUMMARY2_TBL (initiator VARCHAR(100), action VARCHAR(100),result VARCHAR(100),totalcount INT)' );
 
insert overwrite table ACCOUNT_ACTIVITY_SUMMARY2 select initiator,action,result, count(DISTINCT key) from ACCOUNT_ACTIVITY group by initiator,action,result;


The summarized tables (ACCOUNT_ACTIVITY_SUMMARY1_TBL and ACCOUNT_ACTIVITY_SUMMARY2_TBL) will be created in BAM_STATS_DB  which is configured in bam-datasources.xml under the <BAM_HOME>/repository/conf/datasources.

12. Generate a gadget based on the summarized data.

JDBC URL*: jdbc:h2:repository/database/samples/BAM_STATS_DB;AUTO_SERVER=TRUE
Driver Class Name : org.h2.Driver
User Name* : wso2carbon
Password*  : wso2carbon

 




 

13. Generate gadget , copy the url.



 14. Go to the dashboard and add new gadget.