Tuesday, February 7, 2017

Enabling SSL for Oracle SOA Managed Server

SSL or TLS is a transport level security, a de-facto standard for secured communication over web ( HTTPS) or file systems( sFTP/secure File transfers). It secures communication by providing message encryption, integrity, and authentication.


  • Authentication enables the server and client to check that the other party is who it claims to be.
  • Integrity ensures that a message sent by a client is received intact by the server, untampered.
  • Encryption provides confidentiality by allowing only the intended recipient to read the message.


For more details about this refer to http://docs.oracle.com/cd/E25054_01/core.1111/e10105/sslconfig.htm#autoId1

The below picture depicts succintly and clearly how SSL session is conducted:



Two kinds of SSL options exist -

  • One Way  
  • Two way. 

This is often misleading term with out mentioning the source and destination. One-Way usually refers to the source trying to be able to call a secured service at the destination. Two Way refers to destination also being able to consume the  secured services at the source in addition to one -way, better represented using pictures below. Source: ibm.com

To configure 2 way SSL on on Oracle SOA 11g Managed server the below are high level steps:

  1. Generate public key/ private key pair in a java identity key store.
  2. Generate either  
  3. Copy the self signed certificate to trust keystore.
  4. Add the third party certificates ( root/intermediate) into the trust storeModify the setDomainEnv.sh to refer to correct trust store.
  5. Refer to the above custom identity and trust stores from SOA managed server in weblogic.
  6. Enable SSL on a port.

The detailed steps for the same are clearly mentioned in the below posts:

Monday, August 4, 2014

Calling external REST services supporting JSON messaging standard in Oracle SOA 11g

SOA Suite 11g doesn't provide an out of box way to call JSON based REST services. In this blog we shall the alternative options to achieve the same.

HTTP binding in SOA Suite 11g  has ability to make calls to endpoints based on HTTP verbs(GET, POST only) in request-response (or) one way modes as depicted below.
  



However Oracle's Documentation states that -"An outbound HTTP binding reference supports only XML as a response from an external HTTP endpoint. The response should contain the correct XML part name according to outbound expectations."

In other words Http binding supports only XML as output for either GET/POST. For input it supports only url-encoded and xml. Though the url-encoded input has to be expressed/defined in an xsd format as shown below.



 Another major hindrance is that - Majority of the REST services created in IT world use JSON rather than XML as their messaging standard dues to its light weight and simplicity ( the same reason to use REST as against SOAP). HTTP adapter/binding has no support for calling REST endpoints which use JSON as a message standard as against XML. 
Does this leave us no way to call REST services using JSON ? Well not really. We always have Java to fall back on to do something that can't be done ( or provided) in SOA Suite.

The below Java code can be used to call REST service using JSON. The input of the methods is an xml format represented in a string ( to make it easier to interact with bpel). The following are the key points of this code

a. Takes input in XML string format.
b. Converts the XML string to JSON object.
c. Creates a HTTPClient and makes a call to post/get method using the REST URI providing the required basic http authentication credentials.
d. Extra code to bypass the SSL layer ( only to be used in dev mode). In production one should import the right certificates to authenticate themselves.

RestService.java

package oracle.soa.rest;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;

import net.sf.json.JSON;
import net.sf.json.xml.XMLSerializer;

import org.apache.http.auth.AuthScope;
import org.apache.http.auth.Credentials;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.methods.HttpGet;

public class RestService
{
    public HttpResponse callPOSTService(String xmlStr,String url) throws ClientProtocolException, IOException
    {
          // Convert the input XML to the JSON String
   XMLSerializer xmlSerializer = new XMLSerializer();  
   JSON json = xmlSerializer.read(xmlStr);  
   System.out.println(json);
   // Call POST of REST Service with JSON input

                        DefaultHttpClient client = new DefaultHttpClient();
                        client = WebClientDevWrapper.wrapClient(client);

                        Credentials creds = new UsernamePasswordCredentials("username", "password");
                        client.getCredentialsProvider().setCredentials( new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT), creds);

                        HttpPost post = new HttpPost(url);
   StringEntity se = new StringEntity( json.toString());
   post.setEntity(se);
                        
   HttpResponse response = client.execute(post);
   return response;
 }
    public HttpResponse callGETService(String url) throws ClientProtocolException, IOException
    {   
                        // Call GET of REST Service 
                        DefaultHttpClient client = new DefaultHttpClient();
                        client = WebClientDevWrapper.wrapClient(client);

                        Credentials creds = new UsernamePasswordCredentials("username", "password");
                        client.getCredentialsProvider().setCredentials( new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT), creds);

                        HttpGet get = new HttpGet(url);
                        HttpResponse response = client.execute(get);
                
                        return response;
        }

    public static void main(String[] args) throws ClientProtocolException, IOException
    {
        RestService rs = new RestService();
 /*       String inputString="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" + 
        "<root>\n" + 
        "   <description>Product New one</description>\n" + 
        "   <sku>146-4100-901</sku>\n" + 
        "</root>";
        
        // Create the product with SKU mentioned above.

        HttpResponse response = rs.callPOSTService(inputString,"https://abc.com/api/v1/products");
        // Get the URL to update the product - Eg: "https://abc.com/api/v1/products/18"
        Header header = response.getFirstHeader("Location");
        String update_url=header.toString().substring(10);
        System.out.println(update_url);*/

       /* // Update the product
        String updatedinputString="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" + 
        "<root>\n" + 
        "   <description>Update Product New one</description>\n" + 
        "   <family>Items</family>\n" + 
        "</root>";
        System.out.println(rs.callIPOSTService(updatedinputString,"https://abc.com/api/v1/products/22"));
        */
        // Get Product
        HttpResponse response = rs.callGETService("https://abc.com/api/v1/products/22");
        String xmlResponse = "";
        BufferedReader rd = new BufferedReader (new InputStreamReader(response.getEntity().getContent()));
        String line = "";
        while ((line = rd.readLine()) != null) {
          xmlResponse+=line;
        }
        System.out.println(xmlResponse);
    }

}

WebClientDevWrapper.java - Used to bypass the SSL

package oracle.soa.rest;

import java.io.IOException;

import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLException;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.conn.ssl.X509HostnameVerifier;
import org.apache.http.impl.client.DefaultHttpClient;

/*
*/
public class WebClientDevWrapper {

    public static DefaultHttpClient wrapClient(DefaultHttpClient base) {
        try {
            SSLContext ctx = SSLContext.getInstance("TLS");
            X509TrustManager tm = new X509TrustManager() {

                public void checkClientTrusted(X509Certificate[] xcs,
                                               String string) throws CertificateException {
                }

                public void checkServerTrusted(X509Certificate[] xcs,
                                               String string) throws CertificateException {
                }

                public X509Certificate[] getAcceptedIssuers() {
                    return null;
                }
            };
            X509HostnameVerifier verifier = new X509HostnameVerifier() {

                @Override
                public void verify(String string,
                                   SSLSocket ssls) throws IOException {
                }

                @Override
                public void verify(String string,
                                   X509Certificate xc) throws SSLException {
                }

                @Override
                public void verify(String string, String[] strings,
                                   String[] strings1) throws SSLException {
                }

                @Override
                public boolean verify(String string, SSLSession ssls) {
                    return true;
                }
            };
            ctx.init(null, new TrustManager[] { tm }, null);
            SSLSocketFactory ssf = new SSLSocketFactory(ctx);
            ssf.setHostnameVerifier(verifier);
            ClientConnectionManager ccm = base.getConnectionManager();
            SchemeRegistry sr = ccm.getSchemeRegistry();
            sr.register(new Scheme("https", ssf, 443));
            return new DefaultHttpClient(ccm, base.getParams());
        } catch (Exception ex) {
            ex.printStackTrace();
            return null;
        }
    }
}

The following are the libraries added to the class path:



I googled and found all of these in the net and added to the class path.

The main() method provides a way to
1. perform GET to retrieve the records using the URL
2. perform POST to create.
3. perform POST to update.

The Java code can be called via a BPEL/Mediator component using Java Embedding ( or more cleaner way by using a Spring component). The code in main method could replace the most of the code in the Java embedding. Most importantly we need to create an XML schema containing the request/response in accordance to the JSON request/response schema of the REST URI.
Eg : In JSON

{
    "sku":"130-01-099",
    "description":"Test"
}

equivalent XML for the same

<?xml version="1.0" encoding="UTF-8" ?>
<root>
   <sku>130-01-099</sku>
   <description>Test</description>
</root>

You could use the following tool for this conversion - http://www.utilities-online.info/xmltojson/#.U9_NevldWAg
You can create an XSD based on the above XML and accordingly use it in the BPEL process to call the Java embedding code/Spring Context component that calls REST service using JSON messaging standard.

Conclusion:  

 While Oracle SOA Suite 12c provides an out of box solution for interacting with REST services with JSON data standard using HTTPAdapter, 11g still need to seek the help of Java code to achieve the same. The way of  creating a Java class that takes XML as input ,converting it to an equivalent JSON and  then calling REST service using JSON ,provides us an interoperability between SOA suite components those primarily still rely on XML messaging with REST services relying on JSON.


Wednesday, April 30, 2014

Understanding the need of Whole Server Migration in Oracle Fusion Middleware

Whole server migration (WSM) is the mechanism built into Oracle WebLogic server that makes it possible to move an entire server instance, along with all of the services it owns, to a different physical machine when triggered by a platform failure event. WSM is probably overkill for most service requirements, because in the vast majority of cases, services can be set up to run on all managed server instances in a cluster, and failover of services from one server to another is transparent and automatic without the need to migrate an entire server instance to another platform. A few critical services, however, with JMS and JTA transactions being the prime examples, have a different relationship with managed server clusters.  These services cannot be pulled out of a failed server instance and restarted in a different operational server instance; the server instance, IP address and all, has to be migrated together with these special services.  Such services normally fall under the purview of SOA managed servers.

Architects and the administrators of the HA design should be aware of the need of WSM based on the needs of the services that are being designed at the client. The following article tries to explain the need of the WSM for various service scenarios and needs. This would probably give some ( if not complete) fair idea on whether to consider setting up the Server migration or not. 

Ability to recover Instances using service engine:

 WSM is not needed ( unnecessary and costly) whenever the instances can be recovered by the service engine ( lets say BPEL). Recovery settings in EM can be set to automatically or manually recover instances at a scheduled interval or at system/server start. Hence Its important to know when an instance can / can not be recovered. In simple an instance can be recovered if it is dehydrated ( persisted in database) before it failed.
 Consider a asynchronous BPEL process 'A' with default transactional and onewaydeliverypolicy ie., "requiresNew" and "async.persist" set. Before the instance is created the message is persisted into the dlv_message table with in the parent transaction if one existed. Because this message is persisted into the database any failure / rollback of the instance would make it still possible to recover.


Ability to "Fail over":
Homogeneous services that are deployed cluster wide will be transparently be able to serve the requests from either of the two nodes in case any service on a node fails or goes down.
Consider a scenario where Service A calls Service B synchronously. Lets say based on round robin approach the Service B of Server 2 is given a chance to serve the request. While the instance is in progress if the service B fails for some reason ( service issues or server shutdown), the transaction is rolled back. Upon retrial it would be assigned the Server1's Service B and hence the transaction is smoothly fulfilled. This is called fail over. In case if the service B is asynchronous you may need to recover it manually from service engine console as explained above.
Server migration is not really needed for these services as they are homogeneous ( not heterogeneous or pinned to a particular server) and hence fail over with some disciplined retry and recovery methods would be sufficient.

Purpose of the Server Migration:
We do not need whole server migration if your service fall in either of above categories ( ability to recover or ability to fail over) However if your process is 

synchronous or Asynchronous  with oneWayDeliveryPolicy is set to 'sync' ( as below)
  <property name="bpel.config.oneWayDeliveryPolicy" type="xs:string"                   many="false">sync</property>
the bpel will participate in the same thread as the caller with out having the message persisted before an instance is created. In these situations a roll back of the instance would leave no trace of the message in the database and hence leaving no possibility to recover them using service engine. If your service is homogeneous a retry of the service ( by using retry option on the partner link or adapter) may kick fail over(ie., kick of the service from other node as explained in prev section). However if your service can not be failed over as the services are pinned to a specific server then the only option to recover them is to bring the server up in the same node or in the other node (server migration). Lets see this practically from an example.

Important Note : If the BPEL is
Asynchronous with oneWayDeliveryPolicy set to 'async.cache' ( as below)
  <property name="bpel.config.oneWayDeliveryPolicy" type="xs:string"                   many="false">async.cache</property>
the message is stored in in-memory cache and it will vanish upon server failure. As the message is not persisted there cant be any recovery to act upon. In this case the new thread and new transaction are just created , not persisted anywhere if the instance is roll backed for any reason , you have neither a chance to recover from BPEL engine nor using Whole server migration. That is because your thread and transaction are rolled back thus losing the message.So be careful when you design the service with async.cache.

Whole Server Migration Example : 
A sample Service Design that leaves no choice (to recover) other than Server migration.


Background :
  • WLS_SOA1 , WLS_SOA2 are two managed servers on machines SOAHOST1,SOAHOST2 clustered ( SOA_Cluster) in a weblogic domain with SOA installed and configured on them.
  • The transaction and jms logs / persistent stores are stored at common location accessible to both these machines.
  • Whole server migration is NOT setup.
  • DemoOutQueue,DemoInQueue are Uniform Distributed Queues. 
  • Service A is an asynchronous service with onewaydleiverypolicy set to 'sync'. It consumes messages placed on the DemoOutQueue and places it onto the DemoInQueue. A delay of approx 30 secs is introduced between consuming and producing using some for loops. It is deployed on the SOA Cluster.
  • Auto recovery on start up and for scheduled intervals are set to off.

Experiments :

Test 1:
Action 1 : 
Five custom messages ( compatible with the jms adapter design) with id's 1,2,3,4,5 were placed on to the DemoOutQueue.
Wait for 15 seconds and shutdown the WLS_SOA2 server.

Observation for Action 1:
Messages 1,3,5 are processed successfully by WLS_SOA1.
Messages 2,4 went to Non Recoverable state as the server WLS_SOA2 processing them was abruptly shutdown.
In few seconds after the WLS_SOA2 is shutdown, the messages 2.4 are picked up by the Service A of WLS_SOA1 and were processed successfully.
All messages 1,2,3,4,5 were placed in DemoInqueue.

Test 2: Repeat the TEST Case 1 again with different message id's for clarity and no confusion.

Action 1 : 
Five custom messages ( compatible with the jms adapter design) with id's 6,7,8,9,10 were placed on to the DemoOutQueue.
Wait for 15 seconds and shutdown the WLS_SOA2 server.
Observation for Action 1:
Messages 6,8,10 are processed successfully by WLS_SOA1.
Messages 7,9 went to Non Recoverable state as the server WLS_SOA2 processing them was abruptly shutdown.
Wait for some time to see if anything happens.
Waited but nothing else happened.
Only messages 6,8,10 were placed in DemoInqueue. 

Action 2: 
Start the WLS_SOA2 server.
Observation for Action 2: 
Messages 7  was picked up by the Service A on WLS_SOA1 and Message 9 was picked up by the Service A on WLS_SOA2 and were processed successfully by placing them on to the DemoInQueue.

Note : The results of Test 2 may be just like Test 1's shown above. You may need to run for a good number of times to get the result of Test 2 as shown above. vice versa..ie., The results of Test 1 above may be like Test 2 as shown above and you may need to run considerable amount of times to get what we got above. But if your web logic cluster is round-robin load balanced you are most probably going to see the results in just 2 runs.

Quick understanding of the UDQ behaviour in cluster before we jump on to the analysis:

Uniform distributed queue is a queue distributed across the different servers cluster wide ie., for an client (internal or external) the queue appears as one lets say DemoOutQueue. However internally the DemoOutQueue UDQ is/are two queues each targeted to JMS servers on WLS_SOA1 and WLS_SOA2,
lets say DemoOutQueue1 ,DemoOutQueue 2 respectively for simplicity.You can see this in Admin console -> JMS Modules - Distributed Queue -> Monitoring

When a message/ set of messages are placed on a UDQ ( here DemoOutQueue) , the messages are placed either on DemoOutQueue1 or DemoOutQueue2 but not on both for Data consistency. This is called the singleton or pinned behaviour of the JMS.

Analysis :

Test 1 - Observation 1 - Reasoning :  Messages 1,2,3,4,5 when placed on DemoOutQueue, the cluster load balancing chose DemoOutQueue1(ie., DemoOutQueue hosted on JMSServer of WLS_SOA1) and placed all the messages on DemoOutQueue1.
Service A of both WLS_SOA1 and WLS_SOA2 servers( lets call ServiceA1,ServiceA2 for simplicity) picked the messages in the round robin fashion ie., ServiceA1 picked 1,3,5 and ServiceA2 picked 2,4 and created instances respectively.When the WLS_SOA2 is shutdown, the Service A2 instances for messages 2,4 were rolled back ( as they were not yet finished with processing) and as the Service A is sync, the messages rolledback to the original queue from where they have come from. In this case it is DemoOutQueue1.However as still WLS_SOA1 is up and running the Service A on WLS_SOA1 ie., ServiceA1 picks these new messages 2,4 and processes them. Hence we see all the messages were successful.

Test 2 - Observations 1,2 - Reasoning : Messages 6,7,8,9,10 this time were placed on DemoOutQueue2 ( ie, DemoOutQueue of JMS Server targeted to WLS_SOA2). ServiceA1 picked 6,8,10 messages. Service A2 picked 7,9 messages and instances were created respectively. However as WLS_SOA2 was shutdown the instances being processed by ServiceA2 ie., for messaged 7 and 9 were abruptly rolledback to the DemoOutQueue2. But now as the WLS_SOA2 is shutdown any consumers of DemoOutQueue2 will not get any messages as DemoOutqueue2 is literally not there ( down because of its server shutdown) for giving the messages. Hence nothing happens.
When the Server WLS_SOA2 comes back the messages 7,9 are picked by ServiceA1,ServiceA2 respectively in round robin fashion and are processed successfully placing the messages on to the DemoInQueue.

Quick Summary : As you can now understand, if there are any messaged placed on to the queue that was targeted to a server that went down, those messaged are not available to be failed over. Only solution is to get the queue up. The only possible approach for this is either get the Server up or get it migrated in case of multiple unsuccessful retrials on a particular node. 

Conclusion : 
Whole server migration involves a tightly-orchestrated sequence of events with participation from WebLogic, Coherence, O/S, network, and database components gets triggered if a SOA server goes down in a Fusion Applications infrastructure. This is costly , time consuming and also needs an extensive amount of hardware sizing and capacity planning to host the migrated server. One needs to really see the possibility of making things work with out WSM by designing services to fall under recovery of service engines or fail over scenarios. However if you extensively need to use JMS distributed destinations and XA transactions spanning multiple resources ( multiple databases) on a two phase commit then WSM is the way to go.

Note : Service Migration is another possible approach but it is not yet supported with Oracle BPEL/OSB. Refer articles NOTE:1306533.1 and NOTE:1407715.1.For service level migration support on SOA, there is a enhancement requests (SOA 11g: BUG:13447082) raised and for OSB (OSB 11g: BUG:13446665) as well. There are still under development and will be included in future releases.

This is still only one use case explaining the need for the Whole Server Migration where JMS services are pinned to server. In my next blog I shall talk more about JTA recovery service use cases.

Disclaimer: The above experiments are based on my own observations and setup of the environment on my local machines. A possibility of a different behaviour may be possible in a real production infrastructure, although highly unlikely to differ from this.

References:
http://docs.oracle.com/cd/E14571_01/web.1111/e13709/migration.htm
http://www.ateam-oracle.com/floating-ip-addresses-and-whole-server-migration-in-fusion-applications/#comment-98
https://community.oracle.com/thread/3527201
Oracle EDG
https://blogs.oracle.com/soabpm/entry/soa_suite_11g_-_transactions_b_1

Wednesday, March 12, 2014

Questions one should have answers for while setting High Availability in Oracle SOA

The customers who are looking to embark on Oracle SOA for the first time especially on a high available mode are mostly daunted looking at various deployment topologies and administrative tasks involved with in. While the Oracle Enterprise Deployment Guide ( EDG) is very popular and is used as a reference to setup the high availability there would be still a lot of questions lingering around like why certain things are being setup or setup in a particular way. In fact, sometimes,its not clear why we are doing somethings in the entire process. This post attempts to help resolve those vague concepts.

Question : What is the use of external load balancer(software or hardware) when we plan to implement a web logic cluster that does load balancing anyway.?

Answer :  External Load balancers provide an additional layer of security and more reliability of load balancing as they are not affected by the internal systems performances but are used only to load balance across Http servers or some cases ( not suggested) application servers using well defined algortihms. The need for load balancing at cluster level is for clustered object stubs when no algorithm is specified at the object level. The algorithm at weblogic cluster level supports load balancing for RMI objects and EJBs.

Question : Can Managed servers continue to run in case admin server comes down ?
Answer : Yes of course they do. They try to contact admin server at a particular frequency that is setup int he configurations, but then will continue on their own. Another point to note is that managed servers can also start by themselves ( with help of node manager) with out Admin server being up. This is called as MSI mode or Managed Server Independence mode.The problem with this mode is that managed servers can not get the latest update on the configurations done in the domain as it runs with its own copy of the configuration which is collected upon its start when admin was up and running.

Question : What is whole server migration ? Why do I need it when my cluster is performing a "fail over" of various services when one of its members is down ?

Answer : In a WebLogic Server cluster, most services are deployed homogeneously on all server instances in the cluster, enabling transparent failover from one server to another. In contrast, "pinned services" such as JMS and the JTA transaction recovery system are targeted at individual server instances within a cluster—for these services, WebLogic Server supports failure recovery with migration, as opposed to failover.

Question : Why does JMS and JTA are targeted to single server ?

Answer : Weblogic services are divided into two categories , one Clustered Services and two Singleton Services. The JMS,JTA services come under the Singleton services that can be targeted only to a single instance in a cluster and more importantly will be managed by a single persistent store ( Note : single persistent store is the key).
The reason for this setup is that it offers high QOS by ensuring message consistency and reliability. For example a JMS message on a topic or a queue should not be stored in multiple places for data consistency. Also the information related to the message like transactions, acknowledgement etc should not be shared across.

Question : Is Whole Server migration the only way to ensure JMS and JTA "Singleton" services are recovered in case of failures?

Answer : No. We can do service migration too. ie., migration of individual JMS and JTA services to other server. The problem is that only JTA service migration supports the migration to other server using the default persistent store. Other services such as JMS needs to use custom persistent stores for them to be able to migrate to other servers.

Question : Would the problem be same for any other Singleton services in my server?
Answer : Yes. The singleton services as the name indicates, means that they are meant to be targeted to a single server instance and will use one persistent store allocated for those services in that server instance.

Question : What is the impact if Server/Service migration is not setup ?
Answer  :  Your instances that was running crashes and it may be retried based on various parameters. If its an asynchronous instance that is dehydrated by the time it crashed, the server may try to recover it. Manual recovery sometimes is still the option in certain cases.

This still sounds theoretical although it clears a cloud of vagueness regarding some important concepts of HA architecture. Understanding of the above in more practical way with examples will be done my upcoming blog.