12c SOA - Oracle Mediator part10 - Java Callout

• Java Callout is used when we need to have certain validation or want to manipulate the message. We need to implement the oracle.tip.mediator.common.api.IjavaCallout interface to use below defined function for Java Callout.
• The Java Callout is invoked by the Mediator on a number of times, prior to and after performing the Routing Rule and each of the cases in it. The Java Callout class can implement a number of methods, one for each specific event or stage in the Mediator process. These methods get access to the input message as well as the transformation result. That means that the callout class can inspect, validate, log, audit and even manipulate these messages, their payloads, headers and properties.

We have below methods in the oracle.tip.mediator.common.api.IjavaCallout interface.

Initialize : This method is invoked when the callout implementation class is instantiated for the first time.
preRouting : This method is called before Mediator starts executing the cases. You can customize this method to include validations and enhancements.
preRoutingRule : This method is called before Mediator starts executing any particular case. You can customize this method to include case-specific validations and enhancements.
preCallbackRouting: This method is called before Mediator finishes executing callback handling. You can customize this method to perform callback auditing and custom fault tracking.
postRouting : This method is called after Mediator finishes executing the cases. You can customize this method to perform response auditing and custom fault tracking.
postRoutingRule: This method is called after Mediator starts executing the cases. You can customize this method to perform response auditing and custom fault tracking.
postCallbackRouting: This method is called after Mediator finishes executing callback handling. You can customize this method to perform callback auditing and custom fault tracking.

UseCase 1: Using a postRouting method, the javacallout will log the response in the soa_server.log/out file.

Implementation:

Create a SOA project and open a mediator component

Select Synchronous interface.

Call a synchronous service

Create a Java class

Add oracle.tip.mediator.common.api.IjavaCallout interface

Modify the postRouting method

    public boolean postRouting(CalloutMediatorMessage calloutMediatorMessage,

                               CalloutMediatorMessage calloutMediatorMessage2,

                               Throwable throwable) throws MediatorCalloutException {

        System.out.println("Mediator has invoked Java Callout; executing postRouting");

        String sPayload = "null";

        for (Iterator msgIt = calloutMediatorMessage2.getPayload().entrySet().iterator();msgIt.hasNext(); ) {

          Map.Entry msgEntry = (Map.Entry)msgIt.next();

          Object msgKey = msgEntry.getKey();

          if (msgKey.equals("reply")) {

              Object msgValue = msgEntry.getValue();

              sPayload = XmlUtils.convertDomNodeToString((Node) msgValue);

              System.out.println("Mediator sends reply: " + sPayload);

          }

        } //for

        return true;

    }

Select the created class from call out section

Map the request and response

Deploy and test

Sample class for modifying the input request:
public boolean preRouting(CalloutMediatorMessage calloutMediatorMessage) throws MediatorCalloutException {
        System.out.println("Pre routing...");
        String sPayload = "null";
        String sPayload_changed = "null";
        for (Iterator msgIt = calloutMediatorMessage.getPayload().entrySet().iterator();          msgIt.hasNext(); ) {
            Map.Entry msgEntry = (Map.Entry)msgIt.next();
            Object msgKey = msgEntry.getKey();
            if (msgKey.equals("request")) {
                Object msgValue = msgEntry.getValue();
                sPayload = XmlUtils.convertDomNodeToString((Node)msgValue);
                try {
                    XMLDocument xmlpayload = XmlUtils.getXmlDocument(sPayload);
                    NodeList node = xmlpayload.getChildNodes();
                    Node item = node.item(0);
                    System.out.println("the value of the request element = " +
                                       item.getTextContent());
                    sPayload_changed = sPayload.replaceAll(item.getTextContent(),
                                                "changed text");
                    XMLDocument xmlpayload_changed =
                        XmlUtils.getXmlDocument(sPayload_changed);
                    String mykey = "request";
                    calloutMediatorMessage.addPayload(mykey,
                                                      xmlpayload_changed);
                    System.out.println("Mediator sends request: " +
                                       sPayload_changed);
                } catch (Exception e) {
                    System.out.println(e);
                }
            }
        }
        return true;
    }

12c SOA - Oracle Mediator part9 - Write a XML file using file adapter

Here i will show how to write a XML file to a local directory.

First create a Schema.

 Open a composite project and select mediator and used the schema.

 Drag and drop the File adapter

 setup the configurations.

 Here I mentioned .txt, you can also make it .xml and other extension needed.

 choose same schema or can use different schema

 Open the mediator and do the transformation

Deploy and Test.

12c SOA - Oracle Mediator part8 - Schematron or Payload validation

Schematron Basics:

• Schematron is an XML schema language, and it can be used to validate XML contents in an XML payload. Schematron defines a set of rules and checks that are applied to an XML instance. 
• Schematron relies almost entirely on XPath query patterns for defining these rules and checks. 
• The schematron validation allows loose coupling of Semantic validation of the incoming/outgoing mediator payload.This is accommodated while defining the routing rule for mediator payloads.
• The Schematron file is created with a file type having an .sch extension.

Schematron Structure:

• A Schematron XML document consist of a schema element in the Schematron namespace: http://www.ascc.net/xml/schematron. 
• The schema element contains one or more pattern elements. Pattern elements allow the user to group schema constraints logically. Some examples of logical groupings are: Text Only Elements, Valid Root Element, Check for ID Attribute. Pattern elements have a name attribute. 
• Rule elements define a collection of constraints on a particular context in a document instance (for example, on an element or collection of elements). This is very similar to XSLT templates, which are fired with respect to a node or group of nodes returned by an XPath expression.
• The Schematron standard defines two types of rules: assert and report. The assert rule returns your error message when the test condition fails (returns false), whereas the report rule returns your error message when the test conditions succeeds (returns true). The Schematron implementation in the SOA Suite only supports the assert rule, so you have to rewrite your report rules. This is not much of a problem, because the assert rule is just the logical complement (inversion) of the report rule, so the logical “not” function solves this issue. In case of just a logical comparison in the test condition, you also can invert it by replacing a = operator, etc.

The namespaces of our input payload (“http://www.emp.org”) have to be declared. The solution is to use the “ns” element with attributes “uri” and “prefix”:

<ns uri="http://www.emp.org" prefix="emp" />

XSD used:

<?xml version="1.0" encoding="windows-1252" ?>

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.emp.org" targetNamespace="http://www.emp.org"

            elementFormDefault="qualified">

  <xsd:element name="Employees">

    <xsd:complexType>

      <xsd:sequence>

        <xsd:element name="Employee" maxOccurs="unbounded">

          <xsd:complexType>

            <xsd:sequence>

              <xsd:element name="Name" type="xsd:string"/>

              <xsd:element name="Id" type="xsd:integer"/>

              <xsd:element name="Salary" type="xsd:integer"/>

              <xsd:element name="Type" type="xsd:string"/>

              <xsd:element name="ManagerName" type="xsd:string"/>

              <xsd:element name="ManagerId" type="xsd:integer"/>

            </xsd:sequence>

          </xsd:complexType>

        </xsd:element>

      </xsd:sequence>

    </xsd:complexType>

  </xsd:element>

</xsd:schema>

 Schematron created:
<?xml version="1.0" encoding="UTF-8" ?>
<schema xmlns="http://www.ascc.net/xml/schematron">
    <ns uri="http://www.emp.org" prefix="emp" />
    <pattern name="Employee may not be the manager of himself">
        <rule context="/emp:Employees/emp:Employee">
            <assert test="emp:Id != emp:ManagerId">Employee may not be the manager of himself</assert>
        </rule>
    </pattern>
    <pattern name="Validate Employee Type">
        <rule context="/emp:Employees/emp:Employee">
            <assert test="emp:Type = 'PERM'">Not a permanent employee</assert>
        </rule>
    </pattern>
</schema>

Rule1: if Emp id != manager id condition fails then it throws the error "Employee may not be the manager of himself"

Rule2: if EMp type = 'PERM' fails then it throws the error "Not a permanent employee".

Double Click on the Mediator component. Check the ‘Validate Syntax (XSD)’ option. Click on the image icon at ‘Validate Semantic’ and use the schematron.sch file for Schematron validation.

Deploy and test.
Tested with keeping the following:
-Employee1 is TMP type
-EMployee2 is sharing same id with manager.

Error details:

The selected operation execute could not be invoked.

A fault occurred while invoking the webservice operation. The fault is : <env:Fault xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<faultcode>env:Server</faultcode>
<faultstring>oracle.tip.mediator.infra.exception.MediatorException: ORAMED-01301:[Payload custom validation]Schematron validation fails with error "<ns1:ValidationErrors xmlns:emp="http://www.emp.org" xmlns:ns1="http://xmlns.oracle.com/pcbpel/validationservice">
<error>Employee may not be the manager of himself</error>
<error>Not a permanent employee</error>
</ns1:ValidationErrors>
".Possible Fix:Fix the payload.</faultstring>
<faultactor/>
<detail>
<exception/>
</detail>
</env:Fault>

oracle.sysman.emInternalSDK.webservices.util.SoapTestException: Client received SOAP Fault from server : oracle.tip.mediator.infra.exception.MediatorException: ORAMED-01301:[Payload custom validation]Schematron validation fails with error "
Employee may not be the manager of himself
Not a permanent employee

12c SOA - Oracle Mediator part7 - Sequential vs parallel routing rules

A routing rule can be executed either in parallel or sequentially. To specify an execution type for a routing rule, select the Sequential or Parallel execution type in the Routing Rules section.

Basic Principles of Sequential Routing Rules:

• Mediator evaluates routings and performs the resulting actions sequentially. Sequential routings are evaluated in the same thread and transaction as the caller.
• Mediator always enlists itself into the global transaction propagated through the thread that is processing the incoming message. For example, if an inbound JCA adapter invokes a Mediator, the Mediator enlists itself with the transaction that the JCA adapter has initiated.
• Mediator propagates the transaction through the same thread as the target components while executing the sequential routing rules.
• Mediator never commits or rolls back transactions propagated by external entities.
• Mediator manages the transaction only if the thread-invoking Mediator does not already have an active transaction. For example, if Mediator is invoked from inbound SOAP services, Mediator starts a transaction and commits or rolls back the transaction depending on success and failure.

Basic Principles of Parallel Routing Rules:

• Mediator queues and evaluates routings in parallel in different threads.
• The messages of each Mediator service component are retrieved in a weighted, round-robin fashion to ensure that all Mediator service components receive parallel processing cycles. This is true even if one or more Mediator service components produce a higher number of messages compared to other components. The weight used is the message priority set when designing a Mediator service component. Higher numbers of parallel processing cycles are allocated to the components that have higher message priority.

How the Parallel Routing rule works:

By using parallel routing rules, services can be designed to be truly asynchronous. However, the service engine executes these requests in a rather unique fashion.

Let's say you have 3 mediator services deployed to your SOA server and each of these has a single parallel routing rule. When the mediator service received a message,

• The message would be inserted in Mediator store by the dispatcher.
• Its message metadata would be written to the MEDIATOR_DEFERRED_MESSAGE table.
• The payload goes into MEDIATOR_PAYLOAD table. 

All these would occur on the original calling thread.

The mediator service engine has one thread called Locker thread. The locker thread would surface the message metadata from the MEDIATOR_DEFERRED_MESSAGE table into an internal memory queue. The locker thread does this in its own transaction. The MEDIATOR_DEFERRED_MESSAGE table also has a state listed below:
0 – READY
1 - LOCKED
2 - COMPLETED SUCCESSFULLY
3 - FAULTED

Hence, it is important to understand how the locker thread works behind the scene, as this would affect your design decision:

The locker thread has an algorithm to process the message stored by the dispatcher, and there is only 1 locker thread per managed server. After the dispatcher has stored the message data in the MEDIATOR_DEFERRED_MESSAGE table, the locker thread will lock message(s) that has a state=“0”, the number of messages to be locked by the locker thread will be dependent on the Parallel Maximum Rows Retrieved parameter that you have set in the EM->SOA-INFRA->SOA Administration->Mediator Properties. The locker thread will cycle through 1 mediator component at a time and check if there are any requests to process from the internal queue. It will process the message(s) by changing its state to “1” for that component for processing and then sleep for the configured interval defined in the "parallel locker thread sleep" setting before moving on to the next mediator component. If it finds no message to process, it would move on to the next, and the next, until it loops back to the first composite, where it would then process its next request.

For example: If there are 3 mediator components m1, m2, m3 with priority 1, 2, 3 then the algorithm goes as lock m1 -> sleep -> m2 -> sleep -> m3 -> sleep -> m2 -> sleep -> m3 -> sleep -> m3, in 6 iterations of locker thread, m1 messages are locked once, m2 messages are locked twice and m3 messages are locked 3 times as per the priority. All this happens in a single Locker cycle. Only after the locker thread locks the retrieved messages will they be queued for the worker threads to process the messages. So if you have many mediator components (e.g. 50 mediator components) with parallel routing rule, it will take a considerable amount of time for the locker thread to lock the message to complete one locker cycle. If you have mediator components with lower priority, it will take a longer time for the locker thread to lock the message for the low priority mediator component. The locker cycle will be reset if you undeploy or deploy a new mediator component with parallel routing rules, this is to ensure the mediator component with higher priority will be processed in the next cycle.  You will be able to observe these behaviors when you set the logging level to the Trace:32 FINEST level in Oracle Enterprise Manager Fusion Middleware Control.

oracle.soa.mediator.common

oracle.soa.mediator.service

oracle.soa.mediator.dispatch

oracle.soa.mediator.serviceEngine


After the locker thread locked the message, the worker thread will retrieve the message from the in-memory queue and process the message. The number of worker threads can be tuned by changing the Parallel Worker Threads property in EM->SOA-INFRA->SOA Administration->Mediator Properties. Once the message is processed, the worker thread will change the state of the message to either “2” –Completed successfully or “3” – Faulted.

The engine was designed in order to prevent starving of threads caused by load on a single composite. What the engine wants to avoid is that, if you have a Mediator service that has received hundreds of thousands of requests and another one having received two requests, each service is given a fair amount of time to be serviced, otherwise the two requests may have to wait for hours to execute. Thus, the three settings to consider in asynchronous Mediator services are the following:

The Parallel Locker Thread Sleep setting: This is defined at the Mediator Service Engine level
The number of threads allocated to the Mediator Service Engine: This is defined by the Parallel Worker Threads parameter

The Priority property: Which is set at design time and applicable only to parallel routing rules

Another important point to note is when a mediator service engine is started, it registers itself in the database table called MEDIATOR_CONTAINERID_LEASE and gets a container ID. This is important because when the row is inserted into the MEDIATOR_DEFERRED_MESSAGE table, it round-robins the deferred message to one of its containers, the engine will then assigns the ID of the container that should process the message.

 List of design considerations designing your composite using mediator parallel routing rules:

• The priority property is only applicable to the parallel routing rules, so you need to consider the mediator component priority base on your business requirement.
• The locker thread will cycle through all mediator components with parallel routing rules deployed in your environment regardless of whether it has been retired or shutdown.

• Use sequential routing rules if latency is important and that you are expecting the message to be processed without delay.
• If you have well over 100 parallel mediator components deployed in your environment, the time to complete the locker cycle grew exponentially and could not be further tuned because there is only 1 locker thread and the lowest parallel locker thread sleep time that you can set is 1 second.

• If you have a mediator component that contain both sequential and parallel routing rules, sequential routing rules will be executed before parallel routing rules.
• Fault policy is only applicable to parallel routing rules only. For sequential routing rules, the fault goes back to the caller and it is the responsibility of the caller to handle the fault. If the caller is an adapter, then you can define rejection handlers on the inbound adapter to take care of the errored out messages, that is, the rejected messages.