Do you have automated processes or workflows in your company. Then this is for you

SAP is pleased to offer a new and focused platform to listen to customer needs and apply this feedback to the software development of Goal-Driven Smart Process Apps. This is a new set of domain-specific applications that are both intelligent and pro-active dealing with both standardized and unforeseen business situations to reach specific business goals. They will be built on the capabilities of SAP Operational Process Intelligence on HANA.

As with previous Customer Engagement Initiatives, this enables customers to influence the software development based on their specific needs and show customers how this feedback is incorporated into the software development as it happens.

Seats are limited, and registration closes on the 9th of April so if you know of a customer who may be interested, please ask them to sign up at this site as soon as possible.

Here's where you can register!

Hoping to see both new and familiar faces,

Alan

Adapter Module: AdapterModuleDCBypass

Use

ReplaceString2.0 module is used to replace strings within the xml text of the payload of the message. It can be used to modify the message structure before it gets in PI and also before it gets out. It is useful when there are strings like namespaces, types, etc that PI can’t “understand” or that PI is unable to include in outbound messages.

The module obtains a pair of string : <OldString>;<NewString> as parameter. Reads the payload of the message and converts it to a string. Then it searches <OldString> within the message and replace each occurrence with <NewString>. Finally it returns the message modified. If OldString can’t be found in the message nothing is modified and the original message is returned.

The module accepts multiple string pairs to allow multiple replacements in a single execution.

Deployment

Enterprise Java Bean Project: ReplaceString-ejb

Enterprise Java Bean Application: ReplaceString-ear

Integration

The module can be used in any kind of Sender and Receiver adapters.

Activities

This section describes all the activities that have to be carried out in order to configure the module.

Entries in processing sequence

Insert the module in front of the adapter module as shown in the picture below.

Entries in the module configuration

The table below shows the possible parameters and values of the adapter module.

(*) The parameter name can be any string: param1, parameter, aaa, etc.

Example

Payload data before execution:

<MT_ROOT>     <RECORDSET>          <DATA>               <FIELD1>value</FIELD1>               <FIELD2>value</FIELD2>               <FIELD3>value</FIELD3>          </DATA>          <DATA>               <FIELD1>value</FIELD1>               <FIELD2>value</FIELD2>               <FIELD3>value</FIELD3>          </DATA>     </RECORDSET></MT_ROOT>

Module parameters:

separator = ‘;’

param1 = ’RECORDSET;ROW’

param2 = <MT_ROOT>;<MESSAGE type=TYPE>

param3 = </MT_ROOT>;</MESSAGE>

param4 = <DATA>;emptyString

param5 = </DATA>;emptyString

Payload data after execution:

< MESSAGE type=TYPE>     <ROW>          <FIELD1>value</FIELD1>          <FIELD2>value</FIELD2>          <FIELD3>value</FIELD3>          <FIELD1>value</FIELD1>          <FIELD2>value</FIELD2>          <FIELD3>value</FIELD3>     </ROW></MESSAGE>

ReplaceStringBean

/**
*
*/
package com.arms.integrations;
import java.rmi.RemoteException;
import javax.ejb.EJBException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import com.sap.aii.af.lib.mp.module.Module;
import com.sap.aii.af.lib.mp.module.ModuleContext;
import com.sap.aii.af.lib.mp.module.ModuleData;
import com.sap.aii.af.lib.mp.module.ModuleException;
import com.sap.engine.interfaces.messaging.api.Message;
import com.sap.aii.af.service.auditlog.Audit;
import com.sap.engine.interfaces.messaging.api.MessageKey;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditLogStatus;
import java.io.*;
import java.util.*;
/**
* @author Roger Allué i Vall
*
*/
public class ReplaceStringBean implements SessionBean, Module {    private SessionContext myContext;    private static final String C_SEPARATOR_STRING = "separator";    private static final String C_AUX_SEPARATOR = "|";    private static final String C_MODULEKEY_STRING = "module.key";    private static final String C_BLANK_STRING = "blankString";    private static final String C_EMPTY_STRING = "emptyString";    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbActivate()     */    public void ejbActivate() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbPassivate()     */    public void ejbPassivate() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbRemove()     */    public void ejbRemove() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#setSessionContext(javax.ejb.SessionContext)     */    public void setSessionContext(SessionContext arg0) throws EJBException,            RemoteException {        // TODO Auto-generated method stub        myContext = arg0;    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#afterBegin()     */    public void afterBegin() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#afterCompletion(boolean)     */    public void afterCompletion(boolean arg0) throws EJBException,            RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#beforeCompletion()     */    public void beforeCompletion() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    public void ejbCreate() throws javax.ejb.CreateException {         }    public ModuleData process(ModuleContext moduleContext, ModuleData inputModuleData)    throws ModuleException {             InputStream payloadIn = null;        ByteArrayOutputStream payloadOut = null;        int inputByte = 0;        String payloadStr = "";        Message msg;        Enumeration paramList;        String sep, paramKey,param, strArray[];                  try {             msg = (Message) inputModuleData.getPrincipalData();             MessageKey amk = new MessageKey(msg.getMessageId(), msg.getMessageDirection());             payloadIn = new ByteArrayInputStream(msg.getDocument().getContent());                       Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,".-Module beginning");                  while((inputByte = payloadIn.read()) != -1) {                 payloadStr = payloadStr + (char) inputByte;             }                      Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Payload before execution:" );            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,payloadStr );                 sep = moduleContext.getContextData(C_SEPARATOR_STRING);                  if ( sep == null ) {                sep = C_AUX_SEPARATOR;                Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Default separator used: " + sep);             }             else {                Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Separator found: " + sep);             }                                          paramList = moduleContext.getContextDataKeys();               while( paramList.hasMoreElements()) {                    paramKey = (String) paramList.nextElement();                 //Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"ParamKey: " + paramKey);                 param =  moduleContext.getContextData(paramKey);                 //Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Param: " + param);                               if ( ! paramKey.equals(C_SEPARATOR_STRING) && ! paramKey.equals(C_MODULEKEY_STRING) ){                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"ParamKey: " + paramKey);                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Parameter: " + param);                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Separator: " + sep);                                     strArray = param.split(sep);                    if (strArray != null){                        if ((! strArray[0].equals(null)) && (! strArray[1].equals(null))){                        if ( strArray[1].equals(C_BLANK_STRING)){                            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Blank String");                            strArray[1]=" ";                                                 }                        else if (strArray[1].equals(C_EMPTY_STRING)){                            strArray[1]="";                            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Empty String");                        }                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution strArray[0]: " + strArray[0]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution strArray[1]: " + strArray[1]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution payloadStrA : " + payloadStr);                                 payloadStr = payloadStr.replaceAll(strArray[0],strArray[1]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution payloadStrB : " + payloadStr);                       }                    }                 }             }                  Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Payload after replacement:" );             Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,payloadStr );                      payloadOut = new ByteArrayOutputStream();             payloadOut.write(payloadStr.getBytes());                  msg.getDocument().setContent(payloadOut.toByteArray());             inputModuleData.setPrincipalData(msg);             }        catch(Exception e) {            ModuleException me = new ModuleException(e);            throw me;        }           return inputModuleData;    }
}

Hello colleagues!

I saw a bunch of good articles on this topic, for example:

Generic PI Async-Sync Bridge Configuration for Any Adapters by William Li

But we had some interesting conversations on our local russian xi/pi forum recently that shows me some gaps in the information about bridges.

This is translation from Russian, this text was initially created for my web-site. Sorry for copyrighted pictures.

Source: http://sap.pitroff.ru/navodim-mostyi-ili-kak-svyazat-mezhdu-soboy-asinhronnyiy-i-sinhronnyiy-interfeysyi-bez-pomoshhi-ccbpm/

-------------------

Contents:

Sometimes SAP Process Integration development gives you a challenge. For example, you want to receive a file from a system (asynchronous mode), convert it to an RFC-call for SAP ERP ( synchronous mode ), and upload the response as a file to an FTP server ( asynchronous mode) .

Or another task: your ABAP program needs to request additional information from the external system (synchronous mode). The problem is that external system using web-services in asynchronous mode only, so you need to call external system and provide some web-service to get answer. More than that – you need to transfer this response to the initial ABAP program, waiting for the answer.

These problems can be summarized as follows :

Figure 1 : Asynchronous - synchronous bridge

Figure 2 : Synchronous - Asynchronous Bridge

There are two solutions possible for this kind of problems:

• Use ccBPM. To do this you need to create the integration process in SAP PI and use it to build the bridge elements .
  - the bridge can be used in a complex process with multiple systems.
  - development and testing time, the complexity of monitoring and debugging , ccBPM is not available in Single Stack installations.

I recommend this way only if you want to use a bridge as a part of complex ccBPM processes.
You can read more about ccBPM async-sync and sync-async bridges on help.sap.com: «Example: Sync / Async Communication».

• Use standard adapter modules.

  - speed of development is comparable with ordinary interface uses standard design elements;  simple debugging and monitoring.

  - you need to be careful filling module parameters; it is difficult for beginners;  source code of modules is not available;  confusing ( and sometimes contradictory ) information on the Internet .

Let's take a closer look to the second method for constructing bridges.

Let's start from the asynchronous-synchronous bridge, this is the case when the sender system is operating in asynchronous mode, i.e. sending a request and waiting for a response are handled by two different (independent) tasks. Target system synchronizes request and response, and sends a response to the same channel which the request came from (Fig.1).

The bridge using two standard modules for using in communication channels:

• RequestResponseBean is responsible for converting an asynchronous message to the synchronous and for transferring it to the next module in the adapter modules chain or to send it to the messaging subsystem of PI.

• ResponseOnewayBean– the module, which converts a synchronous response message to the asynchronous and redirects it to the receiver.

The modules can be used in the sender communication channel or in the receiver communication channel.

Additional Information: adapter modules are executed one after another in the stacking order (top-down) on the «Module» tab of the Communication Channel settings.  Output of each module is the input for the next one. The list of modules usually includes standard adapter module - CallSAPAdapter.

Module parameters can be configured on the «Module» tab too - in the «Module Configuration» section.

For using modules in the Communication Channel you should go to the tab «Module» and enter the following values:

«Module Key» can be anything - so long as it is unique and the keys would be difficult to confuse - it will help us later to determine the parameters of modules.

Further settings depends on the location of our modules: sender communication channel or receiver communication channel.

If we use Sender Communication Channel for bridge's modules, the scheme of message processing would be next:

Figure 3 : The logic of the asynchronous-synchronous bridge with modules in the Sender CC.

1. Receiving an asynchronous message from external system via communication channel, sending it to the module RequestResponseBean. The module changes the message type from asynchronous to synchronous by changing the message header.
2. Depending on the parameter passThrough, module pass the message down the chain of modules or directly sending it to the message processing subsystem of PI (skipping step 3).
3. Message processing by the standard adapter module, transfering it to PI message processing subsystem.
4. Sending a request via synchronous communication channel.
5. Synchronous call to the external system, obtaining the response.
6. Receiving a response from the synchronous communication channel.
7. Return the response to the module ResponseOnewayBean, changing the message type from synchronous to asynchronous. Detereming the receiver system for the response (using module parameters).
8. Sending response to the selected channel.
9. Delivery of the message to the external system via asynchronous communication channel.

Module parameters for RequestResponseBean when placing it in the Sender Communication Channel:

    Please remember that all parameters in Java are context-dependent and there is no verification of parameters names/values during saving and activation of Communication Channel!

    Errors in spelling and register of letters will only emerge during testing, and sometimes it's very difficult to understand the cause of an error.

Parameters of ResponseOnewayBean module (if it defined in a Sender Communication Channel):

* - Theoretically , you can skip parameters at all.  Will only need to create an "artificial » receiver agreement, which will be given:

system sender = synchronous system

reciever = target asynchronous system

interface = original request interface

I named it "artificial " because PI will not let you choose such set of parameters in agreement - you will have to put these parameters  in the fields of an object manually (by copy-paste). It is this " mess " is formed in the header of message returned from a synchronous system as a result of the bridge.

If adapter types of the initial asynchronous system and the final destination are not the same - you will still have to configure adapterType and adapterNamespace in the communication channel.

In practice - accurate definition of all necessary parameters would do the job better.

You can also handle application error messages (optionally):

Once we have established all necessary modules in the Sender Communication Channel, we should only create routing rules that bind the initial asynchronous system with the synchronous system.

This can be done in two ways:

• create a bunch Receiver Determination -> Interface Determination -> Receiver Agreement

• create Integrated Configuration

Routing of the response from a synchronous system is automatic; bridge module will send a message to the endpoint.

Ok, if you  have broken through a lot of words above - let's try to do it in the real world.

The schema of the bridge:

Figure 4 : Schematic of the asynchronous-synchronous example bridge  -  SOAP-RFC-File.

The initial system requests additional information about the person / partner on some ID. Technically, the system using an asynchronous web-service.

Requested information is stored in the ERP ( in this particular example - it is ABAP-part of PI installation). Information request could be done via synchronous RFC call to the Z-function module.

The answer must be saved to the file system (in this case - it's a folder on the PI server) in XML format, where ("according to the legend" ), it will be taken by an external system's agent.

To implement this interface , we need following development objects:

Figure 5 : Objects in the Integration Repository Development

All objects are created as usually(see Fig. 6-8) :

Figure 6 : Asynchronous - request

Figure 7 : Asynchronous - Reply

Figure 8 : RFC-module, imported from ABAP. Simple Z-Code to get some info from Z-table.

The only exception is the Operation Mapping:

Figure 9: Syncronous Operation Mapping between the two async interfaces and synchronous RFC

The problem is that the system will not give you possibility to create a two-way synchronous mapping, as the source interface SI_ExtSystem_Request_Async is asynchronous . But we know that the bridge will "create" visibility of synchronous operation

and the mapping will be carried out in both directions during runtime.

But (thanks to William Li who realised this possibility) we can create this mapping using a small trick:

1. Create Message Mapping MM_ExtSystem_Request_to_RFC for SI_ExtSystem_Request_Async and ZZ_EXT_REQUEST interfaces.
2. Create Message Mapping MM_RFC_to_ExtSystem_Response for ZZ_EXT_REQUEST.Response and SI_ExtSystem_Request_Async
3. THE TRICK: edit the source interface SI_ExtSystem_Request_Async: change the type to synchronous, add the message type from the interface SI_ExtSystem_Response as a response message type. Save, but do not activate.
4. Create Operation Mapping, source interface - SI_ExtSystem_Request_Async ( it's synchronous now), the target interface would be RFC module. Use Message Mappings from 1 and 2 as request and response mappings respectively. Save and activate mapping.
5. Reject the interface changes ( return it to its previous state - asynchronous).
6. Activate all changes.

Using this trick, we have got the mapping OM_ExtSystem_Request_to_RFC, it will perform all transformations in our interface.

Now we need to configure our interface - run the Integration Builder and create following objects:

        Fig.10: Configuration objects

Communication channels CC_IntegrationServer_RFC_Receiver and CC_ExtSystem_File_Receiver have no special features, you can get adapter type from their technical names.

Most important step is to configure SOAP Sender Communication Channel CC_ExtSystem_SOAP_Sender and use bridge modules in it:

Figure 11 : The communication channel SOAP Sender

Configuring modules and RequestResponseBean ResponseOnewayBean:

Figure 12 : The communication channel SOAP Sender - Modules

Then create the routing from the source system to the ERP. I made ​​it through the Receiver Determination + Interface Determination + Receiver Agreement ( you can also use the Integrated Configuration):

Figure 13 : Receiver Determination and other objects routing

Don't forget to add our mapping - OM_ExtSystem_Request_to_RFC - to the Interface Determination:

Figure 14 : Interface Determination and Operations Mapping

That's it!

Now let's check our bridge. Use the menu item «Display WSDL» of the Sender Agreement, then press «Save» and save WSDL of our service request to the local file.

Figure 15 : Uploading the WSDL for Sender SOAP Comm. Channel

Then create and send SOAP-request to PI using any tool (I used a freeware tool SOAPUI).

Figure 16 : Sending SOAP- request from SOAPUI in PI

If everything was set up correctly - we can find the answer in the target directory:

Figure 16 : File with additional information from ERP

So, we have set up an example of asynchronous-synchronous bridge using modules in the sender communication channel.

Congratulations to all who is still reading it - you did a half of the way!

Or you skip it - that's another option.

2.3 Asynchronous - synchronous bridge with modules in Receiver Communication Channel.

Suppose that we do not want (or cannot) affect the Sender Communication Channel. It is possible to place same modules in a synchronous communication channel of the receiver system.

Here is the logic scheme of the asynchronous-synchronous bridge with modules in the receiver channel:

Figure 17 : The logic of the asynchronous- synchronous bridge when placing adapters in the communication channel receiver.

1. Receiving the asynchronous messages from external system.
2. Transfer the message to PI messaging system.
3. Message processing in PI, transfer it to a synchronous communication channel - to RequestResponseBean module. RequestResponseBean module change the message type from asynchronous to synchronous. Checks the parameter passThrough to forward the message to the next module in the modules chain or directly send it to the module ResponseOnewayBean (and skip steps from 4 to 6).
4. If passThrough=true the message goes to standard CallSapAdapter module.
5. Synchronous call to an external system, obtain the result.
6. Return the response to the module ResponseOnewayBean, which changes the type of message from synchronous to asynchronous .
7. Return the response message to message processing subsystem , which conducts header-based routing. The header was changed by RequestResponseBean module in step 3.
8. Sending the message to the selected channel.
9. Delivery of the message to an external system via asynchronous communication channel.

Bridge can be build using the same modules - RequestResponseBean and ResponseOnewayBean.

Module parameters for RequestResponseBean:

Module ResponseOnewayBean, included in the Receiver Communication Channel have not so many parameters. We can only change an Interface name in the message header or change the delivery of application error message:

NOTE: There are some conflicts in the documentation (official and unofficial from help.sap.com and sdn.sap.com) about settings for modules in Async-Sync Bridge!

Example: «Configuring the Async / Sync Bridge Using the JMS Adapter» on help.sap.com describes some ResponseOnewayBean module parameters; it's subchapter - «Adding ResponseOnewayBean in the Module Processor» - describe another set of parameters.

In SDN articles connected to the bridges there are a lot of specific examples based on «Configuring the Async / Sync Bridge Using the JMS Adapter». Sometimes there are comments, that it works with one adapter and not working for the other (with the same settings).

Actually, you can't use parameter replaceInterface for ResponseOnewayBean with RFC-adapter as synchronous receiver - it does not work! However, the same substitution works fine when using synchronous request through SOAP-adapter.

Be careful , and analyze all possible sources of information in case of errors.

If we finished with communication channels - we need two routings to configure:

1. External System -> Sender Agreement ( if needed ) -> Reciever Determination -> Interface Determination -> Receiver Agreement -> Synchronous communication channel with modules
2. Synchronous System -> Sender Agreement ( if needed ) -> Reciever Determination -> Interface Determination -> Receiver Agreement -> Asynchronous communication channel.

Both routings also can be configured via Integrated Configuration.

2.4 Example : Asynchronous-synchronous bridge with modules in Receiver Communication Channel.

From theory - to practice!  Let's see how to build a bridge, if you add modules to the receiver communication channel.

The interface schema looks like this:

Figure 18 : Schematic example of asynchronous- synchronous bridge SOAP-RFC-File

We need some preparations does not related directly to the configuration of the bridge.

At this time, synchronous SOAP-service would be the recipient. For simplicity, I used the function module from the previous example as a basis for SOAP-service (transaction SE80 -> Enterprise Service and SOAMANAGER were used for this).

Figure 19 : Creating a SOAP-service in transaction SE80

Figure 20 : SOAP-service configuration in SOAMANGER transaction

Figure 21 : SOAP-service configuration in SOAMANGER transaction

Download WSDL of our service and import it into the PI ESR repository - External Definition DOC_3I_ZZ_SOAP_EXT_REQUEST.

Then create a synchronous interface SI_SOAP_ADDINFO_Sync, using the message types from the previous import:

Figure 22 : Synchronous interface based on external WSDL

Synchronous part of the bridge is ready!

For async part we need following development objects:

Figure 23 : Objects in the Enterprise Service Repository

In total:

• Two asynchronous interface for asynchronous systems (Request and Response): SI_ExtSystem_Request_Async and SI_ExtSystem_Response

• Synchronous interface SI_SOAP_ADDINFO_Sync.

• Asynchronous SI_ERP_SOAP_Response_Async (transmits a message from ZZ_SOAP_EXT_REQUEST.Response from synchronous SI_SOAP_ADDINFO_Sync).

• Asynchronous Operations Mapping for request: OM_ExtSystemRequest_to_SOAP_Request

• Asynchronous Operations Mapping for response: OM_ERP_SOAPResp_to_ExtSystem_Response

Objects are created in the standard way, no technical tricks needed. There would be only one alert - during  activation of OM_ExtSystemRequest_to_SOAP_Request mapping. System will alert about interfaces type mismatch, but the object would be activated so you can ignore it.

Now all development objects are ready - let's go for settings in Integration Directory:

Figure 24: Configuration objects

Let's build routing via Integrated Configuration this time (as opposed to 2.2). Create an object to send the request :

Fig. : 24 : Integrated Configuration - Request

Create a synchronous communication channel for SOAP-service - CC_IntegrationServer_SOAP_Receiver:

Figure 25 : The communication channel SOAP Receiver

Configure the modules in the communication channel :

Figure 26 : The communication channel SOAP Receiver - setting modules

ResponseOnewayBean module parameters specify the name and the namespace of interface to be replaced in the message header . As a result of the module processing response message from the SOAP changing the header and become asynchronous, sent from the interface SI_ERP_SOAP_Response_Async.

WARNING:such substitution will not work with  synchronous RFC!  To create a bridge with synchronous RFC you need to use same technique as in Example 2.2 - with synchronous mapping and Receiver Determination (or Integrated Configuration).

You need two channels - sender and receiver - to create an Integrated Configuration. But we defined SOAP channel as the receiver , and use it for response routing is not possible. The solution is to create a dummy channel, which will take place in the Integrated Configuration on Inbound Processing tab. Type of adapter for this channel is not important, I chose SOAP:

Figure 27 : SOAP communication channel as a placeholder in the Integrated Configuration

Then adjust the reverse route in the usual way :

Figure 28 : Integrated Configuration - Reciever

Figure 29 : Integrated Configuration - Reciever Interface

Figure 30 : Integrated Configuration - Outbound Processing

Well that's it. Activate all the objects, test using SOAPUI, as in Example 2.2.

Result pleases us by XML-file with the requested information :

  Figure 31 : File with additional information from the SOAP- service

--------------------------------------------------------------------------------

Thanks for you attention!

I have two more articles on same topic in the queue, so stay tuned!

With best regards,

Alexey Petrov

                   The message size directly influences the performance of an interface processed in SAP PI. The size of a PI message depends on two elements,the PI header and the actual payload. The Payload can vary greatly between interfaces or over time for example, larger messages during year-end closing.The size of the PI message header can cause a major overhead for small messages of only a few kB and can cause a decrease in the overall throughput of the interface.The larger the message payload, the smaller the overhead due to the PI message header. On the other hand, large messages require a lot of memory on the Java stack which can cause heavy memory usage on ABAP or excessive garbage collection activity will also reduce the overall system performance.

Large Message Queue on PI ABAP :

                   In this case the interfaces using the ABAP Integration Server,uses the large message queue filters to restrict the parallelization of mapping calls from ABAP queues. To do so set the parameter EO_MSG_SIZE_LIMIT of category TUNING to e.g. 5.000 to direct all messages larger 5 MB to dedicated XBTL* or XBTM* queues.The value of the parameter depends on the number of large messages and the acceptable delay that might be caused due to a backlog in the large message queue. To reduce the backlog the number of large message queues can also be configured via parameter EO_MSG_SIZE_LIMIT_PARALLEL of category TUNING. The default value is 1 so that all messages larger than the defined threshold will be processed in one single queue. Naturally the parallelization should not be set higher than 2 or 3 to avoid overloading of the Java memory due to parallel large message requests.

Large message queues on PI Adapter Engine :

                    Handling of large messages in the Messaging System for the Java based Adapter Engine. Contrary to the Integration Engine, it is not the size of a single large message only that determines the parallelization.Instead the sum of the size of the large messages across all adapters on a given Java server node is limited to avoid overloading the Java heap. This is based on so called permits that define a threshold of a message size. Each message larger than the permit threshold is considered as large message. The number of permits can be configured as well to determine the degree of parallelization. Per default the permit size is 10 MB and 10 permits are available. This means that large messages will be processed in parallel if 100 MB are not exceeded.

To discuss more in detail, for an example, using the default values. Let us assume we have 5 messages waiting to be processed (status “To Be Delivered” on one server node. Message A has 5 MB, message B has 10 MB, message C has 50 MB, message D 150 MB, message E 50 MB and message F 40 MB. Message A is not considered large since the size is smaller than the permit size and is not considered large and can be immediately processed. Message B requires 1 permit, message C requires 5. Since enough permits are available processing will start (status DLNG). Hence for message D all available 10 permits would be required. Since the permits are currently not available it cannot be scheduled. If blacklisting is enabled the message will be put to error status (NDLV) since it exceed the maximum number of defined permits. In that case the message would have to be restarted manually. Message E requires 5 permits and can also not be scheduled. But since there are 4 permits left message F is put to DLNG. Due to the smaller size message B and message F finish first releasing 5 permits. This is sufficient to schedule message E which requires 5 permits. Only after message E and C have finished message D can be scheduled consuming all available permits.

The example above shows a potential delay a large message could face due to the waiting time for the permits. But the assumption is that large messages are not time critical and therefore additional delay is less critical than potential overload of the system.The large message queue handling is based on the Messaging System queues. This means that restricting the parallelization is only possible after the initial persistence of the message in the Messaging System queues. Per default this is only done after the Receiver Determination. Therefore if you have a very high parallel load of incoming large requests this feature will not help. Instead you would have to restrict the size of incoming requests on the sender channel (e.g. file size limit in the file adapter or the icm/HTTP/max_request_size_KB limit in the ICM for incoming HTTP requests). If you have very complex extended receiver determination or complex content based routing it might be useful to configure staging in the first processing step of the Messaging System (BI=3) as described in Logging / Staging on the AAE (PI 7.3 and higher).

The number of permits consumed can be monitored in PIMON-->Monitoring-->Adapter Engine Status. The number of threads corresponds to the number of consumed permits

General Hardware Bottleneck :

               During all tuning actions discussed,please keep in mind that the limitation of all activities is set by the underlying CPU and memory capacity. The physical server and its hardware have to provide resources for three PI runtime: the Adapter Engine, the Integration Engine, and the Business Process Engine. Tuning one of the engines for high throughput leaves fewer resources for the remaining engines. Thus, the hardware capacity has to be monitored closely.

Note: Please for more information, give a glance on the SAP Note : 1727870 - Handling of large messages in the Messaging system and SAP Note 894509 – PI Performance Check.

        Some of the tables in SAP PI grow very quickly and can cause severe performance problems if archiving or deletion does not take place frequently. For more information on troubleshooting - SAP Note 872388 – Troubleshooting Archiving and Deletion in PI.

Log in to the Integration Server and call transaction SE16. Enter the table names as listed below and execute. In the following screen simply press the button “Number of Entries”. The most important tables are

Table :SXMSPMAST, cleaned up by XML message archiving/deletion,If the switch procedure is used, please check SXMSPMAST2 as well.

Table :SXMSCLUR / SXMSCLUP ,cleaned up by XML message archiving/deletion.If the switch procedure is used, please check SXMSPCLUR2 and SXMSCLUP2 as well.

Table :SXMSPHIST ,cleaned up by the deletion of history entries.If the switch procedure is used, please check SXMSPHIST2 as well.

Table :SXMSPFRAWH and SXMSPFRAWD, cleaned up by the performance jobs,For more information, go thru SAP Note 820622.

Table :SWFRXI*, cleaned up by specific jobs, For more information, go thru SAP Note 874708. for deleting Message Persistence Data in SWFRXI*

Table :SWWWIHEAD, cleaned up by work item archiving/deletion

PI Message tables: BC_MSG and BC_MSG_AUDIT,when audit log persistence is enabled.

PI message logging information when staging/loging is used, Table : BC_MSG_LOG_VERSION

Table :XI_AF_MSG ,stores incoming and outgoing messages in Adapter Engine

Table :XI_AF_MSG_AUDIT,audit log entries for messages stored in XI_AF_MSG in Adapter Engine

Table :XI_SEC,for security message archiving in Adapter Engine

In general, check for all tables if the number of entries is reasonably small or remains roughly constant over a period of time. If that is not the case, check your archiving/deletion set up.

Reference: http://wiki.scn.sap.com/wiki/display/XI/XI+Tables   (  Special thanks to Kenny Scott, for contribution )

Hi ,

Recently I have worked on a requirement of sending mail in a tabular format using mail package .I want to share my knowledge on this as it took longer time for me to fine tune the code ,hope it might be useful for others to quickly develop interfaces with similar functionality .

Functionality: PI will receive employee pension reports in the form of IDocs from SAP ECC .PI need to send these details in mail (tabular format) to concerned parties .

I have used two mappings to achieve the above functionality

1)Mapping1 (Graphical Mapping) :To capture the employee pension report details from IDoc and put those in a temporary structure .

2)Mapping2 (XSLT Mapping) :To send the above details in tabular format to concern business parties .

Mapping1:

Mapping2:(Mapping1 O/P will be given as input to Mapping2)

<?xml version="1.0" encoding="UTF-8" ?><xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"  xmlns:ns0="YourNameSpace"><xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes" />  <xsl:variable name="break">&lt;br&gt;</xsl:variable>  <xsl:variable name="tableB">&lt;table BORDER="5"&gt;</xsl:variable>  <xsl:variable name="tableE">&lt;/table&gt;</xsl:variable>  <xsl:variable name="trB">&lt;tr&gt;</xsl:variable>  <xsl:variable name="trE">&lt;/tr&gt;</xsl:variable>  <xsl:variable name="tdB">&lt;td&gt;</xsl:variable>  <xsl:variable name="tdE">&lt;/td&gt;</xsl:variable>  <xsl:variable name="nbsp"> </xsl:variable>  <xsl:variable name="thB">&lt;tr BGCOLOR="#CCCC99"&gt;</xsl:variable><xsl:template match="/"><ns1:Mail xmlns:ns1="http://sap.com/xi/XI/Mail/30">  <Subject>Error Report of XYZ interface</Subject><From>  <xsl:value-of select="ns0:PensionErrorReports/FROM"/>  </From><To>  <xsl:value-of select="ns0:PensionErrorReports/TO"/>  </To>  <Content_Type>text/html</Content_Type><Content><b><xsl:text>Hi Team,</xsl:text></b><xsl:value-of select="$break" /><xsl:value-of select="$break" /><b><xsl:text>There was an error from SAP ECC while processing IDOC for the interface XYZ. Below are the details. Please take required action immediately.</xsl:text></b>  <xsl:value-of select="$break" />  <xsl:value-of select="$break" />  <xsl:value-of select="$tableB" />  <xsl:value-of select="$thB" />  <xsl:value-of select="$tdB" /><b>  <xsl:text>Payroll number</xsl:text>  </b>  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" /><b>  <xsl:text>Name</xsl:text>  </b>  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" /><b>  <xsl:text>Formatted Name of Emp(or)Applicant</xsl:text>  </b>  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" /><b>  <xsl:text>COMMENT</xsl:text>  </b>  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" /><b>  <xsl:text>Date of Transfer</xsl:text>  </b>  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$trE" /><xsl:for-each select="ns0:PensionErrorReports/Pension_ErrorReports">  <xsl:value-of select="$trB" />  <xsl:value-of select="$tdB" />  <xsl:value-of select="PayrollNumber" />  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" />  <xsl:value-of select="EMPLOYEE_INITIALS" />  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" />  <xsl:value-of select="EDIT_NAME" />  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" />  <xsl:value-of select="COMMENT" />  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$tdB" />  <xsl:value-of select="DateOfTransfer" />  <xsl:value-of select="$nbsp" />  <xsl:value-of select="$tdE" />  <xsl:value-of select="$trE" />  </xsl:for-each>  <xsl:value-of select="$tableE" />  <xsl:value-of select="$break" />  <xsl:value-of select="$break" />  <xsl:value-of select="$break" />  <xsl:value-of select="$break" />  <b><xsl:text>Regards</xsl:text></b>  <xsl:value-of select="$break" />  <b><xsl:text>Appsupport Team</xsl:text></b>  </Content>  </ns1:Mail>  </xsl:template>  </xsl:stylesheet>

Zip and import the above .xsl file as a imported archive in PI.Now create the operation mapping as below

Source:IDoc

Target:Service Interface for standard Mail Package

Here I'm using From,To parameters to get From and To mail id's dynamically.

Now Create the ID objects as below. (You can also use ICO if you are using IDOC_AAE)

For better understanding and readability I'm providing testcases in Part2

Here I'm going to cover Test cases.

Input IDoC XML payload: PI received 2 IDocs in a package

Output Mail generated :

While doing this scenario a common doubt will be raised ,i.e "What if the length of the text is more ,does it gets skipped or will be populated ?"

Here is the answer:It automatically gets adjusted and populates the text .

Here is the link for navigation to Part1

Increasing the number of threads and working on a queue in the messaging system, didn't not solve the performance problem ?.

    Why can't we try Adapter Parallelism....

The below are the three strategies to work around.....

1) Create additional communication channels with a different name and adjust the respective sender/receiver agreements to use them in parallel.

2) Add a second server node that will automatically have the same adapters and communication channel running as the first server node. This does not work for polling sender adapters (File, JDBC, or Mail) since the adapter framework scheduler assigns only one server node to a polling communication channel.

3) Install and use a non-central Adapter Framework for performance-critical interfaces to achieve better separation of interfaces.

Some of the most frequently used adapters and the possible options are:

Polling Adapters (JDBC, Mail, File):
               At the sender side these adapters use an Adapter Framework Scheduler which assigns a server node that does the polling at the specified interval. Thus, only one server node in the J2EE cluster does the polling and no parallelization can be achieved. Therefore scaling via additional server node is not possible. For example, since the channels are doing the same SELECT statement on the database or pick up files with the same file name, parallel processing will only result in locking problems. To increase the throughput for such interfaces, the polling interval has to be reduced to avoid a big backlog of data that is to be polled. If the volume is still too high, think about creating a second interface, for example, the new interface would poll the data from a different directory or database table to avoid locking. At the receiver side, the adapters work sequentially on each server node by default. For example, only one UPDATE statement can be executed for JDBC for each Communication Channel (independent of the number of consumer threads configured in the Messaging System) and all the other messages for the same Communication Channel will wait until the first one is finished. This is done to avoid blocking situations on the remote database. On the other hand, this can cause blocking situations for whole adapters as discussed in section Avoid Blocking Caused by Single Slow/Hanging Receiver Interface. To allow better throughput for these adapters, can configure the degree of parallelism at the receiver side. In the Processing tab of the Communication Channel in the field “Maximum Concurrency”, enter the number of messages to be processed in parallel by the receiver channel. For example, if you enter the value 2, then two messages are processed in parallel on one J2EE server node. The parallel execution of these statements at database level of course depends on the nature of the statements and the isolation level defined on the database. If all statements update the same database record database locking will occur and no parallelization can be achieved.

JMS Adapter:

              The JMS adapter is per default using a push mechanism on the PI sender side. This means the data is pushed by the sending MQ provider. Per default every Communication channel has one JMS connection established per J2EE server node. On each connection it processes one message after the other so that the processing is sequential. Since there is one connection per server node scaling via additional server nodes is an option ( refer SAP Note 1502046 ).By doing so you can specify a polling interval in the PI Communication Channel and PI will be the initiator of the communication. Also here the Adapter Framework Scheduler is used which implies sequential processing. But in contrary to JDBC or File sender channels the JMS polling sender channel will allow parallel processing on all server nodes of the J2EE cluster. Therefore scaling via additional J2EE server nodes is an option.The JMS receiver side supports parallel operation out of the box. Only some small parts during message processing (pure sending of the message to the JMS provider) are synchronized. Therefore to enable parallel processing on the JMS receiver side no actions are necessary.

SOAP Adapter:

               The SOAP adapter is able to process requests in parallel. The SOAP sender side has in general no limitations in the number of requests it can execute in  parallel. The limiting factor here is the FCAThreads available to process the incoming HTTP calls.On the receiver side the parallelism depends on the number of the threads defined in the messaging system and the ability of the receiving system to cope with the load.

RFC Adapter:

               The RFC adapter offers parameters to adjust the degree of parallelism by defining the number of initial and maximum connections to be used. The initial  threads are allocated from the Application thread pool directly and are therefore not available for any other tasks. Therefore the number of initial thread should be kept minimal. To avoid bottlenecks during the peak time the maximum connections can be used. A bottleneck is indicated by the following exception in the Audit Log:com.sap.aii.af.ra.ms.api.DeliveryException: error while processing message to remote system:com.sap.aii.af.rfc.core.client.RfcClientException: resource error: could not get a client from JCO.Pool: com.sap.mw.jco.JCO$Exception: (106) JCO_ERROR_RESOURCE: Connection pool RfcClient … is exhausted. The current pool size limit (max connections) is 1 connection,this should be done carefully since these threads will be taken from the J2EE application thread pool. Therefore a very high value there can cause a bottleneck on the J2EE engine and therefore cause a major instability of the system. As per RFC adapter online help the maximum number of connections are restricted to 50.

IDoc_AAE Adapter:

                The IDoc adapter on Java (IDoc_AAE) was introduced with PI 7.3. On the sender side the parallelization depends on the configuration mode chosen. In Manual Mode the adapter works sequential per server node. For channels in “Default Mode” it depends on the configuration of the inbound Resource Adapter (RA). Via the parameter MaxReaderThreadCount of the inbound RA you can configure how many threads are globally available for all IDoc adapters running in Default Mode.Hence, this determines the overall parallelization of the IDoc sender adapter per Java server node.Currently the recommended maximum number of threads is 10.The receiver side of the Java IDoc adapter is working parallel per default.

The table below gives a summary of the parallelism for the different adapter types.....

Using Mapping-Lookups is a powerful tool to gather data within an integration scenario. But is it possible to perform Mapping-Lookups using ABAP Proxies?

Help content does not provide any information about it, so we gave it a shot. Within a B2B customer order scenario, we want to look up order details out of several SAP ERP clients using ABAP Proxies.

SAP Help content about Lookups...

Good news first, it is possible. You can use a SOAP channel with message protocol XI 3.0 (available since PI 7.11), within your java mapping. Details on how to program the java mapping can be found here.

The important part for Proxy-Lookups is these 2 LOC:

// 4. Set the operation name and namespace; optional step, // only necessary if the used adapter needs these parameters.  accessor.setOperationName("myInterfaceName");

accessor.setOperationNamespace("myInterfaceNS");

These are needed to call the proxy, otherwise the XI header does not provide the information about the interface and operation to call, which is needed by the local integration engine.

During runtime, ABAP proxy lookup calls, can be identified within the local MONIS (SXMB_MONI) by sender component name "Mapping" and the receiver component name is the name of the system which the integrations calls later on. This is the receiver determined within the routing. Afaik, it is not possible to override these header informations. But imho, it's nice to know, where the journey is going to, after the Lookup has been performed.

In several productive scenarios, batch jobs are scheduled and tracked by third party schedulers. A scheduler is responsible for seamless and optimized scheduling and processing of jobs. Control-M (BMC) is one such scheduler that exists in various landscapes to schedule the jobs in PI/PO as well as ECC. The efficiency and throughput of Control-M (or any third party scheduler software) highly depends on the way it is integrated with components like SAP PI/PO, ECC etc. Also, in case of high volume scenarios, unforeseen event like PI/PO messages getting stuck in adapter engine (To be delivered -TBDL/Delivering -DLNG etc) do take place. This kind of situation affects the batch jobs execution (and completion) adversely. It might result into partial execution of PO/ECC jobs and then a lot of reprocessing effort is required both in PI/PO and ECC. The motive behind writing this blog is to consider such undesirable scenarios during third party scheduler (Control-M) integration with PI/PO and ECC.

Premises

1. Integration of Control-M with SAP PI/PO

2.Integration around File based scenarios (File to any-adapter)

Related Content on SCN

There is another valuable blog written by Deepak Shah which provides a file generation based approach of integration.

Integrating Scheduler Software with SAP PI

Why this blog then?

1. The approach used in this blog can open another avenue for people who are looking for a different integration pattern esp. in very high volume scenarios. This solution uses real time message status handling based on PI/PO database.
2. This solution is very helpful in File-to-IDoc scenarios where multiple IDocs are created from one file and the message volume is huge. By huge volumes, I mean the following:

    3. This solution provides mechanism to accommodate sender-side actions once messages are found stuck in intermediate statuses like

        ‘To Be Delivered (TBDL)’, ‘Delivering (DLNG) and ‘Waiting (WAIT)’.

System Details

SAP PO 7.31 Single Stack

(It should work on Dual Stack and other PI/PO versions given the PO Table fields for table BC_MSG remain same. Otherwise, minor adjustment might be required based on different PI/PO versions)

Prerequisites

SAP PO JDBC Driver should be deployed to establish JDBC Connection.

(Please note: JDBC connection to PO database will only involve SELECT query and no CRUD operations will be performed on the DB in any case, whatsoever. Any modification (CRUD) within SAP standard DB is a straight No-Go and is not advised in any case. )

Business Scenario:

Let’s say, a business requires automatic execution of jobs based on interdependence and success of related jobs. Control-M will have to execute both PI/PO and ECC Jobs to fulfill this requirement. By PI/PO Jobs, I mean starting and stopping of channels for various interfaces which in turn will pick the files from configured directories for processing. Subsequently, it has to start other ECC jobs based on successful completion of predecessor/related jobs. ECC Jobs can be batch jobs that are scheduled for batch processing.

Here, the dependency of jobs is crucial because certain jobs cannot be started unless predecessor jobs are successfully completed. The reason behind this is, predecessor jobs might be creating the input for successor jobs. If a job fails, then related jobs can be postponed based on business rules to make end-to-end execution successful.

High volume scenarios make it even important to have a perfect triggering of interdependent jobs in order to successfully complete end-to-end business processes.

Process Flow:

       A typical scenario (File to IDoc – Multi mapping) will include the following process chain:

         Visible Disadvantages in above case:

By ‘Partial Completion’, I am referring to scenarios where messages get stuck in Adapter Engine in To Be Delivered (TBDL), Delivering (DLNG), Waiting (WAIT) statuses. These situation are encountered due to several reasons e.g. Resource Failure, Receiver Bottleneck, System Crashes etc.

These intermediate messages are not captured by alerts because of their non-final status.

So, if the PO job does not return any error back to Control-M, then Control-M will trigger the ECC jobs and it will lead to partial execution in ECC because of partial delivery of messages (i.e. IDocs in File-to-IDoc scenarios) to ECC.

   b.Control-M job scheduling with another SOAP or HTTP_AAE call: (Solution Integrated)

       A typical scenario (File to IDoc – Multi mapping) along with the message status based solution will typically follow the below process chain:

1. Control-M starts PO Job (ZPO_JOB1). This basically starts the sender channel.
2. Files get picked up from source folders and processing gets completed in PI/PO. IDocs get delivered to ECC.
3. Control-M triggers a synchronous SOAP or HTTP call (whichever it is capable of) with a defined Interface ID and some other details viz. Sender/Receiver Business Components, Service Interfaces (outbound and inbound), Interface Start Time, ICO Scenario ID (This is a unique identifier for an ICO and is totally capable of identifying and end-to-end interface execution. This can be integrated as a self-sufficient field and all other details can be avoided) etc.

             Internal Processing:

• Control-M triggers SOAP/HTTP  call.
• Input data is read by PO and a synchronous JDBC call is triggered on PO database on table BC_MSG.
• The conditional mapping maps request data to a JDBC request structure TO get overall message count in TBDL/DLNG/WAIT/HOLD statuses.
•   It can even be tweaked to fire 5-6 JDBC calls (Only SELECT for different statuses, if required)
• JDBC response is appropriately mapped back to SOAP/HTTP response.

   4. Control-M starts related ECC job (ZECC_JOB1) based on the SOAP/HTTP response to identify the extent of interface completion (if required, in terms of percentage) and, it stops the PO Job (ZPO_JOB1) which basically stops the sender channel.

Build

This solution can be realized using two integration patterns, depending on Control-M (or any other third party) capability of firing the request:

1. SOAP to JDBC – Synchronous (Preferred)
2. HTTP_AAE to JDBC – Synchronous

A typical development procedure will include creation of the following:

• ESR: DT/MT/SI/MM/OM
• ID: Business Components, SCC, RCC, ICO, CS

(Details are not provided as the solution can be customized according to the requirement. The approach is something that is highlighted in the blog while the regular object build is not explained owing to its custom nature and varying requirements. Any help in designing will be entertained, if required.)

Testing

Once implemented, it can be tested with Control-M (or third party) integration. Alternatively, it can be tested from SOAP UI to verify the results.

The image below captures execution results in details (status wise). When fired, it will reflect the real time status of messages falling under respective status. The response statuses can be configured in message mapping based on the requirement.

Major advantages of the message status based approach

1. It can save a lot of reprocessing (effort as well as time) and post-processing adjustments/reversal in case there is a failure/bottleneck in PO system.
2. It can improve the performance of Control-M in terms of Jobs Execution. This solution can be leveraged to execute jobs independent of hard-coded wait time. After Control-M fires a status seeking request and the response contains no error (i.e. error count = 0), then subsequent related jobs can be triggered immediately. This can dramatically decrease fixed-time dependency as well as execution timing of batch job sets.
3. It can provide a mechanism to quickly troubleshoot interfaces in production support and during maintenance activities.
4. It can help in identifying PO system bottlenecks. If there are a lot of messages in TBDL/DLNG/WAIT/HOLD/FAIL, a quick system health checkup can be done and remedial steps can be taken. It will subsequently prevent automatic PO system restarts; heap dumps due to out of memory (OOM) issues in very high load scenarios.
5. It can avoid message blacklisting which happens in case of a system restart during message delivery.

Conclusion

This solution provides one way of effective PO/ECC job integration and execution. Reprocessing and Post-processing adjustments/reversal are always tedious in case a link is found broken in batch jobs execution chain. It can be extrapolated that there are numerous possibilities from this point forward for the third party scheduler (e.g. Control-M) to take actions on such undesirable impediments. Based on the message status response, email alerts can be triggered and appropriate remedial steps can be taken.

Last but not the least; this was one way of integrating Control-M with PO/ECC jobs. There can be more ways to achieve it as well. Readers are requested to share different ways, if encountered, and their viewpoints around this approach.

I would like to flag this issue with the community and hopefully this will help prevent you from encountering the "JAR resources in JNLP file are not signed by the same certificate" error when opening the Enterprise Services Repository (ESR) or Integration Builder Directory.

Background:

SAP have changed the certificates used for launching the Process Integration (PI), Integration Builder tools. This change affects all Support Packs (SP's) and/or Patch Levels (PL's) delivered after 05.03.2014.

If required, you can use SAP Note 1381878 How/where to check the patch levels of your XI/PI system, to verify the release date of the relevant Software Components.

Solution:

See the SAP Note 1981572 XI/PI:Jar Files are signed by different certificates in 2014, for details on the Software Components which require patching. Please make your Basis teams aware of this change.

If your Software Components were delivered prior to the 05.03.2014 cut off date then you will not encounter this error. However, if for example, you update one component and not the other required components, you will most likely hit this issue.

It is important to ensure that all required Software Components are updated as updating only one or two will not prevent the "JAR resources in JNLP file are not signed by the same certificate" error.

Steps required after patch deployment:

After the new patches are imported, it may be necessary to clear the caches on your Java console. See Troubleshooting PI ESR and Integration Builder Logon Issues-> Java Cache

Adapter Module: ReplaceString

Use

ReplaceString module is used to replace strings within the xml text of the payload of the message. It can be used to modify the message structure before it gets in PI and also before it gets out. It is useful when there are strings like namespaces, types, etc that PI can’t “understand” or that PI is unable to include in outbound messages.

The module obtains a pair of string : <OldString>;<NewString> as parameter. Reads the payload of the message and converts it to a string. Then it searches <OldString> within the message and replace each occurrence with <NewString>. Finally it returns the message modified. If OldString can’t be found in the message nothing is modified and the original message is returned.

The module accepts multiple string pairs to allow multiple replacements in a single execution.

Deployment

Enterprise Java Bean Project: ReplaceString-ejb

Enterprise Java Bean Application: ReplaceString-ear

Integration

The module can be used in any kind of Sender and Receiver adapters.

Activities

This section describes all the activities that have to be carried out in order to configure the module.

Entries in processing sequence

Insert the module in front of the adapter module as shown in the picture below.

Entries in the module configuration

The table below shows the possible parameters and values of the adapter module.

(*) The parameter name can be any string: param1, parameter, aaa, etc.

Example

Payload data before execution:

<MT_ROOT>     <RECORDSET>          <DATA>               <FIELD1>value</FIELD1>               <FIELD2>value</FIELD2>               <FIELD3>value</FIELD3>          </DATA>          <DATA>               <FIELD1>value</FIELD1>               <FIELD2>value</FIELD2>               <FIELD3>value</FIELD3>          </DATA>     </RECORDSET></MT_ROOT>

Module parameters:

separator = ‘;’

param1 = ’RECORDSET;ROW’

param2 = <MT_ROOT>;<MESSAGE type=TYPE>

param3 = </MT_ROOT>;</MESSAGE>

param4 = <DATA>;emptyString

param5 = </DATA>;emptyString

Payload data after execution:

< MESSAGE type=TYPE>     <ROW>          <FIELD1>value</FIELD1>          <FIELD2>value</FIELD2>          <FIELD3>value</FIELD3>          <FIELD1>value</FIELD1>          <FIELD2>value</FIELD2>          <FIELD3>value</FIELD3>     </ROW></MESSAGE>

ReplaceStringBean

/**
*
*/
package com.arms.integrations;
import java.rmi.RemoteException;
import javax.ejb.EJBException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import com.sap.aii.af.lib.mp.module.Module;
import com.sap.aii.af.lib.mp.module.ModuleContext;
import com.sap.aii.af.lib.mp.module.ModuleData;
import com.sap.aii.af.lib.mp.module.ModuleException;
import com.sap.engine.interfaces.messaging.api.Message;
import com.sap.aii.af.service.auditlog.Audit;
import com.sap.engine.interfaces.messaging.api.MessageKey;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditLogStatus;
import java.io.*;
import java.util.*;
/**
* @author Roger Allué i Vall
*
*/
public class ReplaceStringBean implements SessionBean, Module {    private SessionContext myContext;    private static final String C_SEPARATOR_STRING = "separator";    private static final String C_AUX_SEPARATOR = "|";    private static final String C_MODULEKEY_STRING = "module.key";    private static final String C_BLANK_STRING = "blankString";    private static final String C_EMPTY_STRING = "emptyString";    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbActivate()     */    public void ejbActivate() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbPassivate()     */    public void ejbPassivate() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#ejbRemove()     */    public void ejbRemove() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionBean#setSessionContext(javax.ejb.SessionContext)     */    public void setSessionContext(SessionContext arg0) throws EJBException,            RemoteException {        // TODO Auto-generated method stub        myContext = arg0;    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#afterBegin()     */    public void afterBegin() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#afterCompletion(boolean)     */    public void afterCompletion(boolean arg0) throws EJBException,            RemoteException {        // TODO Auto-generated method stub    }    /* (non-Javadoc)     * @see javax.ejb.SessionSynchronization#beforeCompletion()     */    public void beforeCompletion() throws EJBException, RemoteException {        // TODO Auto-generated method stub    }    public void ejbCreate() throws javax.ejb.CreateException {      }    public ModuleData process(ModuleContext moduleContext, ModuleData inputModuleData)    throws ModuleException {          InputStream payloadIn = null;        ByteArrayOutputStream payloadOut = null;        int inputByte = 0;        String payloadStr = "";        Message msg;        Enumeration paramList;        String sep, paramKey,param, strArray[];            try {             msg = (Message) inputModuleData.getPrincipalData();             MessageKey amk = new MessageKey(msg.getMessageId(), msg.getMessageDirection());             payloadIn = new ByteArrayInputStream(msg.getDocument().getContent());                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,".-Module beginning");               while((inputByte = payloadIn.read()) != -1) {                 payloadStr = payloadStr + (char) inputByte;             }                   Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Payload before execution:" );            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,payloadStr );             sep = moduleContext.getContextData(C_SEPARATOR_STRING);               if ( sep == null ) {                sep = C_AUX_SEPARATOR;                Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Default separator used: " + sep);             }             else {                Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Separator found: " + sep);             }                                 paramList = moduleContext.getContextDataKeys();               while( paramList.hasMoreElements()) {                    paramKey = (String) paramList.nextElement();                 //Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"ParamKey: " + paramKey);                 param =  moduleContext.getContextData(paramKey);                 //Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Param: " + param);                            if ( ! paramKey.equals(C_SEPARATOR_STRING) && ! paramKey.equals(C_MODULEKEY_STRING) ){                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"ParamKey: " + paramKey);                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Parameter: " + param);                    Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Separator: " + sep);                                  strArray = param.split(sep);                    if (strArray != null){                       if ((! strArray[0].equals(null)) && (! strArray[1].equals(null))){                        if ( strArray[1].equals(C_BLANK_STRING)){                            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Blank String");                            strArray[1]=" ";                                              }                        else if (strArray[1].equals(C_EMPTY_STRING)){                            strArray[1]="";                            Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Empty String");                        }                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution strArray[0]: " + strArray[0]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution strArray[1]: " + strArray[1]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution payloadStrA : " + payloadStr);                              payloadStr = payloadStr.replaceAll(strArray[0],strArray[1]);                        Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Substitution payloadStrB : " + payloadStr);                       }                    }                 }             }               Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,"Payload after replacement:" );             Audit.addAuditLogEntry(amk, AuditLogStatus.SUCCESS,payloadStr );                   payloadOut = new ByteArrayOutputStream();             payloadOut.write(payloadStr.getBytes());               msg.getDocument().setContent(payloadOut.toByteArray());             inputModuleData.setPrincipalData(msg);          }        catch(Exception e) {            ModuleException me = new ModuleException(e);            throw me;        }        return inputModuleData;    }
}

Continue of Bridges Construction Site (1/3): Asynchronous - synchronous bridge.

----------------------------------------------

Hello again, colleagues!

Let's continue to review solutions for the problem of integration in heterogeneous interfaces landscape using bridges.

Uff.. most complex part of this post is finished!

Contents:

3. Sync-Async Bridge and its modules - RequestOnewayBean, WaitResponseBean and NotifyResponseBean.

  Let's think about a new problem: we are running the process in the one system and want to send some information to the external system. It's simple. But we also need immediate confirmation of  of successful reception (or an error message).

Our external system works in asynchronous manner only, ie it takes the input by one service (it can be a file, asynchronous SOAP, SQL, HTTP, etc.), and send an acknowledgment by another service after while.

Let me remind you the scheme of such interfaces:

Figure 1 : Synchronous - Asynchronous Bridge

Sync-async bridge using following modules:

• RequestOnewayBean is responsible for converting synchronous message to asynchronous.

• WaitResponseBean - leaves a synchronious communication channel open and waits for the response. When receiving asynchronous response - converts it to a synchronous message and sends it back to the original system. If the answer does not come within the specified time - sends an error message to the source system.

• NotifyResponseBean is used instead of the standard adapter module in the asynchronous response sender channel to transfer the response message directly to WaitResponseBean ( bypassing the processing in the messaging subsystem of PI).

RequestOnewayBean and WaitResponseBean can be used in a sender communication channel or in a receiver communication channel.

RequestOnewayBean must beinserted before the standard adapter module (usually CallSAPAdapter); WaitResponseBean - after.

For using modules in the Communication Channel you should go to the tab «Module» and enter the following values:

«Module Key» can be anything - so long as it is unique and the keys would be difficult to confuse - it will help us later to determine the parameters of modules.

Further settings depends on the placement of modules: in a sender communication channel or in a receiver communication channel.

But we have one more problem connected to synchronous-asynchronous bridge.

Imagine that several messages are going through the bridge simultaneously. So, at some point few queries are waiting for responses. Some response is coming from the target system - but how we know which of pending channels must be used to send a specific answer?

Figure 2 : the task of determining the recipient of asynchronous response

Here comes new entity - a correlation. Correlation is an identifier , the "key" , which is held on the correspondence between the response and awaiting queue.

    The term " correlation " can also be found in the Sync-Async Bridge in ccBPM - there may be a correlation using not only one but a set of identifiers .

So the task is to "remember" some id during the sending of asynchronous message and check this id during the receiving an answer:

Figure 3: Determination of the recipient using the asynchronous response correlation

The correlationID parameter (defined in the PI message header) is used to correlate a response from asynchronous system. The value in correlationID must be equal to the message GUID of the synchronous request message . This parameter belongs to a group of Adapter-Specific Message Attributes and can be changed using the standard module - DynamicConfigurationBean.

Another example in SAP standard documentation («Configuring the Async / Sync Bridge Using the JMS Adapter») correlate messages using standard settings of communication channels JMS ("Set PI Conversation ID" parameter).

In other kinds of adapters similar standard parameter is missing and you have to make some tweaks - in particular , to save Message GUID and fill correlationID parameter using PI or with the help of the target system. In the example below (3.2) , you can find one of the ways to work with correlations in file adapter.

3.1 Synchronous-asynchronous bridge with modules in the Sender Communication Channel.

If we use Sender Communication Channel for bridge's modules, the scheme of message processing would be next:

Figure 4: The logic of the synchronous-asynchronous bridge with modules in the Sender CC.

1. Receiving a synchronous message from external system via communication channel, sending it to the module RequestOnewayBean. The module switch the message type from synchronous to asynchronous by changing the message header.
2. Depending on the parameter passThrough, module pass the message down the chain of modules or directly sending it to the message processing subsystem of PI (skipping step 3).
3. Message processing by the standard adapter module, transfering it to PI message processing subsystem.
4. Sending a request to asynchronous communication channel.
5. Asynchronous call to the external system
6. The response from external system is coming to async sender communication channel
7. Return the response to the NotifyResponseBean, which directly transmits a response to WaitResponseBean, bypassing the messaging subsystem PI.
8. WaitResponseBean changes message type from asynchronous to synchronous and returns it to the source system.

Module parameters for RequestOnewayBean when placing it in the Sender Communication Channel:

Module parameters for WaitResponseBean when placing it in the Sender Communication Channel:

NotifyResponseBean module in this configuration must be located in the recieving communication channel for a response from asynchronous system.

The module must be used instead of the standard adapter module.

NotifyResponseBean have following module parameters:

Once we set up all the necessary modules , we need to configure the request and response correlation.

To do so, we need keep the message GUID somehow and pass it to the target system. The system, in turn, should return a response with a the GUID in some part of message. When the answer is coming, we need to fill out the parameter in the message header responsible for the correlation - correlationId (Dynamic Configuration header section) with stored GUID value.

Solution can be different for each of the adapter types. I'll show you one of possible file adapter solution in the example below.

After setting up the correlation the only thing to do is to create routing rules linking source synchronous interface to the target asynchronous interface.

For the routing of response it is sufficient to create a Sender Agreement only - everything else will be complete by the NotifyResponseBean module.

3.2 Example of synchronous - asynchronous bridge with modules in Sender Communication Channel.

Let's take a look to the specific example.

Business case: we have an external client who gives us an information about new books available in the store using a synchronous web-service .

PI should receive this information and send it to our database. This DB is pretty old, so the Interface of DB is based on the file exchange. The status of operation  is also returned as a file, so PI should get it and return it to the client.

Figure 5: Example diagram.

To implement this interface, we need to create the following development objects:

Figure 6: Development objects in the Integration Repository

Please note the operations interfaces mapping - it synchronous, but "one-way" - it works in one direction only. To create it we need to use a "trick" with a temporary changing of receiver interface type, like it was done in the mapping from the previous article about the asynchronous-synchronous bridges.

After that, create all necessary routing objects in the Integration Builder:

Figure 7: Configuration objects

The question may arise - how the routing of response can be done by Sender Agreement only?

You can configure full routing rule for the answer or make response routing via Integrated Configuration. But in this particular case it does not make sense - the answer still will be transferred to the module NotifyResponseBean and directly to waiting module WaitResponseBean after that, bypassing the messages processing subsystem of PI.

Now let's put the modules into place.

Synchronous SOAP communication channel:

Figure 8: Synchronous SOAP Sender communication channel

Please don't forget to set Quality Of Service = Best Effort, ie identify the channel as synchronous.

Then add modules RequestOnewayBean and WaitResponseBean:

Fig. 9: Modules in SOAP Sender synchronous communication channel

Then set up a file communication channel to submit a request to the database:

Fig.: 10: Asynchronous File communication channel for DB request.

Please note the configuration of file name:

      Filename is formed by a variable that have Message ID value. Filename = message GUID without any extension.

    Why do so? Thus, in our example, we store the GUID of request message to use it in correlation later. In reality, you can do it same way. or you can pass GUID in the message body. The main thing - target system must get it, save it and return it in an accessible manner to PI along with the response.

Next step: set up a file communication channel for the response:

Fig. 11: File communication channel to get a response from the target system.

We assume that the system produces a response in another local directory on the PI-server as an XML-file.

The name of the file must be the same GUID as request.

Modules configuration:

Fig.: 12: Modules in asynchronous file sender comm. channel

Standard module DynamicConfigurationBean writes the file name to the correlationId parameter in the header of PI message - ie fills correlation for further processing of the response by the bridge.

We are using NotifyResponseBean module instead of standard adapter module CallSAPAdapter. NotifyResponseBean forwards the response directly to the waiting module WaitResponseBean. If nobody expects the answer with such correlation value - an error will be written to the log.

Well, that's all settings now, time to check our bridge.

Form a WSDL for web-service (use context menu on Integrated Configuration -> Display WSDL). Then, use WSDL to form and send SOAP request using any suitable SOAP-tool (I used a freeware tool SOAPUI).

Fig.: 13: Sending a synchronous request from SOAPUI

After sending the file appears in the directory with the name of the message GUID and request data inside.

Fig.: 14: File request received by the target system

Now let's simulate the recipient DB system.

Edit file - put the response data in it.

Fig.: 15: File with response

Move it into the directory books/status. If we were quick enough, the file will be processed by the PI and the response in SOAPUI will look like this:

Fig.: 16: Response successfully received by SOAP-client

Synchronous-asynchronous bridge successfully built.

All fine, but ..

Just imagine: our customer came and asked us to add one more field to the answer - timestamp.

We don't have response mapping, the target system also "can't" provide us response in new format.

What should we do?

Fig.17: Additional development objects in Integration Repository

Also define the mapping, which will convert the response from the system BS_ExtDB in new response format for BS_WebClient.

Figure 18: Add timestamp to the response

Now configure the routing in Integration Directory:

• create a new "pseudo" File communication channel CC_Dummy_file_receiver and move bridge's modules DynamicConfigurationBean and NotifyResponseBean there, with all configuration parameters;

• restore the module CallSAPAdapter in communication channel SS_ExtDB_FileSender;

• remove Sender Agreement for SS_ExtDB_FileSender;

• create and configure the Integrated Configuration for response routing from BS_ExtDB to BS_WebClient.

Fig.: 19: Additional configuration objects

Check our bridge as before and we must get the right answer:

Fig.: 20: Request and response with timestamp.

Ok, the customer is satisfied now, our synchronous-asynchronous bridge works perfect.

Congratulations!

3.3  Synchronous - asynchronous bridge with modules in Receiver Communication Channel.

There are nothing specific in this version of bridge comparing to the previous. There are the same three modules: two of them - in asynchronous Receiver Communication Channel, transmitting a request to target system; one more - in asynchronous Sender Communication Channel for proceessing the response.

Figure 21 : The logic of the synchronous-asynchronous bridge with modules in the receiver communication channel.

1. Receiving the synchronous request from external system.
2. Transfer the message to PI messaging system.
3. Message processing in PI, transfer it to a synchronous communication channel - to RequestOnewayBean module. The module switch the message processing type from synchronous to asynchronous by changing the message header.
4. Message goes to the standard adapter module.
5. Asynchronous call to external system.
6. Asynchronous response from external system. NotifyResponseBean processing the module instead of standard adapter module.
7. Transfer request to WaitResponseBean module, correlation check.
8. The module switch the message processing type from asynchronous to synchronous, transfer the message to messaging sub-system of PI.
9. Response message is going to synchronous communication channel.
10. Response to external system.

Modules options are independent from the placement .

Please note: there isanother information in SAP help, there are more options for modules when placing in a Receiver Channel. I have not found any meaningful application of these parameters.

RequestOnewayBean parameters:

WaitResponseBean parameters:

NotifyResponseBean parameters:

You also should take care about correlation.

3.4 Example : Synchronous - asynchronous bridge with modules in Receiver Communication Channel.

Let's rebuild our example from 3.2.

Figure 22: Development objects in Integration Repository

At this time we need two synchronous interfaces - SI_WebClient_SaveBook_sync and SI_ExtDB_SaveBook_sync, as well as mapping between them-OM_SaveBook_WebClient_to_ExtDB. We also need asynchronous interface SI_ExtDB_Status (should be based on the same message type as the response in SI_ExtDB_SaveBook_sync) - we will use it for response from an asynchronous system.

Configure routing in Integration Directory as follows:

Fig.: 23: Configuration settings, Integration Directory

Remove all modules from SOAP sender communication channel and move them to file receiver communication channel:

Fig.: 24: asynchronous file receiver communication channel for request.

Fig.: 25: asynchronous file receiver communication channel for response.

Reconfigure Integrated Configuration - we have new communication channels for sender and receiver, new interface and mapping; all we have from the previous example - target system only.

To get the file with response, we need only the Sender Agreement connected to the channel CC_ExtDB_File_Sender_WithModule and interface SI_ExtDB_Status.

Start SOAP test tool (SOAPUI in my case) and send test request. You should have a file with request in the file system.

Replace its content with the answer, move it to the folder with responses, wait for PI processing - and you should get following picture:

Fig.: 26: bridge at work - request and response in SOAPUI.

If your picture looks the same - congratulations! You have just built live sync-async bridge!

-----------------------------------------------------------------

Last part of bridge posts will follow - we'll talk about some tricks and "exotics" in bridges.

With best regards,

Alexey Petrov

Document Objective

Figure 1: Advantages and flow using BCM

• FIN (Financial Institution Transfer) – International Cross- Border Payments

• FTA (Federal Tax Application) – Domestic Payment within Country

Figure 2: Folder location for FIN(fin) and FTA(fileact) Payments in AL11

BCM & SwiftNet Integration with SAP PI

Figure 6: Flow diagram of Integration of SAP PI with BCM and SwiftNet

Figure 8: SFTP Module Deployed in ESR

All the files created in AL11 folder of ECC system will contain a signature. The verification of the signature is done in PI system using the Module Key and parameter from SWIFT Module.

In order to show the Encrypted and signed file we have taken the below screen as there are no files currently in fileAct folder in ECC system.

Figure 9: Encrypted and signed file

Configuration in Integration Directory for Payments

We have used different channels for different Banks. This makes easy for supporting testing for Banks whenever required. Channels for the banks which are not in scope for testing will be switched off.

The signature is verified using the parameter “VerifyBackendSignature” available in the SWIFT Module as under:

Figure 10: Module Parameters used for integration with ECC

• IsNotificationRequested – This parameter is marked ‘true’ to receive Notifications from Bank for the payments done.
• IsUrgent – The parameter is marked ‘false’ inorder to avoid the urgency.
• KeyId – This is the unique Key Id for every system integrated with SwiftNet in the Landscape. The value format is SWIFT_<SID>. SID – System ID of Development/Test/Production system
• UseLocalSecurity - Messages are securely and reliably exchanged between banks and other financial institutions
• VerifyBackendSignature – All incoming files from ECC will be checked if they are signed by BCM module. The unsinged files will not be processed and the channel will show error message in PI system.
• Exit is the standard module for in SFTP

• Msecs to Wait Before Modification
  Check where PI waits some time from first finding the file, until reading and processing it later, and the file is only processed if it had not changed over
  the waiting period. As shown below PI waits 10 minutes until the file is complete
• Masking – Suppose we want to process all files that have the extension '.nfs', but want to exclude all files that begin with the letter 'GB2'. To do this follow the below:

Figure 11: Channel configuration for polling payment files

 

 

Additionally mention the connection parameters like host name of the sending server, port, UserId and Password for Authentication. The UserId and Password should be maintained at the server Level and should not be a Dialog/System User.

 

We place the processed files in Archive folders in ECC system.

 

Receiver Channel Configuration:

The receiver channel will contain the Third Party information. We have used PGP (Pretty Good Privacy) Encryption also happens when the payment are sent from PI system

Figure 12: Encryption at Target

• armor - If the protocol (or) transmission channel supports only ASCII printable characters, the data to be transferred should be encoded as plain text. This is referred as binary to text encoding. If it is applied on the plain text itself, and decoded on the receiver end is called "ASCII Armoring". To turn on the value we input is “true”.
• encryptionAlgorithm – In cryptography, Triple DES is the common name for the Triple Data Encryption Algorithm (TDEA or Triple DEA) block cipher, which applies the Data Encryption Standard (DES) cipher algorithm three times to each data block.
• publicKeyRing
  - The public key is used to code a digital message in order to make it unreadable without the use of a private key, which typically takes the form of
  a password.
• recipient – Target mail recipient ID.

Standard mappings are used for FIN and FTA payments as shown in below
figure:

Figure 13: Mapping for FTA Payments

Figure 14: Message Processing in PI

Figure 15: Mapping for FIN Payments

Bank Statements and Notifications from Banks/ Third Party System

As soon as the payment reaches the third Party system, that connects with different Banks, a technical acknowledgement is send back to PI system notifying the receipt of payments. These feedbacks files are sent by the Third Party for FTA and FIN Payments. In addition to these feedbacks, Banks sends a payment status notification PAIN back to SAP PI system after receiving the payment. The payment Status are sent to ECC system using Proxy and the status are changed accordingly in transaction BNK_MONI. We also receive notifications from Bank for FIN and FTA payments. These notifications have two different formats, namely TXT and XML file format. This will be discussed in next section.

Payment & Bank Statement Notifications

• The acknowledgement is received from Third Party system when a FIN payment is received at Third Party. There is not encryption done for the  acknowledgement process.
• The mappings used are from the standard Operation mapping SwiftFIN_PaymentStatus and SwiftFTA_PaymentStatus from the SWIFT Module. (Refer Figure: 7)

• The Acknowledgement sent by Bank to PI is in a different format and the same we refer as PAIN.
• No encryption or check done.
• The format for the file sent is a report which can be found in Software Component Version ISO20022 1.0 in namespace http://sap.com/xi/ISO20022
  and report name as PaymentStatusReportV02.
• This report is mapped to CollectivePaymentOrderNotificationMessage that is a structure from SAP APPL 6.04 in namespace http://sap.com/xi/APPL/Global2 .

Bank Statement Notifications

1. The file is send to AL11 directory directly to keep a track of the Banks statements
2. A service message is sent to ECC using SOAP.

XML Bank Statement Notification for FTA payments:

Issues and Resolution

• Duplicate payments are not processed at Third Party end that nullifies the risk.

• The version of the SFTP adapter had a bug of routing the payments to incorrect channels leading to inconsistencies and escalations. A new version
  was deployed that solved the issue.

• The Public Key for validating and handshake mechanism should be deployed at the server level in PI system. This Key is provided by Third Party.

• Due to network issues between SAP PI and Third Party or Third Party and Bank, the status get incorrectly assigned in BNK_MONI. For eg: Accepted By Bank is sent before Received By Bank. Network checks required to be done during connectivity tests to ensure there are no delay

• Lower TCO by becoming independent from proprietary payment standards and bank-specific e-banking products and setting up a single communication
  channel to SWIFTNet, which will enable your company to communicate with all the banking partners through this one channel.
• Minimize bank connections and enhance straight-through processing rates via end-to-end integration with SWIFTNet
• Full implementation of SAP Bank Communication Management application and SAP Integration Package for SWIFT is a SAP Ramp-Up program in less than 6 months

Introduction

In this blog, I will introduce a custom proxy framework with FEH/ECH capability. FEH/ECH is an error handling framework for proxy-based interfaces and is widely used in SAP's pre-delivered Enterprise Services. In my previous blog, FEH/ECH setup for custom asynchronous proxy, I have described the steps to complete the FEH/ECH configuration for custom processes/interfaces. The design and architecture of the framework is similar to the Custom IDoc framework that I have blogged about recently in the ABAP Connectivity space. In this section, the focus will be on the ABAP development objects for this framework.

The main purpose for this framework is to streamline the development of proxy interfaces with FEH/ECH capability, and also to simplify the development effort required. With this framework in place, new interfaces can be developed rapidly - the developer just have to implement interface-specific application logic and do not need to worry about adding the necessary logic in place to enable FEH/ECH.

Source code

The objects and code for this framework can be found in the following public repository on Github.

Github repository for Proxy_FEH_Framework

Framework object

Base input class - ZCL_PROXY_INPUT

This is the root superclass of the framework model. It provides the factory methods to dynamically determine the runtime class used for processing. This class implements interface IF_ECH_ACTION to enable processing from the Post Processing Office (PPO) - retry, confirm, fail.

This class controls the sequence of logic executed during processing from either proxy execution or PPO.

Proxy execution flow

PPO retry flow

It also contains the common methods for inbound processing (validation checks, custom logging.)  PROCESS_INPUT_DATA is empty at this level and has to be redefined in subclass for interface specific processing.

With all the common FEH registration/retry logic implemented at this super class level, there is no longer any need to recode this when creating a new interface.

Steps to implement a new interface

Below is a step-by-step guide to develop a new interface using this framework. The example shows a create purchase order interface.

1) Create service interface in ESR

2) Generate proxy in backend system (i.e. ECC)

3) Create subclass for processing this PO interface

Class ZCL_PROXY_INPUT_PO is created with superclass ZCL_PROXY_INPUT. All the methods are inherited.

PROCESS_INPUT_DATA method is redefined in this class to include the logic to create PO. At this subclass level, the input data from the message can be dynamically retrieve with method GET_INPUT_DATA. Also, if there are any processing error, exception ZCX_PROXY_PROCESS_ERROR is raised to pass the messages collected into the FEH/ECH.

4) FEH/ECH configuration

The setup for custom proxy interface needs to be completed following the steps in FEH/ECH setup for custom asynchronous proxy

In step 3 of the linked blog, the custom class created above is entered as the action class. The entries in these customizing tables are also used by the framework factory methods for dynamic determination of runtime processing class.

5) Code proxy method

The method for the proxy created in step 2 above. Static factory method S_GET_INSTANCE is called to instantiate the dynamic runtime class. The values for the input parameter must match the entry in table ECHC_PROCESSES above. Subsequent, call the method EXECUTE_IN to start processing by the subclass.

6) Configure scenario in ID

Configure the scenario as per normal in integration directory and then the interface is ready to be tested.

Runtime and testing

When a message with error is processed by the proxy, the error will be transferred to PPO and the following will appear in SXI_MONITOR.

The application error can be viewed from the message, and also is available in PPO as shown below.

The order in error can be viewed via transaction /SAPPO/PPO2

The contents of the message can be edited directly from PPO.

After that, the order is submitted for reprocessing and completed successfully.

Finally, the status of the order is updated in both PPO and SXI_MONITOR.

Dear Colleagues,

Background:

This is probably not worth blogging, but I thought I should share this information anyhow with the community which has given me so much. There was a requirement recently on my project where an email report had to be sent out with a CSV atttachment. Now, as part of the data within the CSV file, there were some special characters like Spanish tilde characters (Ñ,Ó etc) in there. When you opened the csv in MS Excel, these characters looked weird. However, when you opened them in a text editor everything was fine. Even though it is not a huge issue, for the purpose of achieving enhanced customer satisfaction I thought I will give this challenge a go.

Solution:

The solution to this issue was easier than I thought. Since I was using mail package in my scenario, I started with changing the content type for the email and the attachments. And as I was using Java mapping, I was declaring each String within the program to be ISO 8859-1 encoded . Each of the combinations resulted in a more funnier email and weirder file.

Finally a bit of research pointed me to the fact that there was absolutely nothing wrong with the file or the email in the first place. This was just a feature of MS excel where it needs a CSV file to have a Byte Order Mark in the beginning for it to consider the file as a UTF-8 encoded one from the scratch.

So, here comes the solution. Just add a BOM at the beginning of the file

Code snippet below:- This code snippet would be helpful in any other case where you need to put in a BOM explicitly.

byte[] bom = new byte[] { (byte)0xEF, (byte)0xBB, (byte)0xBF };
String outputCSV = new String(bom,"UTF-8") + yourActualCSV;

Hope this helps some one in the community.

I have been really irritated at line segments in IDOCS. The build functions have a hard time to make sure it is selecting the right values. They can have a structure as bellow. I have removed values that I’m not using.

This allows for a lot of missing values that have to be tested for. This is the last mapping that I got to create. And it was failing because an element was missing the TDLine. So I knew that I had to find a much better solution to the following mapping. This mapping also caused problems on some data.

I may  have been able to fix it with some standard functions but it will probably end up being too big an issue.

So I wanted to create a more reliable and faster way to make it this type of lines. In my design effort I stumbled of the return as XML that gives the node with child notes as a value.

If this XML option did not exist I would have required 4 input the key input 2 times and the target value 2 times at different contexts. It would have been messy to work with but possible.

The return as XML is something that I not have been using. I have just on one occasion a few weeks ago. I could not see a good benefit of using it.  Here it makes a lot of sense, because it is return the full E1TXTH9 in one element. Then it is just to create a parser for it.

The configuration is simple.

• Key is the place where it should look for a value
• Allowed values is a comma separated list of allowed keys. If any of the values are found the current node and it parent will be selected.
• Fields to return. Name of the field to be returned.

All positive results will be added to a text string that is returned for the context. So there will only be one output for a queue. In the example above ZSSI does not exist in the input data, so it will not be returned.

The function can also be used on partners segments to find values. Both on header but more importantly on line items where it can be a lot more difficult to find the values.

The code is here

  @LibraryMethod(title="selectXMLValue", description="select values from a XML element", category="GF_EDIHELPERS", type=ExecutionType.ALL_VALUES_OF_CONTEXT)   public void selectXMLValue (  @Argument(title="node in return as XML")  String[] xmllist,  @Parameter(title="field where the key is located")  String keyfield,  @Parameter(title="comma seperated list of allowed values")  String keyvaluelist,  @Parameter(title="if the key is found which field values should be returned.")  String resultField,  ResultList result,  Container container)  throws StreamTransformationException{  try{  StringBuffer sb = new StringBuffer();  for (String xml : xmllist) {  if(xml.trim().length()>0){  String[] keyvalues = keyvaluelist.split(",");  DocumentBuilder docBuilder = DocumentBuilderFactory.newInstance()  .newDocumentBuilder();  Document document = docBuilder.parse(new ByteArrayInputStream(xml.getBytes("UTF-8")));  NodeList tagslist = document.getElementsByTagName(keyfield);  for(int i = 0; i< tagslist.getLength();i++) {  Node node = tagslist.item(i);  String nodename = node.getFirstChild().getNodeValue();  for (String keyvalue : keyvalues) {  if(nodename.equals(keyvalue)){  //add the content of the result field to the output  //look from the parrent node  Element parrentNode = (Element)node.getParentNode();  NodeList resultList = parrentNode.getElementsByTagName(resultField);  for(int j = 0; j< resultList.getLength();j++) {  Node resultnode = resultList.item(j);  sb.append(resultnode.getFirstChild().getNodeValue());  }  }  }  }  }  }  result.addValue( sb.toString());  }catch(Exception e){  throw new StreamTransformationException("Unable to parse xml", e);  }  }

Update: 20140508: with xml.getBytes("UTF-8") to avoid com.sun.org.apache.xerces.internal.impl.io.MalformedByteSequenceException

If you want you can also download the Figaf Tools package for free at

http://figaf.com/products/figaftools.html

Overview

Salesforce.com integration with your SAP ERP system is becoming a lot easier with the Advantco SFDC Adapter for SAP PI/PO. The SFDC adapter enables you to read data from Salesforce.com and to update your Salesforce.com data with information from your SAP backend systems. The SFDC Adapter is tightly integrated with the Salesforce API and Security Framework, freeing you from these basic tasks like schema generation, code integration and authentication. The Advantco SFDC Adapter supports BULK, Query and Streaming API for fast and real-time data retrieval and upload.

Figure1: Receiver channel for Object update

Figure2: Sender channel with Topic subscription.

Key features:

• Full integration with SAP PI Adapter Framework, Alert Framework and Monitoring.
• Supports Read/Create/Update/Delete operations to Salesforce objects
• Supports subscription to Salesforce push topics (Streaming API)
• Supports custom objects/fields
• XML/CSV data format supported for data updates
• Multiple parameters provided in queries for precise data selection
• Adjust communication method dynamically base on data volume to achieve best performance
• Workbench for Schema generation and Query validation.
• SAP Certification is scheduled for May 2014.
• Available for SAP PI 7.10, 7,11, 7.30, 7.31, 7.4

Key advantages:

• Significantly reduces implementation effort for integration with Salesforce.
• Easy configuration and setup Fully integrated with SAP PI/PO without need for external tools.

Confession: I HATE using the graphical RFC Lookup function!

Ok, ok... maybe that was a little too strong. In general I try to avoid using RFCs wherever possible due to the lack of trace/logging in the called system.

Even though the graphical RFC Lookup function has been available since PI 7.1, it has been tolerable at best, buggy at worst! There must have been countless times where I've executed "Display Queue" on the function only to get different results on each occasion even though everything remains the same. It would seem as though there is some cache between supposedly independent calls to the RFC. On other occasions, the input values from the mapping just don't get transferred to the RFC parameters (viewed while debugging the RFC call) for unknown reasons. I get to the point where once I get the RFC lookup to work correctly, I am very hesitant to make any future changes to that portion of the mapping.

However, one of the nastiest bugs I've encountered while using the RFC lookup function is one that causes infinite execution of the RFC. This bug is documented in SAP Note 1674730 with the following description:-

Symptom

Infinite number of calls are being triggered to an RFC from a standard RFC lookup function used in message mapping. This can cause memory issues and increased CPU utilization.

Granted that it is possible to avoid the pre-requisites that trigger this bug, but this a high-impact bug that can cause one message/interface to bring down an entire system! It is bewildering to me why there is no patch for this, and the solution in SAP note only suggests to prevent the problem from happening.

When this happens, you can see in the workload statistics of the server (via tcode ST03N) that the RFC calls are insanely high!

Well, life goes on and we just have to find a workaround to prevent this from happening....

To do this, we first need to know under what conditions this bug will be triggered.

More often than not, this happens when the RFC lookup occurs at the item level of the mapping, and we want to optimize it into a single call instead of multiple calls. Then the bug is triggered if the below condition is met:-

Number of values in Parent node is less than number of contexts in Child nodes

Following is an example using the function module SALERT_CREATE. It has a TABLES parameter IT_CONTAINER which allows unbounded input entries. This structure contains 5 fields.

The corresponding structure in the graphical lookup is shown below. Over here, item, ELEMENT and VALUE are configured as input parameters to the lookup function.

item is the Parent node, and ELEMENT& VALUE are the child nodes.

Therefore, we need to ensure that the number of values in item cannot be less than the number of contexts in ELEMENT& VALUE.

Incorrect	Correct
Number of values in item = 3 Number of contexts in ELEMENT = 5 Number of contexts in VALUE = 5	Number of values in item = 5 Number of contexts in ELEMENT = 5 Number of contexts in VALUE = 5

The incorrect entry on the left meets the trigger condition for the bug and the SALERT_CREATE RFC will be executed infinitely!

To PI or not to PI

I tend to end up in some interesting discussion, with other consultants, customers and my self. The last part is often a big problem.

I was working with a client, where we had to do and migration. From SAP business connector to SAP Process Integration. One thing that I realized this process is that all the development that we do, have to be changed at some point.

We do not know how long time is the key is going to support its appeal process integration, if there is going to be an upgrade path all the time. Or if our strategy is to move away from the PI system at some point. Therefore, whenever we develop something, it always have to be replaced later. It can be costly to do and migration from one platform to another. The people that did the original of development is no longer in the organization or may not know the new tool.

When we are developing SOA applications we all played saying that it is a great thing to have products developed using the SOA thought. SOA creates an infrastructure that allows a lot of reuse. But what happens down the line when no one knows what we have reusing. Who know what is going on 10 years from now.

One thing that I was considering was direct connections to SAP ERP. If for instance want to connect at portal to the ERP system. Then we have two options for doing this.

• We can connect directly to the SAP system from the portal. The benefit of this is less latency, less development, and an easy-to-understand architecture. The downside it is difficult to know what is going on, who is calling which services. The security is not optimal since it allow direct access to the ERP system from the DMZ.
• We can also connect through SAP PI, and expose the services using the ESR. By doing this we have a better way to secure who gets access to the SAP system, we can close firewall from the Internet. We have more flexibility of what we want to expose. And we have the whole SOA concepts that we can use. The downside is we have to have this system running all the time, and we have to upgraded whenever it’s needed. You are not going to change the SAP ERP system to ORACLE, so you just can change the connections with little effort.

I recorded video around some of the concepts, and thought that I had on direct connections vs thru SAP PI.

I think one important aspect is if we are dealing with web services, we should at least publish these in the ESR. So no matter what we want do we can always see where the services this exposed. We may not get the locking from the PI if we dealing with direct communication but we will get some more flexibility in other respects.

I still believe that using SAP process integration is still good use in some scenarios, way want to expose which services to a portal or a DMZ. We just have to consider some of the downsides of having an integration broker in in between.

Do you have any recommendations regarding connecting direct to ERP system or thru the PI system.