Skip to Content

Retry Message Processing in Case of an Error

test
0 %
Retry Message Processing in Case of an Error
Details

Retry Message Processing in Case of an Error

2020-02-26
Retry message processing in case of an error, instead of setting the message status to Failed or asking the sender to resend the message.

You will learn

  • How to use Enterprise Messaging
  • How to automate retry of message processing in case of an errors
  • How to model exactly once processing in SAP Cloud Platform Integration

Prerequisites

  • Enterprise Messaging License is bought: SKU 8005999.
  • JMS broker is provisioned - check here.

In the previous chapter, we simply sent an intimation of error through mail and expected the sender to resend the message in case of an error.

There are cases where the error is caused by a temporary reason and the processing usually succeeds after a while, like endpoint temporarily unavailable. In such case, the message need only be reprocessed as is and the processing succeeds.

There are other cases where the error is caused due to bad data or incorrect URLs. No number of retries can help process the message successfully without changing the incorrect data.

Automated retry mechanism will only be helpful in the first case.

In this chapter, we will model an automated retry mechanism.

To enable retry, we will store the failed message in a JMS queue and pick it up repeatedly to retry processing, until it is processed successfully. To accomplish this, we will perform the following steps:

  1. We shall park the incoming data in a property.

  2. In case of an exception, we shall write the message from the property to a JMS queue in the Exception Sub-process.

  3. Create a duplicate flow which reads from the queue and retries message processing. If the exception recurs, the message will not be deleted from the queue. Hence it will be retrieved over and over until processed successfully or placed in the Dead Letter Queue.


Step 1: Park incoming message in a property
  1. Add a Content Modifier immediately after the Start Event:
    • Click on the design Palette.
    • Choose Message Transformers.
    • Click on Content Modifier.
    Add Content Modifier
    • Drag it to the message execution pipeline between the Message Start Event and the Content Enricher step.
    Drag Content Modifier
    • Rename the Content Modifier - click on the Content Modifier step, go to the Properties Sheet –> General tab
      Set the Name to Park incoming message.

    • Go to Exchange Property tab, click on Add and enter the following details:

    Field Value
    Action Create
    Name Input
    Type Expression
    Datatype java.lang.String
    Value ${in.body}
    Default
    Park Input
Log on to answer question
Step 2: Write incoming message to JMS Queue for exceptions

In this step, we assume that an exception has happened. We will now store the incoming message into a JMS queue. For this purpose, we cannot take the message on the pipeline as we can never be sure where an exception can occur. So we need to use the message that we parked in the previous step - this way we are always sure that we are working on the incoming message and can duplicate the entire processing while we retry.

  1. Load the initial message from the property into the body when an exception occurs:

    • Click on the design Palette.
    • Choose Message Transformers.
    • Click on Content Modifier.
    Choose Content Modifier
    • Drag it into the Exception Sub-process, just after the Error Start Event.

    • Click on the Content Modifier, go to the properties sheet and rename it to Retrieve incoming message into body.

    Rename Content Modifier 2
    • Go to the Body tab and choose the following:
    Field Value
    Type Expression
    Body ${property.Input}
    Assign input
  2. Add a JMS receiver in the Exception sub-process.

    • Click on the design Palette.
    • Choose Call.
    • Choose External Call.
    • Click on Send.
    Choose RequestReply
    • Drag it into the Exception Sub-process, just after the Content Modifier.
    Add RequestReply
    • Click on the design Palette.
    • Choose Participant.
    • Choose Receiver.
    Choose Receiver
    • Drag the receiver below the Request Reply step as shown below:
    Drag Receiver
    • Click on the receiver, go to the properties sheet and rename the receiver to JMSQueue.
    Rename Receiver
    • Hover over the Request Reply step use the Connector to connect it to the JMSQueue receiver. Choose JMS from the pop-up.
    JMS Connector
  3. Configure the JMS connector:

    Add the following configuration to the JMS receiver in the Processing tab:

    Field Value
    Queue Name Input
    Retention Threshold for Alerting (in d) 2
    Expiration Period (in d) 90
    Encrypt Stored Message Checked
    Transfer Exchange Properties Unchecked
Log on to answer question
Step 3: Save, deploy and execute the original flow
  1. Save and deploy the integration flow:

    Save Deploy flow 1

    As soon as you deploy the flow, a queue shall be created and this can be see in the Queue Monitor:

    Go to the Operations view –> Manage Store section and click on Manage Queues:

    Open Queue View

    As you can see, a single queue is created. The name of the queue is Input - same as what was configured in the JMS receiver.

    Queue View

    There is currently no data in the queue.

    Let us now execute the flow to write a message into this queue.

  2. Execute the integration flow and check the queue view again:

    Queue View

    We now see one entry in the queue.

  3. Check the configured inbox. You should not have received any mails yet. This indicates that the flow did not complete processing.

    Now let us create a retry logic.

Log on to answer question
Step 4: Create duplicate flow for reading messages from JMS queue

In this exercise, we shall copy the original flow and make the following changes in it:

  • Replace the SOAP sender with the JMS sender: this is because, we now want the integration flow to poll the JMS queue, not expose a SOAP endpoint. This flow shall poll the queue after regular intervals as configured in the JMS channel in the next exercise. Every message entered into the queue through the exception sub-process of the original flow shall be retried as a result of the above process.

  • The Exception sub-process shall be removed - this is because, when an entry is read from the queue, it is deleted. In case an exception occurs, the entry will be written back into the queue (or not deleted). This will make sure that it is retried and later put in the dead-letter queue in case the process fails to process successfully in the configured number of retries.

  • The exception inducing script should be removed - this makes sure that when the message is retried, the exception is not simulated and hence we should be able to process the message successfully in the first retry.

  1. Copy the integration flow:

    • Go to the Integration Package view:
    Package view
    • Click on Edit. Go to Actions for the integration flow Manage Freight Logistics and choose Copy:
    Copy Flow
    • In the pop-up dialog, enter Manage Freight Logistics_retry as the name of the new integration flow:
    Name Flow

    Click Copy.

    The corresponding integration flow gets added to the integration package:

    Flow added
  2. Edit the retry flow:

    1. Open the retry flow and click Edit:

      Edit Flow
    2. Delete the sender SOAP channel:

      Click on the channel and click on the delete speed button:

      Delete SOAP
    3. Hover on the e-CommerceVendor sender, use the speed button for Connector and connect the sender to the Message Start Event:

      Add JMS
    4. Delete the script step - hover on the Script step and choose the speed button for delete:

      Script Delete
    5. Remove the Exception Sub-process: delete the following:

      Delete Exception subprocess
      • Exception Subprocess 1
      • JMSQueue
      • Receiver 1

    The final integration flow looks like the picture below:

    New flow
Log on to answer question
Step 5: Configure retry setting in JMS queue

Configure the JMS sender as follows:

Field Value Description
Queue Name Input Name of the queue to be read
Number of Concurrent Processes 1 How many concurrent processes must read from the queue
Retry Interval (in min) 1 After how many minutes should the process poll the queue
Exponential Back-off Checked When this checkbox is checked, the retry interval is doubled after every retry. This ensures that in case the issue is caused by something that does not resolve quickly, the retry does not burden the system
Maximum Retry Interval (in min) 60 What is the maximum interval after which retry should not be performed. This is only relevant when Exponential back-off is selected
Dead-Letter Queue Checked This is used to place entries that do not succeed after 2 retries in a separate queue, called the Dead-Letter queue
Log on to answer question
Step 6: Simulate and run a retry
  1. Save and deploy the retry integration flow:

    Save Deploy flow

    As soon as the flow is deployed, it starts polling the queue for messages. As a result, the message from the queue should now be picked by the retry integration flow and processed.

  2. Check the queue view: since the message is now picked, we should not see any messages in the queue:

    Queue View 2
  3. Since we do not raise an exception in the retry flow, this time, the flow will execute successfully and you should receive an email in the configured inbox.

    Mail received
What needs to be provisioned when you need to start using Enterprise Messaging in SAP Cloud Platform Integration?
×

Next Steps

Back to top