In the previous chapter, we simply sent an intimation of error through mail and expected the sender to resend the message in case of an error.
There are cases where the error is caused by a temporary reason and the processing usually succeeds after a while, like endpoint temporarily unavailable. In such case, the message need only be reprocessed as is and the processing succeeds.
There are other cases where the error is caused due to bad data or incorrect URLs. No number of retries can help process the message successfully without changing the incorrect data.
Automated retry mechanism will only be helpful in the first case.
In this chapter, we will model an automated retry mechanism.
To enable retry, we will store the failed message in a JMS queue and pick it up repeatedly to retry processing, until it is processed successfully. To accomplish this, we will perform the following steps:
-
We shall park the incoming data in a property.
-
In case of an exception, we shall write the message from the property to a JMS queue in the Exception Sub-process.
-
Create a duplicate flow which reads from the queue and retries message processing. If the exception recurs, the message will not be deleted from the queue. Hence it will be retrieved over and over until processed successfully or placed in the Dead Letter Queue.
Step 1: Park incoming message in a property
- Add a Content Modifier immediately after the Start Event:
- Click on the design Palette.
- Choose Message Transformers.
- Click on Content Modifier.
- Drag it to the message execution pipeline between the Message Start Event and the Content Enricher step.
-
Rename the Content Modifier - click on the Content Modifier step, go to the Properties Sheet –> General tab
Set the Name to Park incoming message.
-
Go to Exchange Property tab, click on Add and enter the following details:
Field |
Value |
Action |
Create |
Name |
Input |
Type |
Expression |
Datatype |
java.lang.String |
Value |
${in.body} |
Default |
|
Step 2: Write incoming message to JMS Queue for exceptions
In this step, we assume that an exception has happened. We will now store the incoming message into a JMS queue. For this purpose, we cannot take the message on the pipeline as we can never be sure where an exception can occur. So we need to use the message that we parked in the previous step - this way we are always sure that we are working on the incoming message and can duplicate the entire processing while we retry.
-
Load the initial message from the property into the body when an exception occurs:
- Click on the design Palette.
- Choose Message Transformers.
- Click on Content Modifier.
-
Drag it into the Exception Sub-process, just after the Error Start Event.
-
Click on the Content Modifier, go to the properties sheet and rename it to Retrieve incoming message into body.
- Go to the Body tab and choose the following:
Field |
Value |
Type |
Expression |
Body |
${property.Input} |
-
Add a JMS receiver in the Exception sub-process.
- Click on the design Palette.
- Choose Call.
- Choose External Call.
- Click on Send.
- Drag it into the Exception Sub-process, just after the Content Modifier.
- Click on the design Palette.
- Choose Participant.
- Choose Receiver.
- Drag the receiver below the Request Reply step as shown below:
- Click on the receiver, go to the properties sheet and rename the receiver to
JMSQueue
.
- Hover over the Request Reply step use the Connector to connect it to the
JMSQueue
receiver. Choose JMS from the pop-up.
-
Configure the JMS connector:
Add the following configuration to the JMS receiver in the Processing tab:
Field |
Value |
Queue Name |
Input |
Retention Threshold for Alerting (in d) |
2 |
Expiration Period (in d) |
90 |
Encrypt Stored Message |
Checked |
Transfer Exchange Properties |
Unchecked |
Step 3: Save, deploy and execute the original flow
-
Save and deploy the integration flow:
As soon as you deploy the flow, a queue shall be created and this can be see in the Queue Monitor:
Go to the Operations view –> Manage Store section and click on Manage Queues:
As you can see, a single queue is created. The name of the queue is Input - same as what was configured in the JMS receiver.
There is currently no data in the queue.
Let us now execute the flow to write a message into this queue.
-
Execute the integration flow and check the queue view again:
We now see one entry in the queue.
-
Check the configured inbox. You should not have received any mails yet. This indicates that the flow did not complete processing.
Now let us create a retry logic.
Step 4: Create duplicate flow for reading messages from JMS queue
In this exercise, we shall copy the original flow and make the following changes in it:
-
Replace the SOAP sender with the JMS sender: this is because, we now want the integration flow to poll the JMS queue, not expose a SOAP endpoint. This flow shall poll the queue after regular intervals as configured in the JMS channel in the next exercise. Every message entered into the queue through the exception sub-process of the original flow shall be retried as a result of the above process.
-
The Exception sub-process shall be removed - this is because, when an entry is read from the queue, it is deleted. In case an exception occurs, the entry will be written back into the queue (or not deleted). This will make sure that it is retried and later put in the dead-letter queue in case the process fails to process successfully in the configured number of retries.
-
The exception inducing script should be removed - this makes sure that when the message is retried, the exception is not simulated and hence we should be able to process the message successfully in the first retry.
-
Copy the integration flow:
- Go to the Integration Package view:
- Click on Edit. Go to Actions for the integration flow Manage Freight Logistics and choose Copy:
- In the pop-up dialog, enter
Manage Freight Logistics_retry
as the name of the new integration flow:
Click Copy.
The corresponding integration flow gets added to the integration package:
-
Edit the retry flow:
-
Open the retry flow and click Edit:
-
Delete the sender SOAP channel:
Click on the channel and click on the delete speed button:
-
Hover on the e-CommerceVendor sender, use the speed button for Connector and connect the sender to the Message Start Event:
-
Delete the script step - hover on the Script step and choose the speed button for delete:
-
Remove the Exception Sub-process: delete the following:
Exception Subprocess 1
JMSQueue
Receiver 1
The final integration flow looks like the picture below:
Step 5: Configure retry setting in JMS queue
Configure the JMS sender as follows:
Field |
Value |
Description |
Queue Name |
Input |
Name of the queue to be read |
Number of Concurrent Processes |
1 |
How many concurrent processes must read from the queue |
Retry Interval (in min) |
1 |
After how many minutes should the process poll the queue |
Exponential Back-off |
Checked |
When this checkbox is checked, the retry interval is doubled after every retry. This ensures that in case the issue is caused by something that does not resolve quickly, the retry does not burden the system |
Maximum Retry Interval (in min) |
60 |
What is the maximum interval after which retry should not be performed. This is only relevant when Exponential back-off is selected |
Dead-Letter Queue |
Checked |
This is used to place entries that do not succeed after 2 retries in a separate queue, called the Dead-Letter queue |
Step 6: Simulate and run a retry
-
Save and deploy the retry integration flow:
As soon as the flow is deployed, it starts polling the queue for messages. As a result, the message from the queue should now be picked by the retry integration flow and processed.
-
Check the queue view: since the message is now picked, we should not see any messages in the queue:
-
Since we do not raise an exception in the retry flow, this time, the flow will execute successfully and you should receive an email in the configured inbox.