Publish Salesforce CDC changes to Azure

Use Case: You have Salesforce with MuleSoft and want to publish Contact changes to a Partner business who is established on the Microsoft Azure cloud, and are currently upgrading their CRM systems.

The Microsoft Azure Service Bus provides a reliable, scalable and cost effective mechanism for handling high volumes of messages between systems.

By leveraging Microsoft Azure Service Bus, you can receive changes raised by Salesforce CDC (Change Data Capture) and store them on the partner’s Queue for use when they are ready. If they are transitioning systems then this has the additional benefit that your integration is not impacted:

The benefits from this design are:

1. Very simple to build and configure end to end. It can be a no-code solution.
2. Very cost effective to use Azure ServiceBus.
3. Reliable. Messages successfully delivered to the Azure Service Bus can be managed by the Partner as they require
4. Avoids Point-to-Point : Your integration with the Partner is not hard set between Salesforce and Dynamics. The Partner can consume the messages in any system they want, and even multiple systems at the same time should they choose

Disadvantages include:

1. You may hit limits at high data change volumes from Salesforce: Change Data Capture Allocations | Change Data Capture Developer Guide | Salesforce Developers
2. Salesforce CDC events can only be picked up for 24 hours — if you have delivery issues then you will need to store changes in MuleSoft somehow (for example, implementing an outbound queue) or use another method to update the Service Bus after that timeframe

Security note: This POC doesn’t include much security at this point, using Salesforce basic authentication and a simple key for Azure. You would want to look at OAuth and secure network channels. This might be for a future article.

The video below shows editing a telephone number in a contact record in Salesforce (top left), having that picked up my MuleSoft Anypoint (lower screen) and then picked up on the Microsoft Azure Service Bus (top right):

+++++++++++++++++++++

How to build:

Step 1: Configure Salesforce

In a Developer Org, go to ‘Change Data Capture’ and choose the ‘Contact’ object (although you can choose whatever you want).

Step 2: Configure Microsoft Azure

Creating a ‘Basic’ level Service Bus is sufficient for the POC (no issue going for ‘Standard’ if you want to try out ‘Topics’ though):

Create one queue (or a Topic) and make note of its name — this will be the destination.

Go to the Shared access policies and copy the ‘Primary Key’:

Step 3: Create MuleSoft Anypoint App

This is the simple Flow that we are creating. To pickup the CDC events, we need to setup the Replay channel listener element. Basic authentication for Salesforce is sufficient for the POC, and if you have configured CDC correctly then it will list the streaming channels available to you:

Add the Azure Service Bus Messaging Connector from Exchange, and add the ‘Send’ element to the Flow. For ease of setup, use the SAS (shared access signature) credentials from Azure Service Bus earlier. ‘Shared access key’ below is the ‘Primary Key’ defined in Azure earlier:

If the configuration is correct, then you will be able to select all available queues (and Topics) defined on the Service Bus under ‘Destination name’. Then add ‘payload’ to the message body:

Here we are simply passing the payload from the Salesforce CDC. Ideally we would use DataWeave to break it down the relevant parts into a new simple JSON payload.

Step 4: Add a ‘’Logger’ element

This is just to output the payload to the console. Optional but very simple to add.

Step 5: Run it

Run you App locally in Anypoint Studio debug mode with the console screen open (in order to see the record changes coming through. Go to any Contact record is Salesforce and change a field value (such as telephone). This should then pass all the way through and sit on the Azure Service Bus message queue.

Notes:

1. The intention of this POC is to show the message passing through. Security must be considered, ideally by securing the ServiceBus behind an Azure Private End Point.
2. I didn’t show the Azure Logic Apps. That would be interesting to try out, but not required for demonstrating the Service Bus
3. It should be possible to skip MuleSoft and build a CometD client on Azure somehow, but it looks like that would require some custom build effort
4. MuleSoft have a Contact broadcast template, which has similarites with my example use case — mulesoft-assets/template-sfdc2msdyn-contact-broadcast: Anypoint Template: Salesforce to MS Dynamics Contact Broadcast (github.com)
5. You can still do Point-to-Point integration between Salesforce and Dynamics: Microsoft Dynamics CRM Connector 3.2 — Mule 4 | MuleSoft Documentation . MuleSoft also provide a bi-directional sync if you wanted — mulesoft-assets/template-sfdc2msdyn-account-bidirectional-sync: Anypoint Template: Salesforce and MS Dynamics Account Bidirectional Sync (github.com)