Creating Data Streams Using Salesforce CRM Connector | Data Cloud Decoded
Introduction
Welcome to the second episode of our Data Cloud video series, where we discuss the process of creating data streams using the Salesforce CRM connector. In this episode, we explore the essential steps involved in connecting data sources and setting up data streams within Salesforce Data Cloud.
Data Cloud Video Series: Episode 2 - Connecting Data Sources and Data Ingestion
-
Welcome to the second episode of our Data Cloud video series, where we delve into the process of connecting data sources and data ingestion.
-
In the previous episode, we provided a high-level overview of how Data Cloud works. Now, we will focus on the first step in the process, which is connecting your data sources and bringing in the data.
-
The process of bringing in data from different sources involves two main phases: data ingestion and data modeling, also known as harmonization.
-
During the data ingestion phase, data from a source is brought in ‘as is’, meaning that the fields and their data types are imported without any transformation.
-
To achieve this, a connection between the data source and Data Cloud needs to be established using a connector. Salesforce offers a variety of out-of-the-box connectors to simplify the integration process with external systems.
-
Once a connector is set up, a data stream is created to retrieve different data sets from the source system. Each data stream writes its data to a corresponding Data Lake object (DLO).
-
After the data is written to the Data Lake object, it is refreshed at regular intervals, and each stream may have its own refresh schedule based on the source system.
-
For example, data from a Salesforce org is incrementally updated every hour and fully refreshed every other week.
-
Once the data has been ingested, the process moves on to the data modeling phase, where the data inside a Data Lake object is mapped to different
Data Model Objects in Customer 360
-
The customer 360 data model consists of data model objects, which play a crucial role in data organization and management. These objects are the building blocks of the data model and are central to understanding the data modeling phase.
-
Data model objects are accessible through the data Cloud app, where users can navigate to the data model tab to view and create relationships between different objects. This allows for a comprehensive understanding of the data structure and its interconnections.
-
Harmonizing the data involves utilizing the identity resolutions tab to establish matching and Reconciliation rules. This process unifies records and ensures data consistency, laying the foundation for effective data analysis and utilization.
-
Once the data has been harmonized, users can leverage the data Explorer and Profile Explorer tabs to visualize the ingested and unified data. This offers transparency and insights into the quality and completeness of the data.
-
Furthermore, the calculated insights tab enables the creation of custom metrics using SQL queries. These metrics can provide valuable information such as customer lifetime loyalty and popular products, empowering businesses to make informed decisions based on data-driven insights.
-
Segmentation and action-driving capabilities are facilitated through the segments and data actions tabs, where users can filter individuals into segments and trigger events based on the ingested data or calculated insights.
Connecting and Setting Up Data Streams in Salesforce Data Cloud
-
In Salesforce Data Cloud, events can be used to trigger automations downstream. For instance, data actions can fire a platform event that then triggers a flow or an apex class.
-
To set up data Cloud features like connectors, users can click on the setup icon and navigate to the Data Cloud setup. This provides a glimpse into the user interface.
-
To ingest data from a Salesforce org, a fictitious company called Solar Circles, which manufactures and sells solar panels, needs to connect their org to data Cloud. This org has already been provisioned with data Cloud, and now they want to fetch data from another org used by their manufacturing department.
-
The first step is to connect the org to data Cloud from the Data Cloud setup. Clicking on ‘Salesforce CRM’ reveals the different orgs that are already connected.
-
To connect to a new org, users can click ‘New’. There are two types of orgs to connect to: the home org, which is the current org where data Cloud is provisioned, and external orgs, including sandboxes. After connecting the home org, the ‘Connect’ button no longer shows up. However, clicking ‘Connect’ allows users to connect to an external org such as the manufacturing org for Solar Circles.
-
After entering the credentials of the org and logging in, users will be prompted to allow access. Once this is done, the connection is created, and the new org is visible in the list. Users can optionally rename this connection for easier identification.
-
Next, users need to set up a data stream for data to start flowing in from the connected org. From the Data Cloud app, they can click on ‘Data Streams’ to view all the different data streams.
Setting Up Data Streams in Salesforce Data Cloud
-
When configuring streams in Salesforce Data Cloud, you can view the last refresh time and the number of processed records for already configured streams.
-
To create a new stream, click on ‘New’ and you will be presented with a variety of connectors to choose from.
-
Select ‘Sales’ for CRM and proceed to choose the organization you want to extract data from. The list will display the orgs that are already connected. In this case, let’s choose ‘Solar Circles Manufacturing’.
-
After selecting the organization, you have the option to choose a data bundle or select a single object. A data bundle creates multiple data streams and corresponding data mappings for predefined objects.
-
Salesforce provides standard bundles, and you can also create custom bundles or download them from the App Exchange.
-
For this guide, let’s choose a single object. A table will then display all the objects available in the org. In this example, the ‘Contact’ object is chosen for import.
-
The next step requires you to choose a category for the selected object. There are three categories: ‘Profile’, ‘Engagement’, and ‘Other’.
-
Select ‘Profile’ if the data set contains information about a person, business, or account. Choose ‘Engagement’ if the data set contains behavioral or engagement data, always time-stamped with a date and time field. If the data doesn’t fit into either category, choose ‘Other’.
-
Certain features in Data Cloud work only on specific object categories. For instance, segments can only be created on profile data, and querying with a time filter is restricted to engagement data.
-
It’s important to note that the category selected for an object cannot be changed after saving.
Data Stream Review and Decision Making
-
When setting up a data stream, it is important to carefully review all the available types and make an informed decision.
-
In the data stream setup, the first step is to choose the type of contact identification. For this, selecting ‘profile’ is recommended as it identifies a person.
-
After selecting the contact identification, the next step is to choose the fields that need to be imported. By default, all the fields are checked, but the user can deselect any fields not required.
-
Once the field selection is done, the user can proceed to the next step. Here, the data stream needs to be given a name. It is also possible to choose a pre-created name, if available. After naming the data stream, the user can click on ‘deploy’ to initiate the setup process.
-
Upon deployment, the user is directed to the detail page of the newly created data stream. Here, the name of the data lake object associated with the stream is displayed. All the incoming data from the stream is inserted into this data lake object.
-
With the Salesforce CRM data stream, data Cloud performs a full refresh of the data during deployment or modification. Additionally, hourly upsets are conducted by data Cloud to ensure data currency.
-
Regular full refreshes, occurring every other week, are automatically performed by data Cloud, requiring no action from the user. However, manual refresh of CRM data streams can be initiated using the refresh button.
-
Following the data import, the user can view the refresh date and the total number of records processed. The refresh history tab provides a comprehensive log of all data refresh instances, distinguishing between full refresh and updates.
-
To inspect the imported records, the user can navigate to the data Explorer tab. Here, the data lake object is chosen as the object type, and the specific data lake object (DLo) to be queried is selected. The records contained within the chosen DLo are then displayed on the screen.
-
In upcoming episodes, the utilization of other connector types such as marketing Cloud ingestion API and web and mobile SDK will be explored. Viewers are encouraged to stay tuned for these insights.
-
If you found this video helpful, please show your support by giving it a thumbs up. For more valuable content, don’t forget to subscribe to our Salesforce Developers YouTube channel and enable notifications for new episode releases.
Conclusion
In this episode, we’ve covered the fundamental process of establishing data streams using the Salesforce CRM connector in Salesforce Data Cloud. By following these steps, you can effectively connect data sources and initiate data ingestion, paving the way for comprehensive data analysis and utilization.