You decided you want to analyze the data you have on Intercom and do that on Google BigQuery. This post aims to help you define a pipeline and load data from Intercom to Google BigQuery for further analysis. Alternatively, you can use RudderStack that can automatically handle this kind of problem for you.
You will learn to access and extract your data from Intercom through its API and how to load it into BigQuery. This process requires you to write the code to get the data and ensure that it will run every time new data is generated.
Extract your data from Intercom
Intercom exposes a rich REST API for interacting with its services. There are two reasons someone would like to use the Intercom REST API:
To pull data from it and use it for analytic purposes.
To push data to enrich the information it contains for customers and enhance its service.
This post will focus mainly on pulling data from the API and using it to enrich our data warehouse with data generated from our interactions with our customers.
The main entities of the Intercom API are the following:
Users – a primary way of interacting with Intercom.
Leads – represent logged-out users of your application.
Companies – allow you to represent commercial organizations using your product.
Tags – A tag allows you to label your users and companies and list them using that tag.
Segments – A segment is a group of your users defined by rules that you set.
Notes – Notes allow you to annotate and comment on your users.
Events – Events are how you can submit user activity to Intercom.
Counts – You can get counts of users and companies filtered by specific criteria.
Conversations – Conversations are how you can communicate with users in Intercom.
Note that you cannot pull out all of the above entities from the Intercom API. For example, you can push Events inside Intercom; however, it’s challenging to extract them again. So if you plan to use Intercom to track your users’ behavior, keep that in mind because contrary to services like Mixpanel, it’s not possible to pull the user events from the system. An excellent strategy to ensure that your user activity is pushed on all the different services and that you will always have access to the data is to use a service like Segment.
Intercom Exposes a RESTful Web API, which means that we interact with its resources through HTTP verbs like POST and GET by using an HTTP client. Intercom also offers several SDKs built around an HTTP client for a number of popular languages:
To pull your data out of Intercom you need to fetch all your users together with all their conversations. You can then load this information to your data warehouse and enhance your analytic capabilities with other interactions with them. To do that, you first need to get all your users, with CURL, in the following way: