By Rudderstack Team

How to load data from Google AdWords to SQL Data Warehouse

Marketing is becoming heavily data-driven. How can you get all marketing data like Facebook Ads or Bing Ads or Google AdWords in one place like SQL Data Warehouse? This post is a guide to help you define a process or pipeline, to load data from Google AdWords to SQL Data Warehouse for further analysis.

We will see how to access and extract data from Google AdWords through its API and load it into SQL Data Warehouse. This process requires you to write the code to get the data and ensure that it will run every time new data are generated. Alternatively, you can use RudderStack that can handle this kind of problem automatically for you.

About Google AdWords

Google AdWords is an online advertising service by Google for businesses wanting to display ads on Google and its advertising network. At its core, Google AdWords is a Real-Time Bidding system where advertisers compete to display their advertising material to web users using Google products like its search engine. Programmatic and instantaneous auctions are performed, similar to how financial markets operate. Among the benefits of AdWords are:

  • Pay-per-click – advertisers pay only for ads that the user has clicked.
  • Any budget – You can start with any budget, although you have to be aware of AdWords’ Real-Time Bidding nature, which means that the effectiveness of your campaigns is linked to what your competitors are also willing to pay.
  • Reach – you can reach billions of people worldwide.

Additionally, Google AdWords, just like every other product from Google, has excellent support, and it exposes a rich ecosystem of tools and APIs that you can use to get the most out of their services.

About Microsoft Azure SQL Data Warehouse

SQL Data Warehouse is the data warehousing solution you can use if you are a user of Microsoft Azure. It’s an elastic data warehousing as a service solution, emphasizing its enterprise focus. It also speaks SQL like the previous two solutions, and it supports querying both relational and non-relational data. It offers a number of enterprise-class features like support for hybrid cloud installations and strong security. It’s probably the less mature solution compared to the two others though, it’s still in “Preview” mode although accessible to existing Azure subscribers.

Extract your data from Google AdWords

The AdWords API allows applications to interact directly with the AdWords platform. You can build applications to more efficiently manage large or complex AdWords accounts and campaigns. Contrary to the rest of the APIs that we have covered in this series of posts, the Google AdWords API is implemented only using the SOAP protocol. It doesn’t offer a RESTful web implementation.

Nevertheless, they offer a number of client libraries that you can use for your language or framework of choice. They officially support clients in the following languages:

  • Java
  • .Net
  • PHP
  • PERL
  • Python
  • Ruby

The AdWords API is a complex product that exposes a lot of functionality to the user, ranging from reporting to do the bidding and programmatic advertisement. As this post’s scope is the extraction of data from it, we’ll focus only on that part of the Google AdWords API to load the data to a data warehouse for further analysis.

There are many ways of interacting with the data that adWords API gathers. One way is to link your Google Analytics and AdWords accounts and actually enrich your analytics data with data coming from AdWords. If you have the luxury to afford a Google analytics premium account, the other possible way is to load your data directly to Google BigQuery. From there, you can either do your analysis from BigQuery or export your data to another data warehouse.

We’ll assume that you do not have a Google Analytics premium account, to be honest, if you had you wouldn’t be looking at this post anyway, but you still want to extract data and load it to your own data warehouse solution. To do that, we’ll utilize the Report related functionality of the AdWords API. The API supports a huge number of reports that you can request, and it is possible to change the granularity of your results by passing specific parameters. Defining what kind of data you want to get back as part of your report can be done in two different ways.

  • Using an XML-based report definition.
  • Using an AWQL-based report definition.

If you want to use an XML-based report definition you have to include a parameter named “__rdxml” that will contain an XML serialized definition of the report you want to retrieve.

<reportDefinition xmlns="https://adwords.google.com/api/adwords/cm/v201509