Tag Archives: dfp_reports

Changes to DFP Reporting, now with more deprecation!

After our last round of spring cleaning, we've gone back to the drawing board to take another look at how we could make reporting better. There currently is an abundance of Dimensions, DimensionAttributes, and Columns (and more coming with each release), so in an effort to simplify the list of fields, we will be sunsetting the following reconciliation-related dimension attributes / columns in all versions. This will happen on September 1, 2015.

Columns with equivalent replacements:

Columns without equivalent replacements:

While the first two have equivalent replacements, the latter ones are not being replaced, as they don’t exist in core product reporting either.

This means that usage of these columns / dimension attributes will begin throwing errors in all versions starting September 1st. If your network is actively using any of these, please update your reports to switch to the supported fields or remove them entirely. If you have any questions, comments, or concerns about this, you know where to reach us!

Important upcoming breaking changes to DFP reporting

Now that it’s spring again (in the Northern Hemisphere at least), it’s time for DFP’s annual spring cleaning! In this edition, we’ll be doing some pruning of our ReportService. What does this mean for you? We’re sunsetting some reporting dimensions, attributes, and metrics in existing versions (before the version is fully sunset), so your reports will break if you don’t migrate before the shutoff dates. I know what you’re wondering: “should I panic?”. Absolutely not. This type of behavior rarely occurs, so as long as you phase out usage for these particular fields, you should be fine moving forward.

Merged Metrics

Remember when Doubleclick for Publishers was called DART? I, too, get nostalgic about our old ad server, but it’s been a couple of years since we transitioned to the new DFP platform, and it’s just about time when the merged reporting columns are no longer useful (these columns only existed so you could continue reporting on delivery that spanned DART and DFP). In all versions after v201502, we will no longer provide merged reporting columns and dimension attributes in the API, that is, anything starting with 'MERGED_' or contains '_LIFETIME_MERGED_.' After August 1, 2015, these columns and dimension attributes will stop returning data entirely and will return INVALID_COLUMNS in all versions that still include them.

There are three scenarios in which you’re using these columns:

  1. Just for fun.
  2. Because you forgot you’re using them.
  3. Because you have lifetime line items that have carried over from DART (in which case you’ll have to recreate these). To give you an example, if the metric you care about is impressions, you can get the DART delivery portion by subtracting the portion of delivery from DFP Premium (AD_SERVER_IMPRESSIONS) from the MERGED value (MERGED_AD_SERVER_IMPRESSIONS) which represents the aggregate DART and DFP Premium volume. Additionally, you should make the switch to the non-merged columns and dimension attributes as soon as possible.

Dimension Filters

But wait, there’s more! Our next API version (v201505) will be the last to support some of our infrequently used dimensionFilters.

  • MOBILE_LINE_ITEMS
  • WEB_INVENTORY_UNITS
  • MOBILE_INVENTORY_UNITS
  • WHOLE_NETWORK
  • PARTNER_STATS_TYPE_ESTIMATED
  • ACTIVE_ADVERTISERS
  • PARTNER_STATS_TYPE_RECONCILED
  • WEB_LINE_ITEMS
  • ALL_SALESPEOPLE

In each of the cases above, the filters either no longer provide meaningful information (as is the case with mobile vs. web line items and ad units with platform unification complete), or weren’t being used at all.

Similar to the changes above, after August 1, 2015, these dimension filters will return an INVALID_DIMENSION_FILTERS error in any version that still includes them.

So if you’re using any of the reporting features above, consider this an early heads up (and an opportunity) to refactor some of your code for spring cleaning.

As usual, if you have any questions, comments, or concerns, don’t hesitate to let us know on the forums.

Deprecation of active view NOT_* report columns in the DFP API

In the coming weeks, we will be deprecating all ACTIVE_VIEW_NOT* report columns in v201405, v201403, v201311 and v201306 of the DFP API. These columns are no longer supported in the DFP query tool, and the DFP API is following suit. The following columns will be affected:

Column.TOTAL_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.TOTAL_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.AD_SERVER_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.AD_SERVER_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.ADSENSE_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.ADSENSE_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.AD_EXCHANGE_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.AD_EXCHANGE_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Migration
Normally all features are supported until the API version is sunset. This deprecation is a rare case where these report metrics are being disabled for existing versions due to product changes related to viewability. If you are using v201408 or later, you will not be affected by this deprecation. If you are currently using these columns in v201405 or earlier, you can replace them with their logical opposites. Alternatively, to retain the same metrics, you can calculate them from the logical opposite and the rate. For example, Column.TOTAL_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS is equivalent to

Deprecation errors
If you do not migrate, your reports will return the following error:

ReportError.COLUMNS_NOT_SUPPORTED_FOR_REQUESTED_DIMENSIONS

If you have any questions or migration troubles, please reach out to us on our developer forum.

Filtering report data by custom targeting key ID in the DFP API

A lot of our DFP API developers have been asking recently about how to filter report data by custom targeting key ID. Currently the DFP API allows you to filter report data by custom targeting value ID only. Until we have official support for filtering by custom targeting key ID in reports, you can use the CustomTargetingService and the ReportService together to achieve this goal.

Step 1: Use CustomTargetingService to get your keys and values

You will first need to use getCustomTargetingValuesByStatement and filter by the custom targeting keys you’re interested in to obtain all the corresponding values. For example:

    WHERE customTargetingKeyId IN (17, 18, 19)

If you have a lot of keys and values in your network, a better approach is to store these in a local database and do nightly syncs. Use getCustomTargetingKeysByStatement to obtain all the keys in your network, and then iterate through them, calling getCustomTargetingValuesByStatement for each key to obtain their values. Our client libraries all have examples of this. For instance, the Java example can be found in our ads Java client library GitHub repository. This way, you can look up the values associated with a custom targeting key more quickly and not do an additional API call.

Step 2: Use the values in a report query

Once you have the values for the custom targeting key you’re interested in, you can then use the ReportQuery.statement with the PQL IN clause to filter on the CUSTOM_TARGETING_VALUE_ID dimension. For example, say you were interested in filtering on custom targeting key ID of 7777. You look up the values of 7777 in Step 1, which gives you three value IDs: 3211, 88990, 123456. You can then use the IDs to effectively filter your report data by custom targeting key ID 7777.

    WHERE CUSTOM_TARGETING_VALUE_ID IN (3211, 88990, 123456)

However, please be aware that if you have a lot of custom targeting value IDs to filter on, you should batch them by querying for no more than 500 IDs at a time in the PQL IN clause. For example, you will run your report filtering on the first 500 IDs you’ve collected and save that report. Then you will run the same report on the next page of 500 IDs you’ve collected and so on until you have no more IDs. You can then combine the reports locally so that you have all the data for those custom targeting IDs.

If you have any questions about this, feel free to drop us a line on the DFP API forums or Ads Developer Google+ page.

New PQL tables in the DFP API

In the recent DFP API releases, we announced the addition of more tables to the PublisherQueryLanguageService, starting with Line_Item and Ad_Unit. These tables are an alternative to retrieving entities from their respective services’ get***ByStatement methods. They allow you to retrieve sparse entities containing only the fields you’re interested in. For example, the following select statement retrieves the first page of only the ID and name of line items that are missing creatives.
SELECT Id, Name from Line_Item WHERE IsMissingCreatives = true LIMIT 500 OFFSET 0
In this blog post, we’ll go over some situations where this feature can be utilized to speed up entity retrieval times from hours to minutes.

Entity synchronization


The first major use case that benefits from these new tables is entity synchronization. For example, if you’re synchronizing line items on your network into a local database, you’re most likely using LineItemService.getLineItemsByStatement and hopefully taking advantage of the LineItem.lastModifiedDateTime field to only filter out line items that have changed since the last time you synchronized. But even with lastModifiedDateTime, this synchronization can still take a while, depending on how many line items you have on your network, and how complex their targetings are. If you don’t need to synchronize all the fields in your line item objects, you may be able to use the Line_Item PQL table to perform this synchronization instead.

If you do need to synchronize fields not yet available in the Line_Item table, such as targeting, you can still take advantage of this table for computed fields that don’t affect lastModifiedDateTime, such as LineItem.status. What you can do is synchronize your line items as usual with getLineItemsByStatement filtering on lastModifiedDateTime. Then update your local statuses with selected line item statuses from the Line_Item table (a very quick process):
SELECT Id, Status from Line_Item LIMIT 500 OFFSET 0

Match tables for reports


Local copies of line item information can also be used as match tables to construct more detailed reports. Sometimes, you may want more information in your reports than what is currently available as a dimensionAttribute. For example, if you run a report by line item ID, you may also want other line item information like isMissingCreatives to show in the report. Because LineItem.isMissingCreatives is unavailable as a DimensionAttribute, you can create a local match table containing line item IDs and additional columns to be included in the report. Then you can merge this match table with the report by the line item ID to obtain a report with those additional columns.

For example, let’s say you run a report with the following configuration:
Dimension.LINE_ITEM_ID
DimensionAttribute.LINE_ITEM_COST_TYPE
Column.AD_SERVER_IMPRESSIONS
The report in CSV_DUMP format looks something like this:
Dimension.LINE_ITEM_ID, DimensionAttribute.LINE_ITEM_COST_TYPE,
Column.AD_SERVER_IMPRESSIONS
1234567, CPM, 206
1234568, CPD, 45
1234569, CPD, 4
To also include LineItem.isMissingCreatives in the report, you would fetch a match table and save it (as a CSV file for example) by retrieving ID and isMissingCreatives from the Line_Item table.
SELECT Id, IsMissingCreatives from Line_Item LIMIT 500 OFFSET 0
Full examples of how to fetch match tables are available in all our client libraries. For instance, Python’s is here. Then using a script or a spreadsheet program, merge the match table with the report to produce something like this:
Dimension.LINE_ITEM_ID, DimensionAttribute.LINE_ITEM_COST_TYPE,
Column.AD_SERVER_IMPRESSIONS, LineItem.isMissingCreatives
1234567, CPM, 206, true
1234568, CPD, 45, false
1234569, CPD, 4, false
If you have any questions on these new PQL tables, or suggestions on what PQL tables you want in the next release, please let us know on the API forum, or on our Google+ Developers page.