Prebid Server, AMP, and CCPA

[bretg]

Prebid Server, AMP, and CCPA

Researching how AMP actually supports CCPA consent, found that there's not a clear way to determine whether a given CONSENT_STRING is GDPR, CCPA, or other format. See ampproject/amphtml#26220

In other words, I don't think the enhancement in #1125 is where we want to be. It assumes that

a) we're going to be getting a us_privacy parameter from AMP.
b) the value, if provided, will always be formatted as a CCPA string

Unfortunately, neither of these are true.

Instead, we're stuck in a situation where we're going to be receiving a CONSENT_STRING that's formatted in an unknown way. i.e. we have to parse the string to determine how to process it.

Currently the AMP CONSENT_STRING comes in on gdpr_consent, but in reality, that string isn't guaranteed to be formatted as an IAB TCF1.0 GDPR string. It could be IAB US_PRIVACY formatted.

Proposed path-forward has been reviewed with the committee and approved:

Phase 1 - Combined GDPR+CCPA Support on two synonym keys

1. PBS should start looking for AMP parameters gdpr_consent and consent_string, treating them as the same. (consent_string won't come in initially, but after a transition phase, we'll convert to it so we don't get confused that the contents of gdpr_consent can actually be CCPA.)

2. PBS introspects consent_string/gdpr_consent:
   2a) If consent_string is exactly 4 chars, starts with "1", and the next 3 chars are in the set [yYnN-] then it's a CCPA-formatted string. Pass it through OpenRTB as regs.ext.us_privacy.
   2b) Pass consent_string through the TCF1.0 parsing function. If it successfully parses, send it through OpenRTB as user.ext.consent.
   2c) Anything other format causes PBS to log a warning into the response and ignore it. Yes, it's understood that warnings in an AMP response are only seen in test mode.

3. Accepting/denying the AMP consent string is completely separate from Prebid Server "enforce" mode. i.e. if PBS determines that the user is in-scope for GDPR or CCPA, then a missing AMP consent_string will be a problem.

(Note that it doesn't make much sense to add us_privacy to the AMP RTC string because then we'll have two copies of the very long TCF1.0 string and still have to parse it to figure out what it really is.)

Phase 2 - Update AMP RTC strings to use consent_string instead of gdpr_consent

Update https://github.com/ampproject/amphtml/blob/master/extensions/amp-a4a/0.1/callout-vendors.js to replace &gdpr_consent=CONSENT_STRING& with &consent_string=CONSENT_STRING&

We may as well add PAGEVIEWID at this time to address #1167. Here are the proposed updates to https://github.com/ampproject/amphtml/blob/master/extensions/amp-a4a/0.1/callout-vendors.js

prebidappnexus: {
    url:
      'https://prebid.adnxs.com/pbs/v1/openrtb2/amp?tag_id=PLACEMENT_ID&w=ATTR(width)&h=ATTR(height)&ow=ATTR(data-override-width)&oh=ATTR(data-override-height)&ms=ATTR(data-multi-size)&slot=ATTR(data-slot)&targeting=TGT&curl=CANONICAL_URL&timeout=TIMEOUT&adcid=ADCID&purl=HREF&consent_string=CONSENT_STRING&account=ACCOUNT_ID&pv=PAGEVIEWID',
    macros: ['PLACEMENT_ID', 'CONSENT_STRING', 'ACCOUNT_ID'],
    disableKeyAppend: true,
  },
  prebidrubicon: {
    url:
      'https://prebid-server.rubiconproject.com/openrtb2/amp?tag_id=REQUEST_ID&w=ATTR(width)&h=ATTR(height)&ow=ATTR(data-override-width)&oh=ATTR(data-override-height)&ms=ATTR(data-multi-size)&slot=ATTR(data-slot)&targeting=TGT&curl=CANONICAL_URL&timeout=TIMEOUT&adc=ADCID&purl=HREF&consent_string=CONSENT_STRING&account=ACCOUNT_ID&pv=PAGEVIEWID',
    macros: ['REQUEST_ID', 'CONSENT_STRING', 'ACCOUNT_ID'],
    disableKeyAppend: true,
  },

Phase 3 - stop supporting gdpr_consent key

This is probably quite far in the future when all AMP caches have been updated and all prebid RTC calls are coming in with consent_string instead of gdpr_consent

Before then, we'll have to support TCF2.0 and hopefully a new "CONSENT_TYPE" attribute which will eliminate the need to guess at the string format. But that's for another issue.

[SyntaxNode]

The approach we took in #1125 was chosen before the CCPA consent string was available in AMP. Clearly, things turned out differently and we need to adjust.

I like that we would parse for all possible formats and not fallback on an assumption of a default format.

I would like to avoid sending an invalid CCPA consent string to our bidders. In addition to checking for the string length and verifying the first character, I'd like to see us validate the other three characters to be on par with the level of validation in the TCF 1.0 parsing function.

Should we consider the consent_state variable for CCPA? Does AMP expect us to ignore the consent_string if the consent_state is rejected? The wording in the docs seems like they might, but you might know better.

[bretg]

My perception is that consent_state is useless to us, but maybe I don't understand it correctly.

I think it's there as a shortcut to AMP entities what want to trust what the CMP says in a tri-state way: user-accepted, user-rejected, unknown. I think it was developed for very simplistic consent mechanisms.

This seems useless in light of the complexity of GDPR particularly, which has much more nuance like Purpose 1, Purpose 3, vendor overrides, legitimate interest, etc.

It makes a little more sense in the CCPA context, but still lacks the nuance of the 3 fields, so I've taken the approach that we need to ignore the simplistic signal and deal directly with the details.

I would like to avoid sending an invalid CCPA consent string to our bidders.

I'm ok with that.

[SyntaxNode]

Gotcha. With the consent-state being a simplified summary of the consent-string, I agree with you it's nothing we need to consider. That information is already available to Prebid Server through the consent string.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bretg]

FYI - this is in progress for PBS-Java

[SyntaxNode]

2c) Anything other format causes PBS to log a warning into the response and ignore it. Yes, it's understood that warnings in an AMP response are only seen in test mode.

We're working on this as well. Where exactly in the response did you envision the warning would go?

[bretg]

ext.debug somewhere? @rpanchyk - where are you guys planning to put this warning?

[SyntaxNode]

The top level error container is my first thought, but it's indexed by bidder. Let's say we use a string like "*" to signify all bidders, we would still be adding a non-fatal error to the mix which may very well be considered a breaking change. We could add a new warning container at the top level to make it obvious. We could also include it in the debug, but the error container at that level is also indexed by bidder. Thoughts?

[bretg]

seems like this isn't the first time we've needed a place to put global (non-bidder specific) errors. bidder * is fine. ext.debug.errors is fine.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bretg]

Done for both Go and Java