Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PBS video impression tracking #1015

Closed
bretg opened this issue Aug 28, 2019 · 26 comments
Closed

PBS video impression tracking #1015

bretg opened this issue Aug 28, 2019 · 26 comments
Assignees
Labels
Intent to implement An issue describing a plan for a major feature. These are intended for community feedback PBS-Go

Comments

@bretg
Copy link
Contributor

bretg commented Aug 28, 2019

As a followup to the event tracking issue #800, we're ready to implement the injection of additional tracking into VAST XML. Initially this is just to count impressions, but will be expanded to additional video events at some point. There are two scenarios, both of which are required and could be happening at the same time:

  1. Server-side VAST - this scenario covers the video XML is returned in the response from a server-side adapter. If that bidder allows PBS to modify their VAST, the server then injects an tag into the VAST, pointing to a prebid server event string, and then caches the modified results.

Here's a flow diagram showing how tracking will work. Changed items are in red.

Screen Shot 2019-08-27 at 11 34 35 AM

  1. Client-side VAST - this scenario occurs when the video XML is returned in the response from a client-side adapter. PBJS forwards the XML to PBS on a new endpoint that instructs PBS to update the XML and cache it. If that bidder allows PBS to modify their VAST, the server then injects an tag into the VAST, pointing to a prebid server event string, caches the modified results and responds to the client.

Screen Shot 2019-08-27 at 9 08 02 PM

Proposed modifications

  1. Add a new bidder-level flag in PBS that allows a given bidder to turn off our ability to modify their VAST. e.g. if bidderA doesn't want us to modify their VAST, the host company won't have complete analytics.

We propose a new bidder configuration property modifyingVastXmlAllowed, with the default being false.

  1. When PBS receives VAST from each bidder, it checks the bidder flag and whether the account supports events. If so, it inserts an <impression> tag before storing to PBC

The contents of the <impression> tag are pulled from a new event.url-template property that has macros that need to be resolved. e.g.

event:
    url-template: "/event?t=imp&b=%s&f=b&a=%s"

where b=BIDID, a=ACCOUNT

  1. Algorithm for updating the VAST. (originally this was over-simplified. Changed May 13, 2021)

Scan the 'adm' on video bids:

  • If adm contains <InLine> (case insensitive), it's "Inline.
  • Otherwise, if adm contains <Wrapper> (case-insensitive), it's a "wrapper".
  • Otherwise if adm is not empty but doesn't contain either Inline or Wrapper, don't modify the results.
  • Otherwise if adm is empty and nurl is present, create a VAST wrapper:
<VAST version=\"3.0\"><Ad><Wrapper>
<AdSystem>prebid.org wrapper</AdSystem>
<VASTAdTagURI><![CDATA[" + bidNurl + "]]></VASTAdTagURI>
<Creatives></Creatives>
</Wrapper></Ad></VAST>
  • Otherwise if neither adm nor nurl are present, it's an invalid video bid.

Once we know what type of VAST is present:

  • Inline VAST - search for <Impression>. Add PBS Impression tag(s) after existing tag. If no <impression> tag is found, it's an unexpected error and this VAST is suspicious. Don't add any PBS Impression tags.
  • Wrappers
    • Look for an impression pixel on the Wrapper
    • If impression pixel present, insert <Impression> tags at the same level
    • If impression not present, refer the xsd (for VAST version in the declaration) and find the parent element for impression pixels. Add the impression pixels one level below the parent element
    • If parent element not present, reject the VAST and the corresponding bid.
  • VAST URL - create a PBS-generated wrapper that includes one or more Impression tags.
  1. Client-Side VAST Caching, including PBJS changes

Unlike server-side video, the VAST XML coming into the browser didn't go through Prebid Server, so will not have the tracking strings added. Prebid.js has configuration that allows the publisher to initiate "client-side caching". In order to modify the VAST XML, the page will refer to a new end point on Prebid Server that will perform the same modifications as if it had come through the server.

pbjs.setConfig({
    cache: {
        url: "https://prebid-server.rubiconproject.com/vtrack?a=ACCOUNT",
        vasttrack: true
    }
});

Where /vtrack is a new endpoint that causes the request to go to PBS where the steps 1-3 above will be applied.

If the vasttrack parameter is true, Prebid.js will add a couple of parameters to the POST XML to the specified endpoint with this JSON

{"puts":[{
    "bidid": BIDID,             // new parameter
    "bidder": "BIDDER",    // new parameter
    "type":"xml",
    "value":"<VAST…/VAST>",
    "ttlseconds":3600
}]}

PBS doesn't currently have a /vtrack endpoint -- we shouldn't use /cache because PBC implements that one. The new endpoint would use a similar code path as for server-side VAST:

  • parse the posted VAST and pull out the bidid and bidders parameters
  • use the bidid and the account id to create the additional field in the VAST
    POST the modified VAST to Prebid Cache, wait for the results from PBC, and forward them to the client. e.g. {"responses":[{"uuid":"94531ab8-c662-4fc7-904e-6b5d3be43b1a"}]}

PBS should validate the arguments supplied with /vtrack: if ACCOUNT, BIDID, or BIDDER aren't supplied, it should reject the /vtrack request with 400. This should get caught when the page is being tested.

If the ACCOUNT doesn't exist and PBS is enforcing accounts, that should also result in a 400.

PBS will process /vtrack similar to the server-side VAST response, inserting impression tracking when:

  • account ID allows events
  • the bidder allows VAST modification

Closed Item:

  • We could combine the "/vtrack" into "/cache" but that could make load balancer configuration more complicated. We discussed in the PBS meeting last week and agreed a separate endpoint is the way to go.
@bretg bretg self-assigned this Aug 28, 2019
@bretg bretg added the Intent to implement An issue describing a plan for a major feature. These are intended for community feedback label Aug 28, 2019
@bretg
Copy link
Contributor Author

bretg commented Aug 28, 2019

For load balancer reasons, I'm starting to prefer a separate endpoint /vtrack.

@rpanchyk
Copy link
Contributor

rpanchyk commented Aug 29, 2019

Hi @bretg !
From PBS configuration point of view i propose to use instead of

events:
    urltemplate: "https://prebid-server.rubiconproject.com/event?t=imp&b=%s&f=b&a=%s"

next:

event:
    url-template: "/event?t=imp&b=%s&f=b&a=%s"

So, the changes:

  1. prefix for the property - event.* since PBS-Java has /event endpoint, so can be more acceptable.
  2. name for the property - url-template to be more readable.
  3. value for the property - includes path with query string since both PBS-Go and PBS-Java has ExternalURL configuration property, so it can be simply joined to obtain full url.
@bretg
Copy link
Contributor Author

bretg commented Sep 3, 2019

Thanks @rpanchyk - updated.

@bretg
Copy link
Contributor Author

bretg commented Sep 8, 2019

Discussed and approved in PBS PMC

@bretg
Copy link
Contributor Author

bretg commented Nov 8, 2019

This is done in PBS-Java. Assigning to @hhhjort for PBS-Go implementation.

@SyntaxNode
Copy link
Contributor

This is largely complete in PBS-Go. @laurb9 noted two remaining things to implement:

  • wurl insertion in bids
  • VAST rewriting when pbs caches

@danielguedesb Are you able to continue contributing to add those areas of functionality?

@laurb9
Copy link
Contributor

laurb9 commented Nov 12, 2020

This older ticket did not define the entire functionality needed for events and I can't find another one that does (edit: or maybe #1470 ), so I will add it here from https://docs.google.com/document/d/1ry0X4C2EV-R0pMrm1IQk9BstxaT395UCl3KKqTGa5c8/edit# so we are all on the same page. Let me know if this should be another ticket. I'm starting work on this.

  1. If the account doesn't support events, we're done - no need for any type of event logic.
  2. Otherwise, if the bid request was video and the response is VAST XML:
    1. If we're allowed to modify the bidder's VAST, then inject an tag with this URL -- https://PBS_HOST/event?t=imp&b=BIDID&f=b&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION
    2. If ext.prebid.cache.vastXml is specified, then cache the VAST in PBS
  3. else // not video
    1. If bid caching is turned on (ext.prebid.cache.bids)
      1. If the event configuration is on for this account/channel combination or if the ext.prebid.events object is defined in the original request, then add wurl to bids cached in PBC.
        URL is https://PBS_HOST/event?t=win&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION
      2. else if it's a PG bid, then add wurl to bids cached in PBC but with x=0
        https://PBS_HOST/event?t=win&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION&x=0
    2. Finally, consider setting seatbid[].bid[].ext.prebid.events.win and seatbid[].bid[].ext.prebid.events.imp
      1. If the event configuration is on for this account/channel combination or if the ext.prebid.events object is defined in the original request, then add response.seatbid[].bid[].ext.prebid.events.{win,imp} to the openrtb output -- https://PBS_HOST/event?t=win&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION and https://PBS_HOST/event?t=imp&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION
      2. else if it's a PG bid, then add response.seatbid[].bid[].ext.prebid.events.{win,imp} to the openrtb output -- https://PBS_HOST/event?t=win&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION&x=0 and https://PBS_HOST/event?t=imp&b=BIDID&f=i&a=ACCOUNT&ts=TIMESTAMP&bidder=BIDDER&int=INTEGRATION&x=0
@SyntaxNode
Copy link
Contributor

Let me know if this should be another ticket.

I think we're good with continuing the discussion here.

I'm starting work on this.

Sweet. Thank you.

@laurb9
Copy link
Contributor

laurb9 commented Nov 24, 2020

I've implemented most of the logic in #1597 . The goal was to get the unit tests to a point where we can validate the behavior, then do some refactoring. Some of the tests are currently failing in a non-significant way because of timestamp mismatch (until #1584 is merged) but are covering the behavior we need.

I see a few differences between what PBS-Java does and the algorithm above:

  • Only the cached VAST is modified, the VAST returned in adm field is not, in returned or cached bids. Is this intentional ? I have copied this behavior in the PR above.
  • bid caching and vast modification are not exclusive as the else above suggests. PBS-Java will change both VAST and cached bids if enabled. This made sense to me, so that's what I did too.
  • PBS-Java adds wurl to cached bid if both account and request enabled it. The algorithm says either suffices, I implemented the algo.
  • int (integration) does not get a value, so I didn't add it yet.
@bsardo
Copy link
Contributor

bsardo commented Dec 7, 2020

Let's discuss the revised algorithm presented by @laurb9 above at the prebid.org committee meeting this Friday.

@bretg
Copy link
Contributor Author

bretg commented Dec 7, 2020

Only the cached VAST is modified, the VAST returned in adm field is not, in returned or cached bids. Is this intentional ? I have copied this behavior in the PR above.

The algorithm says "If ext.prebid.cache.vastXml is specified, then cache the VAST in PBS. Otherwise, just return the modified VAST." So this is a PBS-Java bug. Opened an internal ticket to resolve. Heads up @rpanchyk

However, I did notice that the algorithm summary didn't check that events were enabled for video before adding the tag. Updated to note that "If events are enabled for this account". See https://docs.google.com/document/d/1ry0X4C2EV-R0pMrm1IQk9BstxaT395UCl3KKqTGa5c8/edit#

bid caching and vast modification are not exclusive as the else above suggests. PBS-Java will change both VAST and cached bids if enabled. This made sense to me, so that's what I did too.

Algorithm summary fixed.

PBS-Java adds wurl to cached bid if both account and request enabled it. The algorithm says either suffices, I implemented the algo.

Another PBS-Java ticket opened to confirm. Thanks.

int (integration) does not get a value, so I didn't add it yet.

Integration is defined in #1428

@laurb9
Copy link
Contributor

laurb9 commented Dec 7, 2020

The algorithm says "If ext.prebid.cache.vastXml is specified, then cache the VAST in PBS. Otherwise, just return the modified VAST." So this is a PBS-Java bug.

I'm going to work on updating the VAST everywhere in #1597 as well, in that case. To do that, I think I might need some clarification on expected behavior for the case below:

when bid.adm is empty, PBS-Go makes up a VAST with <VASTAdTagURI><![CDATA[' + bid.NURL + ']]></VASTAdTagURI> just for VAST cache, but it still caches and returns the bid with empty adm.

What should happen when events are enabled and bid.adm is empty ? I feel should generate the VAST, include the Impression events and return the same VAST everywhere (vast, cached bid, returned bid). That is inconsistent with the above behavior, so maybe we should return the same thing all the time, generated or not ?

@laurb9
Copy link
Contributor

laurb9 commented Dec 9, 2020

More questions:

Should ext.prebid.events and vast rewriting apply to:

  1. only on winning bids and only when targeting is enabled ? (current implementation)
  2. only on winning bids always ?
  3. all bids ?
  4. other ?
@bretg
Copy link
Contributor Author

bretg commented Dec 11, 2020

Thanks for pushing this @laurb9 - this is quite intricate. I've revised (somewhat heavily) the Events doc. I don't think the algorithm has changed much, but trying to structure to document to make the various constraints more clear.

  • Added requirements section for Prebid Server VAST support
  • Clarified the scenario of empty bidresponse.adm when bidresponse mediatype is video: create the wrapper and treat it like VAST that was returned from the bidder.
  • Make the request.ext.prebid.events object consistent and work for VAST as well as banner/native.
  • Updated Prebid Server algorithm to support multi-format requests. Defined how to tell if a bidresponse is mediatype video.
  • Not returning seatbid[].bid[].imp[].ext.prebid.events for video
  • Specified that injecting the VAST impression tag should happen for all bidresponses returned to the client, not just the highest CPM. We don't know for sure which one the client will use.
  • Added references to the already-existing ext.prebid.cache.vastxml.returnCreative option. It had not previously been part of this doc because it's not directly related to events, but adding it here provides clarity to how it relates to VAST processing.

So specifically answering your questions:

What should happen when events are enabled and bid.adm is empty ?
I agree with you: generate the wrapper and operate as if that wrapper is what had come from the bidder in the first place.

Should ext.prebid.events and vast rewriting apply to
#3, because we always return all bidresponses and don't know for sure which one will be chosen by the client.

@rpanchyk - while researching this in PBS-Java code, I reviewed the "analytics_config" database column as implemented and have questions.

  1. what is the format of this DB field? Based on a brief tour in the code, I believe it an example would be:
{"auction-events": {"amp": true, "app": true, "web": true}}

But our database refers to auction_events -- note the underscore rather than the dash. Which is correct?

  1. Does the auction-events JSON also define the win and imp events? i.e. are all 3 types of events tied together? It appears so.
@rpanchyk
Copy link
Contributor

hi, @bretg

  1. "auction-events" is the name PBS expects in DB.
  2. bid.ext.prebid.events.{win,imp} is added to response only if account has enabled events AND incoming request allows events (defined appropriate channel OR has ext.prebid.events field).
@laurb9
Copy link
Contributor

laurb9 commented Dec 15, 2020

Thank you @bretg and @rpanchyk for the updates. I'm working on implementing the changes and updating the PR.

@laurb9
Copy link
Contributor

laurb9 commented Dec 15, 2020

I've tried to make a truth table to clarify some of the situations. How does this look ?

bidType == video account events account events channel ext prebid events bid dealID is PG bidder vastEnable vast modify bid ext prebid events bid.wurl, if caching  
     
    x=0
    x=0
         
       
       
       
  x=0
      x=0
       
@laurb9
Copy link
Contributor

laurb9 commented Dec 16, 2020

@bretg I've had a few more questions while working on adding the unit tests based on the truth table above:

  1. from doc

Make the request.ext.prebid.events object consistent and work for VAST as well as banner/native.

I do not see where this happened, maybe I am looking at a cached doc version ? request.ext.prebid.events only impacts non-video bids as far as I can tell.

  1. from the doc

Video impression Events - always generated if events are enabled for the account

I assume that is a qualified "always" because of this in the algorithm

If we're allowed to modify the bidder's VAST

  1. How do we define a PG bid ? bid.dealID being set ?

  2. What is LINEID ? Is it the GAM lineitem (imp.ID), the bid.dealID or something else ?

  3. from the doc

PG always needs to get notified about bidsWon in order to pace.

Does this override the account settings ? Should this be host-controlled, if they implemented pacing in some way since it's not part of PBS ?

  1. In what scenarios should request (.ext.prebid.events) override account (.events_enabled) ? Because PBS will receive events for an account that has events disabled, and that can be confusing.
laurb9 added a commit to openx/prebid-server that referenced this issue Dec 16, 2020
- all bids have vast modified if account and bidder enabled, not just winning
- all NON-VIDEO bids have events if account or request enabled
- all cached NON-VIDEO bids have wurl patched in if account or request enabled.

prebid#1015 (comment)
laurb9 added a commit to openx/prebid-server that referenced this issue Dec 16, 2020
prebid#1015 (comment)

- all VIDEO bids, cached and returned, have modified VAST if account and bidder enabled - not just winning cached bids.
- all NON-VIDEO returned bids have `ext.prebid.events` if account or request enabled
- all cached NON-VIDEO bids have `wurl` patched in if account or request enabled.
laurb9 added a commit to openx/prebid-server that referenced this issue Dec 16, 2020
prebid#1015 (comment)

- all VIDEO bids, cached and returned, have modified VAST if account and bidder enabled - not just winning cached bids.
- all NON-VIDEO returned bids have `ext.prebid.events` if account or request enabled
- all cached NON-VIDEO bids have `wurl` patched in if account or request enabled.
@laurb9
Copy link
Contributor

laurb9 commented Dec 16, 2020

I've updated the PR to what I think is a potentially releasable functionality. I see exchange code is a refactor target, so I'd like to defer the additional PG and channels functionality as an incremental update for later, to avoid more PR conflicts

DGarbar added a commit to prebid/prebid-server-java that referenced this issue Dec 18, 2020
- Add validation for Video bids. `bid.adm` or `bid.nurl` needs to be present
  - This case now, is not possible `<VASTAdTagURI><![CDATA[null]]></VASTAdTagURI>`
- Bid adm will be updated as cache. (see prebid/prebid-server#1015)
- Bid type is defined by bidder, not our (`imp.video != null`, etc) checks. For example Appnexus use `bid.ext.appnexus.bid_ad_type` to define it. (that's why there are a lot of changed cache jsons. Also add ordering for tests)

Refactored a bit.
- Removed confusing maps
- Removed confusing checks
- Removed several imp to bid matching

Possible improvements (in another ticket bc current PR is too large)
- Extract more event URL to another class
- Use bidInfo in BidResponseCreator for BidResponse
@bretg
Copy link
Contributor Author

bretg commented Dec 21, 2020

@laurb9 - created a somewhat modified version of your truth table at the bottom of https://docs.google.com/document/d/1ry0X4C2EV-R0pMrm1IQk9BstxaT395UCl3KKqTGa5c8/edit# . The only comment for your version:

  • The "bid dealID" column should be "bid is PG". PBS-Go doesn't support Programmatic Guaranteed so you don't need to worry about those cases.

Responding to questions in your comment:

request.ext.prebid.events only impacts non-video bids as far as I can tell

right - that was the original set up -- the proposal here is that request.ext.prebid.events overrides the server account event/analytics config for video as well as banner/native.

assume that is a qualified "always"

Removed the word "always".

  1. How do we define a PG bid ? bid.dealID being set ?
  2. What is LINEID ? Is it the GAM lineitem (imp.ID), the bid.dealID or something else ?
  3. from the doc

PG not supported in PBS-Go.

  1. In what scenarios should request (.ext.prebid.events) override account (.events_enabled) ? Because PBS will receive events for an account that has events disabled, and that can be confusing.

The idea is that request.ext.prebid.events is a request-level override to the server-side config.

defer the additional PG and channels functionality as an incremental update for later, to avoid more PR conflicts

PG may not be built in PBS-Go. That would be a fairly major effort and ought to be driven by demand for the feature.

Channels -- PBS-Java has new account level "analytics_config" that defines analytics on a per-channel basis. Unfortunately I don't think the JSON attribute very well named. Currently it's "auction-events", but it's really more a generic flag for analytics support by channel. Will open a separate discussion on whether we want to make changes there. #1470

laurb9 added a commit to openx/prebid-server that referenced this issue Jan 6, 2021
VAST is modified if *either* account or request setting demanded it.

prebid#1015 (comment)
@SyntaxNode
Copy link
Contributor

@laurb9 We've merged your events PR. Would you say that PBS-Go now implements this feature or are there still gaps?

@laurb9
Copy link
Contributor

laurb9 commented Jan 23, 2021

Out of the things specced in the document, we are only missing PG and integration channels.

PG was said to be not implemented yet in PBS-Go, so that part can be added when PG functionality is worked on.

Integration channels: PBS-Go has a per-account enable flag for events. PBS-Java has that, plus three additional subflags that can control it for amp, app and web individually for each account if set. This was mentioned here. This, I have not implemented because I was left the impression that channel mapping needed further discussion (#1428, #1470), and the channel wasn't readily available in the auction for me to use.

rpanchyk pushed a commit to prebid/prebid-server-java that referenced this issue Feb 23, 2021
* Add modifying of VAST for video bids and add validation

- Add validation for Video bids. `bid.adm` or `bid.nurl` needs to be present
  - This case now, is not possible `<VASTAdTagURI><![CDATA[null]]></VASTAdTagURI>`
- Bid adm will be updated as cache. (see prebid/prebid-server#1015)
- Bid type is defined by bidder, not our (`imp.video != null`, etc) checks. For example Appnexus use `bid.ext.appnexus.bid_ad_type` to define it. (that's why there are a lot of changed cache jsons. Also add ordering for tests)

Refactored a bit.
- Removed confusing maps
- Removed confusing checks
- Removed several imp to bid matching

Possible improvements (in another ticket bc current PR is too large)
- Extract more event URL to another class
- Use bidInfo in BidResponseCreator for BidResponse

* Merged master and fixed corresponding imp can't be null

* Fix after review. (without changing localhost/event)

* Use placeholders in test resources instead of concrete urls where possible (#1084)

* Fix tests and remove ordering

* Replace usage of Wiremock EqualToJsonPattern with custom implementation to prevent incorrect json comparison

* fix openx

* Use equalToBidCacheRequest for consistency and add openrtbCacheDebugComparator() when debug is used (fix flaky tests)

* Remove redundant code

Co-authored-by: Sergii Chernysh <schernysh@users.noreply.github.com>
Co-authored-by: rpanchyk <rpanchyk@rubiconproject.com>
@bretg
Copy link
Contributor Author

bretg commented May 13, 2021

Updated VAST updating algorithm

nickluck8 pushed a commit to prebid/prebid-server-java that referenced this issue Aug 9, 2021
* Add modifying of VAST for video bids and add validation

- Add validation for Video bids. `bid.adm` or `bid.nurl` needs to be present
  - This case now, is not possible `<VASTAdTagURI><![CDATA[null]]></VASTAdTagURI>`
- Bid adm will be updated as cache. (see prebid/prebid-server#1015)
- Bid type is defined by bidder, not our (`imp.video != null`, etc) checks. For example Appnexus use `bid.ext.appnexus.bid_ad_type` to define it. (that's why there are a lot of changed cache jsons. Also add ordering for tests)

Refactored a bit.
- Removed confusing maps
- Removed confusing checks
- Removed several imp to bid matching

Possible improvements (in another ticket bc current PR is too large)
- Extract more event URL to another class
- Use bidInfo in BidResponseCreator for BidResponse

* Merged master and fixed corresponding imp can't be null

* Fix after review. (without changing localhost/event)

* Use placeholders in test resources instead of concrete urls where possible (#1084)

* Fix tests and remove ordering

* Replace usage of Wiremock EqualToJsonPattern with custom implementation to prevent incorrect json comparison

* fix openx

* Use equalToBidCacheRequest for consistency and add openrtbCacheDebugComparator() when debug is used (fix flaky tests)

* Remove redundant code

Co-authored-by: Sergii Chernysh <schernysh@users.noreply.github.com>
Co-authored-by: rpanchyk <rpanchyk@rubiconproject.com>
nickluck8 pushed a commit to prebid/prebid-server-java that referenced this issue Aug 10, 2021
* Add modifying of VAST for video bids and add validation

- Add validation for Video bids. `bid.adm` or `bid.nurl` needs to be present
  - This case now, is not possible `<VASTAdTagURI><![CDATA[null]]></VASTAdTagURI>`
- Bid adm will be updated as cache. (see prebid/prebid-server#1015)
- Bid type is defined by bidder, not our (`imp.video != null`, etc) checks. For example Appnexus use `bid.ext.appnexus.bid_ad_type` to define it. (that's why there are a lot of changed cache jsons. Also add ordering for tests)

Refactored a bit.
- Removed confusing maps
- Removed confusing checks
- Removed several imp to bid matching

Possible improvements (in another ticket bc current PR is too large)
- Extract more event URL to another class
- Use bidInfo in BidResponseCreator for BidResponse

* Merged master and fixed corresponding imp can't be null

* Fix after review. (without changing localhost/event)

* Use placeholders in test resources instead of concrete urls where possible (#1084)

* Fix tests and remove ordering

* Replace usage of Wiremock EqualToJsonPattern with custom implementation to prevent incorrect json comparison

* fix openx

* Use equalToBidCacheRequest for consistency and add openrtbCacheDebugComparator() when debug is used (fix flaky tests)

* Remove redundant code

Co-authored-by: Sergii Chernysh <schernysh@users.noreply.github.com>
Co-authored-by: rpanchyk <rpanchyk@rubiconproject.com>
nickluck8 pushed a commit to prebid/prebid-server-java that referenced this issue Aug 10, 2021
* Add modifying of VAST for video bids and add validation

- Add validation for Video bids. `bid.adm` or `bid.nurl` needs to be present
  - This case now, is not possible `<VASTAdTagURI><![CDATA[null]]></VASTAdTagURI>`
- Bid adm will be updated as cache. (see prebid/prebid-server#1015)
- Bid type is defined by bidder, not our (`imp.video != null`, etc) checks. For example Appnexus use `bid.ext.appnexus.bid_ad_type` to define it. (that's why there are a lot of changed cache jsons. Also add ordering for tests)

Refactored a bit.
- Removed confusing maps
- Removed confusing checks
- Removed several imp to bid matching

Possible improvements (in another ticket bc current PR is too large)
- Extract more event URL to another class
- Use bidInfo in BidResponseCreator for BidResponse

* Merged master and fixed corresponding imp can't be null

* Fix after review. (without changing localhost/event)

* Use placeholders in test resources instead of concrete urls where possible (#1084)

* Fix tests and remove ordering

* Replace usage of Wiremock EqualToJsonPattern with custom implementation to prevent incorrect json comparison

* fix openx

* Use equalToBidCacheRequest for consistency and add openrtbCacheDebugComparator() when debug is used (fix flaky tests)

* Remove redundant code

Co-authored-by: Sergii Chernysh <schernysh@users.noreply.github.com>
Co-authored-by: rpanchyk <rpanchyk@rubiconproject.com>
@AlexBVolcy
Copy link
Contributor

This issue will be closed, as everything requested here has been implemented besides PG. And since PG is a big feature, we decided to branch that off into it's own issue #2312

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Intent to implement An issue describing a plan for a major feature. These are intended for community feedback PBS-Go
7 participants