Reviews
Part of Why programmatic advertising reviews need a scoring method
Before you trust a programmatic advertising review methodology
Most teams score vendors on sales decks rather than evidence. Here is what a programmatic advertising review methodology assesses, and what it leaves out.
What to take away
- The commonest mistake is scoring vendors on their own pitch material and calling the result a review.
- A workable method separates what you verified, what the vendor told you and what you judged.
- Company records, training certifications and advertising rulings are independent inputs; vendor performance figures are not.
- Nothing in a desk review is hands-on testing, and the write-up should say so.
- Scores only help when the weighting and the cut-off are written down before shortlisting.
Fix the evidence tiers before you score anything
Sort inputs into three tiers: tier one is independently verifiable, with registry filings, published rulings and certification records. Tier two is vendor-supplied: case studies, benchmark decks and reference calls the seller arranges.
Three evidence tiers
Tier 1
- Source
- Independent
- Examples
- Registry filings, rulings
- Verifiable
- Yes
- Risk
- Lowest
Tier 2
- Source
- Vendor-supplied
- Examples
- Case studies, decks
- Verifiable
- No
- Risk
- Pitch-shaped
Tier 3
- Source
- Reviewer judgement
- Examples
- Weighing against brief
- Verifiable
- No
- Risk
- Reviewer bias
Tier three is editorial judgement, where a reviewer weighs the documents against the brief. That separates a scored review from a rewritten sales deck; the wider approach sits in Programmatic advertising: reviews and comparison methods.
What gets assessed, and what does not
Assessed: corporate standing, contractual terms, fee disclosure, data handling and enforcement history. Not assessed: live campaign performance, dashboard usability and day-to-day account management. Those need account access and a measurement window, which a desk review lacks. Say so plainly: a reader who assumes the platform was tested decides worse than one told the limits.
Independent checks worth running
Start with the registry: filing history, named directors and accounts status are public, and a mismatch with the pitch is a finding in itself. Corporate records sit on the register at Companies House, the official source for UK company data.
Training claims come next. A certification badge proves a course was completed, not that campaigns worked, and LinkedIn Ads Certifications are credentials a reviewer can confirm.
Then check enforcement, because the ASA rulings database records upheld complaints and a pattern belongs in the review with its date attached.
Separate claims from evidence in the write-up
Use two verbs and keep them apart. "The vendor states" introduces a tier-two claim. "The filing shows" introduces a tier-one fact. Readers can see where the checking stopped. Name the vendor and the date for any supplied number; otherwise use a labelled illustrative example.
Weighting and cut-offs
Write the weighting before scoring. For a typical agency search, corporate standing might carry 25 per cent, contractual transparency 25 per cent, evidence quality 30 per cent and enforcement history 20 per cent. These are illustrative, not a standard. Record the minimum score needed to stay on the shortlist, and the reasons for any override.
How to record an override
An override is a documented decision to keep or drop a vendor despite the score. Log the date, the reviewer, the evidence and the counter-argument. If a vendor fails on enforcement but wins on transparency, the note should say which drove the call.
A worked example with labelled figures
Suppose an English agency reviews three demand-side platform resellers, with reviewer time costed at an illustrative £60 an hour.
Vendor A supplies filings, two certifications and no upheld rulings: 12 hours, about £720. Vendor B supplies a deck and one reference; 8 hours chasing registry records, about £480, turns up a dormant subsidiary. Vendor C declines to name its contracting entity: 4 hours, about £240, ending in a no-bid. Every figure is illustrative.
Keep the method repeatable
Apply the same tiers, weighting and override log to every vendor in a round, or the scores are not comparable. Publish the method alongside the review so readers can audit it. For the same discipline on the agency side, see Programmatic advertising agency reviews in England, which covers the questions to put to an agency before scoring it.
Common questions
Does a review methodology need hands-on testing?
No. A desk review can assess records, contracts and enforcement history without account access. If no testing took place, say so rather than imply it.
How do you treat vendor-supplied performance figures?
As tier-two claims. Name the vendor and date, and mark the figure unverified unless an independent source confirms it.
When should a vendor be excluded outright?
When a tier-one check fails, such as an inability to name the contracting entity or a pattern of upheld rulings. Record the reason first.
How often should the methodology be revised?
Review the weighting annually, or sooner if the brief changes. Log every change so earlier scores remain interpretable.
