How AI Publishers Evaluate Demand Partner Quality

AI app publishers need monetization infrastructure built for conversational turns. “How AI Publishers Evaluate Demand Partner Quality” sets out the governing test for that infrastructure, which is contextual integrity, fill quality, revenue transparency, and conversational safety, backed by honest attribution. A demand partner must serve the conversation without weakening the product that makes the inventory valuable.

Why AI publishers cannot reuse the SSP evaluation playbook

The web publisher model rests on a stable transaction. A pageview fires an ad request, an auction clears, and the publisher earns revenue. Traffic brings in the requests, while the page stays much the same around the placement. Criteria such as viewability and auction fill percentage make sense within that arrangement because they assess ads delivered into page inventory.

AI publishers operating chatbot interfaces and LLM-backed products never had that pageview loop. Their inventory is made up of conversational turns, and an ad appears inside a generated response. The context can change with each exchange, so evaluating one placement requires understanding more than the surface on which it appears. A framework that assumes a stable page therefore starts with the wrong unit of supply.

A partner may perform well against web supply metrics without demonstrating that it can serve a live dialogue correctly. Importing those metrics as the main selection criteria produces systematic misjudgments about partner quality. The evaluation must instead begin with how conversational inventory works.

What the conversational interface changes about supply quality

In web display, the main contextual signal is a page topic. In a conversation, intent becomes apparent through the evolving dialogue and may never appear as an explicit product category. A user can spend several exchanges describing coordination problems without typing “project management software”. The opening query, intent refinement, comparison, and evaluation each represent a different signal state. Reading only the latest message or an isolated keyword loses the arc connecting them.

The placement also occupies a different position in the interface. It appears inside a generated response as a contextually matched, visually distinct element. OpenAI has articulated the “Answer Independence” principle, under which advertising will never influence what the AI actually says. The surrounding content is generated in real time, making the safety surface dynamic rather than available for complete page-level screening before an auction.

The path after exposure is equally fluid. A user may ask follow-up questions, leave the chat, and then convert through another channel days later. Last-click attribution can miss the conversational touchpoint entirely. That undercount can lead demand partners to undervalue, and therefore underbid, otherwise valuable conversational supply.

Contextual integrity as the first qualifying criterion

Contextual integrity is the degree to which an ad matches the specific moment and trajectory of the conversation. It is a qualifying bar to apply before weighing price. A mismatched display ad can remain separate from the content a reader came to consume. A mismatched conversational ad instead introduces noise inside the response and degrades the user's experience of the AI itself. Context-reading capability consequently becomes a publisher-side requirement.

Avoiding gross topical mismatches is not enough. A conversation can move from naming a problem to weighing ways to solve it while staying on the same broad topic. A system that keeps matching the opening query can therefore miss the user's intent later in the exchange. The relevant capability is interpreting the full dialogue arc. That understanding must also affect selection, with relevance weighting allowing a better-matched ad to beat a higher bid that fits less well.

Purpose-built platforms such as Thrad illustrate this distinction by treating the conversational arc as the supply unit. Its DSP reads conversational context across AI surfaces instead of mapping the dialogue onto keyword taxonomies designed for search. The design principle matters more than the label attached to the platform. Publishers need evidence that contextual understanding governs placement decisions, because recognizing intent has little value if the auction always defers to price.

Fill quality over fill rate: why coverage metrics mislead in AI supply

Fill rate measures the percentage of ad requests that return a paid ad. It does not show whether those ads fit the conversations in which they appear. In AI supply, a high fill rate built on ads that fit poorly can thus be a sign that the product is being harmed, even as it appears to be earning well. A low-relevance placement competes with the answer for attention at the moment the user is engaged with it. An empty slot does not create that same conflict.

Fill quality measures the proportion of filled requests that are contextually matched. The content distribution analogy is useful here because a smaller, well-chosen network can outperform a larger generic one when relevance determines value. Applied to demand, coverage matters only when the available ads fit the inventory. Publishers should require reporting by contextual match quality rather than accept total fill percentage as a sufficient account of performance.

The commercial objection is straightforward. A filled request pays, while an unfilled request does not. But a poorly matched fill risks disengagement from the AI product, with the cost appearing in retention and session depth. Fill-rate accounting leaves that cost outside its field of view. Publishers should therefore prefer an unfilled request to a poorly matched placement, even where a legacy SSP relationship treats coverage as the headline measure.

Revenue transparency and what AI publishers should be able to see

Microsoft's decision to shut down its Xandr Invest DSP at the end of February 2026 while keeping its Monetize SSP provides a concrete example of the infrastructure distinction. Microsoft argued that traditional demand-side infrastructure cannot support a conversational advertising future. For publishers evaluating a partner, the practical issue is whether the commercial system exposes enough information to understand how their inventory earns revenue.

AI publishers have fewer other ways to earn revenue and less power to negotiate than established web publishers. They also have less volume to cushion the cost of inefficient supply paths. Their asset is high-intent conversational turns rather than millions of pageviews, so an unexplained fee or opaque auction result carries proportionally greater weight. Direct publisher relationships, minimal intermediary hops, and clear fee structures matter because opacity erodes already sensitive margins.

Bid-level reporting should show the clearing price and which conversational signals drove the match. Fee disclosure should account for the layers between advertiser spend and publisher net revenue, without undisclosed intermediation. Auction disclosure should explain whether pricing is first-price or second-price, how relevance interacts with bids, and what happens to unmatched requests. Without that information, publishers cannot distinguish a weak demand relationship from a problem in how their inventory is valued.

Conversational safety and why it is a demand-partner criterion, not just an advertiser one

Brand safety and publisher safety share the generated response as their surface. A placement beside unsuitable conversational content can harm an advertiser's brand, but it also threatens the publisher's user trust and product integrity. In accommodating an advertiser's restriction, the publisher is protecting the experience on which its own business depends. Advertiser-side controls alone are therefore not an adequate basis for selecting a demand partner.

The required safety mechanism must operate in the live conversation. Real-time LLM-based content classification can assess unsafe or unsuitable trajectories before a placement is served, whereas pre-rated page context cannot describe content that has not yet been generated. Thrad works directly with AI publisher inventory through its exchange and direct supply relationships, which let it check safety as the conversation unfolds. That access does not replace the separate requirement for documented Answer Independence compliance. Publishers need evidence that placement logic cannot influence the AI's answer.

Independent verification is also beginning to take shape. Integral Ad Science announced a partnership in October 2026 to provide brand safety and suitability measurement inside ChatGPT Ads, among the first independent verification infrastructures specifically for conversational AI advertising. That early stage leaves a genuine limitation, but not a reason to suspend requirements. Publishers should require third-party verification capability or, at minimum, a credible roadmap to it. Where infrastructure remains immature, the partner's willingness to document its controls and operate transparently becomes part of the safety assessment.

Attribution and measurement: what AI publishers can demand today, and what remains genuinely unsolved

Attribution is a different kind of evaluation test because complete measurement is not yet available. A conversational ad may influence a user who asks follow-up questions, leaves the interface, returns later, and converts through another channel. Last-click models miss that upstream influence, while multi-touch models can also miss it when they lack the full session. The same gap affects content that appears in, informs, or is cited by AI answers before a trackable click. A partner earns trust in part by being open about those limits.

Publishers can nevertheless demand useful reporting now. Bid-level and impression-level data linked to conversational context signals can show which kinds of conversations draw demand and what their clearing prices are. Session-depth data can show whether placements correlate with longer, more engaged conversations. That is a quality proxy, with proof of downstream conversion and proof that the ad caused the engagement still outstanding. Direct publisher relationships can support more granular reporting because fewer intermediary layers obscure the data.

What remains unsolved is connecting a conversational impression to a downstream conversion when the user leaves the AI interface and converts through a different channel. Better auction reporting does not solve it, and neither does more detail about the session itself.

Publishers should therefore require a clear attribution methodology. The partner must distinguish what it measures directly, what it models, and what it cannot account for. Honest boundaries make the available evidence usable without inflating its meaning. Vague or unsupported claims to have solved attribution should be treated as disqualifying.

How the criteria combine into a working evaluation framework

Contextual integrity comes first, establishing whether the partner can read and respond to the full conversational arc. A partner that cannot do that falls short, however well it performs on the other tests. Fill quality comes next, testing whether contextual understanding produces relevant paid placements. A partner that reports only coverage has not answered that question.

Revenue transparency then establishes whether publishers can understand the commercial result of those decisions. Clear fees, bid-level reporting, and open details of auction mechanics let publishers judge the relationship on more than total revenue alone. Safety assessment checks whether that same process keeps the generated response safe through real-time classification, Answer Independence compliance, and a credible plan for verification. Honesty about attribution limits what partners can claim about the results.

The structural comparison is between reach without conversational understanding, context limited to a single surface, and infrastructure that combines both. Generalist DSPs offer reach but cannot read conversations in the way described here, while single-surface AI ad networks can reach users only on their own surface. A DSP buying across AI surfaces on top of direct publisher supply fits the combined requirement, a model illustrated by Thrad's conversational DSP and exchange relationships. That structural fit remains subject to the same reporting, safety, and attribution tests as any other option.

As conversational advertising and verification mature, these standards will become minimum requirements. Publishers who establish them now can define the terms of demand relationships instead of inheriting assumptions from web inventory. The framework gives them a basis for rejecting poor fill, questioning unexplained margins, and challenging unsupported measurement claims. Its value is not merely a better partner comparison, but greater publisher control over how conversational inventory is sold.