Tannhäuser Gate

Search

SearchSearch

Recent Notes

See 42 more →

Amazon Ads, Deeply

Sep 02, 2026, 20 min read

Why this, why now

Amazon’s advertising business did $56.2 billion in revenue in 2024 [26] - third behind only Google and Meta, and growing faster than either. Unlike the other two, almost all of it is sold one search at a time: a shopper types “mascara for volume”, and in the few hundred milliseconds before the results page renders, an auction runs, a ranking model scores every eligible product against that exact phrase, prices are set, and the winning ads are sewn into the page. The unit of the whole machine is the (query, product, placement) triple, and the machine reprices it billions of times a day.

Two facts from August 2026 make the mechanics concrete. In its response to an FTC lawsuit, Amazon disclosed that roughly 92% of placed ads are not given to the highest bid, that average winning bids on Sponsored Products fell 50% from 2019 to 2025, and that conversion rates on those ads grew 24% from 2021 to 2025 [25]. Read those numbers again: the auction is not a highest-bid-wins market. It is a prediction market where the currency is modeled relevance, and the models have gotten good enough that the platform systematically prefers a lower bid attached to a product shoppers actually buy. Whoever owns the better conversion model owns the auction.

That makes the methodology unusually legible from the outside. The pieces - sponsored-search auction theory, CTR/CVR prediction at scale, bidding under budget constraints, and experimentation inside a marketplace - each have a real literature, much of it published by the platforms themselves (Amazon’s science org included). This note is the deep version: the mechanism, the prediction stack, the bidding brain, and how any of it gets measured, with the actual papers.

There is also a companion interactive lab - The Ad Auction, From the Inside - where you can run the auction, train the Bayesian conversion model, and drive the hourly repricer yourself.

Want the full-day version?

This note is the map. The territory is A Day on Amazon Ads: ten linked pages that go paper by paper through the auction theory of the autobidding world, the 2017-2026 lineage of RL and generative bidding, the modern prediction stack, equilibrium learning, measurement, and a bridge from Noam Brown’s poker and Diplomacy work to bidding under imperfect information.

1. The mechanism: a second-price auction wearing a relevance mask

The stage

Sponsored Products - the workhorse format, and the bulk of spend - are cost-per-click ads targeted to keywords or products, shown in three placement families: top of search results, rest of search, and product detail pages. The advertiser sets a bid per targeting clause (a keyword, roughly) and can layer placement multipliers on top. Every shopper query triggers a fresh auction among every clause whose targeting matches. Scale check: a mid-size brand has thousands of active keywords; Amazon has hundreds of thousands of advertisers; every search on the site is an auction.

GSP, and the price you actually pay

The theoretical spine is the generalized second-price auction (GSP), formalized by Edelman, Ostrovsky and Schwarz [1] and Varian [2] in 2007. Rank ads by a score, allocate slots in score order, and charge each winner the minimum it needed to hold its slot. In the plain version the score is the bid, and the winner pays the next bid plus a cent. Neither party found GSP to be truthful - unlike VCG, bidding your value is not a dominant strategy - but Edelman et al. showed it has “locally envy-free” equilibria whose outcomes and payments coincide with VCG, which is why the market doesn’t collapse into constant re-shading.

Modern sponsored search, and Amazon in particular, weights the score by predicted relevance [3][25]:

scorei = bi · qi,

where bi is the bid and qi is a predicted quality term - essentially P(click | query, product, placement), sometimes blended with predicted conversion. The winner’s price per click backs out of the same inequality as before: to hold rank 1 you need b1 · q1 ≥ b2 · q2, so

price1 = (b2 · q2) / q1 + ε.

This one formula explains the entire FTC disclosure. If your q doubles, your price halves at fixed competition - which is precisely “average winning bids fell 50%” while “conversion rates grew 24%“. And it is why 92% of placements don’t go to the highest bid: the ranker is multiplying. It also quietly changes what a bid is. Under bid-only GSP your bid is your max willingness to pay per click. Under relevance-weighted GSP, the platform’s model decides how much of your bid ever reaches the market; two advertisers bidding the same $2 face different effective prices because the model trusts one of them more. Industry explainers of Amazon’s auction now lead with exactly this point [4].

The mechanism itself is now learned

The research frontier has moved past hand-set scoring rules. Deep GSP (Zhang et al., WSDM 2021) replaces the b·q score with a neural network that maps bids and features directly to an allocation, trained to optimize a constrained mixture of platform objectives - revenue, CTR, CVR, even user experience - with Lagrange multipliers on the constraints [5]. Related work treats reserve prices and other mechanism parameters as a constrained optimization problem over business metrics [6]. Amazon’s own auction group studies what happens to these designs under realistic bidder behavior: in a model where advertisers are adversarial-bandit agents with partial feedback and unobserved competitors, they show “soft floors” (minimum scores rather than minimum prices) improve platform metrics, and that advertiser value distributions can be inferred from observed bids [7].

Section takeaway. The auction is GSP with a predicted-relevance multiplier, drifting toward learned mechanisms. Everything downstream - what the models estimate, how bids are chosen - is in service of the two terms in score = b·q. The platform owns q. The advertiser owns b. The interesting engineering is estimating both.

2. The prediction stack: pCTR and pCVR per (query, product, placement)

The whole auction turns on predicted probabilities, so the prediction stack is where most of the science is. The targets: P(click | query, product, placement) and P(conversion | click) - and their product, P(conversion | impression), which is what value-per-impression is made of.

Era 1: sparse logistic regression, engineered to death

Google’s 2013 “view from the trenches” paper [8] is the canonical account of the first industrial stack: logistic regression over billions of hashed binary features, trained online with FTRL-Proximal (per-coordinate learning rates, L1/L2 for sparsity), with hard-won lessons on calibration and memory. Facebook’s 2014 paper added the other classic trick: gradient-boosted trees whose leaf assignments become sparse features fed into LR - a learned feature crossing, plus an early demonstration that data freshness dominates model cleverness [9]. Two lessons from this era never stopped being true: (1) the pipeline and the features are most of the lift; (2) because the probabilities feed a pricing formula, calibration matters as much as ranking. An AUC-optimal model that overestimates click probability 15% overpays 15% in every auction. Log-loss and expected calibration error, not just AUC, are the operating metrics [8].

Era 2: interaction-aware deep networks

The 2016-2021 wave industrialized deep CTR models, each paper fixing a specific failure of the last:

  • Wide & Deep (Google, 2016): a wide linear side for memorizing explicit crosses (“user installed the app shown”) joint-trained with a deep side for generalization [10]. The template for everything after.
  • DeepFM (Huawei, 2017): replace the hand-built crosses with a factorization machine so pairwise feature interactions are learned embeddings, end-to-end, no feature engineering [11].
  • DCN / DCNv2 (Google, 2017/2021): cross networks that build bounded-degree feature interactions explicitly, layer by layer; the v2 paper is notable for its production lessons - mixture-of-experts decomposition of the cross matrix is what made it cheap enough to serve [12].
  • DIN (Alibaba, 2018): the e-commerce-native move. A shopper’s behavior history is a long sequence; DIN applies target attention - for each candidate product, attend over the history to extract the parts relevant to this candidate. The model’s representation of the user changes with the ad being scored [13].
  • DIEN (Alibaba, 2019): model the evolution of interests, not just their retrieval - a GRU over the behavior sequence with an auxiliary loss that forces each step’s hidden state to predict the next behavior [14].

On Amazon-shaped data (search queries, short-term session behavior, huge sparse catalogs), the transferable ideas are target attention and explicit interaction modeling; sequence models help most where the query is ambiguous and the session disambiguates (“mascara” after browsing waterproof everything).

The CVR problem is three separate problems

Conversion rate is the term that prices bids (margin × pCVR × pCTR), and it is statistically nasty in three distinct ways, each with its own literature.

(a) Sample selection bias. CVR models are traditionally trained on clicked impressions - but scored on all of them. The clicked subset is heavily selected (that’s the whole point of the CTR model), so a naive CVR model is evaluated far outside its training distribution. Alibaba’s ESMM (2018) fixes this with multi-task structure: never estimate pCVR alone; estimate pCTR and pCTCVR over the entire impression space, with pCVR = pCTCVR / pCTR emerging as an intermediate quantity. The CVR tower trains on every impression through the CTCVR term, so selection bias and data sparsity are both defused [15]. This is close to the industry default now.

(b) Delayed feedback. Clicks are logged in milliseconds; conversions arrive hours to weeks later (Amazon attributes sales up to 14 days post-click for Sponsored Products). Train on a window ending “now” and your positives are systematically censored: the model sees fresh clicks with not-yet-arrived conversions as negatives, and the bias is worst exactly for the most recent data - which is also the data that matters most for hourly repricing. Chapelle (Criteo, KDD 2014) modeled the delay distribution explicitly alongside the CVR, correcting the likelihood for censoring [16]; Ktena et al. (RecSys 2019) showed how to keep a continuously training neural model unbiased with importance-weighted duplicated samples - log the click as negative now, then re-inject it as positive when the conversion lands, weighted to keep the estimate honest [17].

(c) Tail sparsity. The head of Amazon’s query distribution is a rounding error. The mass - and, per several analyses, a large share of the inefficiency - is in the tail: keywords with 3 impressions and 0 clicks ever. A per-keyword MLE is useless there. The working tool is Bayesian shrinkage: model the conversion rate of each (keyword, placement) as a draw from a Beta prior shared across a cluster (category, semantic niche), and update the posterior as evidence accrues. A keyword with zero data prices at the cluster mean; each click moves it; the posterior width tells you how much to trust the point estimate - which doubles as an exploration signal (Thompson sampling against the bid landscape is the standard move). This is exactly the shape the problem takes when, as one autonomous-ads platform’s job posting puts it, “conversion rates [are] statistically modeled per search term and placement.” The interactive lab’s second widget is this exact loop.

Upstream of the models: matching, position bias, attribution

Three supporting literatures deserve a paragraph each.

Retrieval and matching. Before scoring, you need candidate ads for the query - exact/phrase/broad keyword match is a manual bottleneck, so the platforms learned it. Amazon’s semantic product search work frames matching as learned dense retrieval over a billion-product catalog [18]; follow-ups cover extreme multi-label matching [19] and cluster-adaptive keyword expansion with relevance tuning [20]. For a bidding agent, this matters because keyword coverage is itself a growth lever: the tail keywords nobody else found are the auctions nobody else priced correctly.

Position bias. Your training labels are poisoned: top-of-search gets clicked partly because it is top-of-search. The position-based model (PBM) factorizes P(click) = P(examine | position) · P(relevant | item), and the right way to learn the relevance term from logged, position-biased clicks is bandit learning-to-rank with counterfactual estimators - Amazon’s group published exactly this (CIKM 2020) [21], plus off-policy evaluation machinery that interpolates between click models [22]. If you skip this, your model learns “products at the top get clicked”, and your bids chase position instead of relevance.

Attribution. Amazon’s default is last-touch within a 14-day window; the same click can be claimed by Sponsored Products, Sponsored Brands, and DSP depending on where it lands. The platform’s 2025 multi-touch attribution work (attention-based models over the full touch sequence) is an attempt to price the marginal contribution of each touch [23]. For a bidder, attribution defines the reward signal your policy optimizes - and last-touch reward systematically overvalues bottom-funnel keywords (brand terms) and undervalues discovery terms. The organic/paid interdependence literature (Yang & Ghose: the two channels positively interact, they are not substitutes) is the economic version of the same warning [24].

3. The bidding brain: from a formula to a policy

Given the prediction stack, what should the bid be? In layers.

Layer 0: value. The value of an impression is v = pCTR · pCVR · m, where m is the margin on the expected basket (or lifetime value, if you model repeat purchase). Note the margin: optimizing revenue-attributed sales (the platform’s default ROAS metric) and optimizing profit are different objectives, and the difference is the bidder’s entire edge over an agent that chases platform metrics. Bids priced from true margins - contribution per conversion, product by product - are the difference between buying revenue and buying profit.

Layer 1: unconstrained. With no budget limit and a second-price payment, bidding near value is the baseline policy - you pay the minimum to hold position, so shading below value mostly loses auctions you’d have wanted. (In equilibrium terms: GSP’s locally envy-free equilibrium supports bids below value, but the safe operating point for an agent with good models is close to truthful [1][2].)

Layer 2: budgets. No real account is unconstrained, and the constraint changes the math from per-auction pricing to inter-temporal allocation: spend is a scarce resource to be spread over the day/week and over thousands of keywords. The theory answer is marginal equalization - at the optimum, the marginal return on the next dollar of spend is equal across every targeting clause, else you should shift budget. Amazon’s OPTIMUS system (2026) implements exactly this as a Lagrangian over forecasted bid landscapes: estimate, per clause, the spend→sales curve from historical auctions, then pick bids so marginal ROAS equalizes, provably, at a scale of millions of campaigns; +2-6% sales in online A/B tests [27]. Balseiro & Gur supply the game-theoretic foundation - how an individual bidder should pace a budget through repeated auctions, and what the market equilibrium looks like when everyone does [28] - and Balseiro et al.’s “field guide” catalogs the pacing algorithms (multiplicative feedback controllers dominate in practice) [29].

Layer 3: reinforcement learning. The auction is interactive and reactive: competitors respond to your bids, and your logs contain no counterfactual (you never observe what an unsubmitted bid would have done). That is the signature of a bandit/RL problem, and the literature took the hint. Cai et al. (2017) formulated RTB as an MDP - state = (time left, budget left, win-rate so far), value = expected conversions over the episode - solved model-based by dynamic programming [30]. Wu et al. (2018) went model-free for budget-constrained bidding [31]; Jin et al. (2018) made it multi-agent, clustering thousands of advertisers into jointly-learning clusters and showing cooperation/competition effects [32]; He et al.’s USCB (KDD 2021) unified the constrained variants (budget, ROI caps, CPC caps) into one RL-plus-Lagrangian framework [33]. Amazon’s own AuctionGym (Jeunen, Murphy, Allison) is the field’s most honest self-assessment: a unified framework showing most “learning to bid” methods are value-based bandits in disguise, plus policy-based and doubly robust formulations, plus a simulation environment precisely because offline evaluation from logged bids alone can’t be trusted [34]. The simulator won a best-paper award [35].

The production answer is a loop, not a policy. Real systems (Amazon’s hourly cadence is disclosed publicly; every serious third-party tool copies it) run a control loop: ingest the last hour’s auction logs → update per-(keyword, placement) posteriors with delay-correction → forecast the bid landscape → solve the budget-constrained allocation → write new bids → repeat. The RL framing describes what the loop approximates; the engineering reality is hierarchical control with Bayesian state estimation, re-solved hourly. The lab’s third widget is a playable version.

4. Measuring anything inside an auction is its own research area

Standard A/B testing assumes your treatment doesn’t leak into control. In an auction it leaks by construction: if the treatment group bids more aggressively, prices rise for the control group too, and every comparison is contaminated. This is marketplace interference, and it is now a literature: Johari et al. (2022) give the formal guidance for two-sided marketplace experiments, showing when first-price vs second-price formats suffer different interference structure and how to design around it [36]. The practical workaround at auction scale is the switchback - randomize over time blocks instead of users - with rerandomization to kill the carryover bias (Procter & Gamble’s version, 2025, is a good industrial account [37]), or shadow-price corrections that subtract the equilibrium effect [38].

Offline, the problem is bandit feedback: logged data only contains outcomes for bids actually submitted, at positions actually won. Inverse-propensity and doubly robust estimators (AuctionGym’s framing [34]) and off-policy click-model correction [22] are the standard tools, and simulation with realistic bidder populations (adversarial-bandit agents, per Amazon’s auction-realism work [7]) is the last line of defense before a policy touches customer money. A legitimate job posting in this space lists, verbatim, “design and run experiments on live auctions, with real spend as the readout” and “build simulations of auction dynamics to test a policy before it touches customer money” - that pairing is not marketing, it is the correct summary of the measurement literature above.

5. The autonomous loop, assembled

Put the pieces together and the full system is legible:

  1. State estimation (hourly): for every (keyword, placement), a posterior over CTR and CVR, delay-corrected [16][17], position-debiased [21], shrunk toward cluster priors in the tail.
  2. Value pricing: margin per conversion (not revenue) times the posteriors gives value per impression; the platform’s multiplier q divides through to translate value into bid.
  3. Allocation: equalize marginal ROAS across clauses subject to budget - Lagrangian over forecasted bid landscapes [27][29].
  4. Exploration: posterior width drives deliberate spend into uncertain tail keywords (Thompson-style), buying information where it’s cheapest.
  5. Measurement: switchbacks and shadow prices for anything causal [36][37][38]; simulator for anything new [34][35].
  6. Feedback: tomorrow’s posteriors inherit today’s experiment.

The flywheel is the compounding part: better estimates → better bids → more profitable spend → more spend to estimate with. The 24% conversion-rate improvement Amazon disclosed [25] is what four years of that loop looks like from the platform side. From the advertiser side, the edge is the same loop run with profit - not attributed revenue - as the reward, and with tail-keyword exploration the platform has no incentive to fund for you.

Open problems

  • Agents against agents. When both sides of the auction are learning algorithms, pricing is a multi-agent game, and the theory says what you’d fear: Q-learning-style pricing agents learn supra-competitive prices without being told to collude (Calvano et al., AER 2020 [39]). Sponsored ads is next.
  • Incrementality over attribution. MTA [23] still prices correlation. The real question - what sales wouldn’t have happened organically - needs experiments, and auction interference makes them expensive [36].
  • Learned mechanisms vs. learned bidders. Deep GSP [5] optimizes the mechanism against modeled bidders; RL bidders [34] optimize against the mechanism. Somebody’s model of the other side is wrong; that gap is where the surplus moves.
  • LLM-native retrieval. Dense/semantic matching [18][19][20] is still keyword-shaped. What happens when “every phrase buyers type, clustered into the niches they actually shop” is an embedding problem end-to-end - and the long tail of keywords stops being sparse because the model generalizes across it - is an open, funded question.

References

  1. Edelman, Ostrovsky, Schwarz. Internet Advertising and the Generalized Second-Price Auction. AER 2007. benedelman.org
  2. Varian. Position Auctions. Int. J. Industrial Organization 2007. berkeley.edu
  3. Ghose & Yang. An Empirical Analysis of Search Engine Advertising: Sponsored Search in Electronic Markets. Management Science 2009. doi.org/10.1287/mnsc.1090.1054
  4. Rel.ai. How Amazon’s Ad Auction Actually Works (It’s Not Highest Bid Wins). Feb 2026. rel.ai
  5. Zhang et al. Optimizing Multiple Performance Metrics with Deep GSP Auctions for E-commerce Advertising. WSDM 2021. arXiv:2012.02930
  6. Practical Constrained Optimization of Auction Mechanisms in E-Commerce Sponsored Search Advertising. 2018. arXiv:1807.11790
  7. Chen, Nabi, Siniscalchi (Amazon). Advancing Ad Auction Realism: Practical Insights & Modeling Implications. 2023. amazon.science
  8. McMahan et al. (Google). Ad Click Prediction: a View from the Trenches. KDD 2013. research.google
  9. He et al. (Facebook). Practical Lessons from Predicting Clicks on Ads at Facebook. ADKDD 2014. doi.org/10.1145/2648584.2648589
  10. Cheng et al. (Google). Wide & Deep Learning for Recommender Systems. 2016. arXiv:1606.07792
  11. Guo et al. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. IJCAI 2017. doi.org/10.24963/ijcai.2017/239
  12. Wang et al. (Google). DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems. WWW 2021. arXiv:2008.13535
  13. Zhou et al. (Alibaba). Deep Interest Network for Click-Through Rate Prediction. KDD 2018. arXiv:1706.06978
  14. Zhou et al. (Alibaba). Deep Interest Evolution Network for Click-Through Rate Prediction. AAAI 2019. doi.org/10.1609/aaai.v33i01.33015941
  15. Ma et al. (Alibaba). Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate. SIGIR 2018. arXiv:1804.07931
  16. Chapelle (Criteo). Modeling Delayed Feedback in Display Advertising. KDD 2014. doi.org/10.1145/2623330.2623634
  17. Ktena et al. (Twitter). Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR Prediction. RecSys 2019. arXiv:1907.06558
  18. Nigam et al. (Amazon). Semantic Product Search. KDD 2019. arXiv:1907.00937
  19. Chang et al. (Amazon). Extreme Multi-label Learning for Semantic Matching in Product Search. KDD 2021. arXiv:2106.12657
  20. Improving Ad Matching via Cluster-Adaptive Keyword Expansion and Relevance Tuning. 2025. arXiv:2505.18897
  21. Amazon. Learning to Rank in the Position-Based Model with Bandit Feedback. CIKM 2020. arXiv:2004.13106
  22. Amazon. Off-policy Evaluation for Learning-to-Rank via Interpolating the Item-Position Model and the Position-Based Model. 2022. amazon.science
  23. Amazon Ads. Amazon Ads Multi-Touch Attribution. 2025. arXiv:2508.08209
  24. Yang & Ghose. Analyzing the Relationship Between Organic and Sponsored Search Advertising: Positive, Negative or Zero Interdependence? ISR 2010. pages.stern.nyu.edu
  25. Amazon. Amazon’s Response to the FTC’s Lawsuit Regarding Sponsored Ads. Aug 2026. aboutamazon.com
  26. Amazon Q4 2024 earnings (advertising services revenue, $56.2B FY2024). s2.q4cdn.com
  27. Mondal, Kandregula, Agrawal, Sembium (Amazon). OPTIMUS: Optimal Offline Bidding Strategy for Manual Targeting Advertising Campaigns. 2026. amazon.science
  28. Balseiro & Gur. Learning in Repeated Auctions with Budgets: Regret Minimization and Equilibrium. Management Science 2019. ideas.repec.org
  29. Balseiro et al. A Field Guide for Pacing Budget and ROS Constraints. ICML 2024. proceedings.mlr.press
  30. Cai et al. Real-Time Bidding by Reinforcement Learning in Display Advertising. WSDM 2017. doi.org/10.1145/3018661.3018702
  31. Wu et al. Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising. CIKM 2018. doi.org/10.1145/3269206.3271748
  32. Jin et al. (Alibaba). Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising. CIKM 2018. doi.org/10.1145/3269206.3272021
  33. He et al. (Alibaba). A Unified Solution to Constrained Bidding in Online Display Advertising. KDD 2021. doi.org/10.1145/3447548.3467199
  34. Jeunen, Murphy, Allison (Amazon). Learning to Bid with AuctionGym. 2022. amazon.science
  35. Amazon Science. Amazon Scientists Win Best-Paper Award for Ad Auction Simulator. 2022. amazon.science
  36. Johari, Li, Liskovich, Weintraub. Interference, Bias, and Variance in Two-Sided Marketplace Experimentation: Guidance for Platforms. WWW 2022. dl.acm.org
  37. P&G / HBS. Reliable Switchback Experiments with Rerandomization for Auction Environments at Procter & Gamble. 2025. hbs.edu
  38. Reducing Marketplace Interference Bias via Shadow Prices. Management Science 2025. ideas.repec.org
  39. Calvano, Calzolari, Denicolò, Pastorello. Artificial Intelligence, Algorithmic Pricing, and Collusion. AER 2020. aeaweb.org

Eternal Horizons · a#evergreen note - permanent, constantly growing. Companion interactive lab: The Ad Auction, From the Inside. Format modeled on the Transformer Circuits article style. Sibling note: Looped Transformers, Deeply.

Graph View

Backlinks