← field notes

The half second auction behind every ad you see

real-time-bidding programmatic-ads cookie-syncing ad-privacy

Open almost any page and somewhere in the layout there is a rectangle that was, for a few tens of milliseconds, empty. You never catch it empty, because by the time the page finishes drawing, an advertisement is resting there as if it had always belonged. The interesting part happened in the gap. In that sliver, a short description of you was written, broadcast to a crowd of companies, auctioned, and sold, and the ad you see is the receipt.

I sit on both ends of this market in a small way. I run a handful of editorial sites, the kind of small publishing that advertising quietly keeps alive, and my day job is mobile proxy lines and rented Android phones, infrastructure that lives adjacent to the ad world’s endless fight over what counts as a real human. Neither vantage point makes the machinery less strange. Watching it closely makes it stranger.

The slot announces you

The publisher sets aside space on the page for advertising because that advertising usually pays for whatever you came to read or watch. What goes in the space is decided nowhere in advance. The slot is a placeholder that says only: something will be sold here in a moment.

When your browser reaches that slot while assembling the page, it fires off a request that amounts to an announcement. A real person is here, on a screen of this size, on this kind of page, and the chance to show them something is for sale. That announcement is the starting gun, and everything that follows exists because an empty rectangle on a page you opened just told the wider advertising world that a human being had arrived and was briefly available.

Why milliseconds

The whole auction runs on a budget often under a tenth of a second, because it has to finish before the page finishes drawing. If the bidding took as long as a spoken sentence, you would stare at a blank box and leave. So the marketplace was engineered to complete inside a blink: the request goes out, many companies receive it, each decides what this moment is worth to it, offers come back, a winner is picked, and the winning ad is fetched and painted into the slot before you register that the page was still loading.

Speed here is a load-bearing fact rather than a detail. Nearly every architectural choice in this system, including the ones that leak, exists because the whole transaction must fit inside the time a page takes to appear.

The dossier

The message that goes out is called a bid request, and it reads like a small dossier of the moment. It carries the type of device you are on and its screen size, the operating system and browser, the general kind of page you are viewing, and a coarse sense of where you are, usually derived from your network address down to about a city. Frequently it carries interest categories too, tidy labels some system attached to you earlier: reads about travel, shops for cars, new parent, looking at loans.

Threaded through all of it is the important part, an identifier. A long string that never spells out your name and does something arguably more useful to the buyers: it ties this exact moment to a history of other moments already collected under the same string. A bid request is no wanted poster. It is quieter and more revealing than that, a machine readable summary of who the page believes you are, composed so that strangers can decide what you are worth.

Read aloud to the whole room

Here is the center of the privacy story. That request goes to no single trusted buyer. It is broadcast, sprayed to every potential bidder at once, sometimes dozens of companies and sometimes far more, so that as many as possible can compete on price.

Every recipient gets the full dossier. The device, the location, the interest labels, the identifier. Whether they bid or decline. Whether they win or lose. The auction only ever needed one winner, yet a description of you was copied to everyone invited, and the losers walked away holding it, having paid nothing and shown you nothing. Less a sealed envelope handed to a buyer, more your details read aloud in a crowded room to see who raises a hand.

Picking a winner

Each company that receives the request runs it through its own logic inside its few milliseconds. Does this person match anyone I have been paid to reach, and if so, what is this single impression worth right now. Back comes a number and the ad it would show. The auction compares the offers, takes the highest, pays the publisher the price, and paints the winner into the rectangle.

Then it is over, and it never generalizes. Every impression on every page load is its own fresh sale, decided in the moment, for an audience of exactly one.

How trackers agree on who you are

One puzzle remains before the most familiar experience in online advertising makes sense. The identifier one company knows you by is usually a different string from the one another company uses somewhere else. Each keeps a private label for you. For your history to travel, two firms have to discover that their two labels point at the same person.

The mechanism is usually called cookie syncing, or id syncing, and stripped of jargon it is two trackers comparing notes. When you land on a page where both are present, they arrange behind the scenes to swap their identifiers for you, so each can record that its label and the other’s label refer to one viewer. Repeated across thousands of sites, the swaps build a vast shared translation table. No single company needs to see your whole life. They only need to agree on who you are, and syncing is that handshake.

Why the ad follows you

Now the haunted feeling has a mechanism. You look at a product once and a system tags your identifier with that interest. Because of syncing, the tag travels. It rides along in the bid requests fired from unrelated pages you open days later, where a bidder translates the identifier back to its own records, recognizes the person who looked at that product, and pays for the chance to show it again.

Nothing is chasing you. The same profile is being read aloud at each new auction, and the same buyer keeps recognizing you and making the same offer. A marketplace with a very long memory produces something indistinguishable from a haunting.

Who else is in the room

Fairness requires naming who benefits, because the room holds more than trackers. This machinery is how a niche blog or an independent creator gets paid at all, selling space into a global market in real time, earning from visitors it will never meet, with no need to strike private deals with every advertiser. Many bidders are ordinary businesses trying to reach people plausibly interested in what they sell rather than shouting at everyone equally. And the exchanges running the auctions act as referees, filtering the considerable volume of fake traffic, because a great deal of money in this market quietly leaks to bots pretending to be human.

Funders, buyers, and referees stand in the same room as the trackers, all pressed into the same fraction of a second. The mixture is what makes the argument about this system genuinely hard.

What is being rebuilt

The leak at the center is now being dismantled, slowly and unevenly. The most consequential shift is the long retirement of the third party cookie, the humble mechanism that made cross site identifiers cheap in the first place. As browsers phase it out, the old style of stitching identity together through syncing gets much harder, and the broadcast model has to be rethought.

The replacement proposals mostly move the sensitive part onto your own device. Your browser holds your interests locally and reveals only coarse, grouped signals to bidders, so no company walks away from an auction carrying a personal dossier. Privacy rules in a growing number of places push from the other side, requiring genuine consent before data like this changes hands and treating the careless spraying of a profile to unknown companies as something a business answers for. None of it is a finished cure, and every version wrestles with the same trade: fund the open web without describing its readers to strangers.

The bind

Both faces of this system are real at once. The half second auction is the financial engine of most of the free internet. The articles you read without paying and the tools you use for nothing are funded, invisibly, by these slots being sold. Remove it and much of what people expect for free gets a price tag or disappears. Yet the same system is an automated marketplace in human attention that discloses you, many times a day, to rooms full of companies you have never heard of, most of which gain nothing from the sale except a fresh copy of your profile. The funding and the leak are currently the same half second. You cannot yet keep one and delete the other, which is exactly what the rebuilding effort is trying to change.

Most people still picture advertising as a poster, chosen in advance and shown to everyone who walks past. That model is gone. Every slot on every page is a private auction for an audience of one, in which the description of you is the commodity and the ad is only the receipt. The free page was paid for in real time, by a fraction of a second in which you were summarized, broadcast, and sold to the highest bidder, before the words you came to read had finished appearing.

More on the machinery running underneath the ordinary internet at The Hidden Internet.

Get new guides and videos first — join the Telegram channel.

watch
The Hidden Internet on YouTube

Every field note starts as a short documentary. Watch the systems in this piece explained on screen.

subscribe →
read on
More field notes

The rest of the series on how the modern internet detects, tracks, and sorts the traffic that reaches it.

browse the archive →