A new report from the thinktank Open Markets Institute scopes out the current state of AI content licensing for news publishers. “Same Gatekeepers, New Tollbooths: Mapping the AI Content Licensing Market” explores the emerging market for content licensing, arguing that news publishers are currently in a “double bind”: The same big tech companies that are developing commercial AI products and stripping news publishers of site traffic are the ones dictating what alternative revenue will look like. As the authors put it, Big Tech is “occupying both sides of the value chain simultaneously.”
“The deal structures, price precedents, intermediary take rates, and governance norms taking shape now will be difficult to revise once they are normalized,” write the authors Courtney Radsch and Karina Montoya, both from the institute’s newly named Center for Media & Digital Governance. (It previously went by The Center for Journalism & Liberty). “The question of whether publishers, journalism, or creators of any sort can make a credible collective claim before market structures crystallize will not stay open indefinitely.”
One of the most interesting sections of the report is a deep dive into new AI content licensing marketplaces, which often take a cut of the revenue they bring in for publishers. This includes new startups like Sphere.ai, ScalePost, Defined, and Tollbit, but also ones operated by Big Tech companies. Last summer, Cloudflare, which services about 20% of global web traffic, launched its “pay-per-crawl” marketplace, which allows publishers to set rates and charge AI companies each time one of their bots crawls their content. In February, Microsoft announced its Publisher Content Marketplace (PCM), which follows a “pay-per-use” model that allows publishers to sell “rights-cleared content” at set prices to Microsoft, and potentially to other AI developers.
Most commercial AI products repeatedly scrape news publications and retrieve up-to-date information from websites in order to answer specific user queries. This is known as retrieval augmented generation (RAG). The promise of these marketplaces is that they are building out new infrastructure that would allow news publishers to earn revenue from RAG systems. But the middleman marketplaces are also taking a big cut of that revenue, the report notes.
A startup like ScalePost takes roughly a 15% cut of the revenue earned by “rights holders,” while competitors TollBit and Sphere.ai take closer to 20–30%. The authors estimate, based largely on interviews with stakeholders, that Cloudflare is taking about a 30% cut of revenue.
Meanwhile, ProRata.ai, a startup that has developed its own answer engine built exclusively on licensed publisher content, shares subscription and advertising revenue with publishers 50/50. However, each publisher is paid proportionally based on attribution, or how often their content appears in the answer engine’s results. As of last summer, over 500 publishers had signed up with ProRata.ai.
It is yet to be seen just how much Microsoft will take from publishers that participate in its PCM.
The report points to Spotify as an important benchmark for evaluating these various “take rates.” Historically, Spotify has taken a 30% cut of revenue from streaming. Despite many drawbacks, that model has allowed music rights holders to earn significant revenue and propped up the industry during its transition to streaming. Still, the report concludes that further scrutiny of these marketplaces is needed, particularly when Big Tech is the one building the scaffolding.
“Regulatory attention is warranted on these platform operators in order to mitigate their data access advantages and ability to set de facto (and potentially coercive) standards for an industry in which no independent standards yet exist,” the authors write.
You can read the full report on Open Markets, including more specific policy recommendations.
