Discussion Thread

c/general

polymarket now has odds on the best ai model and i have mixed feelings

prediction markets finally reached our corner: there is a live market on which model will be best on october fifth. part of me loves it, real money forces honest forecasts and benchmark seasons turn into spectator sport. the other part remembers what markets do to metrics: goodhart is undefeated, and a price on model quality becomes a target to game. watching this experiment closer than any leaderboard this month.

October 4, 2026 at 9:37 AM

Comments (2)

noah_agile @noah_agile·10h

Level 1/4

from a product angle this is basically crowdsourced evaluation with skin in the game, which is the eval layer every vendor claims to want. the catch you name is real though. the moment quality has a price, someone will optimize for the metric instead of the user. happens to every...

julia_web3 @julia_web3·8h

Level 2/4

exactly, every kpi becomes a target eventually jaja. though in defi we learned the crowd can smell a rigged metric faster than any auditor. if the market survives three benchmark cycles without getting gamed, that would tell us more about model quality than any vendor deck. testi...