To what extent are Polymarket prices well calibrated with eventual event outcomes?
HKU Capstone Project · 2025–2026
A Multi-Framework Analysis of Information Efficiency & Trading Signals in Decentralized Prediction Markets
Can crowd-sourced prediction markets like Polymarket serve as a reliable, real-time alternative to traditional forecasting benchmarks? We put that question to the test — using Polymarket & Kalshi as case studies.
01 · Introduction
What are prediction markets — and why do they matter?
Prediction markets are decentralised platforms where participants trade binary contracts on future events, backing their beliefs with real capital. A contract price in the range [0, 1] can be read directly as the market's consensus probability of an outcome.
These markets have expanded far beyond politics and sports into macroeconomic forecasting (Fed decisions, unemployment, inflation) and corporate events (earnings, IPOs). This raises a compelling question:
Can crowd-sourced prediction markets serve as a reliable alternative to traditional macroeconomic forecasting benchmarks?
To answer it, this project combines data collection, cloud database design, monitoring, and quantitative analysis to evaluate market efficiency and trading signals in a structured, reproducible way.
In Scope
- Polymarket (+ Kalshi for cross-market) data collection
- PostgreSQL cloud storage
- Calibration / information-incorporation / consistency measures
- Category comparison across market types
- Simulated strategy & arbitrage evaluation
Out of Scope
- Real-capital live trading
- Full cross-platform coverage of every venue
- Institutional low-latency optimisation
02 · Research Questions
Five questions guiding the analysis
How rapidly do Polymarket prices respond to relevant public information?
Are there observable pricing inconsistencies within Polymarket or across platforms such as Kalshi?
Do efficiency properties differ across categories (politics, sports, finance/macro, corporate)?
Can these observations support useful predictive or simulated trading strategies?
Three dimensions of "efficiency"
Forecast Calibration
Responsiveness
Internal Consistency
03 · Data & Infrastructure
A cloud-native pipeline built for scale
0
Minute-level records
0
Hourly observations
0
Resolved earnings markets
0
Resolved macro events
Cloud-native database
A PostgreSQL instance deployed on the cloud (Onidel) continuously ingests from Polymarket's Gamma API and CLOB API, alongside Kalshi data captured in a dedicated kalshi_trades store.
Four-level data hierarchy
Data follows a clean containment model: Series → Event → Market → Price history (per outcome token). Each market has two CLOB tokens (Yes / No); the Yes price ≈ the implied probability.
Data dictionary (condensed)
| Column | Type | Description |
|---|---|---|
| series_id | bigint | Unique series identifier (top-level grouping). |
| title | text | Human-readable series name. |
| category | text | Domain: politics, macro, corporate, sports. |
| created_at | timestamp | Series creation time. |
| Column | Type | Description |
|---|---|---|
| event_id | bigint | Unique event identifier. |
| series_id | bigint | Parent series (foreign key). |
| title | text | Event description (e.g. "FOMC March 2026"). |
| resolution_date | date | Scheduled/actual resolution date. |
| Column | Type | Description |
|---|---|---|
| market_id | bigint | Unique market identifier. |
| event_id | bigint | Parent event (foreign key). |
| yes_token_id | text | CLOB token id for the Yes outcome. |
| no_token_id | text | CLOB token id for the No outcome. |
| outcome | text | Resolved result once settled. |
| Column | Type | Description |
|---|---|---|
| token_id | text | Outcome token (Yes / No). |
| timestamp | timestamp | Observation time (minute / hourly). |
| price | numeric | Price in [0, 1] ≈ implied probability. |
| volume | numeric | Traded volume in the interval. |
04 · Methodology
From raw prices to rigorous metrics
Forecast construction
Discrete binary bins are converted into a single implied value through probability-weighting:
Implied = Σ(Yes prob × threshold) / Σ(Yes prob)
Evaluation metrics
Accuracy measured with MAE and RMSE across horizons (30d, 14d, 7d, 1d), and Brier score for calibration (0 = perfect, 0.25 = random-at-0.5).
MAE = (1/n) Σ |pᵢ − yᵢ| · RMSE = √[(1/n) Σ (pᵢ − yᵢ)²]
Brier = (1/n) Σ (pᵢ − yᵢ)²
Horizon-aligned panel design
Observations are grouped into aligned windows for fair comparison across markets:
Early · 90–31d
Mid · 30–8d
Pre-resolution · 7d–25h
Resolution · final 24h
Cross-market matching framework core
A three-stage pipeline underpins the latest cross-market work — linking economically-equivalent contracts across Polymarket and Kalshi.
01
Catalog Export
Extract contract catalogs from both venues for comparison.
02
Semantic Matching
Sentence-transformers all-MiniLM-L6-v2, cosine similarity on titles/descriptions.
03
Rule-based Review
Topic-family, date-proximity, contract-family & exclusion filters + human review.
04
Transformation
Map Kalshi thresholds to Polymarket brackets; sum Fed dissent-split contracts.
matching-pipeline.png
Chart coming soon
05 · Findings
What the data revealed
Across four analytical fronts, prediction-market prices proved consistently informative — often beating traditional forecasting benchmarks.
Macro Events — FOMC · Unemployment (U-3) · CPI
Polymarket vs institutional forecasting benchmarks across the macro calendar.
FOMC Brier 0.044 · 82% better than random
U-3 Brier 0.096 · 62% better
CPI Brier 0.107 · 57% better
FOMC vs CME FedWatch
Polymarket consistently outperforms CME FedWatch at all horizons, converging smoothly toward the eventual outcome.
| Horizon | PM MAE | FedWatch MAE |
|---|---|---|
| 30d | 0.061 | 0.084 |
| 14d | 0.043 | 0.062 |
| 7d | 0.028 | 0.041 |
| 1d | 0.011 | 0.019 |
fomc-convergence.pngChart coming soon
U-3 vs Bloomberg Consensus
Polymarket provides continuous 30-day forecasts versus Bloomberg's discontinuous coverage. An OLS of jobless-claims surprises vs next-day repricing gives R² = 0.52 (small sample n=8, suggestive not conclusive).
u3-convergence.pngChart coming soon
CPI vs Investing.com Consensus
Across 16 YoY + 14 MoM releases, results were broadly similar on YoY, with Polymarket better on MoM. A notable curiosity: peak accuracy around the 12-hour mark before release.
cpi-accuracy.pngChart coming soon
brier-by-event.pngChart coming soon
Corporate Earnings
Are decentralised markets informationally efficient for corporate earnings? Binary "beat/miss" contracts (tag 1013), Yes price = P(beat).
Brier 0.120
88% final accuracy
r ≈ 0.18 (weak return corr.)
382 scored events
Pipeline & data
DB export → analyst fetch via yfinance → comparison engine → evaluation. Coverage: 416 resolved / 209 unique markets, 382 scored events.
NVIDIA Feb 2026 — worked example
| Stage | P(beat) |
|---|---|
| Listed | ~0.47 |
| 7 days before | ~0.95 |
| Just before | ~0.9995 |
| Outcome | Beat ✓ |
earnings-brier-baseline.pngChart coming soon
Calibration & returns
Markets outperform both a random 50/50 and the historical beat-rate baseline. Quadrant analysis: correct beat +1.60%, correct miss −2.97%. Calibration is bimodal — the low-Yes bucket realised 19.5%, the high-Yes bucket 90.1%.
earnings-quadrant.pngChart coming soon
earnings-calibration.pngChart coming soon
IPO Timing
Markets: "IPOs before 2027?" (34 active) and "IPOs in 2025?" (25 resolved, all NO). Flagships: OpenAI, Anthropic, SpaceX, Cerebras. Raw 12.2M minute-level → 204,273 hourly.
Event-study: all significant @ 1%
Monotonic Brier decline
Option-like time decay
Finding 1 — rapid news incorporation
An event-study of cumulative abnormal probability change (CAPC) across 8–9 news events — all significant at the 1% level. Largest negative: OpenAI for-profit discussion; largest positive: Anthropic's $4B round.
Finding 2 — option-like time decay
A theta-gamma analogy: near-certain outcomes stay stable while uncertain ones drift.
Finding 3 — progressive learning
Brier score declines monotonically toward zero — the "all-NO" 2025 contract serving as a natural experiment.
ipo-eventstudy.pngChart coming soon
ipo-timedecay.pngChart coming soon
ipo-brier.pngChart coming soon
Fed / FOMC cross-market
High price alignment — mean absolute gaps rise from 1.39% (7d) to 2.97% (final). On paired Brier, Polymarket edges Kalshi at every horizon.
| Horizon | Abs Gap |
|---|---|
| 7d | 1.39% |
| 3d | 2.10% |
| 1d | 2.55% |
| Final | 2.97% |
crossmarket-fomc-gap.pngChart coming soon
US CPI cross-market
Larger gaps (4.1%–6.7%) due to fragmentation and threshold transforms — yet Polymarket consistently posts a lower Brier across all horizons.
Earnings feasibility audit
Of 316 Kalshi rows, 299 are qualitative "earnings_call_mention" and only 17 are possible quantitative beats — so a direct earnings cross-market comparison is currently blocked by contract-design mismatch.
crossmarket-cpi-gap.pngChart coming soon
Key findings
- Prediction-market prices are highly informative versus baselines.
- Prices converge in high-liquidity macro markets → a shared information environment.
- Contract design is critical to feasibility of cross-market comparison.
crossmarket-brier.pngChart coming soon
Trading Signals & Cross-Platform Arbitrage
Translating efficiency observations into simulated, testable trading logic.
Internal signals
- Within-market trend: p(1h) − p(7d)
- Momentum: p(1h) − p(12h)
- Revision: |Δp| > 10pp
- Benchmark-deviation vs historical beat-rate
Cross-platform signals
- Disagreement/gap: |P(PM) − P(K)| > 10%
- Lead-lag: detect Δp > 5% in a short window
- Measure 1–6h reaction on the other venue
Cross-platform arbitrage
Buy the cheaper YES on one venue + the cheaper NO on the other; the pair always pays $1.
Arb_Cost = min(P_poly_yes, P_kalshi_yes) + 1 − max(P_poly_yes, P_kalshi_yes)
Net_Profit = 1 − fee − Arb_Cost · base fee 2% (sweep 2–4%)
Pair A — exactly-25bp July-2026 hike
Efficient market, no exploitable segmentation. Net profit −0.40% (daily). ADF confirms mean-reversion but an unfavourable mean.
| Fee | Net Profit |
|---|---|
| 2.0% | −0.40% |
| 3.0% | −1.40% |
| 4.0% | −2.40% |
Pair B — any 2026 hike
A statistically & economically meaningful edge. Polymarket ~2.5pp cheaper; net profit +1.33% (daily), 61.9% positive days. Edge survives to ~3.3% fee; ADF mean-reverting with a positive mean → suitable for a limit-order grid.
| Fee | Net Profit |
|---|---|
| 2.0% | +1.33% |
| 3.0% | +0.33% |
| 3.3% | ~0.00% |
| 4.0% | −0.67% |
arb-spread.pngChart coming soon
06 · Key Takeaways
The big picture
Bounded-price microstructure
The [0,1] constraint creates mechanical mean reversion — a feature, not a bug.
Continuous information aggregation
Prices refine continuously versus discrete expert surveys → a real temporal advantage.
Cross-event heterogeneity
Quality ranks FOMC > U-3 > CPI; liquidity & participant expertise matter.
Adequate calibration
Estimates stay directionally reliable and beat random guessing — a valuable real-time, crowd-sourced complement to expert forecasts.
07 · Roadmap & Future Work
Where the project is heading
Phase 1
Data infrastructure & database
Cloud PostgreSQL, ingestion pipelines.
Phase 2
Market-efficiency measurement
Calibration, information incorporation, consistency.
Phase 3
Category & cross-market + IPO/earnings
Extension across market categories.
Phase 4 · current
Cross-market analysis & trading signals
Fed/CPI convergence, arbitrage pairs.
Phase 5
Final report & presentation
Consolidation and delivery.
Future directions
- Full lead-lag price-discovery analysis
- Expand to core CPI & corporate revenue-beat markets
- Refine automated contract transformation
- Real-time data streams & cross-venue arbitrage detection
Known limitations
- Modest sample sizes
- Longer-horizon data sparsity
- Daily EOD prices miss intraday moves
- Kalshi hourly data ~70% stale carry-forward (daily treated as primary evidence)