Accountability
Model report card
Every Courtside model graded in public against a simple baseline, with its biggest misses and a log of forecasts made before tip-off.
At a glance
- Win probability0.207Strong 16.6% better than baseline
- Live win probabilityComing soon: not graded yet
- Player projectionsComing soon: not graded yet
- Season simulatorComing soon: not graded yet
- Shot quality0.644Solid 1.9% better than baseline
- Is it real?1.99Solid 9.9% better than baseline
- Impact ratings63.3Even 0.3% better than baseline
Each tile shows the headline error, how many forecasts it covers, and how it compares with the simple baseline for that model. How grading works ยท Forecast log
01Win probability
No 2026-27 games are final yet, so the live grade starts on opening night. Until then, this is the model's 2025-26 backtest, replayed one day at a time so no forecast saw its own result.
2025-26 backtest, regular season
| Forecast | Games | Picked winner | Brier | Log loss |
|---|---|---|---|---|
| This model | 1,230 | 68.0% | 0.207 | 0.603 |
| Plain Elo | 1,230 | 66.3% | 0.212 | 0.612 |
| Home team always | 1,230 | 55.4% | 0.248 | 0.689 |
| This model, Dec 1 on | 931 | 68.4% | 0.209 | 0.607 |
Brier score by forecast, on one scale starting at zero. Shorter is better.
Calibration, 2025-26 backtest
02Biggest misses
The 10 finals where the winner had the lowest pregame chance. The last column is a simple rule, not a judgment: any player who averaged 25 or more minutes over his team's previous 5 games (playing at least 3 of them) and did not play. The model never sees who is playing, so those misses were hard to see coming.
| Date | Game | Winner's pregame chance | Final | Did not play |
|---|---|---|---|---|
| Sat, Nov 22 | SAC at DEN | SAC 2.6% | SAC 128, DEN 123 | Aaron Gordon (DEN, 25.5 min); Domantas Sabonis (SAC, 33.7 min) |
| Mon, Dec 1 | HOU at UTA | UTA 3.6% | HOU 125, UTA 133 | No rotation player out |
| Mon, Dec 1 | DAL at DEN | DAL 4.1% | DAL 131, DEN 121 | P.J. Washington (DAL, 34.5 min) |
| Fri, Nov 21 | BKN at BOS | BKN 6.1% | BKN 113, BOS 105 | No rotation player out |
| Tue, Nov 25 | ATL at WAS | WAS 6.3% | ATL 113, WAS 132 | KyShawn George (WAS, 34.0 min) |
| Sat, Mar 7 | BKN at DET | BKN 7.3% | BKN 107, DET 105 | Cade Cunningham (DET, 38.0 min) |
| Fri, Jan 23 | IND at OKC | IND 7.3% | IND 117, OKC 114 | Ajay Mitchell (OKC, 26.2 min); Aaron Wiggins (OKC, 25.8 min); Jalen Williams (OKC, 25.3 min) |
| Sat, Nov 29 | TOR at CHA | CHA 7.5% | TOR 111, CHA 118 | RJ Barrett (TOR, 27.7 min) |
| Sat, Dec 6 | SAC at MIA | SAC 8.7% | SAC 127, MIA 111 | Tyler Herro (MIA, 31.5 min); Davion Mitchell (MIA, 30.0 min) |
| Sat, Dec 6 | HOU at DAL | DAL 8.8% | HOU 109, DAL 122 | Alperen Sengun (HOU, 32.8 min) |
03Live win probability
Built. Grades start once its forecasts have finished games.
Its own published test: Brier, test model 0.150, Accuracy, test model 0.779, Log loss, test model 0.453. Those numbers come from a replay of past games; the grades here only count forecasts made before the games were played.
04Player projections
Built. Grades start once its forecasts have finished games.
05Season simulator
Preseason projections are logged. Grading starts after opening night.
All 30 preseason projections
| Team | Projected wins | Record | Pace |
|---|---|---|---|
| Spurs | 65.3 | 0-0 | n/a |
| Thunder | 64.3 | 0-0 | n/a |
| Knicks | 64.1 | 0-0 | n/a |
| Pistons | 56.3 | 0-0 | n/a |
| Celtics | 53.2 | 0-0 | n/a |
| Cavaliers | 52.2 | 0-0 | n/a |
| Rockets | 52.2 | 0-0 | n/a |
| Raptors | 51.1 | 0-0 | n/a |
| Hornets | 49.0 | 0-0 | n/a |
| Nuggets | 48.6 | 0-0 | n/a |
| Magic | 48.3 | 0-0 | n/a |
| Heat | 47.1 | 0-0 | n/a |
| Suns | 46.6 | 0-0 | n/a |
| Timberwolves | 45.7 | 0-0 | n/a |
| 76ers | 44.9 | 0-0 | n/a |
| Hawks | 43.8 | 0-0 | n/a |
| Lakers | 43.4 | 0-0 | n/a |
| Trail Blazers | 43.1 | 0-0 | n/a |
| Clippers | 37.2 | 0-0 | n/a |
| Warriors | 33.2 | 0-0 | n/a |
| Pelicans | 30.4 | 0-0 | n/a |
| Mavericks | 27.3 | 0-0 | n/a |
| Bucks | 25.3 | 0-0 | n/a |
| Pacers | 24.7 | 0-0 | n/a |
| Kings | 24.7 | 0-0 | n/a |
| Bulls | 24.3 | 0-0 | n/a |
| Jazz | 23.1 | 0-0 | n/a |
| Grizzlies | 22.6 | 0-0 | n/a |
| Nets | 21.8 | 0-0 | n/a |
| Wizards | 16.1 | 0-0 | n/a |
06Shot quality
Expected make chance for every shot, tested on 43,788 shots from 246 games the model never trained on. It ranks makes above misses with an AUC of 0.644 against 0.632 for a zone-only lookup (0.5 is a coin flip, 1 is perfect), and its log loss is 0.653 against 0.659 for zones and 0.692 for one league-wide rate.
Log loss, lower is better
07Is it real?
| Stat | Players | Missed by, raw 10-game number | Missed by, our blended number | Grade |
|---|---|---|---|---|
| Points per game | 224 | 2.21 | 1.99 | Solid |
| True shooting | 224 | 5.8% | 3.8% | Strong |
| 3-point % | 201 | 7.4% | 4.6% | Strong |
| Free throw % | 198 | 9.7% | 5.9% | Strong |
Predicting the rest of the season from a 10-game stretch: the raw stretch against our number that blends it with the player's earlier games by how fast the stat stabilizes. Lower misses are better.
08Impact ratings
Impact ratings (RAPM) predict the scoring margin of 36,423 lineup stints from who is on the floor. On games held out of the fit, the error is 63.29 points per 100 possessions against 63.49 for assuming every lineup is average: 0.32% better. That is small because single stints are mostly noise. The useful signal is in the player ratings, which line up with plain on/off numbers at r = 0.81.
09Forecast log
Every forecast is written to the log when it is made, with a timestamp, and the log cannot be edited. Once a game tips off, its forecast is frozen.
How this works: how grading works
The scores
A Brier score is the average squared miss of a percentage. Say the model gives a team 70% and it wins: the miss is 0.3, squared 0.09. If it loses, the miss is 0.7, squared 0.49. Averaged over every game, 0 is perfect and 0.25 is what you get by calling every game 50/50. Log loss is similar but punishes confident misses much harder: a 95% favorite losing costs about 3.0, a 60% favorite losing about 0.9. For projections we use the average miss in points, rebounds or wins (mean absolute error), and for the shot model the AUC, the chance it rates a random make above a random miss.
Why every grade has a baseline
A Brier of 0.21 means little on its own, so each model is compared with the obvious guess: the home team at its usual win rate for win probability, the pregame number held all game for live win probability, the player's season average for projections, 41 wins for every team for the simulator, a zone-only lookup for shot quality, the raw stretch for Is it real?, and an average lineup for Impact ratings. The grade word comes from how much smaller the model's error is: Strong at 10% or more, Solid at 3% or more, Even within 3% either way, Behind otherwise (for AUC, where gains are smaller: 3% and 1%). Example: in the 2025-26 backtest, win probability scored 0.207 against 0.248 for always taking the home team, a 16.6% smaller error, so it grades Strong.
Logged before tip-off, never edited
Every time a game forecast moves, the new number goes into the forecast log with a timestamp. The log only accepts new rows: the database rejects any edit or delete. Once a game tips off, the stored forecast is frozen too, and live grades use the last forecast logged before tip-off. Player projections and simulator runs are logged the same way each morning once those models are running.
Where the numbers come from
Until 2026-27 has finals, the win probability grade is the 2025-26 backtest: each day's games were forecast using only games played before that day. The summary tile switches to the new season after 50 finals, because a few opening-week games say little. Shot quality, Is it real? and Impact ratings are graded on data held out of their fits, as each page explains. The biggest misses list uses the stored forecast for each final. Grades for the live chart, projections and the simulator are recomputed early every morning, and switch to 2026-27 on their own once games are final.
What we do not grade yet
Playoff and play-in games, the projected margin, player tiers, similar players and lineup numbers. The misses context is a rule (a regular who did not play), not a reason: it cannot see foul trouble, minutes limits, garbage time or a hot shooting night. One season of backtest is a small sample, so a grade can move a step with a few dozen games.