How accurate is PassCast?
The short answer: seasonal snowfall is barely predictable, and we can prove it with our own numbers. Across 8,712 back-tested resort-seasons, PassCast beats a plain climatological forecast by 0.8% (CRPS skill score) — real, but small. Skill is concentrated in 5 regions with a strong El Niño teleconnection and is effectively zero in the rest. Any ski forecast promising to tell you how deep next winter will be is overselling; the honest product is a probability range and a bust risk.
Skill vs climatologyCRPSS — above 0 means we beat “assume an average winter”
CRPS is the standard score for a probabilistic forecast: 42.08" for PassCast against 42.41" for climatology, lower being better. The middle figure is what this site used to ship — a single ENSO strength applied everywhere, which verified worse than ignoring ENSO entirely. The right-hand figure is the strictest test, where the tuning never sees the season it is graded on.
Is the uncertainty honest?Calibration — does reality land inside our stated range?
When we say 80% of outcomes fall between P10 and P90, 78.2% actually did — slightly overconfident, so treat the published range as marginally too narrow. The distribution of outcomes across our forecast percentiles is close to flat (11 · 10 · 9 · 10 · 9 · 9 · 10 · 11 · 9 · 11%, where a perfectly calibrated forecast reads 10 across), with mild excess in the tails — real winters are a little more extreme than our simulation expects.
Where the forecast has real skill — and where it has none
We tune how heavily each region leans on ENSO by what actually verifies, then switch the signal off where it doesn't earn its place. These are the regions where a season-ahead forecast is worth reading, and the regions where you should simply plan for a typical winter. The skill column is the strict out-of-sample figure: the region's ENSO weight is re-chosen without ever seeing the season it is then graded on.
| Region | Skill (out-of-sample) | ENSO weight used | Mountains | Seasons scored |
|---|---|---|---|---|
| PNW | 2.3% | 50% | 14 | 504 |
| Northern Rockies | 1.1% | 50% | 14 | 504 |
| Interior BC | 0.9% | 50% | 11 | 396 |
| Canadian Rockies | 0.4% | 40% | 6 | 216 |
| Andes | 0.3% | 40% | 10 | 360 |
| Colorado | none | ENSO off | 20 | 720 |
| Utah | none | ENSO off | 11 | 396 |
| Sierra | none | ENSO off | 14 | 504 |
| Northeast | none | ENSO off | 32 | 1,152 |
| Pyrenees | none | ENSO off | 3 | 108 |
| Japan | none | ENSO off | 14 | 504 |
| Australia | none | ENSO off | 5 | 180 |
| Mid-Atlantic | none | ENSO off | 5 | 180 |
| Alps | none | 10% | 42 | 1,512 |
| Western Canada | none | 30% | 4 | 144 |
| Southwest | none | 30% | 8 | 288 |
| Scandinavia | none | ENSO off | 5 | 180 |
| Midwest | none | 20% | 10 | 360 |
| New Zealand | none | ENSO off | 7 | 252 |
| Eastern Canada | none | 20% | 7 | 252 |
Genuine skill: PNW, Northern Rockies, Interior BC, Canadian Rockies, Andes — the regions where El Niño and La Niña reliably move the storm track. No usable seasonal signal: Colorado, Utah, Sierra, Northeast, Pyrenees, Japan, Australia, Mid-Atlantic, Alps, Western Canada, Southwest, Scandinavia, Midwest, New Zealand, Eastern Canada. In these regions PassCast forecasts from the mountain's own 36-season record and says so on the resort page.
How we tested
For every mountain and every season in its 36-year record, we hide that season, build the forecast from the remaining years, and score it against what actually fell — leave-one-season-out cross-validation, 8,712 forecasts in total. Scoring uses CRPS, which rewards a forecast for putting probability near the truth and penalises both overconfidence and vagueness.
Three limitations we will not paper over. First, the back-test hands the model the observed ENSO state for the hidden season, so it measures the teleconnection, not our ability to predict ENSO — real-world skill is lower than the figures above. Second, it scores the analog engine alone; the live site also applies NOAA CPC and ECMWF SEAS5 tilts, which cannot be back-tested without archived historical outlooks. Third, “truth” here is bias-corrected ERA5 reanalysis snowfall, not a resort snow stake.
The back-test is reproducible from this repository's data: node scripts/backtest.mjs and node scripts/tune-analog-weight.mjs. Full method on the methodology page.
Forecast accuracy: straight answers
- How accurate is PassCast?
- Measured honestly: only slightly better than assuming an average winter. Across 8,712 resort-seasons of leave-one-season-out back-testing, PassCast's forecast scores a CRPS of 42.08" against 42.41" for plain climatology — a skill score of 0.8%. Under stricter nested cross-validation it is 0.2%. Seasonal snowfall is close to unpredictable months ahead; anyone claiming otherwise is overselling.
- Can anyone predict how much snow a ski resort will get this season?
- Not with much precision. Day-to-day weather models lose skill beyond about two weeks. What carries real signal months out are slow climate drivers — above all ENSO — and even those only help in regions with a strong teleconnection. Our back-test finds genuine skill in PNW, Northern Rockies, Interior BC, Canadian Rockies, Andes, and effectively none elsewhere. The useful output is not a single number but a probability distribution and an honest bust risk.
- Is an El Niño forecast enough to predict a ski season?
- No, and we can quantify it. Conditioning our forecast on ENSO at a flat strength everywhere actually made it WORSE than ignoring ENSO (skill score -2.9%). Conditioning on a weak predictor costs you effective sample size and buys signal that isn't there. We now vary how much each region leans on ENSO by what verifies, and switch it off entirely where it doesn't.
- What does the skiability rating actually measure?
- It is a 0-100 blend of four things from the same simulation: absolute snow quantity (35%), this season versus the mountain's typical winter (30%), the count of 6-inch-plus powder days (20%), and bust risk (15%). It is not a percentage of average, and it is not a measure of terrain quality, crowds, or grooming.
We publish this because a forecast you cannot audit is worth nothing. If the numbers here get worse, they will still be published.