LearnBlog

We tried to beat the random walk on five currency pairs. Here is the scoreboard

We ran the FX-forecasting test on five years of live data across five pairs. The random walk won. The more interesting result is how easily a couple of defensible choices could have made it look like we had beaten it.

6 Jul 2026 · 8 min read

We took the test from our tutorial and ran it live on five currency pairs, five years of daily data. The random walk held up on every pair. But the honest story is not "our model lost." It is that two ordinary, defensible analysis choices were enough to make one pair look like a real, statistically significant edge, which is exactly why FX-prediction claims deserve a skeptical read.

  • At the setup we fixed in advance (70/30 split, plain standard errors), no pair beat the random walk with statistical significance.
  • Adding 300 features made out-of-sample accuracy worse on every pair, landing at -20% to -25% R². Complexity is a mirage here, exactly as the 2025 Fed paper found.
  • Switching to HAC standard errors, or moving the train/test split, pushes two pairs across the 5% significance line. Nothing changed but the analyst's choices.
  • Corrected for multiple testing (Bonferroni across five pairs, t > 2.33), nothing survives. That is the actual result.
  • Every number is reproducible on the free tier with the code from the companion tutorial.

The setup

Same recipe as the tutorial, applied identically to every pair:

  • Data: daily rates from January 2021 to January 2026 via the /v1/range endpoint, with forward-filled weekend and holiday rows dropped.
  • Split: first 70% to train, the final 378 trading days (the same count for all five pairs) held out. The model never sees the test window.
  • Simple model: Ridge regression on eight cheap features (return lags, one-week and one-month momentum, ten-day realized volatility).
  • Complex model: the same features blown up into 300 Random Fourier Features, one shared random basis for train and test.
  • Benchmark: the driftless random walk. Its forecast for tomorrow's return is always zero.
  • Scoring: out-of-sample R² against the random walk, and the one-sided Clark-West test. A t-stat above 1.645 clears the 5% bar.

We picked the 70/30 split and the plain standard errors before looking at any result, which matters for what follows.

The scoreboard

Out-of-sample results, 70/30 split (dir. acc = simple model)
pair       OOS R2    CW t   p-value    complex R2   dir. accEUR/USD    -0.34%    0.84   0.200      -20.5%       49.5%GBP/USD    -1.23%    0.17   0.432      -23.8%       51.9%USD/JPY    -2.29%   -0.33   0.631      -24.9%       53.2%AUD/USD    -0.52%    1.10   0.136      -24.2%       48.7%USD/CAD    +0.61%    1.50   0.067      -20.5%       52.1%

At the setup we committed to up front, nobody beat the random walk. Four of five pairs post a negative out-of-sample R², meaning the trained model did worse than assuming no change at all. The best Clark-West t-stat in the table is 1.50 (USD/CAD), short of the 1.645 you need for a 5% edge. USD/CAD's +0.61% R² is the only positive cell, and its p-value of 0.067 misses the bar. Read the table as printed and the story is: no edge.

Complexity made it worse, on every pair. The 300-feature model did not uncover hidden structure. With only about three training rows per feature, it memorized the training noise and fell apart out of sample, landing at -20% to -25% R² across the board. On USD/JPY its Clark-West t-stat is -2.05: not just worse than the random walk, significantly worse. In-sample, that same model fit beautifully. It looked brilliant right up until it met data it had not seen, which is the whole cautionary tale of complexity in one pair.

Direction was a coin flip. Predicted up-or-down landed between 48.7% and 53.2% correct. With 378 test days, anything inside roughly 45-55% is statistical noise, and every pair sits there. A coin gets you 50%.

How you could have "beaten" it anyway

Here is the part that should make you distrust any FX-prediction result, ours included. "Nobody beat the random walk" was true for the exact choices we fixed in advance. Change one or two of them, each defensible on its own, and the picture shifts.

Swap the standard errors. We used plain standard errors in the Clark-West test. A reviewer could reasonably ask for Newey-West (HAC) errors instead, since daily forecast errors carry mild autocorrelation. Make that one substitution and two pairs cross the 5% line:

PairClark-West t (plain)Clark-West t (HAC)
AUD/USD1.101.66, significant
USD/CAD1.501.83, significant

Nothing about the data moved. Only the formula for the error bars did, and now there are two "significant" results where a moment ago there were none.

Move the split. We held out the last 30%. Try the last 40%, an equally arbitrary line, and USD/CAD's Clark-West t-stat climbs to 2.26. It stays positive at every split we tried:

USD/CAD across train/test splits
split          60/40    65/35    70/30    75/25    80/20OOS R2        +1.25%   +0.33%   +0.61%   +0.87%   +0.71%Clark-West t    2.26     1.58     1.50     1.43     1.31

With two knobs (which standard error, which split) and five pairs to pick from, a motivated analyst can produce a headline that reads "machine learning beats the random walk on USD/CAD, statistically significant." Every step is defensible in isolation. Stacked together, they are how a spurious result gets into a paper.

The discipline that kills it

Fix your choices before you see the numbers, score out-of-sample, and correct for how many things you tested. Checking five pairs at the 5% level carries about a 23% chance of at least one false positive by luck alone, so the honest threshold is higher: a Bonferroni correction across five pairs pushes it to roughly t > 2.33. USD/CAD's best showing, that 2.26 at a 60/40 split, still falls short. Across every pair, split, and standard-error choice we ran, nothing clears the honest bar. That is the actual result.

Why this keeps happening

Short-horizon major-pair FX is about as close to efficient as markets get. Trillions of dollars change hands daily, and any pattern simple enough for a Ridge model to spot on price history alone gets arbitraged away long before it reaches your notebook. Today's price is the best available point forecast for tomorrow's, because the conditional mean of the next return is essentially no change. The models are not broken. That efficiency is the finding.

Note the word point, though. The conditional mean being unpredictable does not make the whole distribution unpredictable. Volatility clusters, ranges widen and narrow, and that structure is both real and forecastable. It is also where the honest value lives.

What we take from this

We build a rates API, so we will say the quiet part plainly. We are not going to ship a "predicted rate" endpoint, because on daily major pairs we cannot honestly beat return today's rate. What we can do is serve today's rate accurately, fast, and with the metadata (source, forward-fill flags, volatility) that lets you quantify uncertainty instead of pretending it away.

If you want to try anyway, everything above is reproducible on the free tier, using the code from the companion tutorial. The bar we will actually be impressed by: pick your split and standard errors up front, and clear a Bonferroni-corrected Clark-West stat across all five majors out of sample. Manage that and you have done something the literature says is genuinely rare.

Questions

Did any model beat the random walk?

No. Across five major pairs on five years of daily data, no model beat a no-change random-walk forecast with statistical significance once choices were fixed in advance and corrected for multiple testing. Four of five pairs had negative out-of-sample R².

Does adding more features (complexity) help FX forecasting?

No. Expanding the model to 300 Random Fourier Features made out-of-sample accuracy worse on every pair, at -20% to -25% R², because it overfits the training noise. This matches the 2025 Federal Reserve finding that complexity offers only fragile, localized gains.

How can a spurious FX-prediction result look significant?

Researcher degrees of freedom. Switching from plain to HAC standard errors, or moving the train/test split, pushed two pairs across the 5% line in our run without changing the data at all. With several pairs and a couple of defensible choices, an analyst can surface an apparent edge that is really noise, which is why pre-committing choices and correcting for multiple testing matters.

What is a Bonferroni correction and why use it here?

When you run several tests, the chance of at least one false positive rises. Checking five pairs at the 5% level gives about a 23% chance of a lucky "significant" result. A Bonferroni correction tightens the threshold (here to roughly t > 2.33) to keep the overall error rate at 5%. Under it, none of our results survive.

Keep reading

Start with a free API key

5,000 requests a month on the free plan. No credit card.

Get a free key