Shuffle test
Scrambling the signals to see whether the real ones beat random ones.
Take a strategy's signals, the list of which stocks it picked each month, and shuffle them at random among the same stocks, thousands of times. Each shuffle gives a fake strategy that trades the same universe with the same costs and the same market, but picks by chance. Run them all and you get the distribution of results that selection-free chance produces under this procedure.
Then place the real strategy's result on that distribution. The number reported is the share of shuffles that did as well or better than the real picks. A value of 0.37 means thirty seven percent of the random shuffles matched or beat the observed result: the benchmark gave no strong evidence against the null model that the picks carry no information. If the real result sits far out in the tail, the picks carried information the dice could not reproduce.
Read carefully: that share is a statement about outcomes under the shuffle procedure, not the probability that the strategy is luck, and not the probability that the idea is false. The test is honest because it keeps everything except the selection: the market drift, the calendar, the costs. It cannot see anything the procedure does not vary.
Explore the diagram (scroll horizontally on small screens)
Where this word comes up
First needed in the log entry H-044 · Short-term momentum in large caps: no robust edge in our test.
From the same entry: Betting against losers, High turnover, Liquid names, Momentum, Multiple testing, Relative strength ranking, The null.