Over on John Chew’s investing blog, he wrote an article called “Beating an Index or What Works on Wall Street” (http://is.gd/w9dxEO), in which he reported:
Richard Brealey, a respected data analysis, estimated that to make reasonable assumptions about a strategy’s validity (95% confidence level or statistically relevant) you would need 25 years of data.
The subject was on whether it was possible to determine if a fund manager had skill. I then cheekily suggested that if he needed 25 years of data to draw a conclusion, then we should perhaps find a better data analyst. That is highly presumptuous of me, of course, because I stopped statistics in my first year at university.
I don’t know how Brealey does his anaylsis, but I’m presuming he uses a frequentist approach, maybe using a binomial distribution with a null hypothesis of “no skill”. Rather than using a frequentist approach, it is possible to use Bayesian statistics.
With that in mind, I put together a little online app here:
If you input the number of years that you beat the market, and the number of years that you lagged the market, then it will return a probability that the result was due to blink luck. The app doesn’t just have to be used for that; it can be used for anything where there are a number of trials, some of which succeed, and others of which fail.
Have a play.