Backtest example

Backtest Overfitting: An Interactive Example

What is backtest overfitting? And why should I care?

Graphic on left shows an investment strategy (in blue) making steady profit while the underlying trading instrument (in green) gyrates in price. This website is designed to illustrate that this type of "optimal" investment strategy is all too easy to produce if enough variations are tried, yet it is typically ineffective moving forward, a consequence of backtest overfitting.

Graphic on right shows the same investment strategy performing poorly on a different sample of the same trading instrument, which demonstrates that an overfit investment strategy is quite likely to fail in real world investing.

By checking this box, you acknowledge that this web site is a demonstration of a mathematical concept known as "backtest overfitting." The software and material on this site should NOT be interpreted as a recommendation to buy or sell any security or securities; as applicable to the specific investment needs of any particular individual or organization; or as applicable to the forecast of future market prices or trends.
Option 1: Repeat the example above
		Please be patient! The program may take up to two minutes to display the results page after clicking Go! button
Option 2: Try with random parameters
		Please be patient! The program may take up to two minutes to display the results page after clicking Go! button
		This "quick-start" option uses random (system generated) values for all of the inputs available in Option 3.
Option 3: Try with your own parameters You can enter values in one or more of the text boxes below. A more detailed explanation of what each box does is provided in the tutorial document below.
Enter values for these parameters and press Go! button
Maximum Holding Period		an integer; ranges from 5 to 20. The number of days a stock is held before it is sold if no other exits (e.g., the stop loss) have been triggered.
Maximum Stop Loss		an integer %; ranges from 10 to 40. The percentage loss from the initial entry at which point the stock will be sold. Do not enter the '%' sign.
Sample Length		an integer; ranges from 1000 to 2000. The number of days simulated. The sample data considered as the daily closing prices.
Standard Deviation		an integer; the standard deviation of random numbers used to generate daily price changes; any positive integer could be entered; we recommend 1 or 2
Seed		a seed for the pseudorandom numbers used for In Sample Data; it can be any positive integer; enter a nonpositive integer to use the current time as the seed
		Please be patient! The program may take up to two minutes to display the results page after clicking Go! button
Option 4: Try with stock market data You can enter values in one or more of the text boxes below.
Enter values for these parameters and press Go! button
Maximum Holding Period		an integer; ranges from 5 to 20. The number of days a stock is held before it is sold if no other exits (e.g., the stop loss) have been triggered.
Maximum Stop Loss		an integer %; ranges from 10 to 40. The percentage loss from the initial entry at which point the stock will be sold. Do not enter the '%' sign.
Sample Length		an integer; ranges from 1000 to 12000. The number of days simulated. We suggest between 5000-6000 as it makes almost equal length data on IS and OOS sides. The sample data considered as the daily closing prices.
		Please be patient! The program may take up to two minutes to display the results page after clicking Go! button

Default values: If you do not enter a value or enter a value that is outside the range mentioned above, a default value will be used. The default values are: maximum holding period = 7; stop loss = 10; sample length = 1000; and standard deviation = 1.

About execution time: The values for maximum holding period, stop loss and sample length significantly affect the number of iterations performed by the program; the larger these values are, the longer the program will run.

A few recent articles explaining the backtest overfitting in the financial press:

Backtest Overfitting: An Interactive Example

What is backtest overfitting? And why should I care?

Option 1: Repeat the example above

Option 2: Try with random parameters

Option 3: Try with your own parameters

Option 4: Try with stock market data

A more detailed explanation of options is provided in the tutorial.

A few recent articles explaining the backtest overfitting in the financial press:

Related documents:

Tutorial of the online tool:

Questions or comments:

Credits: