SSRN-id2502613

Prévia do material em texto

Electronic copy available at: http://ssrn.com/abstract=2502613 
Marcos López de Prado 
Lawrence Berkeley National Laboratory 
Computational Research Division 
OPTIMAL TRADING RULES 
WITHOUT BACKTESTING 
 Electronic copy available at: http://ssrn.com/abstract=2502613 
Key Points 
2 
• Calibrating a trading rule using historical simulations leads to 
backtest overfitting. 
• Big Data and High Performance Computing make backtest 
overfitting a major source of in-sample performance inflation. 
• In this presentation we describe a procedure for determining the 
optimal trading rule (OTR) without running alternative model 
configurations through a backtest engine. 
• We present empirical evidence of the existence of such optimal 
solutions for the case of prices following a discrete Ornstein-
Uhlenbeck process, and show how they can be computed 
numerically. 
 Electronic copy available at: http://ssrn.com/abstract=2502613 
SECTION I 
Trading Rules 
What is a Trading Rule? 
4 
• Each investment strategy requires an implementation 
tactic, often referred to as trading rules. 
• While strategies can be very heterogeneous in nature, 
tactics are relatively homogeneous. 
 
 
• For example, a position may be: 
– entered when the strategy’s signal reaches a certain value. 
– exited when a profit-taking or stop-loss threshold is exceeded. 
• These entry and exit rules rely on parameters that are 
usually calibrated via historical simulations (backtests). 
Trading rules provide the algorithm that 
must be followed to enter and exit a strategy’s position. 
Overfitting of a Trading Rule 
5 
• Bailey et al. [2014] have pointed out the 
dangers of calibrating a strategy through 
backtests. 
– For a large enough number of trials, a 
backtest can always be fit to any desired 
performance over a given sample. 
– Standard statistical techniques designed to 
prevent regression overfitting, such as hold-
out, are inaccurate in the context of backtest 
evaluation. 
– Under memory effects, overfitting leads to 
systematic losses, not noise. 
“Pseudo-Mathematics and 
Financial Charlatanism” 
explains why most historical 
simulations published in the 
academic & practitioners’ 
financial journals are likely 
to be false positives. 
A very real problem… 
6 
• Lawrence Berkeley National Laboratory has 
developed a website that finds “optimal 
investment strategies” in random series. 
 
 
• Because the series are random, there is no 
signal in the data one could profit from. 
• It is therefore obvious that such investment 
strategies are purely the result of overfitting. 
• The implication is that overfitting is hard to 
avoid when calibrating a strategy or trading 
rule on historical data. 
http://datagrid.lbl.gov/backtest/index.php 
In-Sample performance 
Out-of-Sample performance 
http://www.lbl.gov/about/
http://datagrid.lbl.gov/backtest/index.php
http://datagrid.lbl.gov/backtest/index.php
SECTION II 
The Problem 
The Problem (1/4) 
8 
• Suppose an investment strategy S that invests in 𝑖 = 1,… 𝐼 
opportunities or bets. 
• At each opportunity i, S takes a position of 𝑚𝑖 units of security 
X, where 𝑚𝑖 ∈ −∞, ∞ . 
• The transaction that entered such opportunity was priced at a 
value 𝑚𝑖𝑃𝑖,0, where 𝑃𝑖,0 is the average price per unit at which 
the 𝑚𝑖 securities were transacted. 
• As other market participants transact security X, we can mark-
to-market (MtM) the value of that opportunity i after t 
observed transactions as 𝑚𝑖𝑃𝑖,𝑡. 
• This represents the value of opportunity i if it were liquidated 
at the price observed in the market after t transactions. 
The Problem (2/4) 
9 
• Accordingly, we can compute the MtM profit/loss of 
opportunity i after t transactions as 𝜋𝑖,𝑡 = 𝑚𝑖 𝑃𝑖,𝑡 − 𝑃𝑖,0 . 
• A standard trading rule provides the logic for exiting 
opportunity i at 𝑡 = 𝑇𝑖. This occurs when one of two 
conditions is verified: 
 𝜋𝑖,𝑇𝑖 ≥ 𝜋𝑖, where 𝜋𝑖 > 0 is the profit-taking threshold for opportunity 
i. 
 𝜋𝑖,𝑇𝑖 ≤ 𝜋𝑖, where 𝜋𝑖 < 0 is the stop-loss threshold for opportunity i. 
• Because 𝜋𝑖 < 𝜋𝑖, only one of the two exit conditions can 
trigger the exit from opportunity i. 
• Assuming that opportunity i can be exited at 𝑇𝑖, its final 
profit/loss is 𝜋𝑖,𝑇𝑖. 
The Problem (3/4) 
10 
• At the onset of each opportunity, the goal is to realize an 
expected profit 𝐸0 𝜋𝑖,𝑇𝑖 = 𝑚𝑖 𝐸0 𝑃𝑖,𝑇𝑖 − 𝑃𝑖,0 , where 
𝐸0 𝑃𝑖,𝑇𝑖 is the forecasted price and 𝑃𝑖,0 is the entry level of 
opportunity i. 
 
 
 
• One way to calibrate the trading rule is to overfit: 
– Define a set of alternative values of R, Ω ≔ 𝑅 . 
– Simulate historically (also called backtest) the performance of S under 
alternative values of 𝑅 ∈ Ω. 
– Select the optimal 𝑅∗. 
 
 
 
DEFINITION 1 (Trading Rule): A trading rule for strategy S is 
defined by the set of parameters 𝑅 ≔ 𝜋𝑖 , 𝜋𝑖 , 𝑖 = 1, … 𝐼. 
The Problem (4/4) 
11 
• More formally, 𝑅∗ = arg max
𝑅∈Ω
𝑆𝑅𝑅, 𝑆𝑅𝑅 =
𝐸 𝜋𝑖,𝑇𝑖 𝑅
𝜎 𝜋𝑖,𝑇𝑖 𝑅
, where 
𝐸 . and 𝜎 . are respectively the expected value and standard 
deviation of 𝜋𝑖,𝑇𝑖, conditional on trading rule R, over 𝑖 = 1, … 𝐼. 
This typically leads to overfitting. 
 
 
 
 
• Intuitively, an optimal in-sample (IS) trading rule 𝑅∗ is overfit 
when it is expected to underperform the median of alternative 
trading rules 𝑅 ∈ Ω out-of-sample (OOS). 
DEFINITION 2 (Overfit Trading Rule): 𝑅∗ is overfit if 
𝐸
𝐸 𝜋𝑗,𝑇𝑗 𝑅
∗
𝜎 𝜋𝑗,𝑇𝑗 𝑅
∗
< 𝑀𝑒Ω 𝐸
𝐸 𝜋𝑗,𝑇𝑗 𝑅
𝜎 𝜋𝑗,𝑇𝑗 𝑅
, where 𝑗 = 𝐼 + 1,… 𝐽. 
SECTION III 
Our Framework 
The Framework 
13 
• We are interested in providing an OTR for those scenarios 
where overfitting would be most damaging, such as when 𝜋𝑖,𝑡 
exhibits serial correlation. 
• In particular, suppose a discrete Ornstein-Uhlenbeck (O-U) 
process on prices 
𝑃𝑖,𝑡 = 1 − 𝜑 𝐸0 𝑃𝑖,𝑇𝑖 + 𝜑𝑃𝑖,𝑡−1 + 𝜎𝜀𝑖,𝑡 
such that the random shocks are IID distributed 𝜀𝑖,𝑡~𝑁 0,1 : 
– The seed value for this process is 𝑃𝑖,0. 
– The level targeted by opportunity i is 𝐸0 𝑃𝑖,𝑇𝑖 . 
– 𝜑 determines the speed at which 𝑃𝑖,0 converges towards 𝐸0 𝑃𝑖,𝑇𝑖 . 
𝜋𝑖,𝑡~𝑁 𝑚𝑖 1 − 𝜑 𝐸0 𝑃𝑖,𝑇𝑖 𝜑
𝑗
𝑡−1
𝑗=0
− 𝑃𝑖,0 , 𝑚𝑖
2𝜎2 𝜑2𝑗
𝑡−1
𝑗=0
 
Numerical Solution (1/2) 
14 
1. Estimate the input parameters 𝜎, 𝜑 from 
𝑃𝑖,𝑡 = 𝐸0 𝑃𝑖,𝑇𝑖 + 𝜑 𝑃𝑖,𝑡−1 − 𝐸0 𝑃𝑖,𝑇𝑖 + 𝜉𝑡 
2. Construct a mesh of stop-loss and profit-taking pairs, 𝜋𝑖 , 𝜋𝑖 . 
3. Generate a large number of paths (e.g., 100,000) for 𝜋𝑖,𝑡 
applying our estimates 𝜎 , 𝜑 . 
4. Apply the 100,000 paths generated in Step 3 on each node of 
the mesh 𝜋𝑖 , 𝜋𝑖 generated in Step 2. 
 For each node, we apply the stop-loss and profit-taking logic, giving us 
100,000 values of 𝜋𝑖,𝑇𝑖. 
 For each node we compute the Sharpe ratio associated with that 
trading rule. 
 
Numerical Solution (2/2) 
15 
5. Compute the solution: 
 We determine the pair 𝜋𝑖 , 𝜋𝑖 within the mesh of trading rules that 
is optimal, given the input parameters 𝜎 , 𝜑 and the observed initial 
conditions 𝑃𝑖,0, 𝐸0 𝑃𝑖,𝑇𝑖 . 
 If strategy S provides a profit target 𝜋𝑖 for a particular opportunity i, 
we can use that information in conjunction with the results in Step 4 
to determine the optimal stop-loss, 𝜋𝑖. 
 If the trader has a maximum stop-loss 𝜋𝑖 imposed by the fund’s 
management, we can use that information in conjunction with the 
results in Step 4 to determine the optimal profit taking 𝜋𝑖 within the 
range of stop-losses 0, 𝜋𝑖 . 
 
SECTION IV 
Some Cases 
Cases with Zero Long-Run Equilibrium 
17 
• From left to right, these figures show the Sharpe ratios for parameter combinations 
𝜇, 𝜏, 𝜎 = 0,5,1 , 𝜇, 𝜏, 𝜎 = 0,50,1 , 𝜇, 𝜏, 𝜎 = 0,100,1 . 
• For small half-life, performance is maximized in a narrow range of combinations of small 
profit-taking with large stop-losses: The optimal trading ruleis to hold an inventory long 
enough until a small profit arises, even at the expense of experiencing 5 or 7-fold losses. 
• This is in fact what many market-makers do in practice, and is consistent with the 
“asymmetric payoff dilemma” described in Easley et al. [2011]. 
• The worst possible trading rule in this setting would be to combine a short stop-loss with 
large profit-taking threshold, a situation that market-makers avoid in practice. 
Cases with Positive Long-Run Equilibrium 
18 
• From left to right, these figures show the Sharpe ratios for parameter combinations 
𝜇, 𝜏, 𝜎 = 5,5,1 , 𝜇, 𝜏, 𝜎 = 5,50,1 , 𝜇, 𝜏, 𝜎 = 5,100,1 . 
• Because positions tend to make money, the optimal profit-taking is higher than in the 
previous cases. 
• As we increase the half-life, the range of optimal profit-taking widens, while the range of 
optimal stop-losses narrows, shaping the initial rectangular area closer to a square and 
then a semi-circle. 
• Again, a larger half-life brings the process closer to a random walk. 
Cases with Negative Long-Run Equilibrium 
19 
• From left to right, these figures show the Sharpe ratios for parameter combinations 
𝜇, 𝜏, 𝜎 = −5,5,1 , 𝜇, 𝜏, 𝜎 = −5,50,1 , 𝜇, 𝜏, 𝜎 = −5,100,1 . 
• Results appear to be rotated complementaries of what we obtained in the Positive case 
(like rotated photographic negatives). 
• The reason is, that the profit in the previous figures translates into a loss in these 
figures, and vice versa: One case is an image of the other, just as a gambler’s loss is the 
house’s gain. 
SECTION V 
The Carr - de Prado Conjecture 
A Conjecture Worth A Lot Of Money… 
21 
• While in this paper we do not derive the closed-form solution 
to the optimal trading strategies problem, our experimental 
results seem to support the following OTR conjecture: 
 
 
 
 
 
• We believe that solving this conjecture would have substantial 
economic value in a trading world where a few milliseconds 
separate winners from losers. 
“Given a financial instrument’s price characterized by a 
discrete O-U process, there is a unique optimal trading rule 
in terms of a combination of profit-taking and stop-loss 
that maximizes the rule’s Sharpe ratio.” 
http://arxiv.org/abs/1408.1159
SECTION VI 
Conclusions 
Conclusions 
23 
• We have shown how to determine experimentally the optimal 
trading strategy associated with prices following a discrete 
Ornstein-Uhlenbeck process. 
• Because the derivation of such trading strategy is not the result 
of a historical simulation, our procedure avoids the risks 
associated with backtest overfitting. 
• Depending on factors such as the frequency at which trading 
takes place, the holding period, etc., the time it takes to run our 
numerical procedure may be too lengthy. 
• It would be beneficial to count with a closed-form solution that 
computes the Sharpe ratio of every combination 𝜋𝑖 , 𝜋𝑖 , which 
we could then optimize analytically to determine the optimal 𝑅𝑖
∗. 
THANKS FOR YOUR ATTENTION! 
 
 
 
24 
SECTION VII 
The stuff nobody reads 
Bibliography 
• Bailey, D. and M. López de Prado (2012): “The Sharpe Ratio Efficient Frontier,” Journal of 
Risk, 15(2), pp. 3-44. Available at http://ssrn.com/abstract=1821643. 
• Bailey, D. and M. López de Prado (2013): “Stop-Out Under Serial Correlation,” Working 
paper. Available at http://ssrn.com/abstract=2201302. 
• Bailey, D., J. Borwein, M. López de Prado and J. Zhu (2014): “Pseudo-Mathematics and 
Financial Charlatanism: The Effects of Backtest Overfitting on Out-Of-Sample 
Performance,” Notices of the American Mathematical Society, 61(5), pp. 458-471. 
Available at http://ssrn.com/abstract=2308659. 
• Bailey, D., J. Borwein, M. López de Prado and J. Zhu (2013): “The Probability of Backtest 
Overfitting.” Working paper. Available at http://ssrn.com/abstract=2326253. 
• Bertram, W. (2009): “Analytic Solutions for Optimal Statistical Arbitrage Trading.” 
Working paper. Available at http://ssrn.com/abstract=1505073. 
• Carr, P. and M. López de Prado (2014): “Determining Optimal Trading Rules Without 
Backtesting”, Working Paper. Available at http://arxiv.org/abs/1408.1159 
• Easley, D., M. Lopez de Prado and M. O’Hara (2011): “The Exchange of Flow-Toxicity.” 
Journal of Trading, Vol. 6(2), pp. 8-13, Spring. Available at 
http://ssrn.com/abstract=1748633. 
26 
http://ssrn.com/abstract=1821643
http://ssrn.com/abstract=1821643
http://ssrn.com/abstract=2201302
http://ssrn.com/abstract=2201302
http://ssrn.com/abstract=2308659
http://ssrn.com/abstract=2308659
http://ssrn.com/abstract=2326253
http://ssrn.com/abstract=2326253
http://ssrn.com/abstract=2326253
http://ssrn.com/abstract=1505073
http://ssrn.com/abstract=1505073
http://arxiv.org/abs/1408.1159
http://arxiv.org/abs/1408.1159
http://ssrn.com/abstract=1748633
http://ssrn.com/abstract=1748633
Bio 
Marcos López de Prado is Senior Managing Director at Guggenheim Partners. He is also a Research 
Affiliate at Lawrence Berkeley National Laboratory's Computational Research Division (U.S. Department 
of Energy’s Office of Science). 
 
Before that, Marcos was Head of Quantitative Trading & Research at Hess Energy Trading Company (the 
trading arm of Hess Corporation, a Fortune 100 company) and Head of Global Quantitative Research at 
Tudor Investment Corporation. In addition to his 15+ years of trading and investment management 
experience at some of the largest corporations, he has received several academic appointments, 
including Postdoctoral Research Fellow of RCC at Harvard University and Visiting Scholar at Cornell 
University. Marcos earned a Ph.D. in Financial Economics (2003), a second Ph.D. in Mathematical 
Finance (2011) from Complutense University, is a recipient of the National Award for Excellence in 
Academic Performance by the Government of Spain (National Valedictorian, 1998) among other awards, 
and was admitted into American Mensa with a perfect test score. 
 
Marcos is the co-inventor of four international patent applications on High Frequency Trading. He has 
collaborated with ~30 leading academics, resulting in some of the most read papers in Finance (SSRN), 
three textbooks, publications in the top Mathematical Finance journals, etc. Marcos has an Erdös #2 and 
an Einstein #4 according to the American Mathematical Society. 
27 
Disclaimer 
• The views expressed in this document are the authors’ 
and do not necessarily reflect those of the 
organizations he is affiliated with. 
• No investment decision or particular course of action is 
recommended by this presentation. 
• All Rights Reserved. 
28 
Notice: 
 
The research contained in this presentation is the result of a 
continuing collaboration with 
 
Dr. Peter P. Carr 
Morgan Stanley & Courant Institute (NYU) 
 
The full paper is available at: 
http://arxiv.org/abs/1408.1159 
 
For additional details, please visit: 
http://ssrn.com/author=434076 
www.QuantResearch.info 
http://arxiv.org/abs/1408.1159
http://arxiv.org/abs/1408.1159
http://ssrn.com/author=434076
http://www.quantresearch.info/