K
Knandi
Guest
How realistic are backtest results compared to real world results on BWB delta neutral strategies?
Technically BT results reflect what technically happens in real trading, not always, but for long dte trades, if you have good market data, if you know of potential pitfalls of BT, and you do bactesting right - it's pretty close.
Problem is that irl you don't trade solely on what BT gives you. News, market events, your emotions, your expectations, opinions of others etc. do influence your trading decisions.
I'm not opposed to BT (unless its stupid "BT said so, so it must be true" bias), especially if you're starting with trading and want to familiarize yourself with some strategies.
BT can be also used to extract some data about option structures but you need some knowledge how to do it properly and have adequate BT engine.
So, BT can be p&l realistic but you will not learn from it how to trade.
Problem is that irl you don't trade solely on what BT gives you. News, market events, your emotions, your expectations, opinions of others etc. do influence your trading decisions.
I'm not opposed to BT (unless its stupid "BT said so, so it must be true" bias), especially if you're starting with trading and want to familiarize yourself with some strategies.
BT can be also used to extract some data about option structures but you need some knowledge how to do it properly and have adequate BT engine.
So, BT can be p&l realistic but you will not learn from it how to trade.
K
Knandi
Guest
Curious.. most of the strategies like M3/Rhinos/Kevlar etc.. have certain rules on min/max deltas allowed in a trade. Can’t we just create an algorithm at get automatic conditional fill with TOS? I suppose exiting trade isn’t as easy especially if the trade has gotten messy.. (probably can come up with an optimal algorithm to optimize the peeling of the messy trade while also keeping deltas flat)..
For fast moving environments, I suppose getting fills might be difficult?
What am I missing here? If backrests can be trusted and human influence is removed from the trade, why isn’t every top hedge fund out in the Wall Street doing these? Or are they?
For fast moving environments, I suppose getting fills might be difficult?
What am I missing here? If backrests can be trusted and human influence is removed from the trade, why isn’t every top hedge fund out in the Wall Street doing these? Or are they?
I doubt hedge funds are playing around with BWB strategies in SPX That's for us the small time retail traders with smaller accounts
They have millions so they need to buy big chunks of stock or futures to hedge their long exposure so for that the back testing might be as simple as using some ma crossovers if they even do any back testing at all They do some research on a company or react to some news and they pile into that company there is no need to mess around with backtesting
For BWB back testing is not so simple because sometime you have some nuances and subjectivity that you cannot account for using back testing
If you saw Amy's presentation you have seen that any small change might affect the p/l over the long run Also let's say you find a back test that works in a certain environment so if the environment changes now the trade no longer works So I would not trust any back testing on BWB
or any complex options
K
Knandi
Guest
Im sure there are some small prop shops that do options trading, using delta neutral strategies, not necessarily BWB. All of them started of somewhere? Maybe BWBs?
I got ONE free trial and did a small six-month backtest.
What I noticed is that,
1) even if I have a loss in one month, usually that loss is recovered by the next month, especially now that vols are in your favor for next trade..
2) We are very Vega sensitive, although we say we collect theta, Vega is what gives us the best bang.
Have you had these insights as well? What have been your experiences in real trading that differed from backtests?
Also can you point me to Amy’s presentation?
I got ONE free trial and did a small six-month backtest.
What I noticed is that,
1) even if I have a loss in one month, usually that loss is recovered by the next month, especially now that vols are in your favor for next trade..
2) We are very Vega sensitive, although we say we collect theta, Vega is what gives us the best bang.
Have you had these insights as well? What have been your experiences in real trading that differed from backtests?
Also can you point me to Amy’s presentation?
Last edited by a moderator:
I have not used any back tests since I don't find it reliable plus I have not seen any good websites for free that do back tests
If you sign up as a gold member it is available here on this website It was done not too long ago as part of the 0 DTE workshop
She used Option Omega to do the back testing during that presentation
If you sign up as a gold member it is available here on this website It was done not too long ago as part of the 0 DTE workshop
She used Option Omega to do the back testing during that presentation
Backtesting does not work the way you think or rather what your question suggests you think. This is why creating algo on simple BT (a la ONE or OV) will fail (although it may work for some time - giving false impressions that everything is fine). Creating trading algos is possible but it requires much more work and experience than we can afford.
If BT alone would work then such algo wold work as well and everybody be using it. Why not?! I gladly settle for smaller pl if I can fire up algo and relax or pursue other interesting adventures. But... even if... such algos would work, it wont be for long because thinking traders will take advantage of it's predictability, which will quickly lead to "reverse profits'.
I know many traders, both pro educators and enthusiasts, do flash before our eyes plots of steady growing accounts or good expectancy numbers or some other impressive ratio valuations - all based on BT. They try to lure. BT numbers are only beginning and in itself mean very little. If they, presenters, don't explain why those numbers should be practically ignored, they count on your and mine ignorance - letting us think that their strategy is the holly grail, or they themselves are not aware what those numbers really are.
I'm talking in very general terms - don't have any particular examples in mind.
Think of it other way. If you want to get position as a trader in some prop desk, do you think they ask you how does your strategy's bactestest equity curve looks like, or what BT says about expectancy? No, they care little about that. They want to see your trading track records. Only after that they may look at equity growth or some ratios of live trading but not in isolation from other factors (time, absolute risk, risk management etc).
What I said doesn't mean that BT results are necessary bad. It means that those results are not conclusive. They carry little more weight for some (not all) long dte trades but still I wouldn't take them for face value.
I don't know what Amy's video Status1 refers to but I think it is related to some shorter dte where factors he mentioned play bigger role, aka environment is a big factor and env can't be simply read from BT data (well.. it depends, but let's skip over it). Here, in shorter dtes, simple sets of rules just wont work, except maybe for limited time period.
That said... You mentioned M3. I didn't take the course and know it only from promotional materials from good few years back but suspect they may be doing something good there, as I see traders successfully using M3 to this day. If those courses are good, they must be teaching more than just explaining well bactested set of rules.This is not a recommendation though, traders are smart folks and can figure out much on their own. Your question about putting some automated trader based on well backtested set of rules is a good example of that.
And remember: think for yourself what I said can be completely false.
If BT alone would work then such algo wold work as well and everybody be using it. Why not?! I gladly settle for smaller pl if I can fire up algo and relax or pursue other interesting adventures. But... even if... such algos would work, it wont be for long because thinking traders will take advantage of it's predictability, which will quickly lead to "reverse profits'.
I know many traders, both pro educators and enthusiasts, do flash before our eyes plots of steady growing accounts or good expectancy numbers or some other impressive ratio valuations - all based on BT. They try to lure. BT numbers are only beginning and in itself mean very little. If they, presenters, don't explain why those numbers should be practically ignored, they count on your and mine ignorance - letting us think that their strategy is the holly grail, or they themselves are not aware what those numbers really are.
I'm talking in very general terms - don't have any particular examples in mind.
Think of it other way. If you want to get position as a trader in some prop desk, do you think they ask you how does your strategy's bactestest equity curve looks like, or what BT says about expectancy? No, they care little about that. They want to see your trading track records. Only after that they may look at equity growth or some ratios of live trading but not in isolation from other factors (time, absolute risk, risk management etc).
What I said doesn't mean that BT results are necessary bad. It means that those results are not conclusive. They carry little more weight for some (not all) long dte trades but still I wouldn't take them for face value.
I don't know what Amy's video Status1 refers to but I think it is related to some shorter dte where factors he mentioned play bigger role, aka environment is a big factor and env can't be simply read from BT data (well.. it depends, but let's skip over it). Here, in shorter dtes, simple sets of rules just wont work, except maybe for limited time period.
That said... You mentioned M3. I didn't take the course and know it only from promotional materials from good few years back but suspect they may be doing something good there, as I see traders successfully using M3 to this day. If those courses are good, they must be teaching more than just explaining well bactested set of rules.This is not a recommendation though, traders are smart folks and can figure out much on their own. Your question about putting some automated trader based on well backtested set of rules is a good example of that.
And remember: think for yourself what I said can be completely false.
IMHO: Another perspective on Back testing! Backtesting done properly (which will vary widely) can inform you what would/could have happened in the past within some bounds, with trying a fully unambiguous trading plan. You must consider slippage (beware that most backtests ASS-U-ME fills at mid) and should consider commissions if your edge is small. A backtest is useful once you have your solution in hand and need to double-check your work. Using backtests to develop a trade is like "pushing a string"! (assume string has no wax for this analogy) .
A preferred method (for me) is similar to backtest, but devoid of the decisions you have yet to determine! Make no assumpitions. Solve for the criteria using unbiased data. -- If the process is fun, you are likely drinking KoolAid that should be taboo. -- This is likely an iterative process to work out a single variable at a time.
A preferred method (for me) is similar to backtest, but devoid of the decisions you have yet to determine! Make no assumpitions. Solve for the criteria using unbiased data. -- If the process is fun, you are likely drinking KoolAid that should be taboo. -- This is likely an iterative process to work out a single variable at a time.
T
TiborK
Guest
Full disclosure: I'm the creator of MesoSim, which is an automated option backtester.
I wouldn't feel comfortable trading without having some proof that my trade plan works. Backtests are one way to reach that conviction.
Another requirement for me is that the trade must have an explainable/understandable edge, such as Volatility Risk Premium.
I spent quite some time using and developing backtesting solutions (pyalgotrade, zipline-live, quantconnect) and found
them very valuable. Initially, I was focused on automated trading to remove the emotional element. It worked to some extent:
I had some successful years running Pairs Trading (Stat Arb) on equity.
Then I started learning about options and found OptionNet Explorer a very useful tool for understanding how options behave.
Simulating positions helped me learn how Greeks affect my trades and how PnL varies with the move of the underlying.
As manual backtesting is a slow process, I started to create an abstraction on options strategies, starting with Amy's A14,
BoxCar, NetZero, Space Trip Trade, and its variations. I suck at discretionary trading, so a mechanic / systematic strategy
is a must for me.
Having an automated backtester helped me to iterate faster on ideas which gave me further confidence as I
was able to do a very quick sensitivity analysis of trades:
While manually backtesting 1 year in OptionNet took me 1 hour at my fastest pace, now it is a matter of seconds.
Even though the 10s of hours put into manual backtesting seems like a waste of time, I think it helped me to understand better
and I'd encourage everyone to spend time with (at least) manually running through the trade plan before taking a trade live.
I wouldn't feel comfortable trading without having some proof that my trade plan works. Backtests are one way to reach that conviction.
Another requirement for me is that the trade must have an explainable/understandable edge, such as Volatility Risk Premium.
I spent quite some time using and developing backtesting solutions (pyalgotrade, zipline-live, quantconnect) and found
them very valuable. Initially, I was focused on automated trading to remove the emotional element. It worked to some extent:
I had some successful years running Pairs Trading (Stat Arb) on equity.
Then I started learning about options and found OptionNet Explorer a very useful tool for understanding how options behave.
Simulating positions helped me learn how Greeks affect my trades and how PnL varies with the move of the underlying.
As manual backtesting is a slow process, I started to create an abstraction on options strategies, starting with Amy's A14,
BoxCar, NetZero, Space Trip Trade, and its variations. I suck at discretionary trading, so a mechanic / systematic strategy
is a must for me.
Having an automated backtester helped me to iterate faster on ideas which gave me further confidence as I
was able to do a very quick sensitivity analysis of trades:
- Shifting entries and exits throughout the day
- Randomly picking entries (sort of Monte Carlo) and exits
- Shifting deltas and distances
- Having proper quantitative statistics (ONE is very weak in this area) of the backtest: Sharpe, Max DD, Alpha, etc.
While manually backtesting 1 year in OptionNet took me 1 hour at my fastest pace, now it is a matter of seconds.
Even though the 10s of hours put into manual backtesting seems like a waste of time, I think it helped me to understand better
and I'd encourage everyone to spend time with (at least) manually running through the trade plan before taking a trade live.
Last edited by a moderator:
Tibor, backtesting is a broad topic. I answered leading questions: "how realistic BT results are". My understanding of that question was that Knandi asked, more or less, if BT training is good enough to prepare a person for live actions and if BT results can predict live trading results. I didn't want to touch anything else.
In particular I didn't want to say BT is bad. I know it can be very helpful in early stages of learning options trading. You mentioned that and I have no objections.
Problem emerges, imo, when trader starts taking BT too seriously (usually he pays a lot of money for software, so it has to be serious, isn't it?). I think that at some point in trading, BT can be completely dropped off. Of course one may hang out to it to get some statistical numbers (which are still of lesser quality than numbers from live trading or rathed have different purpose) or to practically test trading ideas in areas where his knowledge is not solid (ie when switching from long to short dtes) but there BT is used to confirm thesis new trade is build on (edge if you will) and not to develop ones. You mentioned it too. There are other areas that BT can be used - but are not necessary for successful trading (even can have negative effects).
All that may sound complicated but it is not, if you realize what BT provides and what it does not. I'm of opinion that hanging on to BT for too long do more harm than good for a trader as he tends to focus on factors available in BT software (mostly mathematically calculated) and neglects other factors important for trading, that BT can not provide.
I've seen some use ONE as a mean for tracking/recording their trades. I dont understand that but whatever it is, it is not BT.
Point: I don't plainly condemn BT.
In particular I didn't want to say BT is bad. I know it can be very helpful in early stages of learning options trading. You mentioned that and I have no objections.
Problem emerges, imo, when trader starts taking BT too seriously (usually he pays a lot of money for software, so it has to be serious, isn't it?). I think that at some point in trading, BT can be completely dropped off. Of course one may hang out to it to get some statistical numbers (which are still of lesser quality than numbers from live trading or rathed have different purpose) or to practically test trading ideas in areas where his knowledge is not solid (ie when switching from long to short dtes) but there BT is used to confirm thesis new trade is build on (edge if you will) and not to develop ones. You mentioned it too. There are other areas that BT can be used - but are not necessary for successful trading (even can have negative effects).
All that may sound complicated but it is not, if you realize what BT provides and what it does not. I'm of opinion that hanging on to BT for too long do more harm than good for a trader as he tends to focus on factors available in BT software (mostly mathematically calculated) and neglects other factors important for trading, that BT can not provide.
I've seen some use ONE as a mean for tracking/recording their trades. I dont understand that but whatever it is, it is not BT.
Point: I don't plainly condemn BT.
As we speak about it.
Tibor, you mentioned ONE being not good in producing usable statistics. Of course, it's true.
We also agreed that BT is good for new traders to learn about options.
Imo, as with statistics, ONE ( I use it as an example as I know it more than other publicly available BT tools) is ok for learning purposes but not ideal. You should not use it as a single source of knowledge, neither direct or via teacher.
From my example (disclaimer below): I wanted to study impact of IV has on some structures. You know: vol levels, steepens, convexity for structures in different dte and different locations... ONE has a very basic IV plots that are hard to use for anything serious. I know, I know ( in case you want to defend ONE ) if one knows what he is looking for, he can, maybe, find data in ONE ( but if one knows what to look for, he wont be using ONE for that purpose anyway ). It was really hard to get data I needed from ONE. Theoretically it was possible but required very, very tedious work ( not for me ). Similar situation to trying to get more useful stats out of ONE - theoretically it's possible... So I bulid a tool to do exactly what I needed. Then, after 'getting a picture' I'd turn back to ONE to confirm or refute my observations (except I've already dropped ONE at this time).
My point is that ONE in itself does not encourage nor allow you to conduct such explorations, at least not without a lot of very tedious, manual (prone to errors) work. The more time you spend inside any particular BT tool, the more restrictions you put on yourself, the narrower you field of view becomes. Ergo: even if BT can be good aid in learning process, relaying on it alone is a mistake.
Disclaimer
I wrote the tool I mentioned good few years back for personal use. I shorty presented it on one of TG1 meetings and published, with limited features, for free on Heroku. I had no feedback whatsoever but also did not advertised it. Now Heroku doesn't offer free hosting anymore, and it's gone ( i think, bc I didn't check ).
I wrote simplified version of it for android. It is 'paid app' but you can take take a look at app page to have an idea.
Tibor, you mentioned ONE being not good in producing usable statistics. Of course, it's true.
We also agreed that BT is good for new traders to learn about options.
Imo, as with statistics, ONE ( I use it as an example as I know it more than other publicly available BT tools) is ok for learning purposes but not ideal. You should not use it as a single source of knowledge, neither direct or via teacher.
From my example (disclaimer below): I wanted to study impact of IV has on some structures. You know: vol levels, steepens, convexity for structures in different dte and different locations... ONE has a very basic IV plots that are hard to use for anything serious. I know, I know ( in case you want to defend ONE ) if one knows what he is looking for, he can, maybe, find data in ONE ( but if one knows what to look for, he wont be using ONE for that purpose anyway ). It was really hard to get data I needed from ONE. Theoretically it was possible but required very, very tedious work ( not for me ). Similar situation to trying to get more useful stats out of ONE - theoretically it's possible... So I bulid a tool to do exactly what I needed. Then, after 'getting a picture' I'd turn back to ONE to confirm or refute my observations (except I've already dropped ONE at this time).
My point is that ONE in itself does not encourage nor allow you to conduct such explorations, at least not without a lot of very tedious, manual (prone to errors) work. The more time you spend inside any particular BT tool, the more restrictions you put on yourself, the narrower you field of view becomes. Ergo: even if BT can be good aid in learning process, relaying on it alone is a mistake.
Disclaimer
I wrote the tool I mentioned good few years back for personal use. I shorty presented it on one of TG1 meetings and published, with limited features, for free on Heroku. I had no feedback whatsoever but also did not advertised it. Now Heroku doesn't offer free hosting anymore, and it's gone ( i think, bc I didn't check
).I wrote simplified version of it for android. It is 'paid app' but you can take take a look at app page to have an idea.
Last edited:
I think you refer to other app, not the one from the link.
I linked Options Structure Analyzer which does not use any market data but is based solely on Black Scholes model.
Kiwi Trader is a game I put as a side project where trying to find answers to some questions re Technical Analysis. I was using sort of BT engine there to get stats - not necessary ONE's style BT. Actually the app itself was for me more testing waters with coding for Android platform
than anything else. There is no BT engine of any sort in the app.'Prerecorded data', in free version, means that charts are indeed pre-set (only one chart, as I remember but still very much playable imo). In paid version there is unlimited 'sets' of charts where you have control over some "market" characteristics. App uses Monte-Carlo simulations. It is of course possible to add feature to use data provided directly by users but this is more technical than practical possibility. I don't see practical purpose of that.
Last edited:
IMO, it may be the case that backtested strategies I think look good are not since if actually profitable the masses would have exhausted the edge. I think it's far more likely that strategies I think look good in backtesting aren't actually good because I failed to include transaction fees.
IMO: If you think about what is happening with markets, there remain exploitable trades that can be discovered via backtesting! -- Just note that if you find one, others will eventually (if not before you) find it and the edge will get arbitraged away, but that is nature of the beast! -- Just look at the 0DTE frenzy and how the profitable trading is continually morphing! -- If you are not accounting for transaction fees and slippage, and you have relatively high trading, you may consider either including the commissions and slippage, or look for lower trading frequency and/or higher profitable trades (so those costs may be safely ignored). -- Just because you found something that looks "too good to be true" does not mean you have a flaw (likely a flaw, but these nuggets still exist and should still pop up ). -- It is tedious work to weed out the chaff.
Having backtested with Optionvue extensively in the pre-mentor era, I have only 3 observations:
1. Backtesting is an excellent tool for learning to make adjustments in a strategy as volatility and price action change, and seeing how they "work out".
2. It has limited use in predicting eventual outcomes of individual trades or a single strategy, but has some utility in establishing the expected returns of a strategy when a sufficient number of tests are completed. Also, a single trade can be allowed to run to completion with no adjustments in order to get a feel for the results of probability alone. Compare the actual results to the expected move at trade launch.
3. Backtesting long-dated strategies (greater than 30 DTE) is less reliable than shorter-dated strategies as a consequence of completely unpredictable price action, volatility, and geopolitical events.
1. Backtesting is an excellent tool for learning to make adjustments in a strategy as volatility and price action change, and seeing how they "work out".
2. It has limited use in predicting eventual outcomes of individual trades or a single strategy, but has some utility in establishing the expected returns of a strategy when a sufficient number of tests are completed. Also, a single trade can be allowed to run to completion with no adjustments in order to get a feel for the results of probability alone. Compare the actual results to the expected move at trade launch.
3. Backtesting long-dated strategies (greater than 30 DTE) is less reliable than shorter-dated strategies as a consequence of completely unpredictable price action, volatility, and geopolitical events.