Objectives of trading strategies, and statistical significance · David Bush

Chat with traders episode 111 this is your key to the minds of tradings elite performer those who profit in relentless market here on the chat would trader Todd Jack you'll hear about the skill sets and tactics that lead winning traders to win so you can level up and become a better trader here's your host Aaron playfield this week's episode is brought to you by squarespace calm squarespace gives you the ability to Make an awesome website a blog or even set up an online store and you can do this all with zero web design experience make

your big ideas no into the world stay your free trial today at squarespace.com and enter the coupon code traders to get 10% off your first purchase what's up crew welcome back welcome to episode 111 this week returning to chat with traders for a second time is David Busch who was first on episode 23 David first began as A discretionary trader more than 20 years ago but over time he's divided in to almost purely a quantitative and he's exceptionally good at what he does David's been the first-place winner of two real money trading competitions in recent

years last time David was on we spoke fairly extensively about his past as a trader and a high level overview of his process this time around we covered plenty of New ground exploring David's process and greater depth also I particularly liked David's comments towards the end about intensity not time that would probably make more sense when you get to it but anyway for show notes for this episode can be found at chat with traders comm / 1 1 1 and I really hope you can gain something from this please welcome to the podcast David Bush

I'd I've have a guy man good how you Doing today I'm doing very well you're a bit under the weather are you I am yet it's been a stretch man it's been a stretch you know just busy and pushing it so under the weather but you know just I sound horrible but otherwise I'm alright now you guys can see that all right good what is it just a bicycle yeah it's a basic hold yeah I totally normally avoid it or it's very light this you know this year there this winter and uh you know I've

been sick Twice like this in the last like three weeks that's just this one was self induced I think from just trying to fit this our workshop in at the end of the week I mean you know I'm glad I have that knowledge now but I'm paying for it you know what I mean because I had to fit everything around it and you were you were caught in my whirlpool of rescheduling and anyway so yeah that's induced problem with you've got me interviewing you it's mmm on a Sunday Morning uh a year the man I'm

so sorry I was about to ask and then I yeah cuz I didn't do the calculation of my head but Wow note of are not aware I appreciate and I place I'd rather be yeah have you ever seen this time of day before in a Sunday morning or cause of course I'm already up before that anyway so alright very good so how was the workshop how was that yeah it was you know it's an introduction but it was two days of an Introduction you know so very - very full day's my you know there is

this tooth there's two ways I look at it there's my goal which is first of all just to litter that are you know I'd really knew not not a lot and then you know other than yes statistics focused and whatever but I my goal is really just to be a better manipulator of data I mean we might have talked about this the other day briefly but you know just have my data set up in a much more you Know usable automated you know updating way and then to be able to actually hang just more quickly

you know manipulated so so basically I feel like I right now have the skills I mean it would be very slow at first but that's how I'm going to cut my teeth I have the skills to you know just import and join a ton of Excel files and it's you know it's so quick you know an R it's just like it's instant you know and you can hyper thread if you want and all that stuff so So that's you know I feel like great mission accomplished I'm going to you know I meet a sit down

and spend hours doing it but if I can just become awesome at mutating and joining Excel files that would be great you know I mean and then I got a peek into the the other levels which are like well you can do anything and out I mean just like most code you know you can do insane machine learning you can use it for graphic portfolio analytics whatever you Know and obviously there's a lot of packages that people have already created which is cool so I'm going to avoid the temptation that go deep into the deep

into the weeds like you know trying to solve the world's problems through our and just like have that one skill you know so I thought it was great oh that's excellent man that's very good yeah so you hadn't coded an R previously before this like at all no exactly exactly no I've never you know I didn't Have our installed until you know like 24 hours before the thing so no didn't know our and but I feel you know like that data manipulation thing I feel like I mean I know I'm going to run it all

in the walls if you always do when you're coding you know anything it's always longest at the front which makes it tough to go through you know through the barriers but we we did the in class you know it was very hands-on thing you know basically the guy would talk for like Five or ten minutes and then be like alright here's what you're going to do now you know and that's the way the whole two days was which was good you know so I already succeeded it you know doing probably 75% of what I want

to do anyways now it's just getting good at it and you know setting up some scripts and yeah yeah just putting in the work yeah we've had it Yemen uh excellent so I know you've got a programmer who you work with yeah Do you deprive to this did you have any programming knowledge is so like we codon in a different language beforehand well what I've done I always think of it like he's a capital P program and I mean like he's trained he sets his whole schooling that says PhD he knows probably tons of languages

and this facile and most you know for me you know I started coding in just proprietary trading platform languages like trade stations easy language or mechanicus Language you know what outs trading blocks you know so in other words they're all just kind of proprietary trading specific language is a pure language something I I have not ever learned I mean I have some Python scripts and so forth and some utilities that use Python but that's because the PhD programmer wrote them for me so but so I'm excited I'm actually you know I mean that's another level

of just well learning are and being able to do things Obviously it's much easier to grasp this language you know having known the language of other platforms and syntax and stuff like that so it becomes you know felt like familiar actually which was kind of nice right I mean there's a first language it's securely its own thing with many applications not trading specific in yet you know it felt familiar so that's pretty cool yeah absolutely I think are is quite similar to Python in many ways so if you've seen A few Python scripts previously then

I'm sure you know you could probably interpret ah reasonably well like sort of on a basic level I presume interesting yes I was a little disturbed when somebody asks a question that was in my head I mean I just want to ask it but I was going to ask him for the class and waste everyone's time but you know one guy was really pretty good he's picking up then quickly obviously had a lot of Programming knowledge already and you know he at the very end is just like so what's it Institute between Python and art

in your opinion you know and the good instructors I don't really know I don't know Python that's like really you know like are you just so unincluded because I was wondering yeah many singularities I think so what was your what was your preference to learn like why did you decide to pick up I you know if this business analytics Center University of Cincinnati if it had offered intro to Python to day work showed up I would have done that you know what I mean they just did intro to our person for whatever reason so that

was it I just either one I would have gone to it wasn't like I sought out our I knew that I'd get a really quality experience because I've took a two day tableau you know data database workshop so now also in other news since we last spoke I Think that was episode 23 I meant to check before we actually got on the call but it was a 20s anyway so for anyone listening if you want to go back check it out early 20s when David Bush was first on the podcast I was speaking to you

not long after did just one I think it was battle thin if I'm getting that correct correct right now correct me if I'm wrong but you've also won another quant trading competition since then is that right it is right yes and I would Like to think now yeah thank you I appreciate that yeah it was first place in this name which is an acronym National Association of active investment managers they had a strategy competition so actually I trying to think whether it was purely a systemic or quantitative contest I actually think it was not but

certainly that that was certainly the focus of those who are presenting and yeah that was that was a great it's a great group in a great Organization great very welcoming people and and the competitors actually were you know we're very impressive I was certainly pleased to win but you know happy to meet the other guys so how many other people we up against Thurman but battle Finn was something like between three to three thousand people oh that was a lot of people yeah metal tons of applicants and then it got narrowed down into three groups

of maybe nine competing over a few months something Like that and involved the presentation in Manhattan as well this was quite different this and I don't know how many applicants that were but they were present there a couple series of presentations and the first presentation was in the fall of maybe 2015 I think and that was maybe 15 to 20 people it might have been 16 rings Bell so I'm not exactly sure and then that got narrowed down to maybe six if I'm remembering correctly and then we the six or so of Us presented in

in the spring of last year and you know it's an instant group I mean you had individual you know traders with a strategy they created you had you know small money management firms with proprietary strategies a group of strategies you had I think to actively managed mutual funds which alone is funny because on one hand some traders so say uh mutual funds you know what what you know what good is a mutual fund I'm going to trade the market and My self-directed way so on and so forth but to actually launch and you know take

a active strategy put it in the mutual fund wrapper and get that at the market and and make it at a viable reality it's actually quite challenging so you know I was impressed with all the bottom line right and during the competition was it all trading real money that's a good question my strategy is real money it has a model but it it's approaching six year anniversary of real Money trading in March of this coming of this year now in terms of the other strategies obviously the mutual funds real money naturally and millions of dollars

and then you know there were a couple of strategies I you know I think predominantly there might have been one that wasn't I think that was something like that okay so the strategy that you traded during the competition was the same strategy that you've been trading For the past like you said six years that's correct yeah it's an equity strategy so it's blue chip stocks mega cap equities and that's been live since 2011 in the spring or late winter spring March 2011 right very interesting I always thought that I don't know why I thought this

but like a lot of traders who enter into these competitions we're trading like purpose-built strategies to try and win that particularly that particular competition but obviously not The case well it's interesting I think there are I think you're spot on that are these competitions that are exactly that where they're essentially orienting the whole approach the whole investment proach or trading approach to winning you know drawdown be damned whatever long-term efficacy who cares in now just win the competition but you know these competitions funny because I never set out to be a competition winner I'm I'm

thrilled to be won a couple times now but really these just kind of fell in my lap in the sense that I learned about them by one means or another and decided to enter usually at the last minute often right right at the deadline I think maybe maybe in each case and you know why not you know this this could be interesting and you know they were not that kind of competition they were really about the whole picture you know so to the credit of each of These groups battle in a name their competitions were

rounded in the sense that this presentation you had defend you had to defend your returns and risk and your thinking your your model you had to defend all of that and much more and of course you know had to have you know good returns or I suppose for those who didn't have real money returns you know had to really defend their hypothetical models but it wasn't about yeah it wasn't about gaming a Competition and just winning you know it was really about the strategy and the integrity of the strategy so maybe that's a bit different

in the competition world yeah no I mean I think that sounds much better it's much more realistic is probably one way to describe it so you know how you said you had to like defend your strategy was there like a panel of judges that you presented to and did they ask you any questions oh yeah yes yeah both cases There there was a panel so in the you know in the first competition you know the battle pitting competition which I'm sure has evolved I don't know how it's done right now but at that time it

was you know go into a Wall Street shop and there's a meeting room and you know you walk in and put your slides up and you're on the hot seat you know and you're just going to get you know make a little presentation and then again get grilled and it was similar for name Although it's a bit more formalized you know there was a whole you know under 200 or however many people were there in the you know in the room in the theatre or whatever and then there was a panel of maybe four or

five judges something like that and people have come to know actually since and you know some sharp some sharp people for sure and very very probing questions you know really just how and I'll tell you here's my here's my takeaway from watching others in one Case I got to watch others present in the other case it did not but where I got to watch other you know strategy creators traders managers whoever they were present and defend their strategies really where people would fall all from from from a great height maybe they've been doing great after

that point it was always related to risk every single time they would just unwrap because if you start if the if the panel would start to peel that onion around How they thought about risk and this kind of risk and that kind of risk and maybe SEC the risk and market risk and correlation risk and on and on and on you start to peel back that onion and in some of these very you know sharp guys either just weren't as prepared as it could have been because they probably have answers but they didn't articulate them

well which you know didn't work in their favor or or you know in some cases may have been a little light on the risk Defense and the risk knowledge and really having thought through every facet of that diamond so that's something that I think about a lot is I think that that did help me because I've I thought a lot about that and certainly that was part of my development process it made my development process longer actually for that strategy because I was constantly looking at another facet of the diamond to use that same analogy

yeah that's really interesting actually That's very cool to hear I've made a note here to ask you a bit more about risk controls as we get going but um yeah I'm definitely going to pick back up on that so last time when you were on Dave we spoke mostly about your path from starting out as a discretionary trader to becoming a quantitative trader now this time I think we're going to pick up on a few of those topics and go a little deeper as well as cover some new grounds so one of the first things

I'd really like to talk to you about or ask about is things around strategy development so starting out right at the very beginning when you come to develop a new strategy do you have any predetermined metrics for what will be a good strategy in your mind like goals or objectives before even trying to find something right great question and that is that's something I if I could go back I'm perfectly happy with my strategy but there are times Where I've thought hmm you know I wish I could go back and maybe rethink you know we

think a couple aspects of the strategy you know obviously with a lot of perspective having managed it now for almost six years again mostly I'm very happy but you do learn you know you do gain a perspective on you know what a great objective function might be and so in many other metrics which I can get into basically one thing that I think that you know a new program or a new Developer and I certainly talk to a number of people aspiring to either make that transition from discretionary to systemic or simply finally deepen their

their knowledge of quantification and develop a model and so on there's some questions I think that those people should ask is first of all who is the model for is this for pure personal or personal entity or family trading for example or is this for other people's money that's that's a huge question Right away and one that really should be answered because if it is you know for other people's money then one has to be thinking about just a broad regime of questions from regulations to you know that that can inform what asset classes one

might choose to model and that's usually not a question because most people have you know some you know leaning towards you know Forex or you know futures or stocks or whatever it is but you know from regulations to you Know drawdown tolerance your trot your personal drawdown tolerance might be way have a much greater threshold for drawdown then you know then a client for example those people who are either or managing other people's money so you know that's one consideration very specifically having a an objective function you know this is kind of the other extreme

of now just the strategy itself how are you going to determine your your end result Let's say you've gone through here towards the end of the development process and you're choosing between you know a couple different sizing methodologies or you're stepping across your parameter stepping various sizing approaches and finding where what sizing of your portfolio of systems for example is optimal for for you well how do you know how do you do that that can be a number one can develop a specific objective function of formula is all That is you know a simple formula

that's just scores essentially all of these results from various sizing approaches and therefore is rank able according to your objective so for instance to make it a bit more specific if one is looking for a an annualized return that is some multiple of the max drawdown across all years of the backtest for example that you know that's a number that every strategy run is going to have and that could be that formula could be part of a Objective function with maybe two other similar metrics right they all gets rolled in the one formula it out

pipe is one number that's then your objective function that's what you're trying to maximize and of course you're trying to maximizing it without burning up the past that's a whole other issue burning up your data you know finding a clever way through the past instead of a robust set of rules and a strategy that can generalize well going forward walking Forward in the future obviously that's the that's the real objective but just in terms of of your question and hopefully I'm answering it at least in part you can have an objective function that actually is

a metric and it's a formula that can be pretty simple usually made up of a few different metrics and essentially it's a score and you can score your score your end results okay so in that in your response that you gave an example about a metric Of draw down compared to return something along those lines let's say you also had a couple other metrics that you wanted to achieve from a strategy so let's say going into this I want to achieve a Sharpe ratio of 1 or greater you know max drawdown of 20% something like

that how much would you be willing to vary on these metrics so how much would I be able to be willing to vary in terms of well I have this great output here that I feel great about but it has A really crummy Sharpe and I wanted a higher honor was that yeah yeah exactly okay yeah right that's a really interesting question you know and these are you know very great questions and not always ones that I was thinking about my number one you know my objective function was essentially pretty simple it was pretty much

what I articulated along with a couple of things which essentially was you know positive in all years minimal Draw downs and an average annualized return that was a multiple of the worst drawdown figure average drawdown is not really something you should look at in my opinion perhaps there's a case where that is important but max drawdown in related draw tends draw downs that were close those are what you're going to be living through in the future and you're probably going to be living through worse you were strata I was always going to come in the

future so That was part of my max my objective function in terms of other other factors you know that you know other metrics that maybe it didn't live up to as well you know I didn't experience that but I didn't set out with a smorgasbord of requirements and really set out to do what I said which is without having full exposure to the market to be able to stock mark in this case you know benchmark being S&P 500 Total Return Index you know including the dividends And so forth with that as a benchmark you know

obviously I don't want to be well maybe not obviously but I don't want to be fully exposed to that all the time some correlation is okay in in my approach other approaches are you know none correlated and they strive for no correlation that's a different you know whole approach and discussion but I didn't have a big list of requirements so I think it's a discovery process Though in other words one should lead you know it's development is there's going to be discoveries along the way that are probably going to be surprising and especially if you're

coding a cherished notion like you know every time this makes a thirty-day high this market and you know this other factor and this other other factor are happening simultaneously well that's definitely you know that results in a reversion to the downside you know and Then you test that you really start to see you know that's a ridiculous notion that really it's your subjective confirmation bias where you you thought that's what what's happening but you're only noticing the times that happened and actually there's so many other times in fact the majority of times you know it

didn't work or the way you have the you know the entry and exit of the risk management isn't working so that the discovery process may lead you to Abandon some cherished notions it may lead you to abandon some previously cherished metrics to hey guys has a mention for squarespace.com a sponsor of this episode so if making a website or a blog is something you've been thinking about consider making it with Squarespace because if you've never done it before making a site can seem pretty confusing and rightly so like there's a lot of moving parts but

I promise you Squarespace make things dead easy arrange content and select features always the click of a mouse and award winning support is available 24/7 to Squarespace is also super affordable with their simple pricing plans so if you have minimal budget you can easily save yourself a few thousand dollars by taking the deal why approach with Squarespace start your free trial today at squarespace.com and enter the coupon code traders to get 10% off your first Purchase plus a free domain that's squarespace.com also supporting chat with traders this week is the go-to source for men's made-to-measure

shirts proper cloth finding a shirt that fits exactly how you want it too is difficult you and I we've both experienced it right but along comes proper cloth who make it incredibly easy to order a custom-fit shirt online all you need to do is jump on their site answer a few short questions your shirt is then Shipped to you and proper cloth guarantee a perfect fit when ordering you can pick from over 500 premium Italian and Japanese fabrics and prices start at just $85 + GQ called proper cloth their favorite online custom shirt maker so

fix up look sharp go to proper cloths comm slash traders and use the coupon code traders at checkout for $20 off to repeat the link visit proper cloth comm slash traders when developing a strategy you know still during this Development phase can you speak about how you think about simplicity and is there a place for complexity and what you do also right you know simplicity and this probably is touching on our last conversation but simplicity is a huge thing for me because my for two reasons first of all that's just generally philosophically resonates with me

that simple things are more robust you know there's a Mendel Mandelbrot quote which Is very beautiful in this regard related to simplicity simple rules and so forth but also you know my study of kind of first generation commodity trend-following traders and there's some great books out there on this group really is the emphasize of simplicity as well and so I was influenced by that even though I'm not in this strategy I'm not doing futures and I'm not in trend following in the pure sense I'm doing equities and Mostly reversion so almost the opposite in a

sense nevertheless I was influenced by their emphasis on simplicity so that that has served me well because the thing about complexity is what ends up happening is if you're I mean in all power to those who can have really really complex con maybe convoluted is not the right world but ornate code and complex rules and if then you know out the wazoo if they can get that to work that's fantastic all power to them However what typically happens with most people who try to do that is they create an artifact basically a really really elaborate

way through the past right because all financial models rely on the past and so we want to avoid complex travel routes that are overly tied to one place and time you know they're overly specific that that is too complex what you really want is you know going forward you want to be able to generalize well because Data you know markets are not stationary the the data can change regimes can happen you know decimalisation can happen in the stock market regulations can change markets you know there's so many things that that change obviously technology you know

market microstructure is is hugely changed in the last 10 years you know I think the average shares stock stock trade was like 2,000 shares maybe in 2005 or 2006 and you know 10 years later or there Abouts it's like 200 shares why didn't be less but you know Jeff that's the general idea Mike micro structure can evolve so basically long would way of saying that you know it's important to to really have simple rules that can can really walk through and navigate markets that are going to be different in nature in the future than they

are on now and on the data which is yesterday that you test it on if you're overly specific Overly complex you're probably going to just break apart in the future it's going to come unglued because it was just too brittle you know it's funny you can even see this on certain parameters for instance and maybe this is too much detail but you can you can find you can do a parameter step let's say on one piece of logic one variable in the test and you can find a parameter shelf like a think of it imagine

like a Ledge you can find this narrow little ledge you Know above you know a thousand foot precipice that you can walk successfully and for that would have been great in the past if you had use that parameter and then you can step right you know one parameter to the right so to speak of the one you found that worked and you're falling off into the precipice because it didn't work that's not a robust piece of trading logic you know if if you are narrowly finding the rule that worked and you're not seeing that one

wrong Move and you fall off a cliff so that's that's the overcome that's the risk of over complexity all right so I just want to pick up on that so I think just to sort of summarize what you're sort of saying there is let's say your strategy uses a a moving average of twenty for some parameter it's just one of the trading rules it involves a moving average of twenty if that moving average was 19 or 21 or 22 what you're saying is that if that produced completely Different results like vastly different results it's it's

quite a brittle strategy am i understanding you correctly there yeah perfectly said perfectly certain fact you can just replace what I said with that Aaron that was perfectly said and I actually remember when when you're on last time you described these these commodity CTA futures traders as their strategies were so simple that they could actually be written down on a napkin and followed And they made a lot of money doing that so rates one thing that that's sort of stuck with me also when you're on last time you use this term degrees of freedom and

I think I kind of glossed over this I think that's maybe quite an important point which we didn't really go into too much can you talk to us about what you mean when you talk about degrees of freedom in a trading strategy and and how it's a positive thing sure yeah now that relates to statistical Significance so the idea is and it relates exactly to what we just discussed so if one has a 20 data points let's just make it real simple rather than you know thousands of data points if one has 20 data points

and has 19 rules or 20 rules related to a system that works over those 20 data points or let's you know put it in another way yet maybe in a 2,000 data points but you have 2,000 rules there are no degrees of Freedom and you you have created most likely an artifact rather than something that is fact in a sense that can live on its own and work and generalize well going forward so the idea is to have let's say those 2000 data points but have a minimal number of rules so that therefore there are

greater degrees of freedom in and at the risk of being completely non quantitative and non-statistical I always think of a Robert Frost and I might butcher this But he said so many things the effect of oh there was poetry or art but I think it was specifically the poetry I think he said poetry is moving easy and harness because when you're developing this logic it is a harness it is a constraint and the rules are are you know they're they're fastened right they're hard-coded and they're fastened on the horse if you will and yet you

you have they have to be able to you know the strategy has to be able to navigate The different terrain you know you have to be able to move easy and harness if you're if you're too confined you have no degrees of freedom and and the strategy is not going to be viable so kind of a mixed answer to that you know partly referring to the statistics of it you know of statistical significance and you know and partly just the way to imagine that or at least the way I imagined at all so just going

back to when your On last time you you mentioned that our back testing poorly is very easy to do and I think that kind of leads us into the next step after sort of thinking about your strategy idea how do you actually go about back testing that and what are some of the ways that one can back test poorly wow there's so many Aaron how long is the interview as long as you want well I would say having poor data that would be a bad starting place you want High quality data I met somebody brief

anecdote I met a young guy who had an economics degree from a from a prestigious school and he was talking about a system that he would just started to develop on weekly data and I talked you know I asked him about well weekly data okay what about the you know in between those those weekly data points or is there any anything that have you done in research there into that massive gap of time between these Weekly closing data point said you know when he hadn't so you know you have to think about your frequency and

what's what's missing from your data you know if you use end of day data well you don't have that intraday low or high that maybe blew up your system or it was too uncomfortable to keep trading so one has to think right away about the data what's the data I'm using is a good quality and is it what's it missing possibly so that's the first thing I Would I would say secondly if one create a strategy and it looks phenomenal but that developer really really couldn't articulate what's going on you know back to the panel

discussion that you know we had early as you were asking about these you know these competitions if if one couldn't be grilled and come up with really cogent answers as to what the logic is why it's working it'd be able to articulate ones investment process and ones edge if you can't do That that's probably a sign that you just created something that you don't really understand and it probably has some flaws so maybe not necessarily a way to develop poorly but that that would be an issue to me one really has to understand each aspect

you know I have a strategy that has it's a it's a singular strategy but it has nine systems within it and attacks is being added related to adding a positive volatility element and you know each of Those systems was is you know is very familiar to me it was not a hodgepodge of of rules and logic that I don't really understand so you know in my case that works for me and I think that's probably important for most developers you know machine learning processes is is different one should really understand that it's a lot to

understand there may be you know the the the whole process is finding things that you may may never have come up with that's a Different thing but if you're doing kind of traditional development I think you should be familiar with the the ideas and the logic and be able to articulate it you know so those are those are two ways I would say not being statistically significant so you know if you have a if you have 30 trades and it's a phenomenal system that works on the lumber market and yet you only have 30 trades

in your model and you know is that is that meaningful you know that that's a Statistical significance question and a degrees-of-freedom question and a robustness question you know better you have thousands of data points not just thirty so I could go on but those are a few I'll probably think of more later and circle back yeah so just to pick up on that last point there we said you know it needs to have statistical significance you know if there's only 30 trades 30 data points in your back test it's not Very significant it doesn't really

tell you that much every tried is different in the sense but for you what's sort of a rule of thumb for something that is statistically significant well it's it it's a good question I fir for my strategy let's say I think that essentially generally speaking there were about maybe twelve to fifteen thousand data points and you know under uh under fifty rules Perhaps in total you know that's something actually I'm have to go back and and quantify I might be off there a little bit but in other words there is I have a very very

low rule to data point a you know number and ratio so that that that was important to me just to have a lot of data and to have a lot of degrees of freedom you know back to that discussion okay see said some way between twelve to fifteen thousand obviously that's very relevant to you You're trading a universe of many many stocks I'm not sure how many exactly so you've naturally there's going to be a lot of data points in that there's someone who might just be starting out let's say they've got a strategy that

only trades on one particular product or a cross you know a very small universe would you say maybe something like a few hundred trades is going to give you some significance yet my answer to that would be most likely yes and secondly is is The does the logic work well on related markets so if one has a great ten year you know Note futures strategy does it work on the thirty-year futures does it work on the five-year notes you know that this is you know this is something that I would I would investigate that would

you know go for stocks obviously if one had a stock specific strategy can it work over portfolio of stocks not just a single name you know I I'm a believer in the portfolio Approach but but it really differs you know with stocks it's that's quite helpful and working the sectors within within the stock market that rotation in and out and so forth even if it's fairly short term but yeah I would think a few hundred data points in does that same set of logic the same rules do they work on related markets probably not as

well but do they at least have a positive expectation that that's something I'd look for if it falls apart on every Other market except the one that you developed that is probably a red flag okay I'm glad you added that last bit in because that was going to be my next question I was going to say like what if it doesn't work on you know like you said that the 10-year note and then you try it on the 30-year note and it sort of falls apart then that is an obvious red flag so yes thanks

for adding that now here's something I want to ask you a Few questions around and that's curve fitting so we've probably already touched on some things that are relevant to curve fitting already but yeah let's just let's just break this down so what measures do you take to reduce curve fitting well you know I think we've already discussed nearly all of them actually because it's avoiding a to Hyrule to data ratio right so in other words minimize the rules and the logic avoid complexity You know simplicity doesn't have to be simplistic it can be sophisticated

yet simple most most formulas you know that are really powerful are incredibly elegant like the fractal formulas I have a book on and I'm going a little maybe a little you know tangent here and but you know I'm fascinated with really complex or seemingly complex things you know phenomenon and nature and so forth and in the fact that many of them can be reduced to really elegant simple Formulas obviously not not all the time but that is often the case so I think we have talked about a lot of them you know it's really avoiding

the complexity embracing the simplicity having simple logic that can generalize well quality of data in and in terms let's see more specific may be something I haven't said if one is not reserving data and that's something we haven't talked about so this is this is critical if one is not reserving data that never gets touched The trading logic the whole development process never sees this data this data is completely quarantined like it has the bubonic plague you know just sealed off never get seen that data should be seen once and only once purely out-of-sample most

people developers don't you know they don't do that obviously sophisticated more scientifically oriented you know modelers are going to do that every time However you know you're maybe average trader who just says hey I want to get into you know trading logic now and develop something quantitative or systematic might not be rigid about that and that's that's crucial so that's that's very important one has to develop and develop and then flash that data once essentially across that out-of-sample data in you know if it if that doesn't work you know you really have to go back

to the drawing board and You also have to realize that if that doesn't work and then you go back and tweak it some more on your you know your development data your testing data your training data essentially and then you go you know you you are now that data is not out-of-sample anymore because it's informing your changes you know maybe maybe the strategy was great on the out-of-sample data for whoops that you know 9/11 or something you know in market terms is Pretty horrible and other terms of course horrible but and then you go WOW

gosh I just you know change the holding time I could have avoided holding in you know to that day and avoided that draw Deb and suddenly now my results look great well you know you just use your out-of-sample data to you know you just burned that up to because that's no longer out-of-sample it just informed your further development now what David You have second set of data you can test on that would be a good idea to in that case so I would say that's a that's something we haven't talked about that's really important to

avoid the curve fitting or at least to verify that you most likely have not curve fit you know that's a very good point that you do highlight there is you know once you have run your strategy or your idea over your out-of-sample data and then you go to make some changes and do it again That is essentially all now in sample data and so you know it's just something that you need to be aware of for sure are there any telltale signs that you have over fit well a fact that that summary or that equity

curve when you go and and go to the out-of-sample data and it looks completely different then you're very smooth a development equity curve obviously that can be a glaring hard you know impossible to ignore sign that you have developed a curve fit Model you know obviously real performance you you let you launch that strategy on on real money now and you are not seeing you you're immediately breaking metrics that shouldn't have been broken maybe you have maybe the worst data the worst drawdown perhaps in all of your back test period was a you know what

5% or 10% and you're breaking that within the first month of trading that's that's certainly conceivable that that could happen if Perhaps it's a similar market condition type of scenario but that that could also be a very bad sign that you know you have a curve fit model that was just to fit to the past and now that you're running through the present and future you know it is it's simply not generalizing well and and it's falling apart okay so with that in mind how do you begin trading a new strategy so you let's say

you've gone through all your development process you're happy with How it looks how you actually start trading that in a live market do you start with small size do you trade it still on a demo account for a certain period of time or do you just go all in yeah that's a good question obviously that's that's going to differ by individual I don't think there's there's one right answer to it obviously there's just degrees of conservatism versus you know gunslinger really I'm never a gunslinger so I I would wait in to the Water and you

know and test it test it that way on the other hand if one is paralyzed by fear when you know just is still afraid that the model maybe is not going to be effective you know you could find reasons to delay for a really long time and and that you know that that's obviously not a good scenario either so you know I I'd say it's personal preference you know perhaps the smartest way to go though would be to depending on the nature of the demo I mean you Know markets are going to be different and

the demos can be different you know if it's spot forex and suddenly you have the greatest spreads where they really don't exist in you know where you'll be trading that strategy then your demo results are meaningless perhaps unless you can build into those assumptions and that was another thing that we didn't touch on but have very conservative functions don't assume that you'll get fill that your limit if you have limit Orders in your model look for those limit order prices to get exceeded and then count that as an entry especially if you're looking to develop

something with high capacity that would that would be trading a lot of money so anyways short engine on that but I wanted to mention them so in terms of rolling out you know a demo is a good idea but just be honest look at the look at the nature of the demo isn't really modeling what your fills and commissions Will be if it's not a situation where you need it to be handling a lot of capacity a lot of money assets and so forth and that's maybe not as important but of course if you're if

you know one is developing a hedge fund strategy for example it's a high capacity strategy you know how do you model market impact right your very order in in the real market could displace you know like elf in the bathtub kind of situation it could displace the market place they Could have a market impact and your actual fill therefore would be quite different so if you're in a really illiquid market you have to think about that or if you're trading huge size or you intend to be then that's a factor that you have to take

into consideration now most people are probably not going to be you know modeling very for very illiquid market or necessarily trying to develop a you know 1 billion a you know dollar capacity strategy for example but Those are factors and you know as you that my approach in those different scenarios would potentially be different in terms of the roll out in those you know in the scenarios yeah absolutely both are really good points now I don't think I've ever asked this question before but I think it's a good question do you have a life expectancy

for your strategy so once you start trading them do you have yeah do you have an expectancy like do you think that this Is going to last for at least five years do you think it should last forever or you were just sort of thought that you know are you with the sort that at an age will not last forever yeah that's a great question and somewhat unknowable so I submit I'll answer the question in a second I have met - you know I divide them into two buckets in other traders with quantitative systematic strategies

there is one group the fact the hedge Fund that is quite sophisticated and pretty large where essentially they are constantly adding new edges retiring edges modifying you know updating parameters so on and so forth so it's like a constant living breathing evolving strategy and that would be hard potentially for you know for a small player to less very capable with with modeling and and data to keep track of that model over time but but that's one bucket that's one approach You know and I've met I've met some groups like that and some individuals like that

then then there are other groups or individuals that really stick to the you know to the hey yeah this is it this is my logic and I stick with it and I really have modified it you know that was that one tweak back in you know 1992 but since then you know and there's those people out there and I've met them as well so I there's not one approach to to how frequently one is I don't think There's one set of rules that say you have to never alter your your model or you should constantly

alter it there are diverging philosophies around that in terms of my own life expectancy for the strategy you know Mike's based on my model my expectation is in in this performance over almost six years is is that it should navigate continue to navigate markets well because again back to simplicity the simplicity of the logic you know in my in my studying of Testing of marketing it's over a long period you know other periods of time you know data that goes way back you know 60s 50s and in earlier on than that when you can get

that data you know I have reason to to expect that you know that the strategy should continue to perform well but I don't necessarily think that it will be without some revamp you know they're just it just to take one example you know obviously all stock traders know or should know of you Know the 29 crash the 87 crash you know I've met many people who were wiped out in the 87 crash or just took their puts off right before the crash and you know that was a life-changing decision trade you know biggest option trader

in the pit who you know lost it that day great guy and you know and these were life-changing events but essentially I was going to mention 87 essentially that crash there there is a change there was a bit more propensity for momentum prior To that fall of 87 and after the crash reversion worked a little better in equity so you know there are there are these obviously it's a dramatic event 87 crash but you know there are these changes in markets that can produce new tendencies and so I think put it simply that I will

adapt to those hopefully but I expect it in lieu of something like that I expected to continue to perform over time just due to dis implicitly yeah and I just want to throw in there guys if you want to hear more about the 87 crash and someone who actually paid a lot of money on that day listen to the interview with Blair Hall who bought the the lowest tick of the day on the you seven crush yeah pretty good record good recommendation yes phenomenal so with all this thing said how do you monitor your performance

when live trading like how do you know everything is doing as it should be right you know I think that's a keep it simple approach in other words what what metric or metrics should one monitor you know they're the obvious one such as you know rolling returned over various windows of time you know how is that stacking up over time is it deteriorating is it maintaining is a slight deterioration how does that compare to the model they're the same with drawdown you know dr. Howard bandy has some great books on trading I you know I

highly recommend And you know in at least one of them he talks about you know winning percentage can be could be a good monitor for example just in he doesn't employ portfolio approaches I don't think but you know on a trade basis for the system you know what's what's the winning percentage over very slices of time you know is that deteriorating that could be a canary in the coalmine saying you know one edge is decaying and so I so I don't think there's a one-size-fits-all there But I don't think it needs to be incredibly complex

either so there are some basic measurements to essentially look for for edge decay and if I could if I could rent real briefly for a moment there is this there's a subjection to systematic and quantitative trading sometimes you hear people will say hey well I've never seen a bad back test haha meaning you know you know it's a survivorship bias free thing you know or survivors you're by Spring where you know you only is the only the good back test win and they're all the other back tests are deleted or sitting in a you know

in a hard drive somewhere so um yeah there's something to that obviously no one pursues a strategy with a negative mathematical expectation has to be positive that goes without saying but so that objection bothers me sometimes like just because the fact that people pursue winning models is is Somehow inherently flawed but what is flawed and what every system at or not flawed but whatever whatever you wanted quant or systematic trader has to have an answer to is how your question you know how will you know when it's not working anymore you know because some people

do object they say hey you know I had a quant and I had experience with the quantity and you know it worked until it didn't and then it blew up and you know I'll Never I'll never touch one of those again you know obviously had a negative experience and you know it's really crucial for I think every systematic quantitative you know strategy trader or manager to have an answer to to that edge decay question and you know it were it works until it doesn't you know what's your answer to that because if you don't have

an answer to that then that that could be a problem whether you're just trading it for yourself or Whether you're you know trying to you know go out into the world with it Gudrun or lock it is there any other misconceptions you'd like to get off your chest Wow ah you know probably but they don't they don't come to mind at the moment so I know I think that's my singular rant right now okay okay now we were speaking the other day prior to doing this interview right now one of the things you mentioned to

me was that you were Thinking about sort of doing things a little bit differently this year you'd like to kind of explore some new things I'm not saying you're going to ditch your your strategy that you've been writing for six years or so that's going to continue but yeah yeah you're looking to explore and look into some new things as well this year moving forward can you tell us a little bit about what some of those things may be yes absolutely and So it's exciting I think you have to be fascinated with markets to really

succeed in them maybe that's not true maybe they're people who are just cynical and just you know develop strategies and then they make a lot of money and they they don't they're care less about them but I you know hasn't been my experience I think most people are deeply interested in that fascination serves you well especially especially in hard times when maybe You're in a drawdown but and so you know in terms of my development list this year it's it's something that I've really started to codify in the last couple weeks for the rest of

the year is relates to cut you know a couple different areas one is just having a better data infrastructure and that you know this is the decade of data as a lot of people you know now call it have been calling it for a few years where I have you know I forget the staff you might Know Aaron but I mean you know just essentially the bulk of the world's data has been created in the last you know few years so essentially it's really just a phenomenal fact however close to the truth my summary is

there but so essentially a better data infrastructure and more tools to manipulate data in essentially making you know because data prep is really a lot of testing whether you're doing doing data visualization you know maybe it's non market related You know doing word clouds or you know web scraping and then trying to find some intelligence through that or obviously in the trading realm simply looking at correlations and looking for other relationships that might be interesting with you know new volatility indices related to you know strategy you already have I mean there's just so it's just

so there's such a richness of possibility so having the data you know a better infrastructure it's kind of a Boring topic but that's something that that I'm working on this year and really that that will enable me to work through just many ideas more quickly so a lot of them are related to - just analytics or enhancements on my current strategy you know the equity strategy there are you know other asset classes that I'm developing edges in and that's usually how it starts which is developing an edge it works in a lot of currency pairs

or it works across Related features markets and then from there one can you know delve further into you know how can i exploit this you know what capital would I apply to this you know obviously it raises a lot of questions but so I have a deep list there slash far-out stuff perhaps is just looking at various whether it's you know fractal formulas or formulas related to you know biology and so forth those are those are things that I'd like to go deeper into but they're a little Further down the list so you know it's

I think we might have talked about it Aaron just casually the other day but the separating ones you know isolating oneself away from distraction is really I think crucial for development at least for me so that has to be a very you know specific time of day has to be the right day it has to be or a regular schedule whatever it is for for every one's going to be different but I I'm a believer in intensity not necessarily time having The schedule is good having the time set aside is important but then intensity being

undistracted so once you can have long thoughts I think that's really important and I just mean uninterrupted thoughts where one can just go through a hierarchy of logic in one's mind as one's coding keep that all there you know cached influence mine as one's working that's really important to my process because if you allow the constant interruptions then you can you Can lose some really phenomenal things that you might be on the verge you know of developing and catifying so kind of a long-winded answer perhaps your question but you know those are just a few

of the areas that I'm I'm really looking at and and how I think about them ya know I like that I like that I think um we'll have plenty to talk about next time you're on as well by the sounds of things I'm blanking on the the man's name at the moment but he Wrote a book called deep work and it sounds as though I haven't actually read the book I've read like a few reviews and a bit about the subject but it sounds as though that's kind of like what you're talking about there another

word for it is deliberate practice you know it's one thing to spend a lot of time on something but are you actually spending that time in the most efficient way like a actually you know totally engaged with what you're doing or if You've got distractions coming left right and center you know what I mean yes yeah I don't know that book but um but you'll you can tell me about it later I'll google it whatever but yes that's absolutely I'm a believer in that and you know I'm older than you you know I've always had

a serious bent but when you see your own theta you know your life time decay which sounds pretty dark but I just what doing the option terms there for a moment but you you know you You you do dig deeper I mean I do anyway um just dig deeper and go alright you know this this and this other thing and this other thing they have to fall away because they're just they just don't rank anymore you know I saw and I would do them I thought I'd get them but you know I'm jetting them because

I'm doing this and I'm doing this other thing and that's it and you know you said deep work I think you use That title um you know I just think of it as long thoughts and I also that phrase intensity not time because you can have a block of time and do very little with it um but you can have less time and do tremendously well with it you know one trick I think this I saw this I've been using this trick for a while I don't use it every day but I get into periods

where I use it where I had set set a little kitchen timer a little egg Timer and that's a nice trick because you can set that for you know twenty five fifty five minutes whatever it is and that's your focus time I mean you could let nothing go except your task at hand for that period it's a little you know it's just a little kind of a crutch or device obviously for longer things it's not going to work but for for tasks staying on task it can be it can be a good device yeah that's

called something that tactic isn't is it Fedora something like that I probably just got that really wrong but I mean although I did see a young I saw an article within the last year you know as I came you know in my inbox and and he did have it did have a name so I guess it has a name yeah we spend like twenty-five or fifty five minutes or a certain period of time and you just purely focused for that amount of time and then you have a ten or five minute break Where you just

do absolutely nothing you're just kind of zone out and then just smash it again for another twenty five fifty five minutes whatever it is yeah I've read a couple things about that actually but I'm just going back the the man who wrote deep work I can't believe I'm blanking on his name right now but um and she's trying to get him on the podcast not a traitor whatsoever but actually I think you know a lot of what he talks about and is knowledgeable About will be very relevant and beneficial to traders and obviously listeners of

this podcast um just speaking of talk about your life in terms of SATA or decay and that kind of dark you might appreciate this as you appreciate a data visualization but there's this very very simple chart which I've seen floating about online and across the x-axis it's got weeks and a year okay goes from 1 to 52 and then down the y-axis Vertically it's got ages so going from like zero 200 or whatever you live - and each little circle on that chart represents one week of your life and I just think it's so powerful

what you know girlfriends let me go you know mention it and she thinks it's really dark and and kind of strange I think it's like I think it's really motivational I think it's very powerful to look at and see things displayed like that I'm going to Look for that Aaron you know it makes me think of you know I like to sit in zazen you know just a traditional simple sittings and practice and Dogen then master from gosh you know 1200 I think I'm probably off but at any rate he's kind of revered just for

his writings and I think it was him although maybe with somebody else but I may be a Dogon was was quoting him whoever the other you know Zen practitioner was but bottom line just said in sit like your heads on Fire you know which to me is you know sit with intensity and practice now you know it's you know carpe diem essentially so that can be applied in all all aspects of life so I think about that a lot I think about just solutions you know there are so many obstacles to anything great you know

trying to be great that you have to push through obstacles so you know solution solution solutions it's just another mantra I think about That every day at some point I just go you know solutions man just got to find the solution to this and it might not be trading related it might just be you know life but that these things are important I think for the trading mindset I mean we're talking about quantification and in you know in data and logic and so forth but there are there are times that are really challenging as a

trader and you really have to have a resiliency mentally that Can be in a multi-faceted from staying fit to having a you know just doing the things you need to do I chill out incredibly deeply unless I can't but on Sunday at least for the afternoon into the evening and I I don't look at screens usually and you know that's just like that's absolutely like a refresh thing so I can start Monday full force and I I completely you know slowed down so you know they're just little tricks like that everyone has their own way

you Know but I think they're important to have that healthy mind and and so on no doubt no doubt on that point I've let's sign off where can listeners car to find out more about you sure I would say you know alpha Titus calm that is a website that people can check out and you know I'm on Twitter at alpha tative I'm not super active but you know those those would be good places to look okay and you just want to spill out alpha tighter for us by that listeners can either find You on Twitter

sure it is al pH a ta tive calm or alpha tan if they're go at alpha tied up on Twitter or alpha Titus calm and what's the go with your site I've been to a few times you've got a walk you've got to sign up to view it is that correct yeah that's right and that's something I can't really talk explicitly about but in other words you know it is not available unless you read through that and then agree to those terms because you know just the nature Of you know the alpha dative activities exactly

so unfortunately and that that's what I can say so you know just just meeting meeting requirements and regulations there yeah it's there because that's the issue just thank you perfect Dave I'm very grateful for having you back on then it's been absolute life thank you very much Aaron a pleasure as always I love what you're doing keep it up and you know Thanks the opportunity sure thing we'll talk soon beautiful thank you you reach the end of this episode of chat with traders but rest assured there are more episodes loaded with real market insight and

zero hype on the way steam so to stay updated with each great new release subscribe to the podcast in iTunes and we'd love it if you'd leave a rating and review we'll catch you next time on chat with trader