060: Strategy Optimization with Robert Pardo [AUDIO ONLY]

Better system Trader episode 60 welcome to better system Trader the podcast to help systematic traders of all levels improve their trading we'll give you loads of expert tips and practical advice on system design and validation money management trading psychology and many other topics whether you're just starting out or a Savvy systematic Trader we're here to help you improve your trading and find more success this Is better system Trader with your host Andrew swanscott [Music] well hello and welcome to the better system Trader podcast this is episode 60 and I'm your host Andrew swanscott one of

the biggest challenges we face as system developers from beginner to experience can be creating robust strategies that not only look great in a back test but also perform well in live real-time trading now why is it that Some Traders can create strategies that perform well in real-time trading while other strategies Fall Apart how do some Traders keep their strategies fresh and adaptive to market conditions while other strategies they just stop working all together in this podcast episode Our Guest Bob Paro is going to discuss some techniques that can help us to solve these issues Bob

is a president of poter Capital author of the book the evaluation and optimization of trading strategies and is also the creator of walk forward analysis Bob is here to chat about creating and optimizing strategies that are robust and continue to work in the future in our chat today you'll learn common mistakes Traders make that can cause strategies to fail in real-time trading the dangers of traditional optimization techniques and how they can Be reduced and even overcome and how to determine if a strategy really is robust while keeping it fresh and adapting to market conditions also

we have a very special bonus to go with this episode which I'll tell you all about after the chat with Bob so um make sure you listen right through to the end for details on that you don't want to miss it anyway I'm sure system developers of all levels will find this chat with Bob extremely Valuable so get your notepads ready let's get to it hi Bob thanks for joining us today um it's great to have you on the show well thank you my pleasure now can we please start with a little bit of background

on yourself well you know going back to the very beginning you know I start I I started my my my activity with trading I work for a couple of Floor Traders at the what was then the at the CME and the Candle pit um and kind of got you know got got interested from when I discovered how much money they made I I got very interested um and so then it kind of that kind of evolved into um you know I I worked for the last firm I worked for um other than for myself since

that time is was Solomon Brothers when that when it did exist um and we I was I anchored their um operations I was their desk man floor manager um for their trading operation and primarily What we did was we did we did Arbitrage for John Murray weather um for his we did the future side of all of his trades um and that was that that kind of ended um and I got involved with um my own developing Trading technical technical analysis software and by doing you know in doing that of course I got I got

I I acquired a rather exhaustive knowledge of of technical analysis and different kinds of trading systems and different kinds Of methodology so we at that point we were kind of like the we were the we were the 900 pound gorilla in software in those days we were I was a Pioneer and we had we had a program called swing Trader which was the very first trading program which allowed a Trader to actually um you know build a trading strategy test it and then actually generate signals uh we also had a parallel development with our with

a very exhaustive charting and Technical analysis program a lot like um the trade station is now except it was not real time uh that kind of led us into some some large scale lasting uh Consulting projects um with one of the one of them was with with Goldman Sachs and another was with a very large oil Trader um but the one that kind of really stuck was you we built we had developed a joint venture with Dawan Securities to build a multistrategy trading program to manage money um they Basically got clobbered in the crash of

in the in the Japanese crash after ' 87 and cut out all non brokerage commission related business businesses and let me have the let me have all let me have full title to all of the the trading strategies that we took one of those uh a very small small set of those and that became our first uh money management program XT 99 Diversified we ran that for a number of years uh in a joint venture with um a Very large CTA um and that that we ended that relationship back in 12 and uh went off

they they used to handle all of the they handle a lot of the sizing and a lot of the market selection and risk management um and we actually developed we we found that um you know xc99 performed extremely well it generated about just a little bit less than a 20% annualized rate of return over 12 and a half years but it had a lot of volatility you know Back in those days people didn't really care that much if you had big draw downs and you had a lot of volatility as long as you delivered you

know High returns so but then that ever since you know the the crash and then then M off that whole climate has changed dramatically and people really are really much more much they have a much shall we say stingier risk profile than they used to have so we developed we decided we're going to go back to Our Roots which was basically Build a multi strategy uh trading program and we came up with Renaissance which is very it it has it has dozens many dozens of trading strategies in it that are not that are actually really

quite uncorrelated um and so and I'm very pleased with it it really has the it has a draw down and uh you know a sharp ratio that I've never I've never actually seen um in a CTA before so I it was it's it's sort of like proof it's It's it was very satisfying was a very satisfying proof of concept that putting together a bunch of uncorrelated strategies produces um you know a very very lowrisk uh return stream and so and that kind of brings us up to the present yeah and that's the big picture of

you okay so you mentioned there that you've got multiple strategies that are uncorrelated do those strategies follow a particular type of trading approach or are they all completely different styles Of trading well it's kind of like it's it's what there's kind of like there's a couple there's three different families um of and within those families there are a very large number of subfamilies based on using different kinds of filters um and different kinds of risk management profiles um we also generally look at every strategy we ever ever consider we look at it um in a

trend friendly way and in a mean reverting way That's just part of our that's part of our style we just we just can do we just kind I just call it inverting it we were able to always we've never had a problem inverting any trading strategy um and so we so we have a we have a we have Pro we have an equal number of ones that are Trend friendly to those that are mean reverting but we have like a strategy we actually rebalance the strategies on a period periodic intervals um and we have Algorithms

that actually tell us which are the Peak Performing strategies and um and we tend to we we always tend we tend to have a we don't we don't ever I've never actually ever wanted to completely bias it one way one way or the other but since the truth the matter is that most markets only are don't really Trend that often except in in recent years that we've had some unusual Trends um we tend to have a we tend to have slightly Higher mix of mean reverting strategies than we have Trend friendly strategies um and that's

this is what gives us the ability to perform more consistently because you know Trend following is great when there's Trends you can't beat it but it's not so hot when there's not so um you know you know how that goes I mean I we're always it's my nature that I'm always we're always doing research and always coming up with new strategies and you know we um so I I will never Really I seem to have a neck to be a neck for doing that so we're never really going to stop making new strategies because we

haven't even truthfully even begun to really Plumb the depths of all of the strategies that we um developed way back in the 90s x99 and and Renaissance really come from a a small set a small subset of one of one of those families um and we plan to keep on adding strategies uh to both continue to raise returns and reduce risk but Also to to add capacity because you know in the stock market it's easy to run a Ted fund that can that can that can handle 10 20 50 billion dollar because you have 20,000

Stacks um but in Futures you don't have that so um the way we get the way we deal with that is by adding more and more strategies h okay so what uh what markets do you mainly trade in we trade all of them all of them Futures and stocks yeah we don't trade any of the we don't trade any uh no we don't Trade Stacks now we will we will in the in the not too distant future but we trade all of the all this all the domestic Futures we don't trade we don't really trade

when we were with in our joint venture um we traded all we traded everything around the around the world but as you know in Futures everything outside of the states primarily as either an interest rate or a stock index future and that led to a kind of it led to we were very very Bond heavy for Example in the in our former program um and of course that's always great when when bonds are moving well but when they're not then of course it's not so great so I I didn't like the bias so we can

just trade trading all the domestics we have um I think a much better diversification uh by market than we had in the original program yeah okay all right well let's move on to strategy development um now you wrote a book called the evaluation and optimization Of trading strategies um the first edition of that I think was published back in the early 90s and you've you've released an updated version in um 2010 I think it was so the book discusses a lot of um techniques that Traders can use When developing trading um strategies what do you

think are some of the common issues that that Traders face when building um when building trading strategies well I mean if you listen to I mean I've talked I talked to a lot of People who do this and a lot of people I mean among the one a lot of people is have a hard time finding something um that works so there there's always that issue um fortunately we don't have we don't have that problem but there is that issue for some people um there's all but I think the biggest issue really the biggest issue

that I've been able to see with people if they're if they're more beginners they have an extremely hard time um validating it so they don't They don't have ultimately they don't wind up having a lot of confidence in it so when the minute they hit hit a draw down even if it's within the even if it's in alignment alignment with their testing they still tend to Run for Cover um so there there is that whole issue but I I I think to some extent a lot of people still have difficult they still have difficulty coming

up with a a back testing method that they in which they can have confidence so there are there Are those kinds of issues but I think another big issue with a lot of people is that they just don't understand that it's important that either they believe in what it is they have and just take it and accept it and let it do its do its thing um which is going to have good times and bad times um and live with that but they frequently don't have people um you know frequently are have expectations about trading

systems that I think are a little bit unrealistic um and they oftentimes can't find something they're comfortable with in terms of accuracy or because I the number of people who insist on something being really ridiculously accurate um is is you know that's that's kind of ridiculous in my point of view um we this is our system now actually is pretty accurate um but you know with with the original one we're probably about 5% right um and all matters as you Make that your winners are bigger than your losers not that you you know uh so

um there's people have a lot of funny expectations um and I think I don't know how it is with your people but I know that a lot of people that I've talked with who approach me um just are they they really are they they don't have a lot of confidence what they come up with so does that make sense to you yeah yeah so I I get a lot of emails and Questions about um you know people who build these trading strategies and then um you know in life testing or sorry in life trading they

just fall apart so I definitely agree with your statement about um having confidence in a back testing method well you do realize you do realize from your own work that if the if if if you have a back test that looks wonderful um and the minute you start trading it um it doesn't look at anything like what you've what you back Tested you know you just got a curve fitter program yeah it's just the that's the simple conclusion you simply that's that's that's the drop de conclusion you have to walk away with yeah okay before

uh we discuss back testing method a bit further I just want to jump back to a statement uh you just made that you said that you don't have have any problems finding strategies that work what do you think the the key difference is between You having that that kind of ability and other people who who don't I have no idea I've never a I've never been able to understand it there's a guy who worked for me years ago was a he was my assistant he was very young just fresh out of college and very bright

and very hardworking and very industrious um and he even saw a lot of the stuff that he you know he was he knew a he knew a lot more about how the systems we he had Worked in those days than than most people ever do people do now um and because he was actually my research assistant he he ran the research all the and he ran the stuff you know he set up the research you know and had the machines running constantly uh which is a bigger chore in those days than it is now um

but he um never really seemed to be able to come up with you know particularly good ideas um you know I I think one one big advantage that I had Um is when I first got involved with developing software um I really did learn a lot about technical analysis and different kinds of methods um and you know then um I always the other big Advantage I have is I'm just really pretty open-minded um and I'm very eclectic and I don't really you know I'm not even so much I don't even so much necessarily care um

if I fully understand why why a strategy Works in all cases I mean I I always I always do With the stuff we use but um you know if if I didn't really understand I could if I was given a strategy um and even if I didn't know what the strategy was um and I put it through my back testing and it came out the way I it has to come out for me to be willing to trade it I I wouldn't really even particularly care um if I did if I knew what the strategy

was as long as I knew that it was actually sound soundly tested so um I think you know open-mindedness being Eclectic um you know being very I'm very I'm very empirically driven I I would never trade a strategy um that wasn't that wasn't thoroughly back tested um and I and and and I really need a high level of we need a high level of robustness uh to have any confidence um in in the back test and so our methods are really very exhaustive compared to I think what most people do um but um you know

I I you know I mean I think you can just take a Lot of the stuff off the shelf um and some a lot of the stuff off the shelf actually could be made into systems that work um if you just you know but it would be probably it might not have the kinds of you know the risk characteristics you might like um and I think anybody I think it's it's a prudent course of action for somebody when they're starting just like just like anybody learning a profession take something that you know basically works Like

you know the turtle system or something like that um and understand it rather thoroughly and then see what you can't do about just trying to trying to improve it if you're if you're lacking ideas that that's I would always just suggest going back to the basics you yeah yeah that's a good point thanks Bob now I'd just like to move on to back testing method a little bit further your book that um that was originally released in the early 90s I believe was One of the very first that discussed walk forward optimization is that correct

yeah yeah I I I invented it so I think I believe so yeah well um I'd like to ask you some questions about that then because how often do we get to speak to someone who invented something like uh like that so before get into um some Advanced uh questions can you just briefly give us a a description of what work forward optimization is well what it what it is basically is um you it can A walk forward analysis consists um of of a sequence of individual walk forwards rolling rolling walk forwards is about the

easiest way to put it so um I think the very big difference with you know walk forward analysis versus regular optimization is that your you're you're judging the performance of the strategy entirely based upon how it performed um in a large number of experimental out of sample tests so instead of actually just doing an Optimization and looking at that and trying to figure out whether it makes sense we actually just look and see how um it it did when when we walk walked It Forward say 20 or 30 or 40 times yeah and so a

walk forward analysis actually consists of stringing together all of the forward results um of a rolling window does that make sense yeah yeah so how did you actually come up or how did you develop this technique well to be honest with you I Just wanted you know being a a practical person I just wanted some I want to kind of an idiot proof way of optimizing because I actually we we started a business year we started the money management business many many years ago with some other people um and they basically you know use use

one of my programs that didn't have War forward analysis built into it um and they it was working for a while uh but then conditions changed and it sto working I Said look at I know it's a lot of work but you're going to have to do this walk forward testing by hand because there's no other way to be confident because you're having all kinds of problems with curve fitting you're not going to have any you're not going to have any any confidence unless you do unless you do it test it that way so because

that's kind of kind of what we did back you know because in those days the computers were had very low memory and were very Very slow compared to now so we had to we we would simply have to do it but we we would we' do a few of them by hand to see if it held up um and then I realized it's kind of like to there's just no point really um there's just no point in in my opinion in looking simply at an optimization uh when you can really look at the results of

how it performs uh out of sample so um and in reality I never have really you know a lot of people will do a walk Forward test and say they they'll optimize it on 10 years and hold out two years and see how it does and that's a worthwhile thing to do but the simple truth is you know you just like you can actually have um an out of sample window work one out of sample work just by chance by random chance so one of the things I I realiz is that the way people actually

really do this kind of thing I would I was talking to various of my customers and I I realized that a lot of them really basically um one of the guys we actually one of our customers is actually doing very well actually did this process and he would actually Walk It Forward like every every for a week at a time and I thought that was kind of I thought that was way too way too frequent but uh he claimed he was making money so um I so I it occurred to me that there might be

a way to systematize this and so we did and I just kind of call walk forward Analysis um in ID approved way of of of optimization okay so you've mentioned a couple of different sized in Sample out of sample Windows there how do you actually determine the length of those windows that you should use well that's actually um specifically is it's it's proprietary but you have to do it empirically you really you really can't just I mean you to some extent you can take you know if you got a slower program then your in Sample

Windows got To be two three four five years and you're out of sample could be more like you know maybe a year or 6 months or you know 18 months but ultimately um what what the way what really works the length of the windows has got a lot to do with how many trades you make because you need to get um you need to get a fairly good size sample in any optimization window to have any credibility uh even if you're doing Walk forward analysis so there's there's an Element of the trading Pace um and

it's also so um one of the things we found is in in you know I mean we all know markets change so um you I'm never I I'm more comfortable these days with ever shorter Windows only because um I I I like to be able to be I like to be adaptive I like be able to respond quickly to CH to to Market changes so it's kind of like a just kind of a function of the nature of your of your system um how many trades the System and produces um and uh you know your

own uh your own sensitivity to uh to risk because the more adaptive you are um the lower your draw Downs are going to be generally speaking um and if you're more comfortable with a slower pace of trading um and you're not as sensitive to draw Downs um then you can go a little bit slower but ultimately it's up to the person to figure just to determine this determine this imper okay so if there is a shift in the Market conditions say um the last optimization period was run during a bull market and we switched to

a bare Market would you would you do a reoptimization then or how would you handle that type of scenario well we the way we maintain the models is we we we actually we can we be we do the same thing we in in real life as we do in the testing so if if we're looking at a two a two win twoyear a two-year optimization window and a six- month out Of sample window we simply reop we have we have a schedule and we just reoptimize it for over two years every six every six months

so that's the way we do that um you know I don't think you can just I I you know I don't think you can simply I I I it doesn't Ser me as being prudent to simply reoptimize because um the market has gone bare because you're still going to really only have you know you're only going to have bull D bullish Market data And you're in you're testing anyway so this is where you have to you have to have some kind of you have to have some kind of confidence uh that your strategy actually will

perform you know relatively well uh in Bull and bare markets yeah okay so with the software these days it's actually possible to optimize the ins sample out of sample lengths as well by running multiple walk forwards so do you think that determining the optimal window length With that type of approach is actually robust not well if you're saying you'd actually want to op you actually want to optimize your in Sample window and you're out of sample window um over let's say you wanted to look you know and and test it over a range of you

know weekly weekly intervals um I would I would never I would really be very uncomfortable with them yeah now what about um there's a couple of different approaches for these um Windows you can Have a rolling window or an anchored window how do you know which one use and there any benefits or disadvantages to each well I would just tell you that I don't use I would never use an anchored window so what that you kind of know my bias there um it it defeats the whole purpose you really don't I mean we in our

old program they even though we built it with walk forward testing um that was a rolling window they act they Actually impr practice used um they actually use an anchor window and and that was because it was kind of slower um as it turns out their look back was about 10 years um because it was slower uh it and they would reoptimize it once a year it w up being a 1 by10 you know a 1 by1 walk forward and that's I I I we we test that for example and I don't really have a

problem with that for a slower slower system but what actually happened was as years went on uh the 10 Year became 11 by1 then 12 by1 then 133 by1 then 14 by1 um and you know it winds up it wind winds up being far less responsive to change because if you're in a especially there's a real problem with um the fact that in in in the middle 2000s that's when the data becomes a little is a little shaky early in in the early 2000s because of the fact that we're rolling into we're rolling into all

the electronic markets yeah so um And and and now that they're that we most people I think test on the 24/7 markets or 246 and a half electronic markets these days so um I just don't think I just I I honestly you have to really ask yourself um unless you gota kind of unless you really have a a slower paced program um is the in your in your in your mind is the data from 2002 um as valuable as the data from 2008 as valuable as a data from 2012 because you know we we had

such such Change um in economic conditions uh these these these markets have really these markets have really gone through some some tremendous change so I think in Anchor window basically winds up um winds up holding on to a lot of data which becomes essentially um irrelevant to what's currently happening okay does that make sense yeah it does yeah that's a good point thanks Bob so when you're comparing the ins sample and out of sample results is there a a specific Level where you'll say um this outer sample is too far off from the in Sample

results so you won't trade it or how do you actually analyze those type of results here well this you know we don't really look that hard at that we actually used to do much more of an analysis like that um what we really are were um I I think you probably would find that if you really have a system that's making money in the in the uh out of sample and it's doing it robustly in Other words it's not it's not just it's not just doing it in two or three it's doing it in 15

or 20 of them of the out of sample Windows um you're probably going to find that it's actually doing it's it's actually probably got a pretty good ratio we used to have we used to look at a ratio of performance of in Sample out of sample because what it really tells you is if you have a situ what what I can say is if you have a situation where you're looking if you're Say you're looking at five five variables which we don't do uh for pragmatic reasons plus also for reasons to do with curve fitting

but if you're looking at a if you're looking at a situation where you've got five five five five variables and your out of sample is really only about 10% of the of the in Sample um if the out of sample result 10% is is is good enough from your point of view if if it's making enough money there's nothing really Wrong with using it but what it does tell you is if you're if you're looking at 10% versus 10% out of Sample versus in Sample uh you're probably you probably have you do have some elements

of curve fitting in there so you can be count you can count on that if you're I mean ideally you want to see the same a of sample as you do in Sample um I really I really um can't say that I've seen that that often um but what you really want to what you what's really More important in in my opinion is that that the a of sample um be robust and that means that you in the same way you look at an optimization and realize if you made all your money uh in one

year out of six you probably have a real problem uh you know it's possible especially like with what in the energy markets for example with what they did in eight um it is possible to have a walk forward where you already you make most of the money uh in a in a Couple of the other couple of the Walk forwards um and that winds up being being problematic in my you okay when you said that you need to make sure that your out of sample is robust are you just saying that robust means that it

makes continuous profits or how do you judge if it's if something's robust from the OU of sample results you really well we have proprietary methods in our testing that tell us this but if you're looking at one walk forward uh you Really want to be sure that you've you want to have for example you want to have more than three you know walk forward tests because with a long system and a Long window you don't you can get just you you can wind up getting very few um so there's there's that issue number one you

want to have a lot of what I call individual walk forward experiments or walk forwards uh and you also want to be sure that the the majority of them are Profitable um and just in just with ordinary common sense that they're relatively evenly distributed so what about um the optimized values they can sometimes vary wildly between optimization window does that actually tell you anything about the strategy well you know in truth unless the market has been extremely volatile um and had a lot of trend revers Trend changes and so forth um that's kind of a

challenge um if they're wildly different Um meaning like if you have let's say you're looking at a moving average thing and you're looking at uh you know 2 by 10 in one window and you're looking at you know 20 by 20 x 70 the next window um I mean I first I wouldn't never do a walk forward that has such a range of parameters but if you had something like that for example um I would say it's it's you got a problem um and it's I I would be honestly I would be I'd be a

little bit surprised um if you had a Walk forward that was successful in my estimation um and you had extremely high volatility in the parameter CH parameters changing okay but you know but on the other hand sometimes it can it can be it can happen if you had really volatile markets yeah so I guess look at the the uh the results in the context of what the Market's been doing uh you do need you do need to it's got to make sense let's put it that way Okay now if I can just jump back to

a statement you made a few minutes ago about um you wouldn't use five parameters in a walk forward have you found in your research that a specific number of values is ideal well you know that's that's that would be I I'm not really sure you can really say that I mean I wouldn't say that dogmatically um that there's one that two is better than seven or 10 is better Than four or whatever um I would it really would be a matter of the nature of the system so I mean I've seen people come up with

different parameters because there's there's a whole thing about testing whether or not the parameters you actually are using are actually contributing much uh to the overall performance um a lot of times when you have have a lot of variables the parameter really the parameter is really Um You probably if you really look at it hard you might not really need them all but let's say you have 10 parameters that are really different from one another and you really need them all um and they all make sense because they all contribute to the performance relatively equally

then there's nothing wrong with them but trust me on this one the one thing you're going to have to to do it the way we do it you're going to have to have some pretty some pretty powerful Computers to get to to see results in your lifetime with 10 parameters and a strategy yeah okay so if you're running multiple strategies should the the walk forward reoptimization be completed on the same date for all all of them and using the same length or is it best to diversify that across across the the modle strategies uh I

think it's much better if you don't have them all lining up it's much better to have them that that Really I'll be lining up y so on your website I read a statement only markets that demonstr great robust and consistent performance are selected so does this um statement imply that you have some type of Market regime filters to ensure only the optimal optimal type of markets are traded well um we have those kinds of things we are not using them right now um the methods that we use for testing really really really show them when

we Pick a strategy to to trade um we know that it's actually uh it's it's it's robust and we know that it's actually um been working in a Model Market that's actually been consistently profitable uh so for example sugar sometimes is really is one of those it's always been a challenging Market um because there's so much such heavy commercial involvement with sugar more than there is in a lot of other markets in my opinion um so we are very um you know the strategies we We're very that's that's kind of a market that we kind

of put should we say on maybe on high alert we are really quick to uh I'm really really we're very very quick to pull in a strategy um or even the shut shut down trading in it um if things if it gets a little if it gets a little if it gets a little bit iffy and it's been like that the last last year or so actually but like the S&P for example um you know these these things um it just keeps on it keeps on keeping On um and there are just some markets that

um demonstrate very very consistent performance um even after even after they've been elect even after we've picked them to to step into our trading uh pool of strategies we still have we still review the strategies uh to make sure that they they continue to perform as we expect them to perform um in real in real life yeah so when you when you're looking at rebalancing these strategies how do you determine when to Do that that's proprietary and it's it's empirical it's actually it's it's really it's a it's uh yeah it's very it's it's very it's

it's very valuable but it's also not easy so yeah so do you actually rebalance them all at the same time or do you I think you mentioned earlier that that you've got them in group so is there anything that you can share about how you do that process or not well let's go it this way in the same for the Same reason it's good it's better to have um walk forwards at all all starting at different in different different times it's actually better to be able to it's actually better to um balance those over a

Time Horizon versus all at once the thing is what we we have we have built-in stops that'll just shut down a strategy if if it if it if it if it misbehaves so we have we have a we develop proprietary saps that just knock it out um if it if it goes past certain C risk parameters so we don't have to really worry about looking at them um looking at them that closely because the stops knock them out if and it's never really happened but the stops will knock it out if it gets to be

if it gets too far out of line um so you but I it's I it's as a professional money manager it's people really it's expected that we're on top of all this stuff so of course of course we are um but the rebalancing again Um is kind of a fun it's kind of it's some ofat proprietary um and there just sometimes markets just keep behave like a dream um and sometimes they're a nightmare yeah so that's what we that's so we are we're when things are really misbehaving that that shows up really that shows up

very quickly anyway in in our because we review this stuff on on a regular basis but we don't always rebalance on on the same on the same interval okay up until this point we've Been talking about walk forward is for strategy parameters but are there any other applications for this type of approach well you know I I you know I never really thought of it but I think in truth you know we part of the idea of walk forward analysis came from a a statistical method called the cman filter back in back at a long

time ago um so it's kind of like I I do believe for example I personally believe that any kind of forecasting methodology Would benefit from would would would really benefit from um you know using walk forward analysis I think people um you know who I've I was long interested I've long been interested in artificial intelligence and certainly neural Nets back in back in the day seemed like they were the the answer everybody's the answer to everybody's dream for forecasting but I personally believe that a um walk forward analysis would be a very good way um

of determining Whether or not you have something you can actually use um if with with a neural net be it for trading or for any kind of forecasting that's interesting thanks Bob now you've been trading for quite a while how has your approach to trading changed over the years um you know in in truth I don't know that really I'm not really sure it really has in its fundamentals um yeah all it's really all that's really happened is we we wound up Uh being able to pursue this multiat approach which was extremely expensive um and

very labor intensive um and so and a lot of people so it was so we had we had the chance to do that but you know I actually developed that concept back in 1993 so it's not exactly new for me we just never really put it into into production until recently um you I I think you know in the end I i' I'm a great believer that any Strategy that actually works um is working because you're really dealing with some kind of basic Dynamic um in a market uh I mean ultimately you have markets that

you you have four regimes um and you you can actually if you want there's nothing to prevent you from actually building a strategy which is optimal for each of the regimes as long as you have a good regime filter shifter or something that tells you with some kind of some kind of rep some kind Of reli ability when you actually have shifted a regime um so I think you know it's it's ultimately there's only a few ways to really make there's only a few ways to really ultimately every strategy makes money only because you're getting

you're getting onto some Trend um period there's no there's really no other other than by for doing things that are like when you're doing Arbitrage uh and you could argue that you're actually getting under the trend of the of the of the Spread itself But ultimately every every system makes money because it gets gets hold of some Strat it gets hold of some Trend um and you know the mean reverting strategy um is really getting is you're ultimately you're buying let's say in in the in the in the ultimate in the ultimate um one of

the perfect examples of a mean reverting strategy is you're in a trading range um and you're buying in the low end of that range and selling in the high end of that range but still You have a trend you though it's a short Trend um in between the the high and the low of the range what's different is the way you enter so I mean ultimately there's you you buy TR you buy Trends you sell Trends uh you you basically you do your you you do your trading range stuff um and I don't nothing really

in my mind has ever really changed about that I've always kind of been aware of that um so You in all actually happened is that i' I've got better we we keep on developing better and better methods to take advantage of these different kinds of regimes and different kinds of opportunities well you've you've achieved some fantastic returns over the years so what do you think is the main driver behind your success well I think that uh you know ultimately there's this this the the strategy of the strategy in the original Program obviously was a pretty

decent strategy um and I think the fact that we kept it that we kept it f kept it fresh unless as time went on but we kept it fresh by reoptimization uh was very important I think what's really important about this new program um is that we we actually are implementing you know walk forward analysis completely um in in development as well as in as in um maintenance and Trading so um the benefit of that is that it keeps the strategy fresh and it's it's very it's highly adaptive I think one of the thing one

one of the advantages that I've had all along um is that I've never I've never stopped trying to find ways to uh improve the strategy or make better strategies okay all right one final question before we start wrapping up for today so for all those Traders out there Who are Building Systems and um that are struggling with trading in real time what kind of advice would you give those people well if they're struggling with it in real time I guess they kind of need to figure out what they're what they're struggling with I mean are

they struggling with because they just have a lousy system or they don't know how to how to it's not being properly tested or is it got something to do with the fact that they simply are not comfortable With the way it trades um you have to kind of figure out where your problem really is um and ultimately I think a lot of I think most people really have a hard time with trading um it's in a lot of ways it it doesn't really make sense in terms of what people um like to do which is

kind of be right and and not have risk uh so I mean there's a whole temperament to do with what you need to need to have to be able to trade um and that's that's probably even more Important than um than your strategy or the way you test it but you have to figure out what your problem is um and if if if if if you have the right temperament if you can accept um what it what it really means to be involved with making you a living from Trading um and and the risk and

so forth um and and the fact that it there there'll be difficult times as well as there'll be great times if you if you can accept all that you're Comfortable with that and you can also have enough confidence in your system to know that you're not going to basically override it um for whatever reason um then you really have then you have to ask yourself if your system is not really making money um is the system just not that good um or is a system not really being properly tested you really have to know the

what the nature nature of the problem is yeah sure all right thanks a lot for that Bob that's some Great advice there so I'd just like to start wrapping up now with a few quick closing questions sure what's the biggest lesson you've learned through trading well you know in the end trading will really I I think trading will really reveal all of your weaknesses um and probably your strengths as well uh so ultimately I think that um you know I think it's it's forced me to continue to try to become a better person um I

suppose that's probably the biggest Lesson okay that may sound OD to you but that's really that's what it's done that's what it's done for me okay what about the best trading advice you've ever received uh I never really got to be honest with you I mean I to to some extent um you know I when when I first started this uh I would like like anybody somebody' say he' give me a hot tip about something and those all um None of those ever really worked out um so and I I don't really know I mean

truthfully I um most of the people that I know in this business are actually systematic Traders so um you know I think it's one of those just one of those things where you just just um you have to learn what it what it's like what what what what what it really takes to actually make a living in trading what what it take what it means to actually be a Trader um and then you Have to really understand that um and I think instead of looking for you know a hot tip because you know there have

been stories of people making buckets full of money based on you know the crash of 87 and so forth or you know it happens you know the the guys that made a lot of money um you know on the on the on the uh on the on the on the debt re refinancing debacle it happens um but I think ultimately what it comes down to with trading is you really want to be You want to be sure you can actually make a living at it you know month in month out year and year out um

so I think again it kind it kind of comes back to to thy own self be true um and then ultimately you have to make sure that you're not fooling yourself about your system you've actually learned you've actually come up with some approach to trading that'll actually serve you well um come hell or high water okay you've kind of already Touched on this a little bit already but what do you think is the most important ingredient to becoming a successful Trader uh that you are that you're able to be a Trader that you have the

proper character to be be a Trader because I I see a lot of people a lot of I've had a lot of clients who really just um not all but they're really risk averse and they really just don't have the temperament for trading all right thanks Bob so uh what About do you have any favorite trading books well you know um I I like my book I'm very fond of Jesse livermore's book um reminiscences of a stock operator um I like the market wizard books um if anybody really wants to be involved with systematic trading

I think they owe it to themselves to to look at Perry Perry's trading systems and methods Perry cowman's Trading systems and methods um and you know I I actually very fond of um I I like Elliot wave um And pattern recognition so there have been some of the some of the books by on Elliot wave that have been very very good and some of the stuff on chart patterns that have been very good but you know I'm I'm really all about the um validation of it uh because I think there's lots and lots of methods

that work y okay and if listeners wanted to get in touch with you what's the best way that they could do that well the probably the best way is my email bob. Partpart capital.com um that's that's probably the best way to do it uh because I I that's that's the most efficient way yep okay all right well thanks a lot for your time today Bob is there anything else that you'd like to mention before we wrap up no I think um you know I'm glad to have had a chance to talk about this with you

it's it's it's always interesting to me um and uh no I think that's I think you pretty much covered a Lot of the important important basis okay great well thanks a lot for uh spending time with us today it was really interesting to hear your insights into trading and strategy validation so I'm sure a lot of people are going to get a huge amount of value out of what you shared with us today so yeah thanks again and I wish you all the best my pleasure talk soon all right cheers so what did you think

of that good stuff huh a big thanks to Bob for Taking the time to share his knowledge and experience in strategy creation and optimization with us now Bob shared a lot of valuable information with us today so if you want a copy of the transcript you can download that from the show notes page at better system trader.com 60 now as I mentioned at the start we have a special bonus for this episode a couple of days after we recorded that interview I was chatting to Bob about System development a bit more and in addition to

running his own fund Bob does some Consulting and coaching work which uh includes stuff like strategy creation design optimization validation uh I think he also does some risk and portfolio management too so um through that work that he does oh by the way if you're interested in Bob for Consulting Services you absolutely should check him out I think he has one or two spots open At the moment so I'll put a link with some more information on the show notes page for those that are interested anyway as I was saying through that work that he

does he he gets to see the techniques and the results of the strategy creation process um of a lot of other Traders and trading firms and I was asking him about overfitting and and Bob shared some really valuable insights into the mistakes Traders make that can result in Overfitting um now Bob was kind enough to allow me to record that so I've I've created a very special bonus for this episode and it's an additional 10 minutes chatting with Bob about overfitting mistakes and some of the signs that you could be overfitting so if you're new

to system development or if you use optimization in your strategy creation process which I assume a large percentage of us do or perhaps you're not sure how to assess your optimization Results properly or I mean even if you're experienced but are always looking for uh techniques that could help to improve your existing process I think you'll find this bonus both interesting and valuable actually it's quite funny because I think I asked Bob for his top three and he shared quite a few more points than that so there are loads of great insights in in those

10 minutes and even if you just pick up one point From it that can really make a huge difference in the long run so if you want to get your hands on that bonus it's absolutely free you just have to go to the show notes page better system trader.com 60 and you'll see um there'll be a link or an image there or something like that it'll be obvious anyway just uh click on that enter your email address and we'll send that bonus to you and by the way while you're on the show notes page if

you enjoyed this Episode please share it we've got a a couple of links on that page that can use to share on social media so please spread the word and help other Traders with their strategy development too anyway that's it for this episode I hope you enjoyed the chat with Bob today and I look forward to catching you at the next one thanks for listening to better system Trader thanks for listening to the better system Trader podcast the next Step is to head over to bettertrades.com catch us next time for even more great ways to

improve your trading here on better system Trader [Music]