http://www.nytimes.com/2007/10/09/science/09tier.html

This would be one of my concerns with some of the trend following systems, like the turtle system. From my understanding, they tend to have too great a reliance on rare positive events to generate their returns.

As Jay mentioned, taking care to avoid over-fitting is really important as well. Jay, how many variables are we talking about in your system? And how far back do you usually backtest?

First, you need to have tested your system against a wide variety of market conditions. These include high and low volatility markets, and trending (up or down) and flat. It is also beneficial if you’ve tested against catastophe scenarios (perhaps not as bad as 1987, but bad). If your system does well in all of these situations, then you’ve covered a pretty broad part of what the future might be like.

So I think there are two important considerations with respect to efficiency/rationality. First is time scale. I believe that prices have both rational and irrational components, and that the rational component generally operates on a longer time scale than the irrational component. This implies that a buy-and-hold investor is not negatively affected by “noise traders” or speculators, even if their behavior is irrational (not that I think it always is). In the long run, the noise/irrationality tends to cancel itself out.

The second factor is whether the actions of irrational actors in the market tend to cancel each other out. If they do, then perhaps then net effect really is zero, on all time scales. With respect to this, I would say that human psychology is much more prone to exhibit behavior patterns that reinforce, rather than cancel out. People look at what the people around them are doing, and they do the same thing; it feels safer. Monkey see, monkey do, right? So I think that we really do see short time scale irrationality because of collective perceptions that are not accurate, or collective behaviors that tend to propogate.

In my opinion, backtesting is where it’s at. You have to really get to know your system, and yet you also have to remain a bit distant. My approach to trading has been to try to become excellent at backtesting, balancing that “getting to know” and “remaining distant” paradox.

You have to keep some distance to minimize (I don’t think it is possible to entirely avoid) backfitting the data. My actual system, which, oddly enough, I call System, went through a pruning process the past few months. I started with some good ideas but not enough detail. By April of this year I came out of that beta phase with a solid system, well articulated, that traded reasonably well. But I knew I had far too many variables with far too much risk of backfitting. So version 2 of System finally came out around August and had about 70% fewer variables but with similar performance characteristics. It’s like you have to unfocus your eyes and see the trees not the individual leaves.

But you have to have an intimate knowledge of what to expect from your system and what you are going to do in response to the unexpected.

Regarding luck versus a real (e.g. profitable) system, generally the number of trials is considered crucial to ascertaining the validity of a sample (as well as the distribution of the results within the sample). That’s the huge advantage of trading quickly. For all of 2007, I’ve held 10 to 15 positions each day and am entirely in cash at night. You get a lot of samples quickly, and then it’s more a matter of waiting around for new market conditions and gathering yet more samples.

So if you had a method to tilt the odds of rolling a 6 on a die up to about 20%, and you had 10,000 samples to prove it, you’d start to feel confident, until you changed the surface on which the die was rolling, and then you’d have to collect more samples and see where you stood.

But, once again, I have found the most important aspect of all of this is being very good at backtesting. And, to be perfectly honest, it took me probably 3 years and change to get where I had any demonstrable skill at doing useful backtesting.

It spits out a flawless schedule D that can be imported into your tax software such as Turbo Tax or TaxCut. Last year, my schedule D printed out at about 40 pages and took about 20 minutes to prepare. One of those amazing bang for the buck applications.

I prefer the long term investing strategy of Buffet taking advantage of the market psychology that causes some stocks to be priced well beneath their true value. However, this does take considerable time to implement so now I’m finding myself attracted to the leveraged index funds such as QLD and SSO.

Jay, is there some tool you use to ease the tax preparation burden of technical trading?

As an engineer/computer scientist with an interest in finance, using my technical skills to try and develop strategies that outperform the market is as much of a hobby or a game than anything else. With a great way of keeping score…

However, I get the feeling that you and a couple others have in mind a specific notion of what “beat the market” means. And I’m guessing that correlates with what you feel is a worthwhile investment of time.

I’ve loosely followed your web site for quite a while (since Biblical Horizons was hornes.org/biblicalhorizons), and I have enjoyed the trading posts immensely. This last post drew me in enough to venture a few comments.

In response to some of Rusty’s comments: it may not be worthwhile to argue too much about the efficient market hypothesis, but I think there is good reason to believe that it does not largely hold for the financial markets. (I have a number of friends who are professional traders—I haven’t done a poll, but I think I can guess the direction they would go.) One of the foundations of economics is the assumption that people are rational actors; this is a great assumption as assumptions go, because you can make almost no formal conclusions without it. However, the truth is that people are _anything_ but rational, professionals and laymen alike. As an example, consider the current housing market. How many people claim to want to sell their house (i.e., they list it on the market) but refuse to sell for anything except the listed price, even though their house has been on the market for months? I’ve known people who flat out refused to sell their property until they could get what they paid for it; this is not rational behavior. The same behavior was seen widely as the dot-com boom started to bust.

Consider the financial markets in the last couple of months. Were the prices before the meltdown rational? What about after? I would argue that a significant component of each is “collective perception”: everybody seems confident, so I’m confident; once a few begin to panic, it spreads like wildfire. My own analysis is that most of the collective error was before the meltdown, but I have heard people talk more recently about fairly good credit investment products that nobody will touch, just because they’re scared of them now. I think that people collectively reinforce each other’s ideas of good or bad, risky or not risky, going up or going down, and that this leads to strong trends of collective irrationality (even though each individual thinks they’re being rational based on what the people around them are doing).

I recently read “Way of the Turtle”, written by one of the original Turtles, Curtis Faith. It’s a fascinating read, not only from the technical trading system point of view but also due to the historical narrative. It turns out that a lot of the original Turtles didn’t make money using the system, although some did very well. What was the reason? The ones who failed didn’t follow the system; they made “judgment calls” based on their instincts (read: emotions) and it destroyed their profitability. Some even claimed that the successful turtles had been given secret rules that the others hadn’t been taught.

This is really inappropriately long, so I’ll address a few other issues briefly. I think a lot can be said about how to invest when you think the market is significantly irrational (e.g., tulip mania), but that would take too long. I would only wholeheartedly agree that you should be careful shorting unless you believe that the market correction is imminent; good ideas and bad timing can easily lead to bankruptcy. Regarding why others like mutual funds aren’t doing this: there are a lot of factors that go into whether an idea works or not in a fund setting. One is drawdown; funds simply cannot tolerate the kinds of significant drawdowns that historically successful systems have. Investors leave, managers get fired, it just doesn’t work. Second, as far as mutual funds go, the long only restriction really limits what you can do. Third, economies of scale can go either way. A lot of trading is only successful with a lot of money (and sometimes a lot of leverage). On the hedge fund side, take quant funds as an example. The final factor is that the “respectable” position in many groups (e.g., academia, mainstream media, investment advisors) is to dismiss technical strategies. “Noise traders” is just a little derogatory, I think. But maybe that’s a good thing; you are right that if too many people attempt the same strategy, the available returns vanish rapidly.

Jay, I’d love to chat with you sometime about how you’ve developed your systems over the years. As I have the time, I’d like to start playing with some ideas of my own (stocks, futures maybe). My wife and I just moved to Dallas recently (attending Park Cities Pres.); send me an email if you’re interested in corresponding at all.