The efficient market hypothesis (EMH) continues to be embraced as the preeminent model of financial markets. The rapid growth in assets managed by strategies this theory underpins – namely, index tracking, risk premia and smart beta − reflects the consensus it has achieved among investment professionals.

Yet the EMH often breaks down as an effective model of real-world market behaviour. It fails, for example, to explain booms, manias and busts, or the long-term investment results achieved by firms such as Winton.

Clearly every model has its limitations – a model itself is only an idealised description of reality – but we find the widespread adoption of EMH-driven strategies alarming, especially given the role that this flawed intellectual framework has played in past financial crises.

Here, we examine an enticing alternative, inspired by ecology, and we find that it captures properties of real-world financial markets that are absent from models rooted in the EMH.

The efficient market hypothesis – a flawed concept

Despite only coming to prominence in the last two decades of the 20th century, the efficient market hypothesis was postulated more than 100 years ago by French mathematician Louis Bachelier [1], whose work compared the movement of financial market asset prices to the physical processes of Brownian motion and diffusion.

Essentially, the theory holds that the market is always right; it synthesises the wisdom of crowds so that it is never possible to earn steady trading profits by forecasting future price moves. Belief in the theory’s accuracy among investment professionals is, however, generally qualified by a recognition of the paradox that the market must reward some level of security analysis to achieve efficiency.

Efficient market theory makes several forecasts, some of which are borne out in practice, such as it is hard to earn speculative profits, and there is little or no opportunity for risk-free arbitrage profit.

But the theory makes other forecasts that are less consistent with empirical evidence: that return distributions are Gaussian and broadly stationary; that there is no autocorrelation spectrum and no cyclicality. The prices of common stocks have been shown to exhibit a much higher level of historical volatility than would have been justified by an apparently efficient market [2].

Perhaps a reason the theory has lasted so long is that its breakdown is manifested over long time horizons. That is not to say that it is a particularly good model over all short-term horizons, but it can produce reasonable predictions over these timeframes. It thus takes a long time for the model’s deficiencies to become apparent, and there is a tendency for its periodic failures to be excused as exceptional, or as “black swan” events.

From physics to ecology

The study of both price movements and market participants’ behaviour provides ample evidence of the efficient market theory’s over-idealisation [3]. Is there an alternative theory that can better capture reality?

For inspiration, we look away from Brownian motion and diffusion and towards ecology. In ecology, predator-prey models with recursive character were designed to capture the cyclical, crash-prone behaviour of populations such as the celebrated Canadian lynx-snowshoe hare system in North America (Figure 1). Such models produce chaotically fluctuating histories.

It isn’t too hard to imagine financial markets comprising various types of participant, whose populations evolve in response to one another and exhibit idiosyncratic behaviours. Such a genre of agent-based modelling has existed in financial economics for 30 years [6] [7].

The agents in these models do not determine their actions by performing costly or impractical optimisations, based on perfect knowledge of the state of the world. Rather, they use heuristic techniques shaped by experience. The most experimental and rapidly evolving group of organisms would probably be the hedge funds that Andrew Lo labels the “Galapagos Islands of finance” [8].

This type of agent-based model (ABM) would not tend to produce markets in equilibrium, nor time series of returns that are stationary or Gaussian. They would be likely to produce unpredictable behaviour, including apparent trends, sometimes in the form of “phase transitions” between alternative equilibria. These would be the general characteristics of market behaviour.

Working out the correct number, type and behaviour of agents to model a particular market usefully is a formidable challenge, especially when the “forcing” influence of external events is itself evolving. Even a single one of our Galapagos islanders might well be dauntingly complex in character. We can start to look at more generic properties of simple markets with a straightforward model, however.

The aim of this model will be to investigate whether a stylised market − comprising 50 agents interacting with each other according to simple rules − can produce a system with properties similar to real financial markets. We do not seek to match the sophistication of recent models in the academic literature [9] [10] [11]. Rather, we introduce features informed by Winton’s first-hand experience of markets, and emphasise the properties of the price series that emerge as a result. We are interested, for example, in the characteristics of price trends.

A toy model of a simple market

Our market consists of a single market-maker and 50 traders that can buy or sell a single asset. It evolves in discrete timesteps, which is a considerable simplification compared to the continuously updated order books used in many markets.

At each timestep, every trader has the chance to place an order with the market-maker, to buy or sell some number of units of the asset. Traders only interact with each other indirectly, via their orders with the market-maker. Orders influence the asset’s price and, in some scenarios, the behaviour of the market-maker − it is these changes that are “experienced” by the other traders.

The market-maker collects all the orders submitted by traders in a given timestep, and deals with them in a random sequence. For each order, the market-maker reports back to the corresponding trader the price at which the order has been filled. We do not consider incomplete fills and, because we do not maintain an order book, all orders are effectively market orders. This price is a function of the pre-order price and the size of the order: in other words, traders experience slippage in the execution of their order. They must also pay a commission to the market-maker.

In addition, the order has a permanent effect on the price, moving it higher in the case of a buy and lower in the case of a sell. The slippage and permanent effect on prices follow empirical models developed by Winton in its research into execution in a range of futures markets.

By the end of a timestep, the inventory of the market-maker (that is, the number of units of the asset it holds), the market price, and the inventory and capital of each of the traders have been updated. We record this information and move to the next step.

Our 50 traders are grouped into four categories:

• Commercial traders (10 suppliers and 10 end users): If we think of our single asset as a commodity, these are the suppliers and end users of that commodity. The suppliers will sell more of the asset if the price moves higher, while the end users will buy more if the price moves lower. For a futures market, these agents are analogous to traders aiming to lock in future costs or profits via hedging (intending to hold the contract to expiry) − that is, they are on one side of the physical delivery of the underlying asset.
  
  The price at which each of these traders will make half its maximum order evolves over time: for the results below, it follows an AR(1) process, in which the current value in the time series is based on the immediately preceding value.
  
  We choose this simple mean-reverting process to prevent the price diverging without limit, which we would regard as unrealistic. The definition of these traders means that there is a market-clearing price, at which supply and demand in the physical market would balance, and this also evolves with time.

The remaining agent types can be regarded as speculators of one kind or another:

• Value traders (10): This group attempt to infer the market-clearing price, which they manage with some degree of random error (and, perhaps, a deterministic bias). That is, at each timestep, the agent assumes the asset has a “true” value equal to the clearing price for the physical market plus a Gaussian random deviate − the distribution of which depends on each agent’s skill.
  
  If the market price is sufficiently below the (inferred) “true” value, the agent will go long; then, once the market price climbs sufficiently above the market-clearing price, they will close out their position, and vice versa for short trades.
• Technical traders (10): These take positions based solely on past prices. For the purposes of this paper, we consider only trend followers, which generate a signal using an exponentially-weighted moving-average crossover system. Position size depends on the strength of the signal and an estimate of the volatility via a function that scales the maximum possible position according to the trader’s current capital.
  
  
• Noise traders (10): The desired position of noise traders follows a random walk − here, an AR(1) process to prevent the position diverging − and they trade to achieve that position. They provide volatility and volume – and profits for the market-maker – but, in fact, are not necessary to generate any of the interesting behaviour we see in our model.

Even with a small number of agent types, our model still has many dials to turn and levers to pull: for example, the starting capital of the different traders; the level of skill of the value traders; the parameters of the price impact function (which, in a real market, would correspond to its volume and liquidity); the level of variation in supply and demand in the physical market; the speed of trend-following employed by each of the technical traders, and so on.

Previous work has tweaked model parameters to try to match a small number of so-called “stylised facts” about financial markets (see [12], for example, and references therein). Two are particularly clear when looking at market data:

1. The distribution of returns is fat-tailed, with large moves being more common than implied by a Gaussian distribution.
2. Volatility is clustered, with markets going through quiet and noisy periods. This can be related to the autocorrelation function of squared (or absolute) returns, which decays slowly.

Mandelbrot highlights the importance of long memory and heavy tails when challenging his readers of [13] to pick out the real market from a selection of return series. These include one with uncorrelated Gaussian returns, and one generated by his multifractal model. We present a similar challenge in Figure 2, and the Gaussian returns are easy to identify.

However, in many of these models, the autocorrelation function of the returns themselves (rather than the squared returns) is often taken to be indistinguishable from zero: in other words, there is no tendency for markets to trend (or, indeed, to mean-revert).

We do not take this position. On the contrary, we wish to understand the performance of trend following in financial markets, and so we are interested in examining the return distribution conditioned on the direction and strength of an earlier trend.

Results

Figure 3 displays the price history and open interest for a run of our ABM. To help reveal the detail of the market, we only show 30,000 steps of the model’s entire 100,000-step history. The parameters of the model are chosen to allow the technical and value traders to achieve a reasonable level of profit, which enables us to maintain a good mix of traders throughout the simulation without having to step in and to adjust parameters as it runs. We also find that a combination of technical and value trading is necessary to achieve a realistic-looking distribution of returns.

Note that we only introduce the technical traders after 10,000 timesteps: this allows trend followers (for all the trading speeds we consider) to estimate their trend signal and the market volatility. The result is a step up in the open interest at this point, from values of ≈200,000 lots, to the levels observed for the period shown in Figure 3.

Regarding the open interest, note that the market-maker is the counterparty to every contract. In early versions of our model, there could be long-term imbalances between supply and demand in the physical market, leaving the market-maker with very large holdings.

To avoid this, we introduced an asymmetry into the function used by the market-maker to determine the slippage and the permanent price impact of an order: if the market-maker has a net short position, it will impose a larger slippage and price impact for agents’ buy orders (which, in turn, would make the market-maker’s short position larger), and vice versa if the market-maker is net long. This makes the outstanding balance of contracts held by the speculators a mean-reverting process, with a mean of zero.

The notional wealth of the speculators can be calculated by adding their capital to the current market value of the contracts they hold – that is, not accounting for the slippage they would experience in closing out their positions. Figure 4 displays the change in notional wealth between timesteps 10,000 and 100,000 for our value and technical traders in the market shown in Figure 3.

If we consider each timestep to be one day, we can compute annualised Sharpe ratios for these agents, assuming a risk-free rate of zero. The value traders have Sharpe ratios between 1.61 and -0.73, while the technical traders achieve ratios between 2.61 and -0.49. These figures can be measured with an error of around ±0.06. This is only possible for a single market because of the very long experiments we can run: 85,000 daily prices after allowing some time for “burn in”, far more than is possible for a real individual market.

When we compare the performance of the value traders before and after the technical traders are switched on (at timestep 10,000) in Figure 5, we find that the introduction of the latter improves the profitability of the former.

Nonetheless, trend following is profitable at some speeds, and also makes the distribution of returns more like those seen in real markets. With value traders alone, the volatility is low but the distribution of returns is unrealistically heavy-tailed.

Four examples of return distributions can be seen in Figure 6. They are shown for the following markets:

• Coffee futures, for the same period as shown in Figure 2;

• The ABM from the bottom panel of Figure 2, for the period shown;

• The ABM displayed in Figures 3 and 4, starting at timestep 15,000 to allow time for the trend followers to “burn in”. The model is run for 100,000 steps in total.

• For comparison, the past 20 years of returns for 10-year Treasury note futures. These are less heavy-tailed than those for coffee.

Figure 6 shows how, by adjusting the mix of agents in the ABM, we can generate return distributions reminiscent of a wide range of markets. We cannot claim, however, that the differences between parameter choices correspond to differences between the real markets.

Discussion

As we have noted, an ABM with only value traders does not yield a market that looks “efficient”: we see a heavy-tailed return distribution, and price movements that only weakly track the clearing price in the physical market.

The inclusion of technical traders increases volatility, broadening the return distribution and making the fat tails appear less prominent. But they do not “arbitrage” away inefficiencies; instead, the slower trend followers enhance the returns of the faster trend followers and value traders, which appear to profit at the slow trend followers’ expense.

The process by which the fast trend followers profit from the slow trend followers also alters the distribution of returns experienced by the latter group: returns during times of positive, negative and weak trends are different; in ways that we observe in real markets.

That said, this result contradicts the observed decline in the performance of fast trend following, once transaction costs are accounted for (see [14]), despite a general increase in the capital controlled by technical traders.

At this point we also hit a limitation of our approach: it is not possible to probe what happens on shorter timescales than a single timestep, which we have treated as a trading “day”. To continue down to the millisecond timescales of high-frequency trading would need another seven or so orders of magnitude of dynamic range, which is computationally infeasible.

Thus it is entirely possible that our fastest trend followers would be “eaten” by agents operating even more quickly, leaving the slow systems relatively unaffected. We can only try to capture intraday effects by tweaking the market-maker’s price impact function.

The agents’ profitability is also limited by the unpredictability present in the simulation. This comes in two forms:

1. Much of the price movement is driven by random numbers. The level of supply and demand in the physical market evolves randomly, noise traders take random positions, value traders receive noisy information about the physical market and the market-maker processes trades in a random order.
2. Our model also exhibits chaotic behaviour. In Figure 7, we show what happens when we run the same simulation twice, with the same set of random numbers throughout, but with one small difference. In the second run of the simulation, after timestep 50,000, we round the market price to the nearest thousandth of a unit.

The price series start to diverge after the rounding. This happens slowly at first, but at some point, the difference is large enough for the agents’ interactions to take the price series in opposite directions. At long timescales, the same trends are present in either case, but at shorter timescales there is intrinsic unpredictability.

This should be no surprise: chaos can be seen in deterministic ecological systems of as few as three species, modelled by coupled, recursive equations. It is interesting to contemplate, though, how much of the unpredictability in financial markets comes from their being buffeted by random, external events, and how much comes from the interaction between the agents operating in the market, even those driven by cool, rational algorithms.

In reality, these algorithms are not static. This suggests an extension of our model: what happens when our agents can learn from their trading experience and alter their behaviour? At present, the mix of agents only changes as capital flows to the more successful strategies, making the more successful strategies more influential. Is this observation sufficient to explain the changes we have seen in the performance of different speeds of trend following [14]? Or is a more careful treatment of the structure of the market and its external influences required?

Conclusion

Agent-based models provide us with a flexible framework to explore ideas and capture some of the behaviour of real markets. However, we must be very cautious about extrapolating from what is still a highly simplified model to the behaviour of a real financial market. We do not envisage setting up an agent-based model of a particular market and using it to predict returns.

It is, instead, best to think of these ecologically-inspired models as a way to generate hypotheses and develop our intuition about the qualitative features of financial markets and their causes. These hypotheses can then be developed and tested on real market data.

The recent rise of products rooted in the efficient market hypothesis shows a lack of inspiration on the part of the investment management industry. We believe ecological models could be more fruitful hypothesis generators than a picture of a market as a simple physical mechanism, about which we already know all there is to know.

Download the PDF

This article contains simulated or hypothetical performance results that have certain inherent limitations. Unlike the results shown in an actual performance record, these results do not represent actual trading. Also, because these trades have not actually been executed, these results may have under- or over-compensated for the impact, if any, of certain market factors, such as lack of liquidity and cannot completely account for the impact of financial risk in actual trading. There are numerous other factors related to the markets in general or to the implementation of any specific trading program which cannot be fully accounted for in the preparation of hypothetical performance results and all of which can adversely affect actual trading results. Simulated or hypothetical trading programs in general are also subject to the fact that they are designed with the benefit of hindsight. No representation is being made that any investment will or is likely to achieve profits or losses similar to those being shown.