Signal data

The current strategies generate signals from daily prices for futures (ContractType column). Handling stocks and ETFs is already supported but will be the topic of another article.

The system stitches histories for individual futures contracts into back-adjusted series for signal calculations and backtests.

CSI and Norgate together cover more markets than either provides on its own. Both have provided excellent support and corrected data after I reported inconsistencies. That is a good reason to build your own quality checks, even when you are paying someone else to do theirs.

The Provider column selects the extractor used to load an instrument’s daily data. One provider may have several extractors, such as Norgate and Norgate-Delisted. The latter runs only when rebuilding the history from scratch.

Delisted markets have no new daily prices, so their histories can sit out the daily download.

The ProviderSymbol column records the instrument’s symbol at the selected provider. When historical data is unavailable, it contains NA and the missing coverage is documented in Comment. That makes the gap explicit rather than leaving an empty cell for the reader, or the code, to interpret. StartDate records the date the system has historic data for an instrument.

Bid/ask data

Bid and ask quotes allow the system to estimate spread costs and measure slippage by comparing execution prices with the quotes prevailing immediately before order placement.

While delayed bid/ask data at relatively coarse intervals is usually available free of charge from the broker, real-time feeds generally require a paid subscription.

To keep costs down, I estimate spread costs using free broker data wherever possible. To measure slippage in live trading, I use real-time feeds from exchanges with affordable subscriptions. For exchanges with more expensive real-time feeds, such as CME and ICE, I use Databento instead.

Databento provides historical trade and quote data. I use the recorded order-placement timestamp to retrieve the bid and ask quotes immediately before submission, then compare them with the execution price.

The IntradayProvider column specifies the provider used to measure slippage, while IntradayIdentifier identifies the instrument at that provider.

For exchanges that do not have a configured provider, spread costs can only be estimated using delayed data.

If you are starting out, you do not need a dedicated quote-data feed. An initial spread estimate can also be obtained from observing quotes in your broker’s trading platform. With a handful of markets, that is manageable. Across a large universe, you have quietly acquired a second job.

Piecing together the past

The “backfill” step uses rows from the spreadsheet to supply the price history for another instrument. They remain useful for backtesting, but do not represent additional instruments to trade.

Take COCOA: the CSI history used here starts in June 1975, while the stored Norgate history begins in December 1979. Combining them adds about four and a half years without requiring ongoing downloads from both providers.

My CSI subscription allows 150 markets to be updated at a time, so each slot is valuable. The older CSI prices are downloaded once, archived under COCOA_backfill, and read by the CSI-Delisted extractor. That frees the CSI slot while Norgate supplies new data for COCOA.

The same approach works across contract sizes. The history of SP500_micro, for example, is extended with data from the larger E‑mini contract, SP500. That history is extended in turn with the original full-size contract, SP500ORG. The chain reaches back to 1982.

Sometimes the larger contract represents the entire history: TSE60_mini uses the standard contract’s data because the providers do not cover the smaller version. These cases are marked “uses full size data” in the Comment column and are only possible if small-size variants of the contract are always liquid enough.

Where the smaller contract is sufficiently liquid, using the larger contract’s data can also save a provider slot, as with CORN-W_micro. Figure 3 shows how much history this recovers in three examples.

“Wait a minute,” you might say. “The S&P 500 Micro did not exist in 1982. How can it appear in a backtest that goes back that far?”

For the years before the Micro existed, the backtest uses the older S&P 500 contract’s price history and calculates positions using the Micro’s smaller contract size. This reconstructs how a smaller position would have behaved over that period.

The full-size, E‑mini and Micro contracts therefore have very similar histories. My system groups them when allocating risk, so adding another contract size does not automatically give the S&P 500 more weight in the portfolio. Smaller contracts still serve a purpose: they allow positions to be adjusted in smaller steps.

I will return to the details of risk allocation in a separate article on portfolio construction.

Recovering the history of instruments is only half the job. The contracts need a clear handover, and their prices need to speak the same language.

Figure 4 shows three ways of combining price histories.

For COCOA, the backfill InactiveContract is the December 1981 contract taken from CSI. Later contracts come from Norgate. The handover follows contract expiry rather than a single calendar-date cutoff across all price series.

Price conventions also need to agree. CSI quotes WHEAT_micro in dollars, while the broker quotes it in cents. Therefore, its prices are multiplied by 100 before they are combined with the older wheat contracts.

Other migrations need a fixed offset. After the market closed on April 26, 2024, Montréal Exchange converted the outstanding BAX contracts expiring after June 2024 into three-month CORRA futures¹. In the workbook, these are CAD3M and CORRA3. The conversion spread was 32.138 basis points. Because these futures are quoted as 100 minus the interest rate, putting the old BAX prices on the CORRA basis requires adding 0.32138 price points. The backfill applies that adjustment to the predecessor contracts before joining the histories.

If you are now tempted to abandon futures trading altogether, I sympathize. You can lose months untangling this mess before you have tested a single trading idea. With equities, at least a good data provider takes the burden of splits and other corporate actions off your shoulders. Norgate does this well.

And just when the histories seem complete, currencies add another complication. The backtest needs exchange-rate histories to express non-dollar positions and profits in US dollars. Some price histories, like CAC, use euros even for years before the euro existed.

To cover those years, I had to reconstruct a euro proxy from the historical currency basket behind the European Currency Unit (ECU).

Gone, but still in the backtest

An instrument can leave today’s trading universe and still be part of the backtest. Building a backtest strictly from today’s survivors is problematic, as is easy to see in equities: omit the companies that went bankrupt, and some expensive mistakes disappear from the backtest. That is called survivorship bias, and keeping delisted instruments helps address it. The security master must therefore record both what can be traded today and what was available to the strategy in each earlier period.

Retired markets also add data where it is most welcome: in the early years, when far fewer instruments existed. Keeping them broadens the set of markets available in those years and lets a strategy face more market conditions. Related markets still share risks, so every extra instrument is not an independent time series, but the broader sample makes the backtest more representative.

Futures markets can disappear for different reasons. Hedging needs change, liquidity dries up, or a replacement contract attracts more business. The commodity itself need not have become worthless. Take PORKBELLIES, which had their moment of Hollywood fame in Trading Places (1983)². CME delisted frozen pork belly futures in July 2011³ after a prolonged lack of trading volume. Bacon had not gone out of fashion, but trading that particular futures contract had.

Uncle Fred therefore stamps the file “Retired” and puts it back in the cabinet. The trades it generated, including the bad ones, still belong in the backtest. Fred may enjoy a tidy desk, but that is no excuse to put the losers through the shredder.

A more recent example comes from my broker. Interactive Brokers will stop trading its London Metal Exchange lookalikes in November 2026. These are over-the-counter contracts that reference LME futures⁴, giving access to metals such as aluminum, copper and tin. The underlying LME markets continue to exist, but this route to trading them is closing. Their histories remain useful even after they leave the live universe. Here, “retired” describes their status in the system. It does not mean the exchange has delisted the underlying market.

The InactiveContract column records where the retained contract history ends. It contains a contract identifier, rather than a calendar date. For COPPER-LME, for example, 1763CAM-2026M identifies the June 2026 contract. Later contracts are excluded when building the history, and an instrument with this field filled in is omitted from the live configuration. That June cutoff is separate from IB’s November withdrawal. The final simulated trade also depends on the roll schedule and the data it needs.

The same column serves two purposes. A backfill entry supplies older contracts to another instrument, as in the cocoa example above. A retired instrument keeps its own historical identity and can remain in the backtest.

Figure 5 shows the 55 retired instruments with stored price histories. The retirement cabinet holds rather more than a few curiosities.

Figure 6 counts the histories available in each year. Together, these retired instruments contribute ten histories in 1980 and 27 in 1990, reaching a peak of 39. The retirement cabinet is doing useful work long before many of today’s contracts enter the picture.

Keeping a delisted instrument does not automatically make it eligible every day in the historic backtest: The instrument still needs usable data and enough liquidity during the period being tested. The aim is to include markets that could realistically have been traded then, with the information available then.

One market, several contracts

Uncle Fred has barely put his stamp down when the Australian dollar returns to his desk wearing a different exchange badge.

“New ticker” it announces. “More diversification”. Fred looks over his glasses. “I have three stamps that say ‘Rejected’. That doesn’t make them three independent instruments.”

The same exposure can come in several contract sizes or trade on competing exchanges. The workbook excludes AUD-ICE in favor of AUD, for example, and ESTR-CME in favor of ESTR. In both cases, the Comment column documents a cheaper, more liquid alternative. Fred points to it with the expression of a man who suspects you have come all this way without reading the form.

Each candidate still deserves a proper examination. Whether two contracts count as duplicates is a case-by-case decision. I treat COCOA-LDN and COCOA as distinct markets: London cocoa is priced in sterling for delivery in Europe, while New York cocoa is priced in US dollars for delivery in the US⁵. Currency movements and local supply-and-demand conditions can therefore drive their prices apart. Even Fred accepts that “both involve chocolate” is an inadequate basis for a rejection.

For the backtest, another highly correlated instrument may add no diversification at all. Photocopying the same witness statement does not give you ten witnesses. Counting each version as a separate market can also give the underlying exposure more weight simply because an exchange offers another way to package it. Ten wrappers around the same market do not make a diversified portfolio.

The additional contract also needs a useful job. If the preferred contract provides the desired exposure more cheaply and with better liquidity, its rival may never reach the portfolio. Fred compares costs for equivalent exposure. A lower commission per contract is less impressive when you need ten of them. The Blacklist flag records the exclusion, and Comment explains the decision, saving the rejected applicant from returning next year with a fresh haircut and the same spread.

Then AUD_micro arrives. Fred checks the contract size: A$10,000, against AUD’s A$100,000. “You appear to have lost a zero.” In this case, that is the point. A smaller contract allows finer position sizing when positions must be rounded to whole contracts, especially in a smaller account. The S&P 500 variants discussed earlier serve the same purpose. Fred puts down the rejection stamp. “Approved. But your larger relative has already claimed the diversification allowance.”

Correlation weighting prevents retained variants from each receiving the allocation of an independent market. Portfolio construction then decides how to express the exposure. High correlation alone therefore does not settle the application: the “highly correlated” step in Figure 2 removes alternatives that offer no compelling execution advantage, while the final “unique markets” bar groups the retained size variants.

Figure 7 shows the verdict for one example: keep the standard and micro contracts at CME, and exclude the ICE alternative. Three applications, two approvals, one AUD/USD market. Fred has earned a biscuit. He breaks it in half and briefly considers recording two.

Broker access

Fred has approved the paperwork. Unfortunately, the broker has its own rejection stamp, and unlike Fred, it does not stop working at five.

An instrument can have a useful price history and still be unavailable to trade. The DoNotTrade column lets me keep that distinction in the system. With the flag set to 1, an instrument will still generate a trading signal and receive a theoretical position in the optimal portfolio, where fractional contracts are allowed. During portfolio construction, however, its position limit is set to zero. It may contribute information, but the resulting portfolio holds no contracts in it.

This assumes the instrument is not blacklisted. With Blacklist set to 1, its price history is not loaded and no trading signal is generated. The system still checks its bid/ask spreads if DoNotTrade is 0. Figure 8 compares these paths through the system.

I typically refrain from having too many instruments in the universe that generate signals but are not tradable because several experiments have shown that when an instrument generates a signal but is not tradable, the final portfolio does not improve.

Some tradability restrictions arise from regulation. The workbook flags some US sector futures such as US-TECH because of restrictions associated with section 871(m) of the US tax code with regard to non-US investors⁶.

Other obstacles are more mundane. The broker may not support the product for the account, or may not make it available for margin trading. Restrictions can also be temporary. The broker may temporarily block trades that open or increase a position. The market can remain open, with prices arriving normally, while my route into it is blocked. DoNotTrade is a blunt control: it imposes a zero position limit rather than representing every variation of those rules. The Comment column records the reason, and a temporary restriction needs revisiting when the broker changes its policy.

Therefore, broker access is an ongoing maintenance job. Figure 9 shows the current process: the system automatically records whether the broker allows each instrument to be traded, but I update DoNotTrade manually in the security master. Once enough tradability history has accumulated, I plan to feed it automatically into the backtest, with the option of manual overrides.

Anyone on the other side?

Fred has the broker’s approval. He calls out, “Next, please.” Nothing happens. He tries again. For the first time in his career, the empty waiting room is a problem.

Markets with virtually no activity usually go on my Blacklist. Thinly traded markets can stay in the dataset because liquidity changes. The eligibility is reassessed each day within the system with its own flag IsTradable.

This daily liquidity check has two parts. Average daily volume over the previous 20 trading days must exceed 50 contracts. Risk volume must also exceed USD 100,000. This measure multiplies average contract volume by the contract’s value in US dollars and its estimated annualised volatility. The result puts differently sized contracts on a common risk scale.

Figure 10 illustrates these checks using MSCIEU, an MSCI Europe equity-index future. It remains eligible throughout 2021, becomes intermittent as activity weakens, spends much of the second half of 2024 outside the tradable universe, and qualifies for most of 2025. In 2026, it moves in and out again as the two liquidity measures cross their thresholds.

IsTradable also checks minimum volatility and estimated holding costs. Those tests pass throughout this example. To avoid constantly switching eligibility on and off, a passing result is carried forward for up to 30 trading days. The flag can therefore remain 1 after liquidity falls below a threshold. An eligible instrument can thus be included in the live portfolio.

The market can qualify again when activity returns. At the end of the chart, both liquidity measures have slipped below their thresholds, but the flag remains 1 because the last passing observation is still inside the 30-day grace period. Fred approves of the daily review. An application that can be reopened every morning is practically a pension plan.

Cost configuration

Trading involves two main variable costs: broker commissions and bid–ask spreads.

These costs arise whenever a position is opened or closed, including when it is rolled from one contract expiry to the next. Depending on the rolling method, this may involve closing the existing position and opening a new one, or trading the calendar spread directly.

The timing and method of each roll can materially affect the trading system’s total costs. Automating this process properly is a topic in its own right.

In brief, PriceRollCycle defines the contracts used for each instrument, while HoldRollCycle specifies the expiries that the system may hold. RollOffsetDays sets how many trading days before expiry a roll may occur, while CarryOffset determines whether the system calculates an instrument’s carry using the next or the previous contract.

The workbook also specifies a SpreadCost for each instrument. Expressed in the contract currency, this value represents half of the instrument’s typical bid–ask spread. The system uses it as an initial estimate when reliable historical spread data are not yet available. Once enough observations and actual fills have been collected to estimate the spread cost, the field is no longer used.

You may have noticed that the workbook does not specify broker fees for each instrument. My system retrieves the current commission from the broker automatically and maintains a history of each instrument’s fees.

Figure 11 applies the daily cost filter to five live instruments and the blacklisted AEX Mini. For each instrument, it adds the one-way commission to the estimated half-spread, divides that cost by the current annualised risk of one contract, and then multiplies the result by the number of scheduled rolls per year. This common scale reveals what dollar fees can hide: the Nasdaq Nano has a lower dollar fee than its larger relatives, but a much higher cost relative to the risk it carries. Similarly, monthly rolls can push an apparently modest trading cost above the screening threshold.

As Figure 11 shows, the AEX Mini exceeds the 10% ceiling and is therefore marked “too expensive” in the workbook. Passing this cost screen does not by itself make an instrument tradable: the liquidity and volatility checks apply independently, and each strategy also decides which instruments it can trade based partly on its turnover.

Commissions change over time, and realistic estimates are essential for a meaningful backtest. Because the price histories of some instruments extend back to the 1970s, the cost history must include at least reasonable proxies for the broker fees charged at the time. In the 1970s, commissions could be as high as USD 30 per transaction!

I will cover how these settings work together, how to automate rolls efficiently and how to estimate costs for the earlier part of the backtest in a separate article.

Contract quirks

Fred likes the phrase “standardised contracts”. It sounds as though someone has already done the difficult work. He opens the security master expecting a quiet afternoon and finds a petting zoo.

The first animal is GAS-US-WEEKLY, which expires every Friday⁷. Fred’s filing system identifies a futures contract only by year and month. This works beautifully until a month contains five Fridays. “Which Friday?” asks Fred. The spreadsheet maintains a dignified silence (see Figure 12). This instrument is blacklisted because the required history is unavailable, but the problem of weekly expiries would remain even if the data became available.

Then comes European natural gas along: It changes its size without asking permission. The contract sizes of Dutch TTF GAS-W-NL⁸ and UK NBP GAS-W-UK⁹ depend on the number of days in the delivery month, so a March contract is larger than a February contract. TTF also counts the delivery hours, where daylight-saving time occasionally adds or removes an hour.

My solution is to hold only December expiries (hence the suffix ‑W for Winter). December reliably supplies 31 days, giving fixed configured multipliers of 744 MWh for TTF and 31,000 therms for NBP. It is less a triumph of financial engineering than a strategic alliance with the calendar.

Fred is just beginning to admire this solution when ICE announces a minimum trading size of five lots for both products. Ordering one gas contract is not on the menu: The smallest serving feeds a family of five. Fred marks both rows DoNotTrade, stamps the application closed and congratulates himself on another job well done.

Trading on exchanges around the world brings plenty of local peculiarities. Australia is a particularly generous contributor. Until June 2025, ASX cash equities opened in five groups ordered by security code¹⁰. Trading started in alphabetical batches: A–B first, C–F next, and everyone from S–Z waiting at the back. ASX now uses a single opening auction, but its bond futures still keep Fred employed.

The Australian bond futures have a fixed face value of AUD 100,000, yet their prices are quoted as 100 minus the annual yield. Consequently, the dollar value of the same yield move varies with the level of interest rates¹¹. The exact conversion is nonlinear.

My system uses a fixed average multiplier instead. This will not win a prize in numerical analysis, but the variation is small enough for the job and Comment records the approximation for the next person who wonders why the multiplier looks suspiciously hand-crafted.

Hong Kong supplied another lesson when I first traded equities there. At noon, the regular morning session ended. I checked the logs, the internet connection, and, briefly, my sanity. Nothing was broken: the afternoon session begins at 1 p.m. Even an electronic market can put up a sign saying, “Back after lunch.”

Finally, expiry schedules can change during the lifetime of an instrument. For MILLWHEAT, Comment records a cycle change and says that the usable sequence begins with the March 2015 contract, 2015H. The current cycle comprises March, May, September and December¹². Unfortunately, a comment is documentation, not an executable instruction. Writing “start here” in a spreadsheet does not make the loader start there, just as Fred’s note saying “please work” does not repair the office printer.

These oddities point to the same engineering rule: a futures identifier is part of the data model, not just a label. If an exception changes prices, position size, trading access or contract selection, the rule belongs in code and tests. Comment should explain the exception, not be the only place where it exists.

Fred adds “standardised” to his list of words that should require supporting evidence.

Collect, compare, reconcile

The process starts with three automated steps: retrieving the futures universe from Interactive Brokers, collecting CSI’s product catalog, and obtaining Norgate’s futures symbols. Each source provides a different part of the picture. The broker lists products that may be available to trade. The providers list the histories they cover.

Next comes matching. Automated helpers compare the broker catalog with the symbols already recorded in the security master. They flag new products to configure and existing symbols that no longer appear in the broker’s catalog. Resolving these gaps is still a manual process. The description, exchange, currency, contract size, and settlement terms help establish whether a broker product matches a provider’s history. Similar names are useful clues, but they are not proof.

From candidates to decisions

Once the sources are reconciled, the review proceeds through several filters and proposal lists. The first pass removes products outside the intended scope: single-stock and spot-quoted futures, along with markets the system cannot access. The remaining candidates need usable histories and a way to estimate trading costs.

The next pass revisits earlier exclusions. Has a previously illiquid contract attracted volume? Has a short history become long enough? Is bid/ask data now available? Have measured spreads fallen below the cost estimate that kept the instrument out? The notebook identifies these candidates using provider statistics, stored histories, and collected spreads. Where evidence is missing, the work list may point back to the exchange for a manual liquidity check.

Existing instruments need attention too. Missing broker symbols and sustained zero volume can signal deteriorating access or liquidity. Other reports compare highly correlated contracts and suggest sharing histories across sizes. These proposals need review: units and settlement terms can differ. Using a larger contract’s history also omits the smaller contract’s own volume and open-interest data, which can affect roll decisions.

The review ends with a decision for each candidate: add it, revise its configuration, retain it only for historical research, or leave it excluded. The workbook records the decision and its reason, so next year’s review can question an assumption rather than reconstruct it.

The reports identify candidates — I make the changes. Fred’s REVIEWED stamp is only useful if next year’s reviewer can tell what he checked.

Rebuild, check, repeat

After the workbook changes, the data pipeline reloads the definitions, updates the contract histories, applies migrations and backfills, creates roll calendars, and assembles the back-adjusted series. This is where a plausible spreadsheet row has to produce usable data.

Automated data quality checks support each step, as outlined below. A failure sends the relevant definition or source data back for investigation.

The updated master then feeds into strategy recalibration. New markets and longer histories can change signal magnitudes and correlations. The system re-estimates forecast scaling and diversification adjustments, then reruns the portfolio backtest after costs and produces a recalibration report.

The annual review revisits the whole universe. Daily automated checks and monitoring may prompt an ad hoc review. A new product or a broker restriction (as in the LME example above) can therefore restart the relevant part of the process. The aim is to make each update smaller and easier to explain, rather than save a year’s surprises for one heroic spreadsheet session.

Cross-sectional Momentum

Let’s define a trading strategy around this economic intuition:

This continuous trading signal has to be converted to actual trading positions and each of the instruments can be traded separately. Let’s not bog down in further details like only trading instruments that are cheap enough (commission or spread wise) and trade just all of them.

The more instruments you trade, the more diversification you get. Figure 2 shows my current universe of tradeable instruments over time. The historic data of the universe starts in June 1975. Back then there were only a few classic futures like soybeans or US bonds.

Over time the futures market got more and more diverse, new instruments entered the market, some of them successful others have been discontinued (like the famous pork bellies). In aggregate, more and more instruments have been tradeable over time.

Now let’s plug the cables directly in the wall and trade all of them available at each point in time. To do that, we actually need to decide which lookback period s to use for our cross-sectional momentum strategy.

Which lookback period should we choose? Twenty days? One hundred days? Are there time frames that do not work at all? This turns out to be surprisingly difficult.

The Parameter Surface

Without looking at any backtests, it is clear by the definition of the signal that shorter time frames will trade more often because \(\mathsf{R_t}\) will be more volatile due to a shorter relevant history. It will therefore vary more from day to day and may switch between long and short more often.

This will incur higher trading costs and most probably impact performance. To get a better feeling for the signal, let’s run backtests for a range of different lookback periods s.

If parameter diversification is going to work, we first need a strategy whose performance is reasonably stable across parameters. Figure 3 shows the Sharpe ratios for a range of lookbacks between 20 and 350.

Now that is quite something! It is a trading signal that works pretty well: The signal has a Sharpe ratio of about 0.5 or higher for a parameter range from about 30 to 280. That is actually a very important property of any worthwhile strategy. If the parameter surface is very noisy, you either have not enough instruments in your universe, the signal is not defined in a stable way or it is not good in general.

Consider one particular strategy of the whole flock: Lookback period s=215. This is the best strategy with a Sharpe ratio of 0.93. Its equity curve and drawdown is shown in figure 4.

There are several important observations of this graph. The performance seems to deteriorate after about the year 2010. Bad drawdowns are about ‑40% and especially pronounced in duration after 2010.

Let’s have a look at some performance statistics of this parameter from table 1:

Especially the large drawdown duration of the strategy is of some concern. A whopping 1837 days, that’s over 5 years! The small positive skewness of the strategy is an interesting property. On the negative side, the return stream of this lookback period has fatter tails on the downside than the upside.

Selecting Parameters

Should you trade this particular parameter, because it is the best? This is precisely the point where inexperienced quants get it wrong. The lookback of s=215 is the best lookback period in a finite sample of historical data. If you do this optimization over a subset of instruments or only parts of your history, it will not be the best any more.

Nevertheless there seems to be something there. The performance increases steadily from a lookback of around 30 up to the maximum of 215 and then decreases again. Cross-sectional momentum seems to play out on a bigger range of time frames.

A sensible approach is to trade several parameters of this strategy. But which ones? We need a better framework to make a decision.

If you remember the introduction, we may also kill several birds with one stone. As the return streams of different parameters are not perfectly correlated, we get some diversification and can trade a range of cross-sectional momentum without committing us to just one time frame.

We can answer this by computing a correlation matrix of the return streams of the different parameters. Figure 5 shows just that. You may notice that especially larger values of the lookback period s are higher correlated than lower values.

The explanation for this lies in the design of the trading signal. The signal for the lookback period s=345 shares 345 (98%) data points with the signal for s=350. Therefore all the outperformances of these neighboring parameters will be quite similar. Some shorter lookbacks like s=20 on the other hand share only 15 (75%) of their data with their neighbor s=15. The exponential weightning has another decay for each of the parameters but will not transform shared data in something totally different.

Because of this it is not advisable to trade all of the parameters. Doing that results in overweighting the larger parameters.

One approach is to trade parameters that have a maximum correlation between them. Which threshold you choose is not that important. Each has the property that you will trade less of the higher correlated large parameters. I will use a maximum correlation of 95% between neighboring parameters and want to capture strategies with a Sharpe ratio of 0.6 or higher.

The relevant parameter range comes down to lookbacks between 40 and 270. Starting with 270 and then searching for the next parameter with a correlation of 95% or less and repeating this procedure you end up with the parameter set of 270, 230, 195, 165, 135, 110, 90, 75, 60 and 45.

As you are reading an article from Systematic Trading—Made in Germany, I will step into my country’s tradition of over-engineering and also provide you with the algorithm to select parameters to trade from a given range.

params <- c(lower, ... , upper)
result <- c(upper)
param <- upper
while (param >= lower) {
 if (cor(param, result[1]) < 95%) {
  selected <- c(param, result)
 }
 next param
} 

The equity curve and drawdown of the particular set of parameters is shown in figure 6 and table 2 shows some statistics of the strategy.

If you compare table 2 with the statistics of the best strategy of table 1, you will find that the strategy with a mix of lookback periods is worse than the best strategy: The Sharpe ratio drops from 0.93 to 0.83 and the maximum drawdown increases to 50%. The skew and tail properties are about the same, only the maximum drawdown duration decreased by over a year.

So where is the benefit of diversification?

Above comparison was made to show you how dangerous it is to compare anything against some data mined best strategy. The correct comparison is to compare the average Sharpe ratio of our parameter range (0.72) to the parameter mix within this range (0.83).

The correlations of the selected parameters are shown in figure 7. The Sharpe ratio increase of 0.1 corresponds to an improvement of 15%. As can be seen from figure 1, such an improvement can be expected from a combination of return streams that are correlated by 44%. All in all, not bad!

Of course you may argue that such comparisons are not very helpful, because the strategies also have different expected returns, which also play a role in the Sharpe ratio. Diversification in practice is a much more messy concept than in theory.

Nevertheless, the central idea survives contact with reality. Diversifying across carefully selected parameterizations does not create miracles, but it reduces dependence on any single choice.

Instead of betting everything on one lookback period, we allow several reasonable variants to work together. In systematic trading, that is often the closest thing to a free lunch that actually exists.

Day 1: Opening the position

Let’s walk through an example: You bought one DAX mini at a price of 23,000 and €1 is worth $1.15. Let’s further assume the DAX did not move at all until the end of trading on day 1.

Your account balances look like this after nightly settlement:

Table 1 introduces two new terms: “Net liquidation value” (NLV) and “Available funds”. NLV is simply the value of your account if you closed all positions now, non-base cash positions included. As neither the DAX mini nor the EUR/USD rate has moved since the position was opened, the NLV is still $100,000. “Available funds” is the amount you can employ to open further positions and cover their margin requirements: the NLV minus all maintenance margin requirements, with a requirement in a non-base currency valued at the current FX rate to your base currency before it is subtracted. Your margin requirements on day 1 are easily satisfied.

Day 2: DAX move creates EUR cash

Moving on to day 2: The DAX makes a big move up from 23,000 to 24,000. You gained 1,000 index points on the futures position! That translates to €5,000, because the DAX mini contract has a multiplier of 5. Let’s further suppose that the exchange rate did not change at all. The profit of €5,000 will be attributed to your euro cash position at settlement. This is called variation margin and means that all daily profit and loss fluctuations are directly credited or debited to your cash position in the currency of the respective contract: Futures don’t accumulate P&L—they settle it. Your account now looks like the following:

Someone downstairs has quietly moved €5,000 onto your EUR pile, and both your NLV and your available funds now show today’s profit. The NLV is now the sum of your USD cash and your EUR cash, valued in USD at the end-of-day exchange rate; subtracting the margin requirements gives the available funds. In the example, the EUR margin of €9,000 equals $10,350, leaving $95,400.

So far, the mechanics are straightforward. However, an important practical aspect remains: interest. In reality, you pay daily interest on your negative cash positions and receive interest on your positive positions, but I am getting ahead of myself. Optimizing these payments will be part of the interest strategy later on.

Day 3: FX moves

To finish the example, let’s have a look at day 3. On day 3, the DAX does not move at all, but the exchange rate between EUR and USD falls to 1.1. Essentially your euros are now worth less than the day before. You decide to close the position at the end of the trading day. What happens to your account balances?

Well, your cash position in USD is still the same, because you closed out a contract denominated in EUR. Nothing changes here, except that the margin is no longer required and therefore drops to zero. The DAX did not move, so it did not have any price impact. The FX market, however, had other plans: Yesterday, your EUR cash position was worth $5,750, but today, at an exchange rate of 1.1, it is only worth $5,500. Downstairs, nobody cares about the DAX anymore—but the FX desk shaved $250 off your account.

It is worth noting that for futures, FX risk materializes through cash positions created by daily settlement, not through the contract notional. If you are thinking ”Wait, shouldn’t the FX loss be much larger?” you are not alone. You bought the DAX at 23,000 × 5 EUR at an exchange rate of 1.15 USD, which equals $132,250. Today, it is worth only 24,000 × 5 EUR at an exchange rate of 1.1, which equals $132,000. This intuition is misleading. The notional comparison suggests a $250 loss, but that is not how futures accounting works. You already realized the DAX gain through daily settlement. What remains exposed to FX is the accumulated €5,000 cash balance, now worth $5,500. Because futures are marked to market daily, only the accumulated cash balance is exposed to FX movements. In futures, FX risk lives in your cash—not in your position. In the real world, however, a big move in the exchange rate would of course be at least partially reflected in the price of the DAX, because international investors would probably adjust their positions.

Margin Alert Strategy

To avoid any margin calls, we can now formulate the margin alert strategy.

while (margin > 0.95) {
  if (n > 0)
    SELL k <= n ETF shares
  if (n == 0)
    CLOSE future with highest PnL
} until margin < 0.8

Interest Rate Strategy

Next up is the strategy to optimize our interest income. This strategy can be formulated as follows:

addedMargin = (200 * P) / NLV
currentETFMargin = n * P / NLV
if (margin + addedMargin  < 0.8)
  if (currentETFMargin < 0.5)
    BUY 200 ETF shares

With the margin alert strategy, your account should be much less likely to run into a margin deficiency or forced liquidation. It does not eliminate the risk entirely, but it gives the system a disciplined way to respond before the broker does. The interest strategy is designed to improve the return on otherwise idle cash.