Maintaining trendiness under forecast_post_ceiling_cost_SR

[VMatthijs]

When adding a lot of instruments, some of them will clearly be too expensive for some of the trading rules. As discussed here, it is a good idea to conditionally include trading rules based on their expense, even in presence of the DO with shadow cost and buffering.

A neat way to do that while keeping the work (with all the instruments, with changing liquidity and costs) manageable is to use
forecast_post_ceiling_cost_SR in the config. However, I just realised that this can give rise to the following issue:
Most of the negatively skewed strategies (for example, carry, skew) tend to trade much more slowly than most trendy strategies. If you don't watch out, therefore, using forecast_post_ceiling_cost_SR can cause the relative weight of trendy rules for certain instruments to become quite low, which is probably not ideal from a risk-management point of view. Am I right that this may really be an issue?

A possible solution that I had been considering for a while is something along these lines: instead of a flat config like this

forecast_weights:
    accel16: 0.04433333333
    accel32: 0.04433333333
    accel64: 0.04433333333
    assettrend16: 0.02216666667
    assettrend2: 0.02216666667
    assettrend32: 0.02216666667
    assettrend4: 0.02216666667
    assettrend64: 0.02216666667
    assettrend8: 0.02216666667
    breakout10: 0.02216666667
    breakout160: 0.02216666667
    breakout20: 0.02216666667
    breakout320: 0.02216666667
    breakout40: 0.02216666667
    breakout80: 0.02216666667
    carry10: 0.01875
    carry125: 0.01875
    carry30: 0.01875
    carry60: 0.01875
    momentum16: 0.0266
    momentum32: 0.0266
    momentum4: 0.0266
    momentum64: 0.0266
    momentum8: 0.0266
    normmom16: 0.02216666667
    normmom2: 0.02216666667
    normmom32: 0.02216666667
    normmom4: 0.02216666667
    normmom64: 0.02216666667
    normmom8: 0.02216666667
    relcarry: 0.075
    relmomentum10: 0.00875
    relmomentum20: 0.00875
    relmomentum40: 0.00875
    relmomentum80: 0.00875
    skewabs180: 0.0375
    skewabs365: 0.0375
    skewrv180: 0.0375
    skewrv365: 0.0375

you allow for a config with hierarchical structure

forecast_weights:
  trendy:
    weight: 0.7
    accel:
      weight: 0.19
      accel16: 0.33
      accel32: 0.33
      accel64: 0.33
    assettrend:
      weight: 0.19
      assettrend16: 0.167
      assettrend2: 0.167
      assettrend32: 0.167
      assettrend4: 0.167
      assettrend64: 0.167
      assettrend8: 0.167
    breakout:
      weight: 0.19
      breakout10: 0.167
      breakout160: 0.167
      breakout20: 0.167
      breakout320: 0.167
      breakout40: 0.167
      breakout80: 0.167
    momentum:
      weight: 0.19
      momentum16: 0.2
      momentum32: 0.2
      momentum4: 0.2
      momentum64: 0.2
      momentum8: 0.2
    normmom:
      weight: 0.19
      normmom16: 0.167
      normmom2: 0.167
      normmom32: 0.167
      normmom4: 0.167
      normmom64: 0.167
      normmom8: 0.167
    relmomentum:
      weight: 0.05
      relmomentum10: 0.25
      relmomentum20: 0.25
      relmomentum40: 0.25
      relmomentum80: 0.25
  carry:
    weight: 0.15
    relcarry:
      weight: 0.5
      relcarry: 1
    plain_carry:
      weight: 0.5
      carry10: 0.25
      carry125: 0.25
      carry30: 0.25
      carry60: 0.25
  skew:
    weight: 0.15
    skewabs:
      weight: 0.5
      skewabs180: 0.5
      skewabs365: 0.5
    skewrv:
      weight: 0.5
      skewrv180: 0.5
      skewrv365: 0.5

Then, if a rule gets dropped by forecast_post_ceiling_cost_SR somewhere within the hierarchy, we maintain the overall characteristics of the trading strategy (such as the fact that 70% of the weight comes from trendy rules).

You could do something similar for instrument weights. That might make it easier to manage large configs with lots of instruments (with the instrument list changing over time as markets come and go) whose weights you want to handcraft according to some groupings.

I'm curious to hear what you think. How do you currently deal with this in your own systems? Are you not worried about expensive instruments (say, V2X) suddenly getting a forecast that is mostly based on carry and skew without any trend that could serve as a kind of stop-loss?

[robcarver17]

Yes this is a genuine issue, and truthfully one that hadn't occured to me. So this is a good example of the 'many eyeballs' at work. It's also more natural to create a config of the type you describe, rather than diving into excel and painstakingly calculating all the forecast weights.

[bug-or-feature]

Example new style configs in #1162