Support for spot instruments

[opsenar]

Hello Rob, Hello community,

Thank you very much for your books, efforts, and your project.

I was wondering if it makes sense to add support for the spot instruments in the framework. For example, we could think of US, UK, EU equity single names, ETFs, and FX spots to be tradable in the framework.

My motivation is the following:

1. I have tested that Rob's systematic approach works well in many markets, not just futures.
2. We can get exposure to the factors that are either unavailable via futures or too capital-demanding.
3. Some (but not all) spot instruments are getting cheaper for trading. Even ETFs.
4. IB supports trading in most of those markets. BarChart has the data.
5. In the end, the portfolio could have spot as well as futures positions.

The documentation shows well-defined extension points on how to add new market data. I guess that other things would also need to be extended accordingly - execution, margins, reporting..

But maybe I am wrong, and the whole idea is just idiotic, and too many things would need to be changed in the codebase :) So, I would like your (first) opinion about this thought. Does it sound reasonable/meaningful?

I appreciate your thoughts.

Best regards,
Pavel

[robcarver17]

Spot instruments actually work in a backtest (i.e. you don't need to provide a multiple prices data) but to implement in production would require a substantial amount of work which I personally don't have time to do.

[opsenar]

Can I add the spot instruments by myself and submit pull requests to you? I have been using TWS api for a long time, and planning to test the new functionality by backtesting and trading my own PST strats live.

[robcarver17]

I assume you mean PRs adding support for spot in the production code. I mean yes, if you're prepared to do this, but given the massive complexity of what is involved I would like to see a detailed plan of exactly what you are planning to do rather than just a series of hacks. Bearing in mind that everything in pst assumes you are trading futures, this is not as trivial as you might think. It would take me about 6 months I would estimate.

[opsenar]

Thank you for your thoughts and the estimate. I've got time to contribute.

I also prefer to avoid any hacks and unnecessary breaking changes. If it makes sense, we could split the scope into bigger epics and then iteratively progress with them one at a time. E.g., we briefly discuss the new epic in this forum, discuss the changes in the code, then I code/test the new functionality and submit a pull request. Please let me know if it works for you.

As for epics, I see the following larger pieces now:

1. The new data classes for spot instruments as specified in the documentation. This includes behaviours for downloading, storing, backtesting, and production.

2. Costs. Previously I have been using PST for my strats, but I adjusted my code to add some costs specific to spot markets (margin rates, stock borrowing fees, and operating expense ratio (for ETFs)). Those were the game-changers in some markets.

3. Review/extend the functionality in the broker-specific execution classes (sysexecution, sysbrokers). Price data, the journey of orders, exchanges, routing, algos, and fills.

4. Review/extend the production flow: production flow code, and corresponding scripts.

5. Review/extend the dashboard, monitor, and diagnostic use-cases

6. Review remaining classes to see if they need to be touched. This is a "catch-all" piece.

7. Documentation.

The list is quite generic at this stage and is supposed to evolve as we move forward.

It also makes sense not to expose the added functionality to the users before it is ready and tested end-to-end. E.g., we could expose the spot backtesting after p.1 and p.2 are done, and we see that the results are correct.

[bug-or-feature]

hey @opsenar - before starting, make sure you understand how deeply ingrained futures are in the production code. Do that by looking at the code itself, not the docs. The docs kind of imply that the project could be extended and adapted for other data sources, brokers, instrument types. But that is not the case at all; the docs are a bit misleading in this respect.

[opsenar]

Hey @bug-or-feature .. I looked at the code, not just docs.