Automation on windows machine

[meldinman]

Hi all,
I'm curious if anyone else here is automating pysystemtrade on a windows machine. I am not familiar with UNIX so I would prefer to keep things windows based. I plan to run the various daily processes manually for now as I get comfortable with daily workings of a production system however I want to start thinking about how I would best implement an automated process.
As of now I am debating between a windows version of crontab, something like pycron (would presumably make it easier for me to use the already included linux crontab scripts) or writing my own with BAT files and a super light task scheduler like freebyte task scheduler (would mean less dependancy on third party software). Has anyone else had any experience or advice in this regard?
Thanks!

[ChrisAllisonMalta]

For what it’s worth I’m using WSL but it seems to be a lot more complicated than it needs to be. You need to do port redirection (for ssl) and also to make sure your firewall isn’t blocking the WSL access to IB Gateway at each restart and windows as you know tries to restart itself quite a lot Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: meldinman ***@***.***> Sent: Tuesday, April 12, 2022 1:25:26 PM To: robcarver17/pysystemtrade ***@***.***> Cc: Subscribed ***@***.***> Subject: [robcarver17/pysystemtrade] Automation on windows machine (Discussion #576) Hi all, I'm curious if anyone else here is automating pysystemtrade on a windows machine. I am not familiar with UNIX so I would prefer to keep things windows based. I plan to run the various daily processes manually for now as I get comfortable with daily workings of a production system however I want to start thinking about how I would best implement an automated process. As of now I am debating between a windows version of crontab, something like pycron (would presumably make it easier for me to use the already included linux crontab scripts) or writing my own with BAT files and a super light task scheduler like freebyte task scheduler (would mean less dependancy on third party software). Has anyone else had any experience or advice in this regard? Thanks! — Reply to this email directly, view it on GitHub<#576>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC3EL3KSJQWXQXG6BNNSLL3VEWWYNANCNFSM5TIAIVSQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[meldinman]

I think If would go down the linux route I would use a VPS as they are incredibly cheap these days. For now I am planning to keep things on my trading computer. I'd rather have the ability to review what's going on in pysystemtrade in VS code. Running on linux would add additional complexity that I am not yet ready for.

[dr1ver1]

I'm running a spread bet system, based on the Leveraged Trading book and heavily influenced by pysystemtrade. I have a .bat file, run by Task Scheduler that calls my own python scripts. These are written in the Windows environment.

I'm also migrating to full pysystemtrade, and I spent some time working with pysystemtrade initially on Windows 10 home, then WSL on Windows 10 Home. In the end I decided to bite the bullet and buy a separate box that I installed Ubuntu on.

• running WSL under Windows 10 Home means no graphics, so no charts. I don't know if that's the same case for WSL on Win 10/11 Pro
• there have been a few Windows-specific issues in pysystemtrade - that Rob has fixed as people have identified them, but that process isn't straightforward as he can't test in a Windows environment
• I've had to spend time (re)learning Unix, but that's no bad thing, as there's all kinds of other interesting Linux projects out there. There's a vast amount of tutorials to learn from.
  I didn't schedule any of the WSL python scripts.

I'm running VS Code on Ubuntu with all of the same add-ins as I have in Windows. I bought a cheap refurbished business machine off of ebay. I use free software called Barrier so that I have 1 mouse, 1 keyboard controlling both my windows machine and Linux machine.

If you start with Windows or Windows+WSL you can always migrate to Linux later.
The system is designed first with Linux in mind, so you'll have extra work to do on Windows.

[js190]

Hi @dr1ver1 i'm curious about your spreadbetting system and migration to pysystemtrade - are you going to modify pysystemtrade data sources to use spreadbet data? I think there is a project tackling this in one of the pysystemtrade forks.
What breadth of universe do you achieve?
I am running pysystemtrade with small account size on futures and the instrument limit due to capital is frustrating! Well, i'm not acutally runnign full pysystemtrade as i'm in semi-automatic mode, I'm using only the position generation parts of the system. Also I didn't want to use Arctic as my storage backend out of arbitary stubbornness on my part.

[dr1ver1]

Hi @js190
Yes.. my goal is to modify pysystemtrade so that it works for spread betting. The benefit of having created this system is that I will be able to run them in parallel and compare.
In the future, I'm thinking about some kind of combined futures/spread betting portfolio.

Limitations I have with Spread Betting compared to Futures are:

• I can't automate the trading (at least not with CMC Markets UK)
• Managing costs: with CMC they don't allow you to enter a limit trade inside the spread - so for instance I have to enter a Sell which is >= Buy price, and then sit there watching and modifying if it moves against me. So I can't capture slippage accurately, and also difficult to quantity what benefit I achieve by entering my trades as limit orders. Also, I understand that costs are higher with spread bets compared to futures.
• Yes, the universe is limited. For instance not all contract months are provided. So I can't have a Winter Crude trade - just the nearer months that they provide. Not all instruments in the pysystemtrade portfolio are there - but so far this hasn't been an issue for me. Currently I trade Gold, WTI, Bund, GBPUSD and S&P500

The benefits are that you can get more diversification with a small account, and it gives me time to better understand the different trading rules and asset classes.

Good luck with your work on the Arctic alternative.

[js190]

Cool, would you consider opensourcing the effort? A mixed portfolio of futures spreadbets would be interesting for me too so would likely ontribute the occasional PR. Out of interest why choose CMC over IG, given IG have an API, are the costs very different?

Getting a prod setup to work without Arctic is stable but is a massive hack, happy to push it publicly if any interest.
Cheers!

[bug-or-feature]

My fork is an attempt to adapt pysystemtrade for spread betting with IG. Happy to swap notes

[meldinman]

I've been running the various processes for the last few days manually from my environment PowerShell. The main drawback is running stack handler in the morning as usually by the time I get that running here on the American east coast, Asian markets are already closed. Additionally, I am not echoing log files to .txt files this way. I think I will go the batch file route, translating the included linux files to a suitable batch file equivalent.

[meldinman]

so I have choice of running the process python scripts directly through my python environments script or trying to mirror using @robcarver17 script that seem to be using an argfeeder in the included run.py
for example the simple batch file for run_capital_update process is
@echo off
"C:\Users\UserName\anaconda3\envs\sampleenv\python.exe" "...\sysproduction\interactive_diagnostics.py" >> "C:\Users\echos\interactive_diagnostics.txt"
pause
this allows me to schedule a batch file run and output logs to the echo location. Is there any benefit I am missing from not using the run.py argfeeder method?
The alternative is to run something like this in the same directory as run.py (I've made a windows script directory similar to Robs Linux one):
@echo off
"C:\Users\UserName\anaconda3\envs\sampleenv\python.exe" "...\sysproduction\windows\scripts\run.py" sysproduction.run_capital_update.run_capital_update >> "C:\Users\Mendel\echos\run_capital_update.txt"
pause
This way I'm executing run.py in my python environment with the added necessary argument to point to the process. I don't know if this is adequate and what ramifications I would have when running a few of these simultaneously. Fingers crossed!

[bug-or-feature]

There's a new section in the docs about creating executable commands for a non linux OS

https://github.com/robcarver17/pysystemtrade/blob/master/docs/production.md#scripts-under-other-non-linux-operating-systems

[meldinman]

so this would add the required process paths to python path on install?
I'm assuming from there one would still have to run a scheduled script to call those processes and specify echo locations?
Very helpful addition, thanks!

[bug-or-feature]

so this would add the required process paths to python path on install?

Not exactly. It creates a standalone executable file which imports, loads and runs the function defined. It creates this file in the same directory as python itself, so no path manipulation is needed. If you have ever used any python program or command line utility, like flake8, or black, or pytest - its a similar mechanism

I'm assuming from there one would still have to run a scheduled script to call those processes and specify echo locations?

yes