Merge branch 'develop' of https://github.com/robcarver17/pysystemtrade …

…into develop

.github/workflows/slow-test-develop.yml

@@ -1,4 +1,4 @@
-name: Slow test
+name: Slow test (develop)
 
 on:
   schedule:

.github/workflows/slow-test-master.yml

@@ -1,4 +1,4 @@
-name: Slow test
+name: Slow test (master)
 
 on:
   schedule:

docs/IB.md

@@ -297,7 +297,7 @@ You can use this directly if you are familiar with ib_insync eg `conn.ib.positio
 
 ### Make multiple connections
 
-It's possible to have multiple connections to the IB Gateway, each from it's own process, but each connection must have a unique clientid. Used clientid's are stored in a the active database (usually mongoDB) to ensure we don't re-use active clientids.
+It's possible to have multiple connections to the IB Gateway, each from its own process, but each connection must have a unique clientid. Used clientids are stored in the active database (usually mongoDB) to ensure we don't re-use active clientids. 
 
 
 

docs/backtesting.md

@@ -1718,7 +1718,7 @@ system.cache.get_cache_refs_for_instrument("EDOLLAR")
 ## if we change the config
 system.config.forecast_div_multiplier=100.0
 
-## ... then the result will be different without neeting to create a new system
+## ... then the result will be different without needing to create a new system
 system.combForecast.get_combined_forecast("EDOLLAR")
 ```
 

docs/instruments.md

@@ -348,9 +348,9 @@ Operating in the production environment is a bit more complex, due to the intera
 
 ## A note about configuration
 
-When you're running in simulation things are relatively simple; configuration items are defined in defaults_yaml, but can be overriden by your private_config.yaml, and then also by your own backtest.yaml file.
+When you're running in simulation things are relatively simple; configuration items are defined in defaults_yaml, but can be overridden by your private_config.yaml, and then also by your own backtest.yaml file.
 
-Importantly, once we're out of the 'backtesting'' part of a production system, we can't see the backtest configuration (which after all is system specific, whereas generally in the production environment we're working with global parameters). So the priority order is `defaults.yaml`, overriden by `private_config.yaml`. The downstream code that produces strategy orders once the production backtest has generated optimal positions, and then trades those orders, will operate only on the configuration in `private_config.yaml` and `defaults.yaml`. 
+Importantly, once we're out of the 'backtesting'' part of a production system, we can't see the backtest configuration (which after all is system specific, whereas generally in the production environment we're working with global parameters). So the priority order is `defaults.yaml`, overridden by `private_config.yaml`. The downstream code that produces strategy orders once the production backtest has generated optimal positions, and then trades those orders, will operate only on the configuration in `private_config.yaml` and `defaults.yaml`. 
 
 ## Reduce only and other constraints in static systems
 

docs/production.md

@@ -1719,7 +1719,7 @@ Linux script:
 
 Called by: `run_systems`
 
-The code to run each strategies backtest is defined in the configuration parameter in the control_config.yaml file (or overriden in the private_control_config.yaml file): `process_configuration_methods/run_systems/strategy_name/`. For example:
+The code to run each strategy's backtest is defined in the configuration parameter in the control_config.yaml file (or overridden in the private_control_config.yaml file): `process_configuration_methods/run_systems/strategy_name/`. For example:
 
 ```
 process_configuration_methods:
@@ -1763,7 +1763,7 @@ Linux script:
 Called by: `run_strategy_order_generator`
 
 
-The code to run each strategy's backtest is defined in the configuration parameter in the control_config.yaml file (or overriden in the private_control_config.yaml file): `process_configuration_methods/run_systems/strategy_name/`. For example:
+The code to run each strategy's backtest is defined in the configuration parameter in the control_config.yaml file (or overridden in the private_control_config.yaml file): `process_configuration_methods/run_systems/strategy_name/`. For example:
 
 
 ```
@@ -2596,7 +2596,7 @@ Useful things to note about the crontab:
 
 #### Process configuration
 
-Process configuration is governed by the following config parameters (in [/syscontrol/control_config.yaml](/syscontrol/control_config.yaml), or these will be overriden by /private/private_control_config.yaml):
+Process configuration is governed by the following config parameters (in [/syscontrol/control_config.yaml](/syscontrol/control_config.yaml), or these will be overridden by /private/private_control_config.yaml):
 
 -  `process_configuration_start_time`: when the process starts (default 00:01)
 - `process_configuration_stop_time`: when the process ends, regardless of any method configuration (default 23:50)
@@ -2825,7 +2825,7 @@ The following are configuration options that are not in defaults.yaml and *may*
 - `email_server`: this is the outgoing server
 
 
-The following are configuration options that are in defaults.yaml and can be overriden in private_config.yaml:
+The following are configuration options that are in defaults.yaml and can be overridden in private_config.yaml:
 
 [Backup paths](#data-backup)
 - `backtest_store_directory` parent directory, backtests are stored under strategy_name subdirectory
@@ -2864,7 +2864,7 @@ Outside of the backtest code, in production configuration options are pulled in
 
 ### Control config files
 
-As discussed above, these are used purely for control and monitoring purposes in [/syscontrol/control_config.yaml](/syscontrol/control_config.yaml), overriden by /private/private_control_config.yaml).
+As discussed above, these are used purely for control and monitoring purposes in [/syscontrol/control_config.yaml](/syscontrol/control_config.yaml), overridden by /private/private_control_config.yaml).
 
 ### Broker and data source specific configuration files
 
@@ -2942,7 +2942,7 @@ You can also change other values in the interactive tool, but be careful and mak
 
 ## Strategies
 
-Each strategy is defined in the config parameter `strategy_list`, found either in the defaults.yaml file or overriden in private yaml configuration. The following shows the parameters for an example strategy, named (appropriately enough) `example`.
+Each strategy is defined in the config parameter `strategy_list`, found either in the defaults.yaml file or overridden in private yaml configuration. The following shows the parameters for an example strategy, named (appropriately enough) `example`.
 
 ```
 strategy_list:
@@ -2956,7 +2956,7 @@ strategy_list:
 
 ### Strategy capital
 
-Strategy capital is allocated from [total capital](#capital). This is done by the scripted function, [update strategy capital](#allocate-capital-to-strategies). It is controlled by the configuration element below (in the defaults.yaml file, or overriden in private_config.yaml).
+Strategy capital is allocated from [total capital](#capital). This is done by the scripted function, [update strategy capital](#allocate-capital-to-strategies). It is controlled by the configuration element below (in the defaults.yaml file, or overridden in private_config.yaml).
 
 ```
 strategy_capital_allocation:
@@ -2970,7 +2970,7 @@ The allocation calls the function specified, with any other parameters passed as
 
 #### Risk target
 
-The actual risk a strategy will take depends on both it's capital and it's risk target. The risk target is set in the configuration option, `percentage_vol_target`, in the backtest configuration .yaml file for the relevant strategy (if not supplied, the defaults.yaml value is used; this is *not* overriden by private_config.yaml). Risk targets can be different across strategies.
+The actual risk a strategy will take depends on both it's capital and it's risk target. The risk target is set in the configuration option, `percentage_vol_target`, in the backtest configuration .yaml file for the relevant strategy (if not supplied, the defaults.yaml value is used; this is *not* overridden by private_config.yaml). Risk targets can be different across strategies.
 
 #### Changing risk targets and/or capital
 
@@ -3693,7 +3693,7 @@ V2X                            [20201118, 20201216] 2020-10-15 09:43:30  (1, -1)
 
 The strategy report is bespoke to a strategy; it will load the last backtest file generated and report diagnostics from it. On a daily basis it runs for all strategies. On an ad hoc basis, it can be run for all or a single strategy.
 
-The strategy reporting is determined by the parameter `strategy_list/strategy_name/reporting_code/function` in default.yaml or overriden in the private config .yaml file. The 'classic' reporting function is `sysproduction.strategy_code.report_system_classic.report_system_classic`
+The strategy reporting is determined by the parameter `strategy_list/strategy_name/reporting_code/function` in default.yaml or overridden in the private config .yaml file. The 'classic' reporting function is `sysproduction.strategy_code.report_system_classic.report_system_classic`
 
 Here is an example, with annotations added in quotes (""):
 

docs/production_strategy_changes.md

@@ -188,7 +188,7 @@ Under ignore_instruments in the .yaml configuration, I suggest you include only
 
 ### Set shadow cost
 
-The shadow cost is a key variable which is set in the private_config.yaml file (*not* the backtest configuration file, since it is used 'outside' the backtest in the strategy order generation). The default value is 10, but you may want to initially begin with a very large value (eg 500) and gradually reduce it over the first few days. This will produce a more gradual adjustment from old to new strategy positions, although bear in mind that any strategy position with the wrong sign will immediately be closed regardless of the shadow_cost value unless you set this instrument to don't trade.
+The shadow cost is a key variable which is set in the private_config.yaml file (*not* the backtest configuration file, since it is used 'outside' the backtest in the strategy order generation). The default value is 50, but you may want to initially begin with a very large value (eg 500) and gradually reduce it over the first few days. This will produce a more gradual adjustment from old to new strategy positions, although bear in mind that any strategy position with the wrong sign will immediately be closed regardless of the shadow_cost value unless you set this instrument to don't trade.
 
 
 ### Strategy backtest output of optimal positions

setup.py

@@ -98,7 +98,7 @@ def dir_this_file():
         "statsmodels==0.14.0",
         "PyPDF2>=2.5.0",
         "pyarrow>=14.0.1",
-        "scikit-learn>1.3.0"
+        "scikit-learn>1.3.0",
     ],
     tests_require=["nose", "flake8"],
     extras_require=dict(),

sysbrokers/IB/ib_futures_contract_price_data.py

@@ -94,7 +94,7 @@ def has_merged_price_data_for_contract(
         """
         Does IB have data for a given contract?
 
-        Overriden because we will have a problem matching expiry dates to nominal yyyymm dates
+        Overridden because we will have a problem matching expiry dates to nominal yyyymm dates
         :param contract_object:
         :return: bool
         """

syscontrol/run_process.py

@@ -2,7 +2,7 @@
 General class for 'running' processes
 
 We kick them all off in the crontab at a specific time (midnight is easiest), but their subsequent behaviour will
- depend on various rules, as defined in ... attribute of defaults.yaml or overriden in private_config
+ depend on various rules, as defined in ... attribute of defaults.yaml or overridden in private_config
 
 - is my process marked as NO OPEN in process control  (check database)
 - is it too early for me to run? (defined in .yaml)

syscore/pandas/pdutils.py

@@ -294,7 +294,7 @@ def make_df_from_list_of_named_tuple(
     field_name_for_index: str = arg_not_supplied,
 ):
     """
-    Turn a list of named tuplies into a dataframe
+    Turn a list of named tuples into a dataframe
     The first element in the tuple will become the index
 
     >>> T = namedtuple('T', 'name value_a value_b')

sysdata/arctic/arctic_futures_per_contract_prices.py

@@ -72,7 +72,7 @@ def _write_merged_prices_for_contract_object_no_checking(
     ):
         """
         Write prices
-        CHECK prices are overriden on second write
+        CHECK prices are overridden on second write
 
         :param futures_contract_object: futuresContract
         :param futures_price_data: futuresContractPriceData

sysdata/base_data.py

@@ -51,7 +51,7 @@ def __getitem__(self, keyname):
         """
 
         raise Exception(
-            "__getitem__ not defined for baseData class: use a class where it has been overriden"
+            "__getitem__ not defined for baseData class: use a class where it has been overridden"
         )
 
     def keys(self):
@@ -66,7 +66,7 @@ def keys(self):
         """
 
         raise Exception(
-            "keys() not defined for baseData class: use a class where it has been overriden"
+            "keys() not defined for baseData class: use a class where it has been overridden"
         )
 
 

sysdata/config/configdata.py

@@ -74,7 +74,7 @@ def __init__(
 
         """
 
-        # this will normally be overriden by the base system
+        # this will normally be overridden by the base system
         self.log = get_logger(
             "config", {TYPE_LOG_LABEL: "config", STAGE_LOG_LABEL: "config"}
         )

sysdata/csv/csv_futures_contract_prices.py

@@ -114,7 +114,7 @@ def _write_merged_prices_for_contract_object_no_checking(
     ):
         """
         Write prices
-        CHECK prices are overriden on second write
+        CHECK prices are overridden on second write
 
         :param futures_contract_object: futuresContract
         :param futures_price_data: futuresContractPriceData

sysdata/data_blob.py

@@ -237,10 +237,6 @@ def _get_specific_logger(self, class_object):
 
         return log
 
-    def _resolve_names_and_add(self, resolved_instance, new_name: str):
-        attr_name = self._get_new_name(class_name)
-        self._add_new_class_with_new_name(resolved_instance, attr_name)
-
     def _get_new_name(self, class_name: str, use_prefix: str = arg_not_supplied) -> str:
         split_up_name = camel_case_split(class_name)
         attr_name = identifying_name(

sysdata/mongodb/mongo_historic_orders.py

@@ -61,7 +61,7 @@ def __repr__(self):
         return "%s (%s)" % (self._name, str(self.mongo_data))
 
     def add_order_to_data(self, order: Order, ignore_duplication: bool = False):
-        # Duplicates will be overriden, so be careful
+        # Duplicates will be overridden, so be careful
         order_id = order.order_id
         no_existing_order = self.get_order_with_orderid(order_id) is missing_order
         if no_existing_order:
@@ -76,7 +76,7 @@ def add_order_to_data(self, order: Order, ignore_duplication: bool = False):
                 )
 
     def _add_order_to_data_no_checking(self, order: Order):
-        # Duplicates will be overriden, so be careful
+        # Duplicates will be overridden, so be careful
         mongo_record = order.as_dict()
 
         self.mongo_data.add_data(order.order_id, mongo_record, allow_overwrite=True)

sysdata/parquet/parquet_futures_per_contract_prices.py

@@ -67,7 +67,7 @@ def _write_merged_prices_for_contract_object_no_checking(
     ):
         """
         Write prices
-        CHECK prices are overriden on second write
+        CHECK prices are overridden on second write
 
         :param futures_contract_object: futuresContract
         :param futures_price_data: futuresContractPriceData

sysdata/production/capital.py

@@ -309,7 +309,7 @@ def get_total_capital(self) -> pd.Series:
     def get_current_accumulated_pandl(self) -> float:
         return self.capital_data.get_current_pandl_account()
 
-    def get_profit_and_loss_account(self) -> pd.Series():
+    def get_profit_and_loss_account(self) -> pd.Series:
         return self.capital_data.get_profit_and_loss_account_pd_series()
 
     def get_maximum_account(self) -> pd.Series:

sysdata/sim/sim_data.py

@@ -174,7 +174,7 @@ def get_raw_price(self, instrument_code: str) -> pd.Series:
         """
         Default method to get instrument price at 'natural' frequency
 
-        Will usually be overriden when inherited with specific data source
+        Will usually be overridden when inherited with specific data source
 
         :param instrument_code: instrument to get prices for
         :type instrument_code: str
@@ -194,7 +194,7 @@ def get_raw_price_from_start_date(
         """
         Default method to get instrument price at 'natural' frequency
 
-        Will usually be overriden when inherited with specific data source
+        Will usually be overridden when inherited with specific data source
 
         :param instrument_code: instrument to get prices for
         :type instrument_code: str

sysexecution/algos/allocate_algo_to_order.py

@@ -92,7 +92,7 @@ def check_and_if_required_allocate_algo_to_single_contract_order(
     # not used yet, but maybe in the future
     is_roll_order = instrument_order.roll_order
 
-    if algo_allocation_is_overriden_for_instrument(
+    if algo_allocation_is_overridden_for_instrument(
         contract_order=contract_order, config=config
     ):
         contract_order = allocate_algo_for_specific_instrument_with_override(
@@ -143,7 +143,7 @@ def already_has_algo_allocated(contract_order: contractOrder) -> bool:
     return contract_order.algo_to_use != ""
 
 
-def algo_allocation_is_overriden_for_instrument(
+def algo_allocation_is_overridden_for_instrument(
     contract_order: contractOrder, config: AlgoConfig
 ) -> bool:
     instrument_code = contract_order.instrument_code

sysexecution/algos/common_functions.py

@@ -37,7 +37,7 @@ def cancel_order(
     data_broker = dataBroker(data)
     data_broker.cancel_order_given_control_object(broker_order_with_controls)
 
-    # Wait for cancel. It's vitual we do this since if a fill comes in before we finish it will screw
+    # Wait for cancel. It's vital we do this since if a fill comes in before we finish it will screw
     #   everything up...
     timer = quickTimer(seconds=CANCEL_WAIT_TIME)
     not_cancelled = True

sysexecution/order_stacks/broker_order_stack.py

@@ -31,7 +31,7 @@ def find_order_with_broker_tempid(self, broker_tempid: str):
         return missing_order
 
     def get_order_with_id_from_stack(self, order_id: int) -> brokerOrder:
-        # probably will be overriden in data implementation
+        # probably will be overridden in data implementation
         # only here so the appropriate type is shown as being returned
 
         order = self.stack.get(order_id, missing_order)

sysexecution/order_stacks/contract_order_stack.py

@@ -24,7 +24,9 @@ def add_controlling_algo_ref(self, order_id: int, control_algo_ref: str):
 
         existing_order = self.get_order_with_id_from_stack(order_id)
         if existing_order is missing_order:
-            error_msg = "Can't add controlling ago as order %d doesn't exist" % order_id
+            error_msg = (
+                "Can't add controlling algo as order %d doesn't exist" % order_id
+            )
             self.log.warning(error_msg)
             raise missingOrder(error_msg)
 
@@ -48,7 +50,9 @@ def add_controlling_algo_ref(self, order_id: int, control_algo_ref: str):
     def release_order_from_algo_control(self, order_id: int):
         existing_order = self.get_order_with_id_from_stack(order_id)
         if existing_order is missing_order:
-            error_msg = "Can't add controlling ago as order %d doesn't exist" % order_id
+            error_msg = (
+                "Can't add controlling algo as order %d doesn't exist" % order_id
+            )
             self.log.warning(error_msg)
             raise missingOrder(error_msg)
 
@@ -74,7 +78,7 @@ def release_order_from_algo_control(self, order_id: int):
             raise Exception(error_msg)
 
     def get_order_with_id_from_stack(self, order_id: int) -> contractOrder:
-        # probably will be overriden in data implementation
+        # probably will be overridden in data implementation
         # only here so the appropriate type is shown as being returned
 
         order = self.stack.get(order_id, missing_order)