RuntimeWarning: invalid value encountered in log

[ChrisBlitz832]

When I'm running 1_optimize_cpcv.py, 1_optimize_kcv.py or 1_optimize_wf.py the optimising process gets always stuck after the first Trial (Trial 1) finished.

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/1_optimize_wf.py

10
TRAIN_START_DATE: 2022-02-02 04:40:00
VAL_END_DATE: 2022-04-29 23:55:00

Starting WF optimization with:
drl algorithm: ppo
name_test: model
gpu_id: 0

Launched hyperparameter optimization with K-Cross Validation

TIMEFRAME 5m
TRAIN SAMPLES 20000
TRIALS NO. 10
N 5
K slash 2
SPLITS 10

TRAIN SAMPLES 20000
VAL_SAMPLES 5000
TRAIN_START_DATE 2022-02-02 04:40:00
TRAIN_END_DATE 2022-04-12 15:15:00
VAL_START_DATE 2022-04-12 15:20:00
VAL_END_DATE 2022-04-29 23:55:00

TICKER LIST ['AAVEUSDT', 'AVAXUSDT', 'BTCUSDT', 'NEARUSDT', 'LINKUSDT', 'ETHUSDT', 'LTCUSDT', 'MATICUSDT', 'UNIUSDT', 'SOLUSDT']

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/.venv/lib/python3.9/site-packages/optuna/samplers/_tpe/sampler.py:263: ExperimentalWarning: multivariate option is an experimental feature. The interface can change in the future.
warnings.warn(
[I 2023-02-25 13:29:46,027] A new study created in memory with name: no-name-d7b91c2c-25dd-4592-9647-df312bfc1c45

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
0 4.00e+04 0.92 |
0 4.00e+04 0.92 | 0.92 0.0 19997 0 | -0.00 3.55 -0.25 -0.46
0 8.00e+04 0.94 |
0 8.00e+04 0.94 | 0.94 0.0 19997 0 | -0.00 4.23 -0.30 -0.47
| UsedTime: 434 | SavedDir: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k

No. Test Samples: 4999

Test Finished!
episode_return: -0.10315243935446239

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/function_finance_metrics.py:275: RuntimeWarning: invalid value encountered in log
return np.log(1 + pct_returns + 1e-8)
[I 2023-02-25 13:37:01,484] Trial 0 finished with value: 25.167498967428035 and parameters: {'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0, 'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}. Best is trial 0 with value: 25.167498967428035.

Found new best agent!

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
0 4.00e+04 0.96 |
0 4.00e+04 0.96 | 0.96 0.0 19997 0 | -0.00 7.21 -0.28 -0.50
0 8.00e+04 0.96 | 0.95 0.0 19997 0 | -0.00 7.62 -0.30 -0.42
| UsedTime: 158 | SavedDir: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k

No. Test Samples: 4999

Test Finished!
episode_return: -0.21765804701412517

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/function_finance_metrics.py:275: RuntimeWarning: invalid value encountered in log
return np.log(1 + pct_returns + 1e-8)
[I 2023-02-25 13:39:40,762] Trial 1 finished with value: 27.337235005057845 and parameters: {'learning_rate': 0.015, 'batch_size': 3080, 'gamma': 0.99, 'net_dimension': 512, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 60000.0, 'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}. Best is trial 1 with value: 27.337235005057845.

Found new best agent!

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID

[tty666]

yup I confirm my logs have only Trial 0 also :

################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_CPCV_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 19:34:51.382136

######### CPCV Settings #########
Paths : 4
N : 5
splits : 10

TIME START INNER: 2023-03-24 19:34:51.385687################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_CPCV_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:05:39.509931

######### CPCV Settings #########
Paths : 4
N : 5
splits : 10

TIME START INNER: 2023-03-24 20:05:39.513377################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_CPCV_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 262144, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:07:47.835229

######### CPCV Settings #########
Paths : 4
N : 5
splits : 10

TIME START INNER: 2023-03-24 20:07:47.837983################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_KCV_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:14:56.929284
TIME START INNER: 2023-03-24 20:14:56.929692K-Fold: 0################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_KCV_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 262144, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:15:23.115226
TIME START INNER: 2023-03-24 20:15:23.115455K-Fold: 0################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_WF_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:17:02.889066
################################## || ppo || ##################################
MODEL NAME: ppo
TRIAL NUMBER: 0
CWD: ./train_results/cwd_tests/model_WF_ppo_5m_50H_25k
{'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 262144, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0}
{'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}

TIME START OUTER: 2023-03-24 20:17:32.375707

[tty666]

But don't imagine a simple solution for that... well it's simple but it's more a question of BUDGET !
Yes in fact if you get only the first trial it's because it uses HIGH CPU + GPU usage and the process get killed ...
My guess it's something like chatgpt to run against very high capacity GPUs rack ...