Merge pull request freqtrade#1957 from freqtrade/new_release

New release - 2019.6

.pyup.yml

@@ -11,7 +11,10 @@ update: all
 # allowed: True, False
 pin: True
 
-schedule: "every day"
+# update schedule
+# default: empty
+# allowed: "every day", "every week", ..
+schedule: "every week"
 
 
 search: False
@@ -22,7 +25,7 @@ requirements:
   - requirements.txt
   - requirements-dev.txt
   - requirements-plot.txt
-  - requirements-pi.txt
+  - requirements-common.txt
 
 
 # configure the branch prefix the bot is using

CONTRIBUTING.md

@@ -11,7 +11,7 @@ Few pointers for contributions:
 - Create your PR against the `develop` branch, not `master`.
 - New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100).
 
-If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE)
+If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg)
 or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR.
 
 ## Getting started

Dockerfile

@@ -16,7 +16,7 @@ RUN cd /tmp && /tmp/install_ta-lib.sh && rm -r /tmp/*ta-lib*
 ENV LD_LIBRARY_PATH /usr/local/lib
 
 # Install dependencies
-COPY requirements.txt /freqtrade/
+COPY requirements.txt requirements-common.txt /freqtrade/
 RUN pip install numpy --no-cache-dir \
   && pip install -r requirements.txt --no-cache-dir
 

Dockerfile.pi

@@ -27,9 +27,9 @@ RUN wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryco
  && rm Berryconda3-2.0.0-Linux-armv7l.sh
 
 # Install dependencies
-COPY requirements-pi.txt /freqtrade/
+COPY requirements-common.txt /freqtrade/
 RUN ~/berryconda3/bin/conda install -y numpy pandas scipy \
- && ~/berryconda3/bin/pip install -r requirements-pi.txt --no-cache-dir
+ && ~/berryconda3/bin/pip install -r requirements-common.txt --no-cache-dir
 
 # Install and execute
 COPY . /freqtrade/

Dockerfile.technical

@@ -3,4 +3,4 @@ FROM freqtradeorg/freqtrade:develop
 RUN apt-get update \
     && apt-get -y install git \
     && apt-get clean \
-    && pip install git+https://github.com/berlinguyinca/technical
+    && pip install git+https://github.com/freqtrade/technical

README.md

@@ -129,7 +129,6 @@ The project is currently setup in two main branches:
 - `master` - This branch contains the latest stable release. The bot 'should' be stable on this branch, and is generally well tested.
 - `feat/*` - These are feature branches, which are being worked on heavily. Please don't use these unless you want to test a specific feature.
 
-
 ## A note on Binance
 
 For Binance, please add `"BNB/<STAKE>"` to your blacklist to avoid issues.
@@ -142,7 +141,7 @@ Accounts having BNB accounts use this to pay for fees - if your first trade happ
 For any questions not covered by the documentation or for further
 information about the bot, we encourage you to join our slack channel.
 
-- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE).
+- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg).
 
 ### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue)
 
@@ -173,7 +172,7 @@ to understand the requirements before sending your pull-requests.
 Coding is not a neccessity to contribute - maybe start with improving our documentation?
 Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
 
-**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
+**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
 
 **Important:** Always create your PR against the `develop` branch, not `master`.
 

bin/freqtrade

@@ -1,7 +1,13 @@
 #!/usr/bin/env python3
 
 import sys
+import warnings
 
 from freqtrade.main import main, set_loggers
+
 set_loggers()
+
+warnings.warn(
+    "Deprecated - To continue to run the bot like this, please run `pip install -e .` again.",
+    DeprecationWarning)
 main(sys.argv[1:])

config_binance.json.example

@@ -11,8 +11,8 @@
         "sell": 30
     },
     "bid_strategy": {
-        "ask_last_balance": 0.0,
         "use_order_book": false,
+        "ask_last_balance": 0.0,
         "order_book_top": 1,
         "check_depth_of_market": {
             "enabled": false,

config_full.json.example

@@ -22,8 +22,8 @@
         "sell": 30
     },
     "bid_strategy": {
-        "ask_last_balance": 0.0,
         "use_order_book": false,
+        "ask_last_balance": 0.0,
         "order_book_top": 1,
         "check_depth_of_market": {
             "enabled": false,
@@ -56,8 +56,10 @@
     },
     "exchange": {
         "name": "bittrex",
+        "sandbox": false,
         "key": "your_exchange_key",
         "secret": "your_exchange_secret",
+        "password": "",
         "ccxt_config": {"enableRateLimit": true},
         "ccxt_async_config": {
             "enableRateLimit": false,
@@ -107,6 +109,13 @@
         "token": "your_telegram_token",
         "chat_id": "your_telegram_chat_id"
     },
+    "api_server": {
+        "enabled": false,
+        "listen_ip_address": "127.0.0.1",
+        "listen_port": 8080,
+        "username": "freqtrader",
+        "password": "SuperSecurePassword"
+    },
     "db_url": "sqlite:///tradesv3.sqlite",
     "initial_state": "running",
     "forcebuy_enable": false,

config_kraken.json.example

@@ -5,15 +5,14 @@
     "fiat_display_currency": "EUR",
     "ticker_interval" : "5m",
     "dry_run": true,
-    "db_url": "sqlite:///tradesv3.dryrun.sqlite",
     "trailing_stop": false,
     "unfilledtimeout": {
         "buy": 10,
         "sell": 30
     },
     "bid_strategy": {
-        "ask_last_balance": 0.0,
         "use_order_book": false,
+        "ask_last_balance": 0.0,
         "order_book_top": 1,
         "check_depth_of_market": {
             "enabled": false,
@@ -60,8 +59,8 @@
     },
     "telegram": {
         "enabled": false,
-        "token": "",
-        "chat_id": ""
+        "token": "your_telegram_token",
+        "chat_id": "your_telegram_chat_id"
     },
     "initial_state": "running",
     "forcebuy_enable": false,

docs/backtesting.md

@@ -123,11 +123,12 @@ python scripts/download_backtest_data.py --exchange binance
 
 This will download ticker data for all the currency pairs you defined in `pairs.json`.
 
-- To use a different folder than the exchange specific default, use `--export user_data/data/some_directory`.
+- To use a different folder than the exchange specific default, use `--datadir user_data/data/some_directory`.
 - To change the exchange used to download the tickers, use `--exchange`. Default is `bittrex`.
 - To use `pairs.json` from some other folder, use `--pairs-file some_other_dir/pairs.json`.
 - To download ticker data for only 10 days, use `--days 10`.
 - Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
+- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with other options.
 
 For help about backtesting usage, please refer to [Backtesting commands](#backtesting-commands).
 
@@ -220,24 +221,8 @@ strategies, your configuration, and the crypto-currency you have set up.
 ### Further backtest-result analysis
 
 To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
-You can then load the trades to perform further analysis.
+You can then load the trades to perform further analysis as shown in our [data analysis](data-analysis.md#backtesting) backtesting section.
 
-A good way for this is using Jupyter (notebook or lab) - which provides an interactive environment to analyze the data.
-
-Freqtrade provides an easy to load the backtest results, which is `load_backtest_data` - and takes a path to the backtest-results file.
-
-``` python
-from freqtrade.data.btanalysis import load_backtest_data
-df = load_backtest_data("user_data/backtest-result.json")
-
-# Show value-counts per pair
-df.groupby("pair")["sell_reason"].value_counts()
-
-```
-
-This will allow you to drill deeper into your backtest results, and perform analysis which would make the regular backtest-output unreadable.
-
-If you have some ideas for interesting / helpful backtest data analysis ideas, please submit a PR so the community can benefit from it.
 
 ## Backtesting multiple strategies
 

docs/bot-usage.md

@@ -26,7 +26,8 @@ optional arguments:
   --version             show program's version number and exit
   -c PATH, --config PATH
                         Specify configuration file (default: None). Multiple
-                        --config options may be used.
+                        --config options may be used. Can be set to '-' to
+                        read config from stdin.
   -d PATH, --datadir PATH
                         Path to backtest data.
   -s NAME, --strategy NAME
@@ -103,7 +104,7 @@ If the bot does not find your strategy file, it will display in an error
 message the reason (File not found, or errors in your code).
 
 Learn more about strategy file in
-[optimize your bot](bot-optimization.md).
+[Strategy Customization](strategy-customization.md).
 
 ### How to use **--strategy-path**?
 
@@ -146,16 +147,26 @@ Backtesting also uses the config specified via `-c/--config`.
 
 ```
 usage: freqtrade backtesting [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
-                             [--eps] [--dmmp] [-l] [-r]
-                             [--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
-                             [--export EXPORT] [--export-filename PATH]
+                           [--max_open_trades MAX_OPEN_TRADES]
+                           [--stake_amount STAKE_AMOUNT] [-r] [--eps] [--dmmp]
+                           [-l]
+                           [--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
+                           [--export EXPORT] [--export-filename PATH]
 
 optional arguments:
   -h, --help            show this help message and exit
   -i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
                         Specify ticker interval (1m, 5m, 30m, 1h, 1d).
   --timerange TIMERANGE
                         Specify what timerange of data to use.
+  --max_open_trades MAX_OPEN_TRADES
+                        Specify max_open_trades to use.
+  --stake_amount STAKE_AMOUNT
+                        Specify stake_amount.
+  -r, --refresh-pairs-cached
+                        Refresh the pairs files in tests/testdata with the
+                        latest data from the exchange. Use it if you want to
+                        run your optimization commands with up-to-date data.
   --eps, --enable-position-stacking
                         Allow buying the same pair multiple times (position
                         stacking).
@@ -164,10 +175,6 @@ optional arguments:
                         (same as setting `max_open_trades` to a very high
                         number).
   -l, --live            Use live data.
-  -r, --refresh-pairs-cached
-                        Refresh the pairs files in tests/testdata with the
-                        latest data from the exchange. Use it if you want to
-                        run your backtesting with up-to-date data.
   --strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
                         Provide a commaseparated list of strategies to
                         backtest Please note that ticker-interval needs to be
@@ -188,7 +195,7 @@ optional arguments:
 ### How to use **--refresh-pairs-cached** parameter?
 
 The first time your run Backtesting, it will take the pairs you have
-set in your config file and download data from Bittrex.
+set in your config file and download data from the Exchange.
 
 If for any reason you want to update your data set, you use
 `--refresh-pairs-cached` to force Backtesting to update the data it has.
@@ -206,15 +213,26 @@ to find optimal parameter values for your stategy.
 
 ```
 usage: freqtrade hyperopt [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
-                          [--customhyperopt NAME] [--eps] [--dmmp] [-e INT]
-                          [-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
+                        [--max_open_trades MAX_OPEN_TRADES]
+                        [--stake_amount STAKE_AMOUNT] [-r]
+                        [--customhyperopt NAME] [--eps] [--dmmp] [-e INT]
+                        [-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
+                        [--print-all] [-j JOBS]
 
 optional arguments:
   -h, --help            show this help message and exit
   -i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
                         Specify ticker interval (1m, 5m, 30m, 1h, 1d).
   --timerange TIMERANGE
                         Specify what timerange of data to use.
+  --max_open_trades MAX_OPEN_TRADES
+                        Specify max_open_trades to use.
+  --stake_amount STAKE_AMOUNT
+                        Specify stake_amount.
+  -r, --refresh-pairs-cached
+                        Refresh the pairs files in tests/testdata with the
+                        latest data from the exchange. Use it if you want to
+                        run your optimization commands with up-to-date data.
   --customhyperopt NAME
                         Specify hyperopt class name (default:
                         DefaultHyperOpts).
@@ -229,27 +247,39 @@ optional arguments:
   -s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...], --spaces {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]
                         Specify which parameters to hyperopt. Space separate
                         list. Default: all.
-
+  --print-all           Print all results, not only the best ones.
+  -j JOBS, --job-workers JOBS
+                        The number of concurrently running jobs for
+                        hyperoptimization (hyperopt worker processes). If -1
+                        (default), all CPUs are used, for -2, all CPUs but one
+                        are used, etc. If 1 is given, no parallel computing
+                        code is used at all.
 ```
 
 ## Edge commands
 
 To know your trade expectacny and winrate against historical data, you can use Edge.
 
 ```
-usage: freqtrade edge [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE] [-r]
-                      [--stoplosses STOPLOSS_RANGE]
+usage: freqtrade edge [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
+                    [--max_open_trades MAX_OPEN_TRADES]
+                    [--stake_amount STAKE_AMOUNT] [-r]
+                    [--stoplosses STOPLOSS_RANGE]
 
 optional arguments:
   -h, --help            show this help message and exit
   -i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
                         Specify ticker interval (1m, 5m, 30m, 1h, 1d).
   --timerange TIMERANGE
                         Specify what timerange of data to use.
+  --max_open_trades MAX_OPEN_TRADES
+                        Specify max_open_trades to use.
+  --stake_amount STAKE_AMOUNT
+                        Specify stake_amount.
   -r, --refresh-pairs-cached
                         Refresh the pairs files in tests/testdata with the
                         latest data from the exchange. Use it if you want to
-                        run your edge with up-to-date data.
+                        run your optimization commands with up-to-date data.
   --stoplosses STOPLOSS_RANGE
                         Defines a range of stoploss against which edge will
                         assess the strategy the format is "min,max,step"
@@ -267,4 +297,4 @@ in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.
 ## Next step
 
 The optimal strategy of the bot will change with time depending of the market trends. The next step is to
-[optimize your bot](bot-optimization.md).
+[Strategy Customization](strategy-customization.md).