Date mismatch when using external axes object

[Vittorio94]

I have the following dataframe that contains ohlc data:
candles.head()

I want to plot the data using an external axes object and then add additional stuff on the chart using matplotlib. The problem is that there seems to be a mismatch in the way mplfinance and matplotlib handle dates. For example, the following code:

fig = mpf.figure(style='yahoo')
ax = fig.add_subplot(1,1,1)

mpf.plot(candles, ax=ax)

ax.plot(candles.index, candles.open)

should overlay a line connecting all the opens of the candles. However, the result is this:

Am I doing something wrong? Thank you

[DanielGoldfarb]

@Vittorio94
Vittorio,
What you are trying to do can be done easily without external axes mode. As a general rule you should avoid external axes mode, unless what you are trying to accomplish cannot be done any other way. This is because some features of mplfinance must be disabled in external axes mode and, as a user of mplfinance, you will have to do a lot more work to accomplish the same result. Even returnfig=True is preferable to external axes mode.

Here is how you should be doing what you are trying to accomplish above:

ap = mpf.make_addplot(candles.open)
mpf.plot(candles, addplot=ap, style='yahoo')

This accomplishes the same thing with half the amount of code!

---

The reason you are seeing the apparent x-axis mismatch is because you are allowing show_nontrading to default to False. When this value is False, then the x-axis values are actually the row numbers of your DataFrame ( range(0,len(candles) ) even though they are formatted as dates. In your external axes example, if you set show_nontrading=True (in the call to mpf.plot()) it will work.

Again, however, I discourage the use of external axes mode unless absolutely necessary since it requires much more code and it necessarily disables some features of mplfinance. If you explain what else you are trying to accomplish, I can let you know if there is a simple way to do it with mplfinance or if external axes are appropriate. If you have not already read it, I suggest "Adding Your Own Technical Studies to Plots."

[Vittorio94]

@DanielGoldfarb
Daniel, Thank you for the exhaustive answer. The reason why I want to use the external axes mode is that I need to add trade markers to the chart. I have tried using addplot, which works perfectly as long as there is no more than one marker per candle. However, say I want to plot an hourly chart, and I have a position that has been opened and closed within the same hour. I would have to create two separate addplots to represent the entry and exit markers. The same problem arises when a position is closed in the same candle where another position has been opened. I would have to create a different addplot for every position, which is very inconvenient!

[DanielGoldfarb]

@Vittorio94
Vittorio, Thanks for the explanation. I understand that mplfinance will not allow you to plot two markers at the same point in time unless they are in separate arrays. Perhaps that is an enhancement we can consider if there is enough demand for it.

If you don't find it inconvenient to put the markers in separate arrays, you can reduce the number of make_addplot() calls by passing a dataframe into make_addplot(), where any markers that are intended to be plotted at the same point in time are in separate columns of the data frame. If that is still inconvenient for the organization of your code, then by all means use external axes mode, but if so, I would encourage you to first try returnfig=True :

fig, axlist = mpf.plot(data,...,returnfig=True)
axlist[0].scatter(...)  # or whatever else you choose to do here to plot your markers
mpf.show()

Of course, if you find it simpler to just use external axes mode directly, then of course do that.

Thanks for you interest in mplfinance. Please, if you don't mind, post an image of your final result. I very much enjoy seeing the creative things that people are doing with mplfinance. All the best. --Daniel

[Vittorio94]

@DanielGoldfarb
Daniel, I did not know that I could return an axes list from mpf.plot(), that's great.
Using external axes mode, I was able to achieve the result I wanted:

If anyone is trying to do something similar to this, here is how I did it. I have a candles dataframe that contains ohlc data, and a positions dataframe that contains the entry and exit information of each position. Here is the output of positions.head():

After creating the figure with mplfinance, I used standard matplotlib to add scatter plots for the trade markers and line plots for the trade lines. There might be a better way to do this, but this has worked nicely for me. Here is the code:

fig, axlist = mpf.plot(candles, show_nontrading=True, returnfig=True, style="classic", type="candle")
        
for i in range(len(positions)):
    if positions.iloc[i].closedPL > 0:
        lineColor = "limegreen"
    else:
        lineColor = "crimson"
    
    axlist[0].plot(
    [positions.iloc[i].entryDateTime, positions.iloc[i].exitDateTime],
    [positions.iloc[i].entryPrice, positions.iloc[i].exitPrice],
    color=lineColor,
    linewidth=0.5,
    linestyle="dashed",
    )
    
    if positions.iloc[i].direction == "long":
        entryColor = "royalblue"
        exitColor = "crimson"
        axlist[0].scatter(
        [positions.iloc[i].entryDateTime, positions.iloc[i].exitDateTime],
        [positions.iloc[i].entryPrice, positions.iloc[i].exitPrice],
        c=[entryColor, exitColor],
        marker="x"
        )
        
    else:
        entryColor = "crimson"
        exitColor = "royalblue"
        axlist[0].scatter(
        [positions.iloc[i].entryDateTime, positions.iloc[i].exitDateTime],
        [positions.iloc[i].entryPrice, positions.iloc[i].exitPrice],
        c=[entryColor, exitColor],
        marker="x"
        )

mpf.show()

[DanielGoldfarb]

@Vittorio94
Vittorio- Great plot! And thank you for sharing the code. Nice work. All the best. --Daniel

[fxhuhn]

If you don't find it inconvenient to put the markers in separate arrays, you can reduce the number of make_addplot() calls by passing a dataframe into make_addplot(), where any markers that are intended to be plotted at the same point in time are in separate columns of the data frame. If that is still inconvenient for the organization of your code, then by all means use external axes mode, but if so, I would encourage you to first try returnfig=True :

Here as a small example how it can look like if you use multiple scatter plots. Also directly on a point if you take the circles, my "donuts".

[DanielGoldfarb]

@fxhuhn
Markus- Thank you. If you don't mind, would you also be willing to share the code that generated that plot? --Daniel

[fxhuhn]

The source table contains the OHLC data and some signals. I have separated the steps so that I can still understand the code later as well.

First step is to assign the signals with the price. In my case, it is the "Low" or "High", and create a new column with them (prefix 'signal_').

        for bull_signal in ['bull_rot', 'bull_lolly', 'bull_or', 'day_croc_long', 'bull_1_solo', 'bull_blau', 'bull_grau','bull_t',  'bull_braun', 'bull_gruen', 'bull_kreuz']:
            df['signal_' + bull_signal] = df.loc[(df[bull_signal] == 1), 'Low']

        for bear_signal in ['bear_rot', 'bear_lolly','bear_or', 'day_croc_short', 'bear_1_solo', 'bear_blau', 'bear_grau','bear_t',  'bear_braun', 'bear_gruen','bear_kreuz']:
            df['signal_' + bear_signal] = df.loc[(df[bear_signal] == 1), 'High']

Second step is my dict of scatter signals. It contains my signals that i want to see in the chart ('signal' is the name of the colum and the other key/values are scatter plot related):

        day_signals = [
            {'signal': 'signal_day_croc_long', 'size': 40, 'marker': '2', 'color': 'black', 'position': -1},
            {'signal': 'signal_day_croc_short', 'size': 40, 'marker': '1', 'color': 'black', 'position': 1},

            {'signal': 'signal_bull_rot', 'size': 40, 'color': 'red', 'position': -2},
            {'signal': 'signal_bear_rot', 'size': 40, 'color': 'red', 'position': 2},

            {'signal': 'signal_bull_1_solo', 'size': 5, 'color': 'black', 'position': -2},
            {'signal': 'signal_bear_1_solo', 'size': 5, 'color': 'black', 'position': 2},

            {'signal': 'signal_bull_lolly', 'size': 5, 'color': 'white', 'position': -2},
            {'signal': 'signal_bear_lolly', 'size': 5, 'color': 'white', 'position': 2},

            {'signal': 'signal_bull_or', 'size': 40, 'color': 'orange', 'position': -3},
            {'signal': 'signal_bear_or', 'size': 40, 'color': 'orange', 'position': 3},

            {'signal': 'signal_bull_grau', 'size': 40, 'color': 'grey', 'position': -4},
            {'signal': 'signal_bear_grau', 'size': 40, 'color': 'grey', 'position': 4},

            {'signal': 'signal_bull_t', 'size': 10, 'color': 'grey', 'position': -4},
            {'signal': 'signal_bear_t', 'size': 10, 'color': 'grey', 'position': 4},

            {'signal': 'signal_bull_braun', 'size': 5, 'color': 'brown', 'position': -4},
            {'signal': 'signal_bear_braun', 'size': 5, 'color': 'brown', 'position': 4},

            {'signal': 'signal_bull_blau', 'size': 20, 'color': 'blue', 'position': -6},
            {'signal': 'signal_bear_blau', 'size': 20, 'color': 'blue', 'position': 6},

            {'signal': 'signal_bull_gruen', 'size': 20, 'color': 'green', 'position': -7},
            {'signal': 'signal_bear_gruen', 'size': 20, 'color': 'green', 'position': 7},

            {'signal': 'signal_bull_kreuz', 'size': 20, 'marker': 'P', 'color': 'green', 'position': -7},
            {'signal': 'signal_bear_kreuz', 'size': 20, 'marker': 'P', 'color': 'red', 'position': 7},
        ]

Annotation. Calling it signal was not my best idea for readable code.

Now we come to the last step. Build the scatter plot.
Be sure, that signal is part of the chart, if it contains only NaN go to the next signal.

        for day_signal in day_signals:
            if df[day_signal['signal']].notna().sum() > 0:
                new_signal = mpf.make_addplot( df[day_signal['signal']] + day_signal['position'] * offset,
                                            scatter = True,
                                            markersize = day_signal['size'],
                                            marker = day_signal.get('marker','o'),
                                            color=day_signal['color'])
                add_plots.append(new_signal)

With the following calculation, I can ensure the appropriate spacing between signals in each chart so that there is no unnecessary overlap.
df[day_signal['signal']] + day_signal['position'] * offset

Small hint for the calculation of the offset:
offset = 0.013 *(df.High.max() - df.Low.min())

I hope this is all understandable so far. Otherwise just ask ;-)

[grdnryn]

Hi Daniel,

In the above reply:

#364 (comment)

You state:
The reason you are seeing the apparent x-axis mismatch is because you are allowing show_nontrading to default to False. When this value is False, then the x-axis values are actually the row numbers of your DataFrame ( range(0,len(candles) ) even though they are formatted as dates. In #364 (comment), if you set show_nontrading=True (in the call to mpf.plot()) it will work.

Does this (show_nontrading=True) remain the only way to achieve the scatter plot overlay onto of a regular chart? I am asking as the gaps in the chart from the weekend days makes the moving average look really wonky.

I am using external axes to overlay multiple scatter plots (signals) onto my daily chart. I have be able to do the same rather easily with the make_addplot() function, with the caveat that I was unable to add labels or a legend with this method, and when trying to track multiple signals, this feature is a must.
Am I missing something simple with how to add labels and or legends when using make_addplot?

[DanielGoldfarb]

@grdnryn
Regarding

Am I missing something simple with how to add labels and or legends when using make_addplot?

It's difficult for me to answer because you haven't completely described what you did to try to add labels or legends other than

I have be able to do the same rather easily with the make_addplot() function, with the caveat that I was unable to add labels or a legend with this method

That said, I can take somewhat of a guess, and/or at least provide some insight into the issues involved here. Certainly you have not missed something simple because there are definitely some complexities going on.

Before I describe them, let me first point out that as a general rule, if you can do something without using external axes mode then you should. You save yourself a lot of work by allowing mplfinance to do much of the work for you. This does not mean that you can't access the Axes objects at all (for labeling, annotations, etc). On the contrary, using returnfig=True gives you access to the Figure and Axes objects, but it is not considered "external axes mode" because these are Figure and Axes objects created by, and managed by, mplfinance.

The complexity here comes into play (as you mentioned) when you want to avoid gaps in the plot due to non-trading periods. There is no simple way to skip over datetimes that have no data associated with them. Essentially the x-axis then becomes non-linear, and discontinuous, with respect to time. One way to implement such a non-linear, discontinuous time axis, is to plot the data according to the row number in the dataframe (row number being linear and, at least in integer space, continuous). Then maintain a mapping between actual datetimes and row numbers or fractions thereof (for plotting purposes). This is how it is implemented inside mplfinance. For the most part, this is transparent to the user. For example, when using kwarg alines, users can specify dates or datetimes (even though the x-axis is row number) and mplfinance automatically finds the appropriate row number (or fraction thereof) where the aline vertex should be placed.

I say "for the most part" because when a user wants to do something with the Axes objects (from returnfig=True or when in external axes mode) then (as long as show_nontrading is False) then the user themselves must generate and use the datetime:row mapping.

I am currently working on an enhancement that will make this easier, exposing mapping transform functions which will allow users to work entirely in datetime space, which is clearly more intuitive in most situations.

I hope that helps. Let me know if you have more questions or need more clarification. All the best. --Daniel

[grdnryn]

Thanks Daniel,

Since output speed is not a determining factor here, I was able to do a combination of both (make_addplot and external axes).
I created a sort of a dummy scatter plot by feeding 0,0 into the x,y co-ordinates of the external axes scatter plot, hence enabling me to keep the show_nontrading=False argument in the main plot, whilst letting use the label functionality of these dummy scatter plots.
Incredibly inelegant, yet functioning.