Feature Request: Replace ta-lib with ta or pandas_ta #492
Labels
enhancement
New feature or request
Comments
|
Hi, I agree that Talib has become a pain to install. I tried to update the dependencies to create guides for different operating systems, and I had a very difficult time, so I decided to remove Talib. However, I'm not going to use these other libraries either because I don't want to make one mistake twice. Especially now that we have AI and unit tests in place, I'm going to replace the code with our own. We also have the number library for doing optimizations, so I'm going to replace the code for those indicators and then remove Talib. |
|
Hi @saleh-mir, thanks for your hard work, if you want to reimplement the indicators, all of them can be implemented with just numpy without using wrappers to an external C/C++ or Rust based solution, numpy comes with a good set to implement most of the indicators efficiently, here is an example of a 14 period RSI: import yfinance as yf
import pandas as pd
import numpy as np
# Fetch historical data for AAPL
df = yf.download('AAPL', start='2022-01-01', end='2023-01-01')
# Calculate Price Changes
df['Change'] = df['Close'].diff()
# Calculate Gains and Losses
df['Gain'] = np.where(df['Change'] > 0, df['Change'], 0)
df['Loss'] = np.where(df['Change'] < 0, -df['Change'], 0)
# Calculate Average Gain and Loss
period = 14
df['Avg Gain'] = df['Gain'].rolling(window=period).mean()
df['Avg Loss'] = df['Loss'].rolling(window=period).mean()
# Calculate RS and RSI
df['RS'] = df['Avg Gain'] / df['Avg Loss']
df['RSI'] = 100 - (100 / (1 + df['RS']))
# Display the RSI
print(df[['Close', 'RSI']].tail(20))As you can see, we don't really need a TA library, and numpy, because it has an efficient low-level implementation, allows you to manipulate arrays with great efficiency and flexibility. If you need contributors to implement the indicators, I'm willing to help with this migration to make Jesse more accessible and simple to install. Happy coding!😄 |
|
That's exactly my goal too, to use NumPy alone. And preferably, not use pandas, either. But if we have to make a loop from time to time, we can use Numba to improve its performance. Well, yeah, if you could help, that'd be great. And we can finish it faster. But if you're going to, please, tell me which ones you're working on, so that, we don't accidentally do duplicate work. We have unit tests in place. So as long as the new implementation can pass that test, it should be fine. I expect to have very small differences. For example, if one value has two decimals, if the second decimal number is slightly different, that would be fine. |
|
Well, I did some checks on the source code, I looked for the indicators implemented in jesse/indicators and I got these results: Total indicators implemented: 183
Files that import talib: 101And the files that import talib are: jesse/indicators/acosc.py
jesse/indicators/ad.py
jesse/indicators/adosc.py
jesse/indicators/adx.py
jesse/indicators/adxr.py
jesse/indicators/ao.py
...
jesse/indicators/vwmacd.py
jesse/indicators/wclprice.py
jesse/indicators/willr.py
jesse/indicators/wma.py
jesse/indicators/wt.py
jesse/indicators/zscore.py
We'll have some work on this but it's definitely possible and worth it, I need some time to get used to the codebase and also how things work (the tests mainly.) It shouldn't take that long, Do you have any means of communication (aka dm.)? Edit: @nb.njit
def numpy_diff(arr, drift=1):
result = np.full_like(arr, np.nan)
result[drift:] = arr[drift:] - arr[:-drift]
return result
@nb.njit
def rma(data, length):
"""Wilder's Moving Average (RMA)"""
alpha = 1.0 / length
rma = np.empty_like(data)
rma[0] = data[0]
for i in range(1, len(data)):
rma[i] = alpha * data[i] + (1 - alpha) * rma[i - 1]
return rma
@nb.njit
def rsi(close, length=14, scalar=100, drift=1):
"""Indicator: Relative Strength Index (RSI)"""
negative = numpy_diff(close, drift)
positive = np.where(negative > 0, negative, 0)
negative = np.where(negative < 0, -negative, 0)
avg_gain = rma(positive, length)
avg_loss = rma(negative, length)
rs = avg_gain / (avg_loss + 1e-10)
rsi = scalar - (scalar / (1 + rs))
rsi[:length] = np.nan
return rsiUsage example: import ta
import numpy as np
import numba as nb
import yfinance as yf
# RSI function definitions here
data = yf.download(tickers=["AAPL"], start="2020-1-1", end="2021-1-1", progress=False)
impl = rsi(data["Close"].values)
comparison = ta.momentum.RSIIndicator(data["Close"].squeeze()).rsi()
plt.plot(impl, label="Implementation", linestyle="solid")
plt.plot(comparison.values, label="Comparison", linestyle="dashed")
plt.legend()
plt.show()The result is:
I based myself on the code of pandas_ta as you search pure numpy with numba I had some difficulties since I do not know numba. I believe that it is possible to implement all the indicators in a similar and simple way in this code not only has the RSI but also RMA (Wilder's Moving Average.) If you like this implementation and find it acceptable then I can help 😃 |
|
Yes, that's exactly the gist of it. Although you don't have to use Numba if there is no loop in the code, I don't think it will add any benefit in that case. To give you a summary, we're going to use AI heavily for this because this is the type of task that it can do a great job with, especially with the reasoning models such as O3 Mini. As long as the unit test for that indicator is passing, we know the implementation is correct. However, as much as possible, we should avoid using loops. Also, the parameters of the functions and the returning types must stay the same as much as possible, so we don't break the existing API for users. If you're going to help, please start with one indicator. For example, submit a PR for it so I can comment on it before proceeding with more. |
|
Hello @saleh-mir , sorry for my slow response (hard day). I decided to clone Jesse in my WSL workspace (Ubuntu). and I had some problems during installation but not with TA-Lib but with dependency packages, some of them the manager could not find them so I made some changes to make it work in my environment.. The arrow>=1.2.1
blinker>=1.4
click>=8.0.3
numpy==1.24.4
pandas>=2.0.0
peewee>=3.14.8
psycopg2-binary>=2.9.9
pydash>=6.0.0
fnc>=0.5.3
pytest>=6.2.5
requests>=2.32.0
scipy>=1.10.1
statsmodels>=0.14.1
TA-Lib
tabulate>=0.8.9
timeloop>=1.0.2
websocket-client>=1.8.0
wsaccel>=0.6.6
simplejson>=3.16.0
aioredis>=1.3.1
redis>=4.1.4
fastapi>=0.111.1
uvicorn>=0.29.0
websockets>=10.0.0
python-dotenv>=0.19.2
aiofiles>=0.7.0
numba>=0.58.1
PyJWT>=2.8.0
cryptography>=42.0.5
ecdsa>=0.16.0Well, I tried at the beginning to run the tests with the command: $ pytest # I ran this in jesse's root folder.Well, some tests passed but others didn't, below I will be leaving the logs: In a simple way: Well, I have no idea why this error is happening since I imagine that the tested packages import the jesse package locally, that is, they directly access the folders and files, but it seems that it is trying to access the jesse package as installed by pip? I'm not sure what's going on. Well, I need to fix this problem to be able to run the tests in my environment so that I can implement the indicators, if you can help me with this I would appreciate it. |
|
Yeah, it seems like the Jesse package itself isn't installed, but other than that, you're running it on Python 3.8, which isn't supported by us anymore. |
|
This is taken care of since version 1.7 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
Currently, the Jesse framework uses
ta-libfor technical analysis indicators. Whileta-libis widely used, it comes with several limitations:ta-liboften requires system-level dependencies, making it difficult to install, especially on different operating systems.These issues can frustrate users and create barriers for adoption and contribution.
Describe the solution you'd like
I propose replacing
ta-libwith one of the following alternatives:pandas_ta:ta: - Pure Python implementation. (Not maintained)Both libraries provide modern Pythonic APIs, reducing dependencies on external C libraries.
Describe alternatives you've considered
ta-lib: While possible, this does not address the core issues of installation complexity.taandpandas_ta.Additional Context
Key Benefits of the Suggested Alternatives:
taandpandas_taare pip-installable without system-level dependencies.Potential Challenges:
taandpandas_taare straightforward and pandas-compatible.ta-lib.By adopting one of these alternatives, the Jesse framework can provide a more modern, user-friendly, and maintainable solution for technical analysis.
The text was updated successfully, but these errors were encountered: