Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datetime64[ns] to int32 is not supported #1214

Open
2 tasks done
alehanderoo opened this issue Feb 10, 2025 · 0 comments
Open
2 tasks done

datetime64[ns] to int32 is not supported #1214

alehanderoo opened this issue Feb 10, 2025 · 0 comments

Comments

@alehanderoo
Copy link

Contributing guidelines

  • I agree to follow this project's
    Contributing Guidelines
    which, I understand, contain short technical notes on how to best contribute to this project.

Own due diligence

  • I verify my due dilligence—I have went through the tutorials and the API docs and
    have used the search on Issues and GitHub Disucussions, as well all Google,
    with all the relevant search keywords.

Expected behavior

First off all: amazing library!

The issue I'm facing:
I'm trying to perform a back-testing strategy using lower timeframe data: 5min interval.

I have the following ohlcv index dtype:

DatetimeIndex(['2024-01-29 09:30:00', '2024-01-29 09:35:00',
'2024-01-29 09:40:00', '2024-01-29 09:45:00',
'2024-01-29 09:50:00', '2024-01-29 09:55:00',
'2024-01-29 10:00:00', '2024-01-29 10:05:00',
'2024-01-29 10:10:00', '2024-01-29 10:15:00',
...
'2025-01-28 10:05:00', '2025-01-28 10:10:00',
'2025-01-28 10:15:00', '2025-01-28 10:20:00',
'2025-01-28 10:25:00', '2025-01-28 10:30:00',
'2025-01-28 10:35:00', '2025-01-28 10:40:00',
'2025-01-28 10:45:00', '2025-01-28 10:50:00'],
dtype='datetime64[ns]', name='datetime', length=19643, freq=None)

Which is printed after I perform to_datetime:
ohlcv.index = pd.to_datetime(pred_df.index) print (ohlcv.index)

However, I get the error: TypeError: Converting from datetime64[ns] to int32 is not supported. Do obj.astype('int64').astype(dtype) instead

Does this mean the library only works with full day intervals? Setting the type to int32 is not possible since it will cause the datetime to overflow.
A quick work-around for now is to use an index range, but using the datetimes would be neat.

I have added the file for the code sample:

EXPD_ohlcv.csv

Thanks in advance!

Code sample

from backtesting import Backtest, Strategy
from backtesting.lib import crossover

from backtesting.test import SMA

# Load your predictions dataframe
ohlcv = pd.read_csv(f'./data/EXPD_ohlcv.csv', index_col=0)
ohlcv.index = pd.to_datetime(pred_df.index)

class SmaCross(Strategy):
    def init(self):
        price = self.data.Close
        self.ma1 = self.I(SMA, price, 10)
        self.ma2 = self.I(SMA, price, 20)

    def next(self):
        if crossover(self.ma1, self.ma2):
            self.buy()
        elif crossover(self.ma2, self.ma1):
            self.sell()


bt = Backtest(ohlcv, SmaCross, commission=.002,
              exclusive_orders=True)
stats = bt.run()
bt.plot()

Actual behavior

I get the following error:
`
TypeError Traceback (most recent call last)
Cell In[5], line 63
61 stats = bt.run()
62 print(stats)
---> 63 bt.plot()

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\backtesting\backtesting.py:1697, in Backtest.plot(self, results, filename, plot_width, plot_equity, plot_return, plot_pl, plot_volume, plot_drawdown, plot_trades, smooth_equity, relative_equity, superimpose, resample, reverse_indicators, show_legend, open_browser)
1694 raise RuntimeError('First issue backtest.run() to obtain results.')
1695 results = self._results
-> 1697 return plot(
1698 results=results,
1699 df=self._data,
1700 indicators=results._strategy._indicators,
1701 filename=filename,
1702 plot_width=plot_width,
1703 plot_equity=plot_equity,
1704 plot_return=plot_return,
1705 plot_pl=plot_pl,
1706 plot_volume=plot_volume,
1707 plot_drawdown=plot_drawdown,
1708 plot_trades=plot_trades,
1709 smooth_equity=smooth_equity,
1710 relative_equity=relative_equity,
1711 superimpose=superimpose,
1712 resample=resample,
1713 reverse_indicators=reverse_indicators,
1714 show_legend=show_legend,
1715 open_browser=open_browser)

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\backtesting_plotting.py:213, in plot(results, df, indicators, filename, plot_width, plot_equity, plot_return, plot_pl, plot_volume, plot_drawdown, plot_trades, smooth_equity, relative_equity, superimpose, resample, reverse_indicators, show_legend, open_browser)
211 # Limit data to max_candles
212 if is_datetime_index:
--> 213 df, indicators, equity_data, trades = _maybe_resample_data(
214 resample, df, indicators, equity_data, trades)
216 df.index.name = None # Provides source name @index
217 df['datetime'] = df.index # Save original, maybe datetime index

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\backtesting_plotting.py:167, in _maybe_resample_data(resample_rule, df, indicators, equity_data, trades)
160 return f
162 if len(trades): # Avoid pandas "resampling on Int64 index" error
163 trades = trades.assign(count=1).resample(freq, on='ExitTime', label='right').agg(dict(
164 TRADES_AGG,
165 ReturnPct=_weighted_returns,
166 count='sum',
--> 167 EntryBar=_group_trades('EntryTime'),
168 ExitBar=_group_trades('ExitTime'),
169 )).dropna()
171 return df, indicators, equity_data, trades

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\backtesting_plotting.py:154, in _maybe_resample_data.._group_trades(column)
153 def _group_trades(column):
--> 154 def f(s, new_index=pd.Index(df.index.astype(int)), bars=trades[column]):
155 if s.size:
156 # Via int64 because on pandas recently broken datetime
157 mean_time = int(bars.loc[s.index].astype(int).mean())

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\pandas\core\indexes\base.py:1087, in Index.astype(self, dtype, copy)
1085 if isinstance(values, ExtensionArray):
1086 with rewrite_exception(type(values).name, type(self).name):
-> 1087 new_values = values.astype(dtype, copy=copy)
1089 elif isinstance(dtype, ExtensionDtype):
1090 cls = dtype.construct_array_type()

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\pandas\core\arrays\datetimes.py:739, in DatetimeArray.astype(self, dtype, copy)
737 elif isinstance(dtype, PeriodDtype):
738 return self.to_period(freq=dtype.freq)
--> 739 return dtl.DatetimeLikeArrayMixin.astype(self, dtype, copy)

File c:\Users\aless\OneDrive\Documenten\Projects\ARB_extrema_classifier.venv\Lib\site-packages\pandas\core\arrays\datetimelike.py:482, in DatetimeLikeArrayMixin.astype(self, dtype, copy)
480 values = self.asi8
481 if dtype != np.int64:
--> 482 raise TypeError(
483 f"Converting from {self.dtype} to {dtype} is not supported. "
484 "Do obj.astype('int64').astype(dtype) instead"
485 )
487 if copy:
488 values = values.copy()

TypeError: Converting from datetime64[ns] to int32 is not supported. Do obj.astype('int64').astype(dtype) instead
`

Additional info, steps to reproduce, full crash traceback, screenshots

No response

Software versions

  • pandas version: 2.2.3
  • becktesting version: 0.6.1
  • numpy version: 1.26.4
  • bokeh version: 3.6.3
  • OS: Win11 x86 -> jupiter notebook python 3.11.6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant