Error: Unrecognized type: "Duration" (18) #4489

felp99 · 2022-03-10T14:53:58Z

Summary

The dataframe shows an error on the frontend if a duration type (timedelta) is used.

Steps to reproduce

import pandas as pd
import streamlit as st

s = pd.Series(pd.date_range("2012-1-1", periods=3, freq="D"))

td = pd.Series([pd.Timedelta(days=i) for i in range(3)])

df = pd.DataFrame(
    {
        "A": s,
        "B": td
    }
)

st.table(df)

at Ji (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3064094)
at Function.$n.decode (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3065289)
at Ki (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3062354)
at Function.Yn.decode (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3066440)
at e._createHeader (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3059955)
at e.value (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3059100)
at e.value (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3069443)
at n.value (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3130013)
at n.value (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3125043)
at Function.value (http://localhost:8501/static/js/5.cbb83f60.chunk.js:2:3119592)

The text was updated successfully, but these errors were encountered:

kajarenc · 2022-03-10T15:39:18Z

Hello @felp99 .
Could you please provide more context for the issue?

david-huck · 2022-03-12T13:05:34Z

I've had the same issue when trying to use

st.write(df.dtypes.astype(str))
st.dataframe(df)

where the first line prints (among other things

...
Duration       timedelta64[ns]
...

And the error message shown in streamlit is:

Error: Unrecognized type: "Duration" (18)

$n.decode@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3067053
Ki@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3064118
Yn.decode@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3068206
value/a._createHeader@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3061719
value@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3060864
value@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3071207
value@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3131777
value@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3126807
value@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3121356
e@http://192.168.0.102:8501/static/js/main.3fe58c98.chunk.js:2:10639
get@http://192.168.0.102:8501/static/js/main.3fe58c98.chunk.js:2:425090
ln@http://192.168.0.102:8501/static/js/main.3fe58c98.chunk.js:2:457494
ia@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3254431
Gs@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3306889
Ml@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3294065
Cl@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3293993
Sl@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3293854
gl@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3290820
qi/<@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3240199
t.unstable_runWithPriority@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3317618
Ui@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3239976
qi@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3240146
Gi@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3240079
dl@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3288189
enqueueSetState@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3244097
g.prototype.setState@http://192.168.0.102:8501/static/js/5.c7ec7886.chunk.js:2:3190437
3156/n/r.handleDeltaMsg/<@http://192.168.0.102:8501/static/js/main.3fe58c98.chunk.js:2:558852

A naive approach, renaming the column, did not help...

kajarenc · 2022-03-17T15:31:24Z

Thank you @felp99 and @david-huck !

We were able to reproduce the issue and can confirm that the bug exists.

The minimal code snippet that I used for verification is this:

import pandas as pd
import streamlit as st

s = pd.Series(pd.date_range("2012-1-1", periods=3, freq="D"))

td = pd.Series([pd.Timedelta(days=i) for i in range(3)])

df = pd.DataFrame(
    {
        "A": s,
        "B": td
    }
)

st.table(df)

As a workaround for now you can use

dataFrameSerialization = "legacy"

config option to switch to a legacy way of data frame serialization.
I checked it and with dataFrameSerialization = "legacy" it works for time delta serialization.

Please see https://docs.streamlit.io/library/advanced-features/configuration for more information

tarmojuristo · 2022-03-30T11:30:04Z

I had the same problem with timedelta64[ns] value in dataframe resulting in Error: Unrecognized type: "Duration" (18)

However, changing dataFrameSerialization = "legacy" did not solve my problem, I got NotImplementedError: Dtype category not understood instead.

bdthombre · 2022-04-24T18:14:39Z

Same problem.
Temporarily working with:

st.code(df)

GeorgeVJose · 2022-05-27T23:31:31Z

Any update on this? Has anyone found a solution or temporary work-around?

felp99 · 2022-05-27T23:56:40Z

It’s solved in GitHub. Check the link Enviado do meu iPhone

…

Em 27 de mai. de 2022, à(s) 20:31, George V Jose ***@***.***> escreveu: Any update on this? Has anyone found a solution or temporary work-around? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

tvst · 2022-06-07T18:03:30Z

Adding some info I just collected on this, so it's easier to fix later:

Streamlit uses Apache Arrow to serialize dataframes and send them from Python to JS
I don't know if it matters, but on the JS side Streamlit uses Arrow v7.
The upstream version of Arrow JS is already at v8. Upgrading could help.
The Jira board for Arrow JS has an open bug that sounds like this one, though, so it's possible upgrading will not help after all.

lukasmasuch · 2022-06-10T14:09:11Z

I also did a quick check today into this issue. The duration type is only partially implemented in the javascript/typescript version of Arrow. For example, it is missing here: https://github.com/apache/arrow/blob/f5166fe21969d19adff23fc840ed1d7511348bad/js/src/ipc/metadata/message.ts#L440
I assume it just never got fully implemented yet. Feel free to leave an upvote on this Jira Issue to get some more attention on this.

In the meantime, a workaround would be either to use legacy serialization as described here or convert the timedelta value to int via df["delta"].dt.days or df["delta"].dt.seconds. For example:

import pandas as pd
import streamlit as st

data = {
    "dt1": ["2020-01-03 12:23", "2020-04-12 12:34"],
    "dt2": ["2021-03-23 12:13", "2021-12-12 11:04"],
}

df = pd.DataFrame(data)
df["dt1"] = pd.to_datetime(df["dt1"])
df["dt2"] = pd.to_datetime(df["dt2"])
df["delta"] = df["dt2"] - df["dt1"]

# Convert to integer value:
df["delta"] = df["delta"].dt.days

st.dataframe(df)

andrader · 2022-08-26T14:45:36Z

I have the same problem when displaying a pd.DataFrame with a column of datetime.timedelta objects.

nafets33 · 2022-09-02T17:49:13Z

I added timezone to my dates which works! to get around this error for displaying timedeltas

est = pytz.timezone("America/New_York")
s = datetime.datetime.now().astimezone(est)
e = datetime.datetime.now().astimezone(est)
delta = (e-s)

dodid · 2022-11-05T04:27:47Z

Append or column.dtype == 'timedelta64[ns]' to

streamlit/lib/streamlit/type_util.py

Line 613 in d774bed

) or column.dtype == "complex128"

fix the problem for me.
In some cases, also need to add df = fix_arrow_incompatible_column_types(df) in front of

streamlit/lib/streamlit/type_util.py

Line 669 in d774bed

table = pa.Table.from_pandas(df)

to force conversion.

flight505 · 2023-01-03T10:16:26Z

Still hoping for a fix to this issue..

shivan-s · 2023-01-03T21:25:43Z

Yes, running into this issue as well.

HWiese1980 · 2023-03-25T11:32:37Z

Any update here? Currently running into this problem...

Adithya-Shetty100 · 2023-05-06T13:00:43Z

same here, not able to fix this issue when taking both time and date as input

Error: Unrecognized type: "Duration" (18)
at Zi (http://localhost:8501/static/js/main.5e4731c6.js:2:1300567)
at gr.decode (http://localhost:8501/static/js/main.5e4731c6.js:2:1301753)
at Ui (http://localhost:8501/static/js/main.5e4731c6.js:2:1299119)
at Mr.decode (http://localhost:8501/static/js/main.5e4731c6.js:2:1302895)
at e._createHeader (http://localhost:8501/static/js/main.5e4731c6.js:2:1296724)
at e.value (http://localhost:8501/static/js/main.5e4731c6.js:2:1295890)
at r.value (http://localhost:8501/static/js/main.5e4731c6.js:2:1320472)
at Function.from (<anonymous>)
at r (http://localhost:8501/static/js/main.5e4731c6.js:2:3571164)
at a (http://localhost:8501/static/js/main.5e4731c6.js:2:3581091)

bobiq · 2023-05-20T20:58:30Z

same here. I have problems with timedelta type variables. Funny enough the same code runs ok locally but not when commited to github and deployed on Streamlit cloud. Has anyone heard whether it is being worked on and what priority it has been given?

bobiq · 2023-05-21T06:48:26Z

The error I get:

File "/home/appuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script

exec(code, module.__dict__)

File "/app/youtube_channels_by_topic_streamlit/YouTube_channels_by_topic_streamlit.py", line 356, in

video_df['likes_per_second']=video_df['likeCount']/video_df['durationSecs']

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/ops/common.py", line 81, in new_method

return method(self, other)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/arraylike.py", line 210, in truediv

return self._arith_method(other, operator.truediv)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/series.py", line 6108, in _arith_method

return base.IndexOpsMixin._arith_method(self, other, op)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/base.py", line 1348, in _arith_method

result = ops.arithmetic_op(lvalues, rvalues, op)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/ops/array_ops.py", line 224, in arithmetic_op

res_values = op(left, right)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py", line 1993, in array_ufunc

return super().__array_ufunc__(ufunc, method, *inputs, **kwargs)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/arrays/base.py", line 1686, in array_ufunc

return arraylike.default_array_ufunc(self, ufunc, method, *inputs, **kwargs)

File "/home/appuser/venv/lib/python3.9/site-packages/pandas/core/arraylike.py", line 489, in default_array_ufunc

return getattr(ufunc, method)(*new_inputs, **kwargs)

numpy.core._exceptions._UFuncBinaryResolutionError: ufunc 'divide' cannot use operands with types dtype('int64') and dtype('<m8[s]')

Pk13055 · 2023-07-23T21:07:09Z

One hack that seems to work is transposing the data such that instead of a column of TimeDelta values, you have a row, eg:

df.loc[:, 'duration'] = df.time - df.time.shift(-1)
df.head().T  # for example

QCAsystemsbharvey · 2023-10-10T17:59:45Z

As a work around you can convert your timedelta to float using total_seconds()

import pandas as pd
import streamlit as st

s = pd.Series(pd.date_range("2012-1-1", periods=3, freq="D"))

td = pd.Series([pd.Timedelta(days=i).total_seconds() for i in range(3)])

df = pd.DataFrame(
    {
        "A": s,
        "B": td
    }
)

st.table(df)

output from your code:

kmcgrady · 2023-11-07T20:42:56Z

@lukasmasuch This will be fixed in the new version of PyArrow/Arrow.js. correct?

I think the fix is here apache/arrow#37341

lukasmasuch · 2023-11-08T02:30:51Z

@kmcgrady Yes, the new ArrowJS release finally provides support for the Duration type 🎉 However, we still have to do a few things on our side to actually make it work. I'm implementing the changes here: #7689

Alli118 · 2024-04-16T05:05:49Z

Hello,
I am encountering an issue with my Streamlit application[Chatbot] , where a blank page randomly appears for a random query. I've tried using the dataFrameSerialization = "legacy" approach, but the issue persists. When inspecting the page, the errors I'm seeing look like the following:
TypeError: Cannot read properties of undefined (reading 'vertical')
at Di (main, 1661a802.js:2:4213879)
at wa (main. 1661a802.js:2:2230016)
at wl (main.1661a802.js:2:2287129)
at ze (main.1661a802.js:2:2276235)
at Oc (main.1661a802.js:2:2276163)
at Mc (main. 1661a802.js:2:2276026)
at cc (main, 16612802.js:2:2273142)
at Uo (moin. 1661a802.js:2:2213737)
at rc (main.1661a802.js:2:2270229)
at Object.enqueueSetState (main.1661a802.js:2:2220755)
Uncaught TypeError: Cannot read properties of undefined (reading 'vertical')
at Di (main. 1661a802.js:2:4213879)
at wa (main.1661a802.js:2:2230016)
at wl (moin.1661a802.js:2:2287129)
at zc (main.16618802.js:2:2276235)
at Oc (main. 1661a802.js:2:2276163)
at Mc (main.1661a802.js:2:2276026)
at cc (main.1661a802.js:2:2273142)
at Uo (main. 1661a802.js:2:2213737)
at rc (main.1661a802.js:2:2270229)
at Object.enqueueSetState (main, 1661a802.js:2:2220755)
Any insights on how to address this would be greatly appreciated.

Thank you!

itsToggle · 2024-04-24T10:04:15Z

I see the same thing as @Alli118 - at seemingly random points of interaction with my app I get the TypeError: Cannot read properties of undefined (reading 'vertical') or TypeError: Cannot read properties of undefined (reading 'tab')

mimn97 · 2024-05-13T19:39:12Z

@Alli118 @itsToggle I'm also having the same issue of TypeError: Cannot read properties of undefined (reading 'vertical') in the JS console for a random query. But it seems that this error occurs only on the specific page that asks GPT-4 to generate several sentences and asks users to answer some questions regarding the generated sentences by clicking radio buttons. Is it related to the size of state sessions or containers?

lukasmasuch · 2024-05-13T19:43:22Z

@Alli118 @itsToggle @mimn97 I'm pretty sure that the error you are getting is unrelated to this issue. It would be awesome if someone could open a new issue for this error and, if possible, add some minimum reproducible examples to this.

felp99 added type:bug Something isn't working status:needs-triage Has not been triaged by the Streamlit team labels Mar 10, 2022

kajarenc added status:awaiting-user-response Issue requires clarification from submitter and removed status:needs-triage Has not been triaged by the Streamlit team labels Mar 10, 2022

kajarenc added package:arrow Related to Arrow status:confirmed Bug has been confirmed by the Streamlit team and removed status:awaiting-user-response Issue requires clarification from submitter labels Mar 17, 2022

kajarenc added the priority:P2 label Mar 17, 2022

lukasmasuch added feature:st.dataframe and removed feature:st.dataframe labels Jun 10, 2022

lukasmasuch mentioned this issue Jun 10, 2022

Webapp goes blank if one tries to render a dataframe with pd.Timedelta entries #3781

Closed

sfc-gh-kbregula self-assigned this Sep 28, 2022

lukasmasuch added the feature:st.dataframe label Jan 12, 2023

kravets-levko mentioned this issue Jan 22, 2023

[PECO-239] Apache Arrow support databricks/databricks-sql-nodejs#94

Merged

10 tasks

lukasmasuch mentioned this issue May 15, 2023

st.dataframe fails on unknown data types #6686

Closed

5 tasks

cise-midoglu mentioned this issue Jul 15, 2023

Error: Unrecognized type: "Duration" (18) in Player GPS Report page simulamet-host/soccer-dashboard#73

Open

kmcgrady assigned lukasmasuch and unassigned sfc-gh-kbregula Nov 7, 2023

lukasmasuch mentioned this issue Nov 8, 2023

Add support for timedelta type to st.dataframe, st.data_editor and st.table. #7689

Merged

lukasmasuch closed this as completed in #7689 Nov 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: Unrecognized type: "Duration" (18) #4489

Error: Unrecognized type: "Duration" (18) #4489

felp99 commented Mar 10, 2022 •

edited by lukasmasuch

Loading

kajarenc commented Mar 10, 2022

david-huck commented Mar 12, 2022

kajarenc commented Mar 17, 2022

tarmojuristo commented Mar 30, 2022

bdthombre commented Apr 24, 2022

GeorgeVJose commented May 27, 2022

felp99 commented May 27, 2022 via email

tvst commented Jun 7, 2022 •

edited

Loading

lukasmasuch commented Jun 10, 2022

andrader commented Aug 26, 2022 •

edited

Loading

nafets33 commented Sep 2, 2022 •

edited

Loading

dodid commented Nov 5, 2022 •

edited

Loading

flight505 commented Jan 3, 2023

shivan-s commented Jan 3, 2023

HWiese1980 commented Mar 25, 2023

Adithya-Shetty100 commented May 6, 2023 •

edited

Loading

bobiq commented May 20, 2023

bobiq commented May 21, 2023

Pk13055 commented Jul 23, 2023

QCAsystemsbharvey commented Oct 10, 2023 •

edited

Loading

kmcgrady commented Nov 7, 2023 •

edited

Loading

lukasmasuch commented Nov 8, 2023 •

edited

Loading

Alli118 commented Apr 16, 2024 •

edited

Loading

itsToggle commented Apr 24, 2024

mimn97 commented May 13, 2024 •

edited

Loading

lukasmasuch commented May 13, 2024

Error: Unrecognized type: "Duration" (18) #4489

Error: Unrecognized type: "Duration" (18) #4489

Comments

felp99 commented Mar 10, 2022 • edited by lukasmasuch Loading

Summary

Steps to reproduce

kajarenc commented Mar 10, 2022

david-huck commented Mar 12, 2022

kajarenc commented Mar 17, 2022

tarmojuristo commented Mar 30, 2022

bdthombre commented Apr 24, 2022

GeorgeVJose commented May 27, 2022

felp99 commented May 27, 2022 via email

tvst commented Jun 7, 2022 • edited Loading

lukasmasuch commented Jun 10, 2022

andrader commented Aug 26, 2022 • edited Loading

nafets33 commented Sep 2, 2022 • edited Loading

dodid commented Nov 5, 2022 • edited Loading

flight505 commented Jan 3, 2023

shivan-s commented Jan 3, 2023

HWiese1980 commented Mar 25, 2023

Adithya-Shetty100 commented May 6, 2023 • edited Loading

bobiq commented May 20, 2023

bobiq commented May 21, 2023

Pk13055 commented Jul 23, 2023

QCAsystemsbharvey commented Oct 10, 2023 • edited Loading

kmcgrady commented Nov 7, 2023 • edited Loading

lukasmasuch commented Nov 8, 2023 • edited Loading

Alli118 commented Apr 16, 2024 • edited Loading

itsToggle commented Apr 24, 2024

mimn97 commented May 13, 2024 • edited Loading

lukasmasuch commented May 13, 2024

felp99 commented Mar 10, 2022 •

edited by lukasmasuch

Loading

tvst commented Jun 7, 2022 •

edited

Loading

andrader commented Aug 26, 2022 •

edited

Loading

nafets33 commented Sep 2, 2022 •

edited

Loading

dodid commented Nov 5, 2022 •

edited

Loading

Adithya-Shetty100 commented May 6, 2023 •

edited

Loading

QCAsystemsbharvey commented Oct 10, 2023 •

edited

Loading

kmcgrady commented Nov 7, 2023 •

edited

Loading

lukasmasuch commented Nov 8, 2023 •

edited

Loading

Alli118 commented Apr 16, 2024 •

edited

Loading

mimn97 commented May 13, 2024 •

edited

Loading