arcticdb_ext.exceptions.ArcticException: Error while normalizing symbol=test, 'datetime.timezone' object has no attribute 'zone'

[sluo1250]

Describe the bug

I upgraded arcticdb from version 4.3.1 to 5.2.6 and one thing I was able to do now gave me an error. Basically I write a pd.Timestamp as a value in metadata, but seems like it failed for normalizing part.

Is this expected behavior? I couldn't find anything here - https://docs.arcticdb.io/4.3.0/tutorials/metadata/

Steps/Code to Reproduce

Code to reproduce -

import pandas as pd  
import numpy as np  
import arcticdb as adb
ac = adb.Arctic(f's3s://s3.us-east-2.amazonaws.com:testXXX')
library = ac['test']
timestamps = pd.date_range(start="2024-01-01", periods=10, freq="H")  
df = pd.DataFrame({"value": np.random.rand(10)}, index=timestamps)  
library.write('test', df, metadata={'mymetadata':pd.Timestamp.utcnow()})

Above gives me an error

arcticdb_ext.exceptions.ArcticException: Error while normalizing symbol=test, 'datetime.timezone' object has no attribute 'zone'

Expected Results

library successfully writes.

OS, Python Version and ArcticDB Version

Python: 3.11.11| OS: Linux-5.14.0-427.42.1.el9_4.x86_64-x86_64-with-glibc2.34| arcticdb: 5.2.6 | pandas: 2.2.3 |

Backend storage used

No response

Additional Context

No response

[IvoDD]

Thanks for reporting! I can reproduce the issue with the latest ArcticDB release. I'll take a closer look tomorrow morning

[IvoDD]

I've narrowed down the issue. It looks like pandas changed the type they use for the timezone in v2 and we don't correctly handle the new type.
With an old version of pandas:

>>> pd.__version__
'1.5.3'
>>> type(pd.Timestamp.utcnow().tz)
<class 'pytz.UTC'>

whereas with a new version of pandas:

>>> pd.__version__
'2.2.3'
>>> type(pd.Timestamp.utcnow().tz)
<class 'datetime.timezone'>

I'll write up a fix for this to be able to process both versions and we'll probably make it available in next release.

In the meantime your code will start working if you install an earlier version of pandas with:

pip install pandas==1.5.3