Skip to content

Fix: MySQL/MariaDB TIMESTAMP precision for message ordering (#3442) #3443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

seungwone
Copy link

Fixes #3442

The AddBatchPreparedStatement class uses AtomicLong.getAndIncrement() to create sequential millisecond timestamps, but MySQL/MariaDB's default TIMESTAMP precision (0) truncates this information, causing message ordering issues.

Added fractional seconds precision (TIMESTAMP(3)) to MySQL and MariaDB schemas to support millisecond-based message ordering in JdbcChatMemoryRepository.

Changes

  • schema-mysql.sql: Changed TIMESTAMP to TIMESTAMP(3)
  • schema-mariadb.sql: Changed TIMESTAMP to TIMESTAMP(3)

Copy link
Contributor

@sunyuhan1998 sunyuhan1998 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this PR itself is fine, but as @markpollack previously mentioned:

#3154 (comment)

this field is actually being used as an auto-incrementing sequence to ensure message ordering, rather than representing time in a temporal sense. Perhaps we should consider renaming this field in the future.

@YunKuiLu
Copy link
Contributor

YunKuiLu commented Jun 6, 2025

There is a common belief that TIMESTAMP can only be used until 2038. Perhaps it should be changed to BIGINT or DATETIME.

https://stackoverflow.com/questions/409286/should-i-use-the-datetime-or-timestamp-data-type-in-mysql

@markpollack
Copy link
Member

yea, I'm not sure what to do here in a few ways.

  1. I think the 1.1 line should drop the timestamp completely.
  2. if we change the schema 'just like that' in a 1.0.1 release, stuff breaks - though one can argue of course it is already broken.

As for point 2, a migration script can be provided to run to upgrade to 1.0.1 and advertised in the release notes etc.

I'm thinking for the 1.0.x line, we provide the migration script.

thoughts @YunKuiLu @sunyuhan1998 @seungwone @ThomasVitale ?

@YunKuiLu
Copy link
Contributor

yea, I'm not sure what to do here in a few ways.

  1. I think the 1.1 line should drop the timestamp completely.
  2. if we change the schema 'just like that' in a 1.0.1 release, stuff breaks - though one can argue of course it is already broken.

As for point 2, a migration script can be provided to run to upgrade to 1.0.1 and advertised in the release notes etc.

I'm thinking for the 1.0.x line, we provide the migration script.

thoughts @YunKuiLu @sunyuhan1998 @seungwone @ThomasVitale ?

Maybe we can do something like this:

  1. In the 1.0.1-SNAPSHOT version, provide a migration script and support both schemas at the same time. When constructing the JdbcChatMemoryRepository, we can check the current schema. If the table in the DB still uses TIMESTAMP as the column type, we log a warn, and include a link to the upgrade guide.

  2. In the 1.0.x release version, we do the same check. If TIMESTAMP is detected, we log an error instead, and again include a link to the migration guide. Also, users would need to explicitly set the flag -Dspring.ai.chat.memory.repository.jdbc.compatible-version=1.0.0 to bypass the error — otherwise, the app won't start.

  3. In 1.1.x, if the schema still has the timestamp column with type TIMESTAMP, we simply block startup and show an error message with a link to the upgrade guide.

These are just my initial thoughts though — not sure if we really need this many steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MySQL/MariaDB schema TIMESTAMP precision causes message ordering issues in JdbcChatMemoryRepository
5 participants