[Bug]: I performed CDC operations between two standalone clusters of Milvus, but after creating the task, the background program reported an error: "fail to get the ts data".

[waitwindy]

Current Behavior

When creating a CDC synchronization task, the following error occurred: [PANIC] [reader/etcd_op.go:769] ["fail to get the ts data"] [prefix=by-dev/kv/gid/timestamp] [len=0] [stack="github.com/zilliztech/milvus-cdc/core/reader.(*EtcdOp).GetAllDroppedObj\n\t/home/runner/work/milvus-cdc/milvus-cdc/core/reader/etcd_op.go:769\ngithub.com/zilliztech/milvus-cdc/server.(*MetaCDC).newReplicateEntity\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:898\ngithub.com/zilliztech/milvus-cdc/server.(*MetaCDC).startInternal\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:712\ngithub.com/zilliztech/milvus-cdc/server.(*MetaCDC).Create\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:532\ngithub.com/zilliztech/milvus-cdc/server.init.0.func2\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/handle_map.go:45\ngithub.com/zilliztech/milvus-cdc/server.(*CDCServer).handleRequest\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/server.go:130\ngithub.com/zilliztech/milvus-cdc/server.(*CDCServer).Run.(*CDCServer).getCDCHandler.func2\n\t/home/runner/work/milvus-cdc/milvus-cdc/server/server.go:86\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2141\nnet/http.(*ServeMux).ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2519\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2943\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2014"]
[2025/06/26 17:53:23.592 +08:00] [INFO] [server/cdc_impl.go:405] ["create request done"]
2025/06/26 17:53:23 http: panic serving 192.168.125.1:55226: fail to get the ts data
goroutine 177 [running]:
net/http.(*conn).serve.func1()
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:1873 +0xb9
panic({0x1a5f6e0?, 0xc0004439b0?})
/opt/hostedtoolcache/go/1.21.13/x64/src/runtime/panic.go:920 +0x270
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x2?, 0x2?, {0x0?, 0x0?, 0xc0000c6be0?})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:196 +0x54
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0004e5450, {0xc0004b0880, 0x2, 0x2})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.27.0/zapcore/entry.go:262 +0x3ec
go.uber.org/zap.(*Logger).Panic(0xc000068c00?, {0x1d66781?, 0x0?}, {0xc0004b0880, 0x2, 0x2})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.27.0/logger.go:285 +0x51
github.com/zilliztech/milvus-cdc/core/log.Panic({0x1d66781?, 0x5?}, {0xc0004b0880?, 0x2?, 0x2?})
/home/runner/work/milvus-cdc/milvus-cdc/core/log/log.go:87 +0x85
github.com/zilliztech/milvus-cdc/core/reader.(*EtcdOp).GetAllDroppedObj(0xc00075b600)
/home/runner/work/milvus-cdc/milvus-cdc/core/reader/etcd_op.go:769 +0x319
github.com/zilliztech/milvus-cdc/server.(*MetaCDC).newReplicateEntity(0xc000690580, 0xc0006d5000)
/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:898 +0x1522
github.com/zilliztech/milvus-cdc/server.(*MetaCDC).startInternal(0xc000690580, 0xc0006d5000, 0x0)
/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:712 +0x1f5
github.com/zilliztech/milvus-cdc/server.(*MetaCDC).Create(0xc000690580, 0xc000453880)
/home/runner/work/milvus-cdc/milvus-cdc/server/cdc_impl.go:532 +0xe8a
github.com/zilliztech/milvus-cdc/server.init.0.func2({0x1fff358?, 0xc000690580?}, {0x1a265c0?, 0xc000453880?})
/home/runner/work/milvus-cdc/milvus-cdc/server/handle_map.go:45 +0x3a
github.com/zilliztech/milvus-cdc/server.(*CDCServer).handleRequest(0xc00052ac78, 0xc000600168, {0x1feda60, 0xc00073e0e0})
/home/runner/work/milvus-cdc/milvus-cdc/server/server.go:130 +0x336
github.com/zilliztech/milvus-cdc/server.(*CDCServer).Run.(*CDCServer).getCDCHandler.func2({0x1feda60, 0xc00073e0e0}, 0xc0008f0c00)
/home/runner/work/milvus-cdc/milvus-cdc/server/server.go:86 +0x368
net/http.HandlerFunc.ServeHTTP(0x10?, {0x1feda60?, 0xc00073e0e0?}, 0xc000688cac?)
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2141 +0x29
net/http.(*ServeMux).ServeHTTP(0x4138e5?, {0x1feda60, 0xc00073e0e0}, 0xc0008f0c00)
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2519 +0x142
net/http.serverHandler.ServeHTTP({0x1fe9a78?}, {0x1feda60?, 0xc00073e0e0?}, 0x6?)
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2943 +0x8e
net/http.(*conn).serve(0xc0006141b0, {0x1ff53c8, 0xc0008e30b0})
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:2014 +0x5f4
created by net/http.(*Server).Serve in goroutine 1
/opt/hostedtoolcache/go/1.21.13/x64/src/net/http/server.go:3091 +0x5cb

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

[SimFG]

This problem should have been fixed in the latest main branch. What version are you using?

[waitwindy]

The version is Release v2.0.0-rc7. Are there any plans to release a new version recently?

[SimFG]

You can try the latest main branch

[waitwindy]

After compiling with the main branch, this issue indeed no longer occurs. However, a new problem has been found:
[2025/06/27 16:28:44.964 +08:00] [WARN] [kafka/kafka_consumer.go:139] ["consume msg failed"] [topic=by-dev-replicate-msg] [groupID=cdc-def9f29e6b9b4e83ae545c8ee07a5052-8444-by-dev-replicate-msg_2fc494d823ca4b159a02f4510e84f789v0-true] [error="Local: Timed out"] .I verified on the server where the target cluster is located that Kafka can read and write. In the cdc.yml configuration, except for the modified IP, all other settings are default.

[SimFG]

@waitwindy This warning will not have any impact.

[waitwindy]

Under what circumstances will this warning be generated?