Open
Description
Hi team,
The current implementation of toRedisZSET(kvs: RDD[(String, String)])
is useful for simple use cases but is not sufficient for high-throughput or binary-safe workloads.
We’d like to propose support for a new method:
toRedisByteZSET(kvs: RDD[(Array[Byte], (Double, Array[Byte]))])
This enables writing Redis ZSETs using byte array keys and members, which is critical for:
- Binary-safe use cases (e.g., Protobuf, embeddings, hashed IDs)
- Avoiding costly string serialization and base64 encoding
We’ve implemented this function following the same pattern as toRedisByteHASHes
, including host-aware batching and pipelined writes. We'd appreciate it if the team could review and prioritize this addition.
Thank you!
Metadata
Metadata
Assignees
Labels
No labels