[Bug] KV cache quantization is not reflected in context length with TP enabled

[rlwj27-sketch]

Environment

Where are you using Unsloth?

• Studio desktop application
• Studio web UI (unsloth studio)
• Unsloth CLI
• Python package or notebook
• Colab or Kaggle

Operating system and version:

Windows 11

GPU model(s) and accelerator backend:

NVIDIA RTX 5090 + RTX 3070, CUDA, Tensor Parallelism (TP) enabled

Versions:

Unsloth Studio: [v0.1.802-beta](https://github.com/unslothai/unsloth/tree/v0.1.802-beta)

add: v0.1.803-beta also same

What happened?

Steps to reproduce:

1. Use a system with two GPUs (RTX 5090 + RTX 3070).
2. Enable Tensor Parallelism (TP) in Unsloth Studio.
3. Select a model and enable KV cache quantization (Q8 or Q4).
4. Load the model.

Expected behavior:

When KV cache quantization is set to Q8, the available context length should be calculated based on the reduced KV cache memory usage. For this setup, I would expect a context length of around 250k tokens with Q8 KV cache.

Actual behavior:

The KV cache quantization option can now be selected while TP is enabled, but the selected quantization does not appear to be reflected in the available context length.

Instead, the context limit appears to be calculated as if the KV cache were using FP16, or according to the context-length limit detected/calculated by the application without accounting for Q8 KV cache quantization.

In other words, Q8 KV cache is selectable, but the resulting context length still behaves as though the KV cache quantization is not being applied.

Model and operation involved:

Model: Qwen 3.8 27B UD_Q6_K_XL

Operation: Model loading/inference with Tensor Parallelism enabled and Q8/Q4 KV cache quantization selected.

Diagnostics and logs

0.05.997.011 I llama_context: constructing llama_context
0.05.997.015 I llama_context: n_seq_max = 1
0.05.997.016 I llama_context: n_ctx = 125440
0.05.997.016 I llama_context: n_ctx_seq = 125440
0.05.997.016 I llama_context: n_batch = 2048
0.05.997.016 I llama_context: n_ubatch = 512
0.05.997.017 I llama_context: causal_attn = 1
0.05.997.017 I llama_context: flash_attn = enabled
0.05.997.017 I llama_context: kv_unified = false
0.05.997.020 I llama_context: freq_base = 10000000.0
0.05.997.021 I llama_context: freq_scale = 1
0.05.997.021 I llama_context: n_rs_seq = 0
0.05.997.022 I llama_context: n_outputs_max = 1
0.05.997.022 I llama_context: n_outputs_max_per_seq = 1

Additional context

I will attach screenshots showing the selected KV cache quantization setting and the
context-length limit reported by Studio.

I tested the same configuration multiple times, so the exact detected context length shown in the screenshots and logs is slightly different between attempts. This is simply due to repeating the test and is not the issue itself. The main issue is that the available context length is significantly lower than expected even when Q8 KV cache is selected.

[rlwj27-sketch]

I also tested the same configuration with Tensor Parallelism (TP) disabled. In that case, Q8 KV cache works as expected and the available context length is calculated accordingly. The issue appears to occur only when TP is enabled.

Additionally, when TP is disabled, setting the context length to 200k with Q8 KV cache and loading the model correctly applies the 200k context length, although the sidebar displays it as "Auto".

When TP is enabled, however, the context length appears to be forced to the limit calculated for FP16 KV cache instead of taking the selected Q8 KV cache quantization into account.

[tp/off set 200k q8 kv]

[tp/on set 200k q8 kv]