Skip to content

Commit 105505f

Browse files
xiezhq-hermannjimoosciuc
authored andcommitted
bug fix for hicache host eviction (sgl-project#4989)
1 parent c1d8045 commit 105505f

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

python/sglang/srt/mem_cache/hiradix_cache.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -205,9 +205,9 @@ def evict_host(self, num_tokens: int):
205205
# only evict the host value of evicted nodes
206206
if not x.evicted:
207207
continue
208-
assert x.lock_ref == 0 and x.host_value is not None
209208

210-
assert self.cache_controller.evict_host(x.host_value) > 0
209+
num_evicted += self.cache_controller.evict_host(x.host_value)
210+
211211
for k, v in x.parent.children.items():
212212
if v == x:
213213
break

0 commit comments

Comments
 (0)