Skip to content

Commit b1616ff

Browse files
minleminzuilifuhuang
authored andcommitted
doc: fix the erroneous documents and example codes about Alibaba-NLP/gme-Qwen2-VL-2B-Instruct (sgl-project#6199)
1 parent cd17ea0 commit b1616ff

File tree

3 files changed

+39
-7
lines changed

3 files changed

+39
-7
lines changed

docs/backend/openai_api_embeddings.ipynb

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -171,6 +171,14 @@
171171
"source": [
172172
"terminate_process(embedding_process)"
173173
]
174+
},
175+
{
176+
"cell_type": "markdown",
177+
"metadata": {},
178+
"source": [
179+
"## Multi-Modal Embedding Model\n",
180+
"Please refer to [Multi-Modal Embedding Model](../supported_models/embedding_models.md)"
181+
]
174182
}
175183
],
176184
"metadata": {

docs/supported_models/embedding_models.md

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,42 @@ SGLang provides robust support for embedding models by integrating efficient ser
66
They are executed with `--is-embedding` and some may require `--trust-remote-code`
77
```
88

9-
## Example launch Command
9+
## Example Launch Command
1010

1111
```shell
1212
python3 -m sglang.launch_server \
13-
--model-path Alibaba-NLP/gme-Qwen2-VL-2B-Instruct \ # example HF/local path
13+
--model-path Alibaba-NLP/gme-Qwen2-VL-2B-Instruct \
1414
--is-embedding \
1515
--host 0.0.0.0 \
16-
--port 30000 \
16+
--chat-template gme-qwen2-vl \
17+
--port 30000
1718
```
19+
## Example Client Request
20+
```python
21+
import requests
22+
23+
url = "http://127.0.0.1:30000"
24+
25+
text_input = "Represent this image in embedding space."
26+
image_path = "https://huggingface.co/datasets/liuhaotian/llava-bench-in-the-wild/resolve/main/images/023.jpg"
27+
28+
payload = {
29+
"model": "gme-qwen2-vl",
30+
"input": [
31+
{
32+
"text": text_input
33+
},
34+
{
35+
"image": image_path
36+
}
37+
],
38+
}
39+
40+
response = requests.post(url + "/v1/embeddings", json=payload).json()
41+
42+
print("Embeddings:", [x.get("embedding") for x in response.get("data", [])])
43+
```
44+
1845

1946
## Supporting Matrixs
2047

examples/runtime/multimodal_embedding.py

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,7 @@
1010

1111
payload = {
1212
"model": "gme-qwen2-vl",
13-
"input": [
14-
{"type": "text", "text": text_input},
15-
{"type": "image", "url": image_path},
16-
],
13+
"input": [{"text": text_input}, {"image": image_path}],
1714
}
1815

1916
response = requests.post(url + "/v1/embeddings", json=payload).json()

0 commit comments

Comments
 (0)