Skip to content

[New Model]: Snowflake Arctic Embed (Family) #16649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Apr 18, 2025
Merged

[New Model]: Snowflake Arctic Embed (Family) #16649

merged 8 commits into from
Apr 18, 2025

Conversation

noooop
Copy link
Contributor

@noooop noooop commented Apr 15, 2025 •
edited by github-actions bot
Loading

Summary

Background

Snowflake Arctic Embed (Family) has many architectures.

The following model architecture is BertModel, currently vllm supports

https://huggingface.co/Snowflake/snowflake-arctic-embed-xs
https://huggingface.co/Snowflake/snowflake-arctic-embed-s
https://huggingface.co/Snowflake/snowflake-arctic-embed-m
https://huggingface.co/Snowflake/snowflake-arctic-embed-l
https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v1.5 <- is_matryoshka

The following model architecture is XLMRobertaModel, currently vllm supports
https://huggingface.co/Snowflake/snowflake-arctic-embed-l-v2.0 <- is_matryoshka

The following model architecture is NomicBertModel, which vllm does not support
https://huggingface.co/Snowflake/snowflake-arctic-embed-m-long

The following model architecture is GteModel, which vllm does not support
https://huggingface.co/Snowflake/snowflake-arctic-embed-m-v2.0 <- is_matryoshka

And three models of v1.5 and v2.0 support matryoshka

NomicBertModel

  • position_embedding_type: Rotary+ DynamicNTKRotaryEmbedding
  • mlp: gate_up_proj + SiluAndMul
  • All linear layers do not apply bias

GteModel

  • position_embedding_type: Rotary
  • mlp: gate_up_proj + GeluAndMul
  • gate_up_proj do not apply bias

FIX #7792

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@noooop
Copy link
Contributor Author

noooop commented Apr 15, 2025

@DarkLight1337

Currently the following tests can be passed locally:

  • tests/models/embedding/language/test_snowflake_arctic_embed.py

@DarkLight1337
Copy link
Member

To get the model to pass CI, please add this model to the test files as mentioned here: https://docs.vllm.ai/en/latest/contributing/model/tests.html

@noooop noooop force-pushed the snowflake-arctic-embed branch from 4b87ea0 to 5d5886e Compare April 15, 2025 10:34
maxdebayser
maxdebayser reviewed Apr 15, 2025
View reviewed changes
maxdebayser
maxdebayser reviewed Apr 15, 2025
View reviewed changes
maxdebayser
maxdebayser suggested changes Apr 15, 2025
View reviewed changes
Copy link
Contributor

@maxdebayser maxdebayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good to me, thanks for contributing. I left a suggestion and a question in the comments.

@noooop noooop force-pushed the snowflake-arctic-embed branch from 270c966 to c9989a7 Compare April 16, 2025 05:30
maxdebayser
maxdebayser approved these changes Apr 16, 2025
View reviewed changes
Copy link
Contributor

@maxdebayser maxdebayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@noooop
Copy link
Contributor Author

noooop commented Apr 17, 2025

@DarkLight1337

ready to final review

@DarkLight1337
Copy link
Member

That sounds good

@DarkLight1337
Copy link
Member

I think we should also add a guide for vLLM users to set that in the meantime

@noooop
Copy link
Contributor Author

noooop commented Apr 17, 2025

I think we should also add a guide for vLLM users to set that in the meantime

I will draft a document soon

@noooop noooop mentioned this pull request Apr 17, 2025
Merged
DarkLight1337
DarkLight1337 reviewed Apr 17, 2025
View reviewed changes
DarkLight1337
DarkLight1337 reviewed Apr 17, 2025
View reviewed changes
DarkLight1337
DarkLight1337 reviewed Apr 17, 2025
View reviewed changes
@mergify Mergify
Copy link

mergify bot commented Apr 18, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Apr 18, 2025
@noooop noooop force-pushed the snowflake-arctic-embed branch from 18a7763 to 613153e Compare April 18, 2025 05:03
@mergify mergify bot removed the needs-rebase label Apr 18, 2025
DarkLight1337
DarkLight1337 reviewed Apr 18, 2025
View reviewed changes
@DarkLight1337 DarkLight1337 enabled auto-merge (squash) April 18, 2025 05:21
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Apr 18, 2025
auto-merge was automatically disabled April 18, 2025 05:22

Head branch was pushed to by a user without write access

@noooop
Copy link
Contributor Author

noooop commented Apr 18, 2025

Thanks for reviewing

@vllm-bot vllm-bot merged commit 3d3ab36 into vllm-project:main Apr 18, 2025
43 of 47 checks passed
yangw-dev pushed a commit to yangw-dev/vllm that referenced this pull request Apr 21, 2025
@noooop @yangw-dev
[New Model]: Snowflake Arctic Embed (Family) (vllm-project#16649)
ee3b53c 
Signed-off-by: Yang Wang <[email protected]>
jikunshang pushed a commit to jikunshang/vllm that referenced this pull request Apr 29, 2025
@noooop @dbyoung18
[New Model]: Snowflake Arctic Embed (Family) (vllm-project#16649)
f8b5599
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025
@noooop @lk-chen
[New Model]: Snowflake Arctic Embed (Family) (vllm-project#16649)
b5cbed1
adobrzyn pushed a commit to HabanaAI/vllm-fork that referenced this pull request Apr 30, 2025
@noooop @adobrzyn
[New Model]: Snowflake Arctic Embed (Family) (vllm-project#16649)
e9c7dcd 
Signed-off-by: Agata Dobrzyniewicz <[email protected]>
RichardoMrMu pushed a commit to RichardoMrMu/vllm that referenced this pull request May 12, 2025
@noooop
[New Model]: Snowflake Arctic Embed (Family) (vllm-project#16649)
f64c867 
Signed-off-by: Mu Huai <[email protected]>
@ckhordiasma ckhordiasma mentioned this pull request May 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@maxdebayser maxdebayser maxdebayser approved these changes

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[New Model]: Snowflake Arctic Embed (Family)
4 participants
@noooop @DarkLight1337 @maxdebayser @vllm-bot