Skip to content

update models docs #1167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 12, 2024
Merged

update models docs #1167

merged 1 commit into from
Jul 12, 2024

Conversation

ebsmothers
Copy link
Contributor

A bunch of miscellaneous documentation changes to torchtune/models. Main changes:

  • Provide explicit download instructions for all model variants (at least all the ones that have training configs. We can consider adding the others too, but left it out since there's not an easy one-liner to use them anyways)
  • Actually document lora_dropout in builders where it's used (we should add it to the other builders where it's missing, can do that later though)
  • Minor clarifications and formatting in clip_vision_encoder and TiledTokenPositionalEmbedding
  • Making clip_vit_224_transform private since I don't think it's worth documenting rn (and it's not used anywhere atm)
  • Improve rendering of our model-specific SentencePiece tokenizers' tokenize_messages docstrings
  • Remove scale_hidden_dim_for_mlp from public APIs (similarly, I don't think it's worth documenting/exposing rn)
  • Fix return type for Mistral 7B classifier

Copy link

pytorch-bot bot commented Jul 12, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1167

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit d03cb05 with merge base c903040 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 12, 2024
@pbontrager
Copy link
Contributor

Is -hf-token <HF_TOKEN> required for phi3, gemma, and mistral? I thought it was only required for models where you had to get special approval for the weights.

@ebsmothers
Copy link
Contributor Author

Is -hf-token <HF_TOKEN> required for phi3, gemma, and mistral? I thought it was only required for models where you had to get special approval for the weights.

Oh interesting.. I just copied from the example commands in our configs, all of which have it included. Just tested these 3 -- seems like Phi3 does not require it but the other two do. If it's all the same to you, maybe I'll leave it in for now and we can separately update the Phi3 commands across the board later on; what do you think?

Copy link
Contributor

@joecummings joecummings left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to do an overhaul of our entire models page to help people find what models we support, but this is a good improvement for our current release.

@ebsmothers ebsmothers merged commit f292b14 into pytorch:main Jul 12, 2024
29 checks passed
@ebsmothers ebsmothers deleted the models-docs branch July 12, 2024 17:07
maximegmd pushed a commit to maximegmd/torchtune that referenced this pull request Jul 13, 2024
pbontrager pushed a commit that referenced this pull request Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants