Add example scripts to show how to run the model

models/llama3_1/README.md

@@ -34,7 +34,36 @@ Pre-requisites: Ensure you have `wget` installed. Then run the script: `./downlo
 
 Remember that the links expire after 24 hours and a certain amount of downloads. You can always re-request a link if you start seeing errors such as `403: Forbidden`.
 
-### Access to Hugging Face
+## Running the models
+
+You need to install the following dependencies (in addition to the `requirements.txt` in the root directory of this repository) to run the models:
+
+```
+pip install torch fairscale fire blobfile
+```
+
+After installing the dependencies, you can run the example scripts as follows:
+
+```bash
+#!/bin/bash
+
+PYTHONPATH=$(git rev-parse --show-toplevel) torchrun scripts/example_chat_completion.py <CHECKPOINT_DIR> <TOKENIZER_PATH>
+```
+
+The above script should be used with an Instruct (Chat) model. For running larger models with tensor parallelism, you should modify as:
+
+```bash
+#!/bin/bash
+
+NGPUS=8
+PYTHONPATH=$(git rev-parse --show-toplevel) torchrun \
+  --nproc_per_node=$NGPUS \
+  scripts/example_chat_completion.py <CHECKPOINT_DIR> <TOKENIZER_PATH> \
+  --model_parallel_size $NGPUS
+```
+
+
+## Access to Hugging Face
 
 We also provide downloads on [Hugging Face](https://huggingface.co/meta-llama), in both transformers and native `llama3` formats. To download the weights from Hugging Face, please follow these steps:
 

models/llama3_1/api/datatypes.py

@@ -11,8 +11,8 @@
 from pydantic import BaseModel, Field, validator
 
 from typing_extensions import Annotated
-from llama_models.datatypes import *  # noqa
-from llama_models.schema_utils import json_schema_type
+from ...datatypes import *  # noqa
+from ...schema_utils import json_schema_type
 
 
 @json_schema_type