End-to-end working example?

[Rich2020]

It would be great to have a complete working example. How to add models? How to set up a model server? Can we use open source or other models without using 'ask'? Can we use other models with the CLI and if so, how? etc. There are just so many questions and so little documentation & examples. I appreciate that this is a quickly evolving repository, however, so it is somewhat understandable, but it would be wonderful if some complete examples were available.

Or perhaps I have missed it?

[dosubot]

Hey @Rich2020! I'm here to help you with any bugs, questions, or contributions you have. If you need assistance, feel free to ask! Dosu

To provide a complete end-to-end working example, including how to add models, set up a model server, and use open source or other models with the CLI, follow these steps:

1. Install PaperQA2

pip install paper-qa

2. Set Up Environment Variables

Set up the necessary API keys for the LLM and metadata services:

export OPENAI_API_KEY=sk-...
export CROSSREF_API_KEY=your_crossref_api_key
export SEMANTIC_SCHOLAR_API_KEY=your_semantic_scholar_api_key

3. Add Models and Set Up a Model Server

You can use a locally hosted LLM server. For example, using llamafile:

1. Download and set up llamafile:

# Follow the instructions on https://github.com/Mozilla-Ocho/llamafile to download and set up
llamafile -cb -np 4 -a my-llm-model --embedding

2. Configure PaperQA2 to use the locally hosted model:

from paperqa import Settings, ask

local_llm_config = dict(
    model_list=dict(
        model_name="my_llm_model",
        litellm_params=dict(
            model="my-llm-model",
            api_base="http://localhost:8080/v1",
            api_key="sk-no-key-required",
            temperature=0.1,
            frequency_penalty=1.5,
            max_tokens=512,
        ),
    )
)

answer = ask(
    "What manufacturing challenges are unique to bispecific antibodies?",
    settings=Settings(
        llm="my-llm-model",
        llm_config=local_llm_config,
        summary_llm="my-llm-model",
        summary_llm_config=local_llm_config,
    ),
)
print(answer.formatted_answer)

4. Use the CLI

Navigate to a directory with your papers and use the pqa CLI:

cd my_papers
pqa ask 'What manufacturing challenges are unique to bispecific antibodies?'

To use a specific model and settings:

pqa --llm my-llm-model --temperature 0.5 ask 'What manufacturing challenges are unique to bispecific antibodies?'

5. Save and Use Custom Settings

Save custom settings:

pqa -s my_new_settings --temperature 0.5 --llm my-llm-model save

Use the saved settings:

pqa -s my_new_settings ask 'What manufacturing challenges are unique to bispecific antibodies?'

6. Full-Text Search

Save the index from a directory and give it a name:

pqa -i nanomaterials index

Search for papers about thermoelectrics:

pqa -i nanomaterials search thermoelectrics

Or ask a question:

pqa -i nanomaterials ask 'Are there nm scale features in thermoelectric materials?'

This example covers the installation, configuration, and usage of PaperQA2 with both the CLI and Python API, including setting up a local model server and using custom settings ^[1].

To continue talking to Dosu, mention @dosu.

[jamesbraza]

Thanks @Rich2020 appreciate the issue. We are working towards this slowly, please stay tuned