feat: claude models usage, but from bedrock

[marcelovicentegc]

Hey, folks! @abi I've noticed that other people have opened PRs integrating this project with Bedrock (#322, #357 and #367) and I'm opening this PR either way to collect your feedback and whether or not its of your interest to have it merged and be one of the available integrations. Let me know! I'm willing to treat this PR carefully to match the expected standards.

Amazing project by the way. Great work putting this up 🙏

[abi]

Thanks!

I am interested in supporting bedrock. How are AWS credentials set with this PR? I also noticed a lot of copied code between the bedrock function and the regular Anthropic function. It would be nice to introduce less duplicated code.

[marcelovicentegc]

How are AWS credentials set with this PR?

They're supported through AWS's Python SDK (boto3). boto3 accepts credentials in many different ways. On this initial implementation, boto looks for the credentials implicitly on:

1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and/or AWS_SESSION_TOKEN
2. Shared credential file (~/.aws/credentials)
3. AWS config file (~/.aws/config)
4. Container credential provider (if deployed on AWS)
5. Instance metadata service on an Amazon EC2 instance that has an IAM role configured (if deployed on AWS)

We can make that explicit too by passing the credentials directly into the SDK's initialization method. Let me know what works the best for you 🤗

I also noticed a lot of copied code between the bedrock function and the regular Anthropic function. It would be nice to introduce less duplicated code.

Agreed! Will patch this PR with an abstraction to handle this 🙏 (PS: Done!)

[marcelovicentegc]

Not using the latest Claude 3.5 sonnet here (v2, 2024-10-22) because I still don't have access to it on AWS to test it - but should work either way.

@abi we could work on creating a dropdown menu or something on the front-end to allow users to select the model they want to use based on what's available on their environment. Let me know if something like this is already on the radar. It occurs to me that this would be specially useful for Bedrock since equivalent models such as DeepSeek R1 will be available there any time soon.

[abi]

Thanks! I'll take a look soon.