Need way to add proprietary aiContext for knowledgeBase chats via lambda or backend

[lets-getitnow]

I have proprietary information that needs to be passed in the context for an AIConversation that calls the knowledgeBase via an appsync resolver.

The proprietary information (PROPVAR) itself does not directly need to be sent to the knowledgebase resolver, but it does need to be in the context for the main prompt.

I would strongly prefer for that information to be private and not publicly accessible.

The only technical solution I found is to pass the propriety information via the aiContext argument of the AIConversation in the front end which means PROPVAR has to be exposed to the public.

This is far from a desired solution.

I tried putting the AIConversation via client.conversations.chat.create() into a backend lambda but it warned of lack of JWT tokens.

Here is the general shape of my setup:

const schema = a.schema({
  // ********** AI **********
  knowledgeBase: a
    .query()
    .arguments({ 
      input: a.string(),
    })
    .handler(
      a.handler.custom({
        dataSource: "KnowledgeBaseDataSource",
        entry: "./resolvers/kbResolver.js",
      }),
    )
    .returns(a.string())
    .authorization((allow) => [allow.authenticated()]),

  chat: a.conversation({
    aiModel: a.ai.model("Claude 3.5 Haiku"),
    systemPrompt: systemPrompt,
    tools: [
      a.ai.dataTool({
        name: 'backendSearchDocumentation',
        description: '",
        query: a.ref('knowledgeBase'),
      }),
    ]

I can see some potential solutions (unless I've completely misunderstood your API/system, in which case I apologize, but after the amount of days I've spent researching this, I'm slightly confident in this request ).

1. allow execution to occur inside of the

chat: a.conversation({
    aiModel: a.ai.model("Claude 3.5 Haiku"),
    systemPrompt: systemPrompt,

1a. that execution could be limited to model calls and specifically only be to load aiContext. maybe something like

chat: a.conversation({
    aiModel: a.ai.model("Claude 3.5 Haiku"),
    systemPrompt: systemPrompt,
    aiContext: client.models.PROPTABLE.list({
      filter: { userId: { eq: event.user.userId } }).propvar

This probably breaks the simplicity of chat being inside the Schema. Which leads to:

1b. maybe expose a little more of chat: instead of (or in addition to) having it defined in the schema, upgrade it to a resolver so that finer grained control can be had?

2. allow the ai conversation to be run on the backend lambda since it does have IAM permission without the strict JWT check.

I think this request makes a lot of sense. AIConversations + Knowledgebase is one of the only end-to-end RAG solutions out there and it's a reason I want to stick with this architecture rather than roll my own RAG.

Given this, any of the solutions proposed above (or any other that satisfy my feature request) would be huge in unlocking the true potential of this very powerful system, thank you

[nadetastic]

Hi @lets-getitnow thank you for opening this issue. At a high level I believe I understand that you would like a way to run AIConversation on server side and have hit some limitations (mentioned that you got an error related to lack of JWTs).

To clarify, it is possible to run the client from generateClient inside a Lambda function and get the needed permissions. Have you had a chance to take a look at the documentations listed here? - https://docs.amplify.aws/javascript/build-a-backend/data/customize-authz/grant-lambda-function-access-to-api/

Let me know if you have any questions about it.

[lets-getitnow]

To clarify, it is possible to run the client from generateClient inside a Lambda function and get the needed permissions. Have you had a chance to take a look at the documentations listed here? - https://docs.amplify.aws/javascript/build-a-backend/data/customize-authz/grant-lambda-function-access-to-api/

See my reproduction at https://github.com/lets-getitnow/amplify_private_conversation

1. User is clearly logged in
2. Lambda is getting the event.identity
3. Using authMode:'userPool' everywhere

If in the lambda I switch to const client = generateClient<Schema>() which would fall back to the default IAM execution as defined in
).authorization(allow => [allow.resource(functionWithDataAccess)]);

Then the error is "ctx.stash.authFilter is not defined"

I'm not sure what you mean exactly by To clarify, it is possible to run the client from generateClient inside a Lambda function and get the needed permissions My reproduction follows the documentation you provided. Is there something incorrect about my reproduction technique? If so, then this is an easy fix.

I would love for you to be right, and if this is bug in amplify, great, then it can be fixed, and if not, then we are in a different situation. Looking forward to your feedback, thank you.

[lets-getitnow]

@nadetastic wondering if you have any feedback on this?

I have another use case in addition to the one listed above regarding need to keep proprietary information private.

In this second use case, I need to run a conversation in the backend so I can send emails based off the knowedgebase, and now I'm having to research the whole setup of a knowledgebase with agents so I can make a RetrieveAndGenerate call instead of using the conversations API, takes a few days out of my way.

I'm leaning more towards this, the fact that you can't call a conversation in the backend, being a bug rather than a feature request, unless again, there is something wrong in my configuration above, what do you think?

[nadetastic]

Hi @lets-getitnow thanks for your patience. After looking into this, it appears that this is a valid feature request. You are correct that calling the client in lambda utilizes IAM authorization, while a.conversation currently supports user pool authorization. I'm discussing with the team to identify an approach to adding context that should be private.

For the second use case, this would still be more of a feature request, since the conversation can only be configured with allow.owner() permissions, and the Cognito credentials required are not gracefully exposed into the backend/lambda environment. One approach of doing this your self in a backend environment is to use SSR and utilize the supported server side apis to make the requests with Amplify, but you would essentially be building this out yourself and not leveraging AI Kit.