-
Notifications
You must be signed in to change notification settings - Fork 417
Backend: mock unit tests #908
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ee32c24
to
4d862a4
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #908 +/- ##
==========================================
+ Coverage 77.27% 81.29% +4.02%
==========================================
Files 281 281
Lines 11484 11477 -7
==========================================
+ Hits 8874 9330 +456
+ Misses 2610 2147 -463 ☔ View full report in Codecov by Sentry. |
c57fa36
to
f629cbb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a really great PR. I left some comments.
src/backend/tests/unit/model_deployments/mock_deployments/mock_base.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to fix rerank_and_chunk method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me
Mocking Cohere API behaviour to remove API Key requirement for backend Unit Tests
AI Description
This PR introduces several changes to the backend codebase, including updates to the model deployment classes, test fixtures, and utility functions.
Model Deployment Classes
name
,env_vars
,rerank_enabled
,list_models
, andis_available
methods have been converted to static methods in theAzureDeployment
,BedrockDeployment
,CohereDeployment
,SageMakerDeployment
, andSingleContainerDeployment
classes.invoke_chat
,invoke_chat_stream
, andinvoke_rerank
methods now accept additional keyword arguments (**kwargs
) in theAzureDeployment
,BedrockDeployment
,CohereDeployment
,SageMakerDeployment
, andSingleContainerDeployment
classes.invoke_chat
method in theCohereDeployment
class now includes arerank
parameter.invoke_rerank
method in theCohereDeployment
class now returns a list of dictionaries, each containing anindex
and arelevance_score
.invoke_chat_stream
method in theCohereDeployment
class now returns a list of dictionaries, each containing anid
,results
, andmeta
fields.invoke_chat_stream
method in theSageMakerDeployment
class now returns a list of dictionaries, each containing anid
,results
, andmeta
fields.invoke_chat_stream
method in theSingleContainerDeployment
class now returns a list of dictionaries, each containing anid
,results
, andmeta
fields.Test Fixtures
mock_available_model_deployments
fixture now accepts a list of dictionaries (mock_event_stream
) as an argument and returns a list of mocked deployments.mock_event_stream
fixture returns a list of dictionaries, each containing anevent_type
and ageneration_id
.inject_events
fixture returns an empty list.Utility Functions
get_deployment
function in theutils.py
module now catches bothDeploymentNotFoundError
andException
exceptions.chunk
function in thecollate.py
module now includes type annotations for its parameters and return type.to_dict
function in thecollate.py
module now includes a return type annotation.Other Changes
create_organization
function in theorganization.py
module now usesmodel_dump()
instead ofdict()
to create an organization data object.with_deployment_name
,with_user
,with_agent
,with_agent_tool_metadata
,with_model
,with_deployment_config
,with_conversation_id
,with_stream_start_ms
,with_agent_id
, andwith_organization_id
methods in thecontext.py
module now returnSelf
instead of"Context"
.select_deployments_prompt
function in theprompts.py
module now accepts a dictionary ofBaseDeployment
types as an argument.get_documents_to_rerank
function in theconversation.py
module now accepts a list of conversations and returns a list of strings.filter_conversations
function in theconversation.py
module now accepts a list of conversations and returns a list of conversations.test_create_agent
function in thetest_agent.py
module now includes additional arguments (user
andmock_cohere_list_models
) and returnsNone
.test_create_agent_with_tool_metadata
function in thetest_agent.py
module now includes additional arguments (user
andmock_cohere_list_models
) and returnsNone
.test_create_agent_missing_non_required_fields
function in thetest_agent.py
module now includes additional arguments (user
andmock_cohere_list_models
) and returnsNone
.test_update_agent
function in thetest_agent.py
module now includes additional arguments (user
andmock_cohere_list_models
) and returnsNone
.test_create_agent_missing_name
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_missing_model
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_missing_deployment
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_missing_user_id_header
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_invalid_deployment
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_deployment_not_in_db
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_invalid_tool
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_existing_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_agents
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_organization_agents
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_organization_agents_query_param
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_organization_agents_nonexistent_organization
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_private_agents
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_public_agents
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.list_public_and_private_agents
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_list_agents_with_pagination
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_nonexistent_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_public_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_private_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_private_agent_by_another_user
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_partial_update_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_with_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_with_tool_metadata_and_new_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_remove_existing_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_nonexistent_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_wrong_user
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_invalid_model
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_invalid_deployment
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_invalid_tool
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_private_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_public_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_change_visibility_to_public
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_change_visibility_to_private
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_change_visibility_to_private_delete_snapshot
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_delete_public_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_delete_private_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_cannot_delete_private_agent_not_belonging_to_user_id
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_cannot_delete_public_agent_not_belonging_to_user_id
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_fail_delete_nonexistent_agent
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_agent_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_update_agent_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_get_agent_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_delete_agent_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_fail_delete_nonexistent_agent_tool_metadata
function in thetest_agent.py
module now includes an additional argument (user
) and returnsNone
.test_create_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_create_deployment_invalid_class_name
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_get_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_fail_get_nonexistent_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_list_deployments
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_list_deployments_empty
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_list_deployments_with_pagination
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_get_available_deployments
function in thetest_deployment.py
module now includes an additional argument (user
) and returnsNone
.test_get_available_deployments_empty
function in thetest_deployment.py
module now includes an additional argument (user
) and returnsNone
.test_update_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_update_deployment_partial
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_do_not_update_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_delete_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_delete_nonexistent_deployment
function in thetest_deployment.py
module now includes an additional argument (session
) and returnsNone
.test_create_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_get_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_fail_get_nonexistent_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models_empty
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models_with_pagination
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_get_models_by_deployment_id
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_get_models_by_deployment_id_empty
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_get_models_by_deployment_id_with_pagination
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_update_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_update_model_partial
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_do_not_update_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_delete_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_delete_nonexistent_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_create_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_update_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_get_model
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models_empty
function in thetest_model.py
module now includes an additional argument (session
) and returnsNone
.test_list_models_with_pagination
function in thetest_model.py
module now