Skip to content

Backend: mock unit tests #908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jan 30, 2025
Merged

Backend: mock unit tests #908

merged 19 commits into from
Jan 30, 2025

Conversation

ezawadski
Copy link
Collaborator

@ezawadski ezawadski commented Jan 15, 2025

Mocking Cohere API behaviour to remove API Key requirement for backend Unit Tests

AI Description

This PR introduces several changes to the backend codebase, including updates to the model deployment classes, test fixtures, and utility functions.

Model Deployment Classes

  • The name, env_vars, rerank_enabled, list_models, and is_available methods have been converted to static methods in the AzureDeployment, BedrockDeployment, CohereDeployment, SageMakerDeployment, and SingleContainerDeployment classes.
  • The invoke_chat, invoke_chat_stream, and invoke_rerank methods now accept additional keyword arguments (**kwargs) in the AzureDeployment, BedrockDeployment, CohereDeployment, SageMakerDeployment, and SingleContainerDeployment classes.
  • The invoke_chat method in the CohereDeployment class now includes a rerank parameter.
  • The invoke_rerank method in the CohereDeployment class now returns a list of dictionaries, each containing an index and a relevance_score.
  • The invoke_chat_stream method in the CohereDeployment class now returns a list of dictionaries, each containing an id, results, and meta fields.
  • The invoke_chat_stream method in the SageMakerDeployment class now returns a list of dictionaries, each containing an id, results, and meta fields.
  • The invoke_chat_stream method in the SingleContainerDeployment class now returns a list of dictionaries, each containing an id, results, and meta fields.

Test Fixtures

  • The mock_available_model_deployments fixture now accepts a list of dictionaries (mock_event_stream) as an argument and returns a list of mocked deployments.
  • The mock_event_stream fixture returns a list of dictionaries, each containing an event_type and a generation_id.
  • The inject_events fixture returns an empty list.

Utility Functions

  • The get_deployment function in the utils.py module now catches both DeploymentNotFoundError and Exception exceptions.
  • The chunk function in the collate.py module now includes type annotations for its parameters and return type.
  • The to_dict function in the collate.py module now includes a return type annotation.

Other Changes

  • The create_organization function in the organization.py module now uses model_dump() instead of dict() to create an organization data object.
  • The with_deployment_name, with_user, with_agent, with_agent_tool_metadata, with_model, with_deployment_config, with_conversation_id, with_stream_start_ms, with_agent_id, and with_organization_id methods in the context.py module now return Self instead of "Context".
  • The select_deployments_prompt function in the prompts.py module now accepts a dictionary of BaseDeployment types as an argument.
  • The get_documents_to_rerank function in the conversation.py module now accepts a list of conversations and returns a list of strings.
  • The filter_conversations function in the conversation.py module now accepts a list of conversations and returns a list of conversations.
  • The test_create_agent function in the test_agent.py module now includes additional arguments (user and mock_cohere_list_models) and returns None.
  • The test_create_agent_with_tool_metadata function in the test_agent.py module now includes additional arguments (user and mock_cohere_list_models) and returns None.
  • The test_create_agent_missing_non_required_fields function in the test_agent.py module now includes additional arguments (user and mock_cohere_list_models) and returns None.
  • The test_update_agent function in the test_agent.py module now includes additional arguments (user and mock_cohere_list_models) and returns None.
  • The test_create_agent_missing_name function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_missing_model function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_missing_deployment function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_missing_user_id_header function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_invalid_deployment function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_deployment_not_in_db function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_invalid_tool function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_existing_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_agents function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_organization_agents function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_organization_agents_query_param function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_organization_agents_nonexistent_organization function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_private_agents function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_public_agents function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The list_public_and_private_agents function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_list_agents_with_pagination function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_nonexistent_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_public_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_private_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_private_agent_by_another_user function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_partial_update_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_with_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_with_tool_metadata_and_new_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_remove_existing_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_nonexistent_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_wrong_user function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_invalid_model function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_invalid_deployment function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_invalid_tool function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_private_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_public_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_change_visibility_to_public function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_change_visibility_to_private function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_change_visibility_to_private_delete_snapshot function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_delete_public_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_delete_private_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_cannot_delete_private_agent_not_belonging_to_user_id function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_cannot_delete_public_agent_not_belonging_to_user_id function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_fail_delete_nonexistent_agent function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_agent_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_update_agent_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_get_agent_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_delete_agent_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_fail_delete_nonexistent_agent_tool_metadata function in the test_agent.py module now includes an additional argument (user) and returns None.
  • The test_create_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_create_deployment_invalid_class_name function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_get_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_fail_get_nonexistent_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_list_deployments function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_list_deployments_empty function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_list_deployments_with_pagination function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_get_available_deployments function in the test_deployment.py module now includes an additional argument (user) and returns None.
  • The test_get_available_deployments_empty function in the test_deployment.py module now includes an additional argument (user) and returns None.
  • The test_update_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_update_deployment_partial function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_do_not_update_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_delete_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_delete_nonexistent_deployment function in the test_deployment.py module now includes an additional argument (session) and returns None.
  • The test_create_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_get_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_fail_get_nonexistent_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models_empty function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models_with_pagination function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_get_models_by_deployment_id function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_get_models_by_deployment_id_empty function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_get_models_by_deployment_id_with_pagination function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_update_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_update_model_partial function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_do_not_update_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_delete_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_delete_nonexistent_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_create_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_update_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_get_model function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models_empty function in the test_model.py module now includes an additional argument (session) and returns None.
  • The test_list_models_with_pagination function in the test_model.py module now

@codecov-commenter
Copy link

codecov-commenter commented Jan 24, 2025

Codecov Report

Attention: Patch coverage is 99.61240% with 1 line in your changes missing coverage. Please review.

Project coverage is 81.29%. Comparing base (75594fb) to head (fab6516).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/backend/tools/hybrid_search.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #908      +/-   ##
==========================================
+ Coverage   77.27%   81.29%   +4.02%     
==========================================
  Files         281      281              
  Lines       11484    11477       -7     
==========================================
+ Hits         8874     9330     +456     
+ Misses       2610     2147     -463     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ezawadski ezawadski force-pushed the feat/mock_unit_tests branch from c57fa36 to f629cbb Compare January 24, 2025 21:26
@ezawadski ezawadski changed the title [WIP] Backend: mock unit tests Backend: mock unit tests Jan 27, 2025
Copy link
Collaborator

@EugeneLightsOn EugeneLightsOn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a really great PR. I left some comments.

Copy link
Collaborator

@EugeneLightsOn EugeneLightsOn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to fix rerank_and_chunk method

Copy link
Collaborator

@EugeneLightsOn EugeneLightsOn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@ezawadski ezawadski merged commit 083a0af into main Jan 30, 2025
8 checks passed
@ezawadski ezawadski deleted the feat/mock_unit_tests branch January 30, 2025 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants