[UI/Feature] Switch MWAS Section to Use MWAS API

[Stufedpanda]

Overview

The virome section currently includes pre-calculated MWAS (Metagenome-Wide Association Study) data for all SRA runs in the knowledge graph. However, this data is static and would require manual recalculation whenever new SRA runs are added or the knowledge graph is updated. A dynamic MWAS API has been developed to calculate MWAS data on the fly, enabling real-time computation for all SRA runs. This issue involves transitioning the virome section to use the dynamic MWAS API, ensuring seamless updates and scalability as the knowledge graph grows.

Background / Context

The current implementation relies on pre-calculated MWAS data, which has several limitations:

• Static Nature: The data does not automatically update when new SRA runs are added to the knowledge graph/needs to be manually updated.
• Scalability Issues: As the knowledge graph grows, pre-calculating MWAS data for all SRA runs becomes increasingly impractical.

The newly developed MWAS API addresses these limitations by:

• Calculating MWAS results dynamically for all SRA runs in the knowledge graph.
• Automatically incorporating new or updated SRA runs without requiring manual recalculation.
• Providing a scalable solution that adapts to the growing size of the knowledge graph.

Hypothesis

Switching to the dynamic MWAS API will improve the scalability, maintainability, and responsiveness of the virome section, enabling real-time MWAS calculations for all SRA runs and simplifying the process of adding or updating the knowledge graph.

Experiment

1, API Integration:

• Integrate the MWAS API into the virome section backend.
• Ensure the API can handle MWAS calculations for the entire set of SRA runs in the knowledge graph.

2. Frontend Updates:

• Modify the frontend to request MWAS data dynamically from the API whenever the advanced virome section is accessed.
• Display loading states or placeholders while the API calculates results.

3. Performance Optimization:

• Implement caching mechanisms to store frequently accessed MWAS results and reduce redundant API calls.
• Optimize the API response time to ensure a smooth user experience, even with large datasets.

4. Testing:

• Test the integration with the current set of SRA runs to ensure accuracy and responsiveness.
• Verify that the API can handle the addition of new SRA runs without requiring manual intervention.

Controls

• Ensure backward compatibility with existing functionality in the virome section.
• Monitor API performance to avoid timeouts or excessive computational load, especially as the number of SRA runs increases.

Expected Outcome

• The virome section uses the dynamic MWAS API to calculate and display results in real-time for all SRA runs.
• New or updated SRA runs are automatically incorporated into MWAS calculations without requiring manual recalculation.
• The system is scalable and performs well even as the knowledge graph grows.

Open Questions

• How should we handle cases where the API calculation takes longer than expected (e.g., timeouts or progress indicators)?
• Should we implement a hybrid approach, using pre-calculated data for initial loading and the API for updates?
• Are there specific caching strategies we should prioritize to improve performance for frequently accessed data?

References

[placeholder]