Open
Description
Problem
Currently, the DAGs for the creation of filtered indexes (for image and audio) depend on the Ingestion Server. There is no reason we can not leave all that work to Airflow and it would be preferable to have fewer moving parts so it's also easier to debug when things go wrong.
Description
Move the create_and_populate_filtered_index
function out of the Ingestion Server to the create filtered index DAG in the Catalog.
openverse/ingestion_server/ingestion_server/indexer.py
Lines 465 to 471 in 41a1272
Additional context
This will be required down the line for other DAGs in the Search relevancy sandbox project.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
⛔ Blocked