Media metadata not in database

[66Bunz]

Is there a reason why media metadata is not stored in the database when adding it, but the metadata is fetched every time the specific media is retrieved?

@FuzzyGrim

[andrebk]

I'm curious about this as well, there's been things I've found difficult to implement because metadata and tracking information exists in separate worlds. I know in #842 this was handled by getting the metadata and then enriching it with tracking data from the database. But it might be worth considering just keeping it all together in the database.

[FuzzyGrim]

I haven't really seen the need to store metadata in the database, but it might be necessary in the future for some functionalities. Although I suppose if we do store it, we'd need a recurring task to keep the data up to date. I think Trakt has a daily task, but I'm not sure that's feasible for a self-hosted application, given the rate limits.

[66Bunz]

As Jellyfin does, it could be possible to just save metadata when adding media, and then let the user decide every how much time the metadata needs to be refreshed. In Jellyfin the options are never, 30, 60, 90 days, but I think it could be done more often, it depends on the metadata source.
I think that having metadata in the db could decrease the general requests to the providers, especially with the integration with a new API. And could make possible to provide without additional requests more complete data.

[andrebk]

I haven't really seen the need to store metadata in the database, but it might be necessary in the future for some functionalities.

The main thing I've found it is needed for is filtering and sorting. That is currently done by the database queries, so you can't sort or filter on the metadata. So for example you can sort by your own score but not by the TMDB score, you can sort by your finish date but not by the release date, you can filter on your tracking state but you can't filter by genre, and so on.

You also can't do sorting and filtering on the frontend, since e.g. your list of movies is fetched in chunks as you scroll, so you can't sort by anything because you'll miss the movies that haven't been loaded yet.

You might not need to store metadata in the database to do these things. If the movie list is client side rendered it can fetch the movie data from an API and then sort and filter any which way in JS on the frontend. But that would be a major refactor compared to how it currently works, and might even bring its own problems for very large libraries since everything needs to be in memory on the frontend.

Although I suppose if we do store it, we'd need a recurring task to keep the data up to date

I don't know about the other APIs, but at least TMDB has endpoints for movies, TV, and people that gives you all the items that have changed, so you can fetch updated data just for those instead of for all items. Might still need a recurring "update all items" task, but maybe it can run less frequently. If we want to get real fancy the update interval could be dynamic, with more frequent updates for newer media or around a release date, and less frequent updates for cancelled or finished items.

[webysther]

Another point to keep the metadata for make sure if something change in the provider the app will not crash so badly.