Skip to content

Move database cache rebuild to a background task with polling (13) #18922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

AndyButland
Copy link
Contributor

Prerequisites

  • I have added steps to test this contribution in the description below

This was raised as an issue in Umbraco Cloud for large sites where the time to rebuild the database cache is quite long. It may go over the hard limit imposed by Cloudflare and although the work continues, the user has no indication that it's completed.

We already implemented this for 15 in #18496 but it was felt useful to backport to 13 to support customers on that version.

Description

This PR updates the database cache rebuild operation to move it from a single request/response to a pattern where we submit the task, run in in the background and poll for completion.

It is based on what we are already doing for Examine index rebuilds, so it's a pattern and technique already in use in the CMS that I've just applied to this operation.

Testing

To test you can use the UI under Settings > Published Status and click the Rebuild Database Cache button. If you delete all records from cmsContentNu first you can see that they are recreated when the operation is completed.

To simulate a long running rebuild operation, you can add a Thread.Sleep(20000); into PublishedSnapshotService.PerformRebuild().

@Copilot Copilot AI review requested due to automatic review settings April 3, 2025 08:01
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR moves the database cache rebuild operation to a background task with polling to improve user feedback and avoid request timeouts in large sites. Key changes include:

  • Modifying the client controller to poll for rebuild status asynchronously.
  • Adding a new API endpoint that initiates the rebuild operation on a background thread.
  • Updating the snapshot service and interface to support background execution via runtime cache flags.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/Umbraco.Web.UI.Client/src/views/dashboard/settings/publishedsnapshotcache.controller.js Updated API endpoint call and added polling logic for asynchronous status updates.
src/Umbraco.Web.BackOffice/Controllers/PublishedSnapshotCacheStatusController.cs Introduced a new POST endpoint for background rebuild with appropriate status checks.
src/Umbraco.PublishedCache.NuCache/PublishedSnapshotStatus.cs Returns a consistent "Rebuild in progress" message while rebuilding.
src/Umbraco.PublishedCache.NuCache/PublishedSnapshotService.cs Modified rebuild operation to use a background task along with runtime cache flagging.
src/Umbraco.Core/PublishedCache/IPublishedSnapshotService.cs Extended the interface to support background rebuild operations.
Comments suppressed due to low confidence (1)

src/Umbraco.PublishedCache.NuCache/PublishedSnapshotService.cs:36

  • [nitpick] The constant name could be more descriptive. Consider renaming it to something like 'DatabaseCacheRebuildLockKey' to better reflect its purpose.
private const string IsRebuildingDatabaseCacheRuntimeCacheKey = "temp_database_cache_rebuild_op";

AndyButland and others added 3 commits April 3, 2025 10:04
@AndyButland AndyButland requested a review from Migaroez April 3, 2025 08:12
Copy link
Contributor

@Migaroez Migaroez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as described and code is sensible

@Migaroez Migaroez merged commit a3db456 into v13/dev Apr 3, 2025
18 of 19 checks passed
@Migaroez Migaroez deleted the v13/task/backport-long-running-database-rebuild-operation-to-submit-and-poll branch April 3, 2025 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants