Skip to content

Making snapshot store/restore rate dynamic #18069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

kkewwei
Copy link
Collaborator

@kkewwei kkewwei commented Apr 24, 2025

Description

How to use:

1.create repository

POST /_snapshot/my-fs-repository/
{
  "type": "fs",
  "settings": {
    "location": "/Users/xxx/Downloads/snap_data1/1",
     "max_restore_bytes_per_sec": "100mb",
     "max_snapshot_bytes_per_sec": "100mb"
  }
}

2.adjust the store and restore rate

POST /_snapshot/my-fs-repository/
{
  "type": "fs",
  "settings": {
    "location": "/Users/xxx/Downloads/snap_data1/1",
     "max_restore_bytes_per_sec": "50mb",
     "max_snapshot_bytes_per_sec": "50mb"
  }
}

or 

POST /_snapshot/my-fs-repository/
{
  "type": "fs",
  "settings": {
     "max_restore_bytes_per_sec": "50mb",
     "max_snapshot_bytes_per_sec": "50mb"
  }
}

Note:

  • Only max_snapshot_bytes_per_sec and max_restore_bytes_per_sec in settings can be changed, which will adjust the rate of ongoing snapshot. Other parameters within the settings can be left default or kept consistent.
  • If other parameters within the settings are also changed, it will have no impact on the rate of existing snapshots.

Related Issues

Resolves #17488

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for fc25085: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@kkewwei kkewwei force-pushed the snapshot_rate_dynamic3 branch from fc25085 to 214486f Compare April 24, 2025 13:25
@kkewwei kkewwei closed this May 22, 2025
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Storage Project Board May 22, 2025
@kkewwei kkewwei reopened this May 22, 2025
@github-project-automation github-project-automation bot moved this from ✅ Done to 🏗 In progress in Storage Project Board May 22, 2025
Copy link
Contributor

❌ Gradle check result for bd1f43c:

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@kkewwei kkewwei closed this May 23, 2025
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Storage Project Board May 23, 2025
@kkewwei kkewwei reopened this May 23, 2025
@github-project-automation github-project-automation bot moved this from ✅ Done to 🏗 In progress in Storage Project Board May 23, 2025
Copy link
Contributor

✅ Gradle check result for bd1f43c: SUCCESS

Copy link
Contributor

✅ Gradle check result for 17e3708: SUCCESS

Copy link
Contributor

✅ Gradle check result for 32c1425: SUCCESS

Copy link
Contributor

@bugmakerrrrrr bugmakerrrrrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

@github-project-automation github-project-automation bot moved this from 🏗 In progress to 👀 In review in Storage Project Board Jun 30, 2025
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: kkewwei <[email protected]>
Copy link
Contributor

❌ Gradle check result for 9c61127: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for 9c61127: SUCCESS

Copy link
Contributor

github-actions bot commented Jul 1, 2025

❌ Gradle check result for ad45900: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@kkewwei kkewwei closed this Jul 1, 2025
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Storage Project Board Jul 1, 2025
@kkewwei kkewwei reopened this Jul 1, 2025
@github-project-automation github-project-automation bot moved this from ✅ Done to 🏗 In progress in Storage Project Board Jul 1, 2025
Copy link
Contributor

github-actions bot commented Jul 1, 2025

✅ Gradle check result for ad45900: SUCCESS

@bugmakerrrrrr bugmakerrrrrr merged commit d990978 into opensearch-project:main Jul 1, 2025
58 of 60 checks passed
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Storage Project Board Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Snapshots
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

[Feature Request] Making snapshot store/restore rate dynamic
5 participants