Skip to content

Perf: copying dir contents in parallel#1026

Merged
RyanZim merged 1 commit intojprichardson:masterfrom
SukkaW:copy-speed
Nov 28, 2023
Merged

Perf: copying dir contents in parallel#1026
RyanZim merged 1 commit intojprichardson:masterfrom
SukkaW:copy-speed

Conversation

@SukkaW
Copy link
Copy Markdown
Contributor

@SukkaW SukkaW commented Oct 27, 2023

The PR uses a technique similar to #885 to speed up the recursive copy operation.

As fs-extra is built on top of graceful-fs, this change won't cause EMFILE: too many open files as graceful-fs will exponentially backoff and retry.

@RyanZim RyanZim requested review from RyanZim and manidlou October 27, 2023 17:59
RyanZim
RyanZim approved these changes Nov 10, 2023
View reviewed changes
Copy link
Copy Markdown
Collaborator

@RyanZim RyanZim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change looks good to me; but would like a second pair of eyes here to make sure I'm not missing something.

@RyanZim
Copy link
Copy Markdown
Collaborator

RyanZim commented Nov 10, 2023

@manidlou bump

manidlou
manidlou approved these changes Nov 18, 2023
View reviewed changes
Copy link
Copy Markdown
Collaborator

@manidlou manidlou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@RyanZim RyanZim merged commit d293448 into jprichardson:master Nov 28, 2023
@RyanZim
Copy link
Copy Markdown
Collaborator

RyanZim commented Nov 28, 2023

Published in v11.2.0!

@SukkaW
Copy link
Copy Markdown
Contributor Author

SukkaW commented Nov 28, 2023 •
edited
Loading

@RyanZim ❤️

Too bad I can't port this change to Node.js' fs#cp. Node.js doesn't have graceful-fs, thus EMFILEcan not be avoided.

@SukkaW SukkaW mentioned this pull request Nov 28, 2023
Merged
@RyanZim
Copy link
Copy Markdown
Collaborator

RyanZim commented Nov 28, 2023

Too bad I can't port this change to Node.js' fs#cp. Node.js doesn't have graceful-fs, thus EMFILEcan not be avoided.

You'd have to implement a concurrency limit yourself, I guess.

valentineus pushed a commit to valentineus/strapi-plugin-checkbox-list that referenced this pull request Feb 5, 2026
@valentineus
Update dependency fs-extra to v11 (#6)
9012c5c 
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [fs-extra](https://github.com/jprichardson/node-fs-extra) | [`^10.0.0` → `^11.0.0`](https://renovatebot.com/diffs/npm/fs-extra/10.1.0/11.3.3) | ![age](https://developer.mend.io/api/mc/badges/age/npm/fs-extra/11.3.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/fs-extra/10.1.0/11.3.3?slim=true) |

---

### Release Notes

<details>
<summary>jprichardson/node-fs-extra (fs-extra)</summary>

### [`v11.3.3`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1133--2025-12-18)

[Compare Source](jprichardson/node-fs-extra@11.3.2...11.3.3)

- Fix copying symlink when destination is a symlink to the same target ([#&#8203;1019](jprichardson/node-fs-extra#1019), [#&#8203;1060](jprichardson/node-fs-extra#1060))

### [`v11.3.2`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1132--2025-09-15)

[Compare Source](jprichardson/node-fs-extra@11.3.1...11.3.2)

- Fix spurrious `UnhandledPromiseRejectionWarning` that could occur when calling `.copy()` in some cases ([#&#8203;1056](jprichardson/node-fs-extra#1056), [#&#8203;1058](jprichardson/node-fs-extra#1058))

### [`v11.3.1`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1131--2025-08-05)

[Compare Source](jprichardson/node-fs-extra@11.3.0...11.3.1)

- Fix case where `move`/`moveSync` could incorrectly think files are identical on Windows ([#&#8203;1050](jprichardson/node-fs-extra#1050))

### [`v11.3.0`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1130--2025-01-15)

[Compare Source](jprichardson/node-fs-extra@11.2.0...11.3.0)

- Add promise support for newer `fs` methods ([#&#8203;1044](jprichardson/node-fs-extra#1044), [#&#8203;1045](jprichardson/node-fs-extra#1045))
- Use `fs.opendir` in `copy()`/`copySync()` for better perf/scalability ([#&#8203;972](jprichardson/node-fs-extra#972), [#&#8203;1028](jprichardson/node-fs-extra#1028))

### [`v11.2.0`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1120--2023-11-27)

[Compare Source](jprichardson/node-fs-extra@11.1.1...11.2.0)

- Copy directory contents in parallel for better performance ([#&#8203;1026](jprichardson/node-fs-extra#1026))
- Refactor internal code to use `async`/`await` ([#&#8203;1020](jprichardson/node-fs-extra#1020))

### [`v11.1.1`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1111--2023-03-20)

[Compare Source](jprichardson/node-fs-extra@11.1.0...11.1.1)

- Preserve timestamps when moving files across devices ([#&#8203;992](jprichardson/node-fs-extra#992), [#&#8203;994](jprichardson/node-fs-extra#994))

### [`v11.1.0`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1110--2022-11-29)

[Compare Source](jprichardson/node-fs-extra@11.0.0...11.1.0)

- Re-add `main` field to `package.json` for better TypeScript compatibility ([#&#8203;979](jprichardson/node-fs-extra#979), [#&#8203;981](jprichardson/node-fs-extra#981))

### [`v11.0.0`](https://github.com/jprichardson/node-fs-extra/blob/HEAD/CHANGELOG.md#1100--2022-11-28)

[Compare Source](jprichardson/node-fs-extra@10.1.0...11.0.0)

##### Breaking Changes

- Don't allow requiring `fs-extra/lib/SOMETHING` (switched to `exports`) ([#&#8203;974](jprichardson/node-fs-extra#974))
- Require Node v14.14+ ([#&#8203;968](jprichardson/node-fs-extra#968), [#&#8203;969](jprichardson/node-fs-extra#969))

##### New Features

- Add `fs-extra/esm` for ESM named export support; see [docs](https://github.com/jprichardson/node-fs-extra#esm) for details ([#&#8203;746](jprichardson/node-fs-extra#746), [#&#8203;974](jprichardson/node-fs-extra#974))
- Add promise support for `fs.readv()` ([#&#8203;970](jprichardson/node-fs-extra#970))

##### Bugfixes

- Don't `stat` filtered items in `copy*` ([#&#8203;965](jprichardson/node-fs-extra#965), [#&#8203;971](jprichardson/node-fs-extra#971))
- Remove buggy stats check in `copy` ([#&#8203;918](jprichardson/node-fs-extra#918), [#&#8203;976](jprichardson/node-fs-extra#976))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4wLjUiLCJ1cGRhdGVkSW5WZXIiOiI0My4wLjUiLCJ0YXJnZXRCcmFuY2giOiJtYXN0ZXIiLCJsYWJlbHMiOlsiYXV0b21hdGVkIiwiZGVwZW5kZW5jaWVzIl19-->

Reviewed-on: https://code.popov.link/valentineus/strapi-plugin-checkbox-list/pulls/6
Co-authored-by: renovate[bot] <renovatebot@noreply.localhost>
Co-committed-by: renovate[bot] <renovatebot@noreply.localhost>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manidlou manidlou manidlou approved these changes

@RyanZim RyanZim RyanZim approved these changes

Assignees

No one assigned

Labels

feature-copy

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@SukkaW @RyanZim @manidlou