Description
Describe the feature
The tree.json file should be able to be arbitrarily large. Currently it is limited to ~512mb, node's max string size!
Use Case
I have a growing CDK application which is running into limitations of node.js when the tree.json file is being written during synthesis. I'm hitting the max string size!
RangeError: Invalid string length
at JSON.stringify (<anonymous>)
at TreeMetadata._synthesizeTree (/<redacted>/node_modules/aws-cdk-lib/core/lib/private/tree-metadata.js:1:1457)
With either of my workarounds, my tree.json ends up being ~630mb.
Proposed Solution
I tried patching aws-cdk-lib to use json-stream-stringify, and while this appeared to resolve my synthesis failures, it didn't pass all the unit tests, presumably because the file write no longer happens synchronously. I'd be willing to spend more time opening a pull-request and iterating on this approach if ya'll think it's reasonable.
The way I've actually unblocked myself before developing the above library change was patching my node.js build - I went this way because I knew I'd be able to validate it faster than learning how to modify and test and contributions to the aws-cdk. Obviously I don't want to maintain either patch long term :)
diff -ruN dist/node-v18.16.1/deps/v8/include/v8-primitive.h dist/node-v18.16.1-patched/deps/v8/include/v8-primitive.h
--- dist/node-v18.16.1/deps/v8/include/v8-primitive.h 2023-06-20 12:50:23.000000000 +0000
+++ dist/node-v18.16.1-patched/deps/v8/include/v8-primitive.h 2023-07-27 21:13:09.852080196 +0000
@@ -123,7 +123,7 @@
class V8_EXPORT String : public Name {
public:
static constexpr int kMaxLength =
- internal::kApiSystemPointerSize == 4 ? (1 << 28) - 16 : (1 << 29) - 24;
+ internal::kApiSystemPointerSize == 4 ? (1 << 28) - 16 : (1 << 30) - 24;
enum Encoding {
UNKNOWN_ENCODING = 0x1,An alternative solution to my problem would be making the generation of the tree.json optional. I'm new-ish to CDK development and don't really know what it's for 🤷♂️ - my light googling makes me think it's an optional bit of metadata for use with visualizers
Other Information
No response
Acknowledgements
- I may be able to implement this feature request
- This feature might incur a breaking change
CDK version used
v2.91.0
Environment details (OS name and version, etc.)
macOS