[core] Flatten imports to speed up webpack build & node resolution

[anthonyalayo]

What's the problem? 🤔

Background

While playing around with Next.js, I installed a package that was using @mui. I immediately noticed huge delays running next dev and the large module count (10K+) listed in the terminal. Searching for an answer, I landed on a long list of requests for help:

• Super slow page load times in development environment vercel/next.js#17977
• SWC is slow when importing directly from @mui/material vercel/next.js#37614
• SWC doesn't tree shake library vercel/next.js#42343
• NextJs compiling extremely slow vercel/next.js#29559
• Slow perfomance & Compile zounds of modules vercel/next.js#43285
• https://www.reddit.com/r/nextjs/comments/swciuj/next_dev_is_slow/

But none of those had an actual answer with an actual solution, just guesses and workarounds. One of the popular answers was: https://mui.com/material-ui/guides/minimizing-bundle-size/

After attempting to fix the library I pulled in by following the first option on that guide:
https://mui.com/material-ui/guides/minimizing-bundle-size/#option-one-use-path-imports

I noticed that the module count was still in the thousands. Why?

After reading everything on webpack, modules, tree shaking, etc, I made a demo application using CRA and a single import to @mui to showcase the problem.

Problem

Reproduction Steps

1. Run npx create-react-app cra-test to get the latest with Webpack 5
2. Ejected from CRA so that I could modify the webpack config for more metrics
3. Using webpack dev server, I modified this to see what modules traversed in development
4. Using webpack prod builds, I modified this to see modules traversed in production

I used the following configurations for stats.toJson() to balance the verbosity of the output.
I tested with and without default imports for a single icon and component.

1. With default import {assets: false, chunks: false, modulesSpace: 6, nestedModules: false, reasonsSpace: 10}
2. With path import: {assets: false, chunks: false})

Demo App 1 - Default Import Icon

This is the scenario that we are told to avoid when using @mui, and for good reason.
I created this as a baseline to see what webpack had to traverse.

import { AbcRounded } from '@mui/icons-material'

function App() {
  return (
      <div>
        <AbcRounded />
      </div>
  );
}
export default App;

Demo App 1 - Webpack Metrics Output

As expected, using the default import for AbcRounded ended up pulling in all icons.

Webpack Module Summary

orphan modules 6.28 MiB [orphan] 10860 modules
runtime modules 28.5 KiB 14 modules

Demo App 2 - Path Import Icon

Following the docs, you would expect an import like this to be lightweight. It isn't.

import AbcRounded from '@mui/icons-material/AbcRounded'

function App() {
  return (
      <div>
        <AbcRounded />
      </div>
  );
}
export default App;

Demo App 2 - Webpack Metrics Output

This is the problem. Importing a single icon resulted in:

1. Importing ./utils/createSvgIcon
2. Importing @mui/material/utils
3. Importing a lot more...(doesn't fit in a single screenshot)

Webpack Module Summary

orphan modules 534 KiB [orphan] 274 modules
runtime modules 28.5 KiB 14 modules

Demo App 3 - Path Import Component

This problem isn't unique to @mui/icons-material either. Here's performing a path import of a button.

import Button from '@mui/material/Button'

function App() {
  return (
      <div>
          <Button>Hello World</Button>
      </div>
  );
}
export default App;

Demo App 3 - Webpack Metrics Output

Again, the problem. Importing a single button resulted in:

1. Importing ./buttonClasses
2. Importing @mui/utils
3. Importing a lot more...(doesn't fit in a single screenshot)

Webpack Module Summary

orphan modules 573 KiB [orphan] 291 modules
runtime modules 28.5 KiB 14 modules

We should not be importing hundreds of modules from a single icon or button.

But... Tree Shaking? Side Effects?

This was confusing for me too, and I had to go into the details to find the answer.

1. Webpack does not tree shake at the code level, it only tree shakes at the module level.
2. Terser performs dead code elimination when minifying, but webpack still has to traverse all those imports!
3. This happens whether you are building for development or production.
4. We "cue up" our code for deletion by using ESM, but it isn't done until the minification happens.

Yes the bundle will still be minimized successfully when following ESM practices, but thousands of modules being traversed bloats memory and slows down development servers.

What are the requirements? ❓

Importing from @mui should always result in a minimal dependency graph for that particular import.

What are our options? 💡

Option 1 - Proposal

Apply transformations to the @mui build process to ensure a minimal dependency graph for all files.

Option 2 - Alternative

Remove all barrel files from @mui. This option isn't great as the developer experience that they provide is desired by both library maintainers and library users alike.

Proposed solution 🟢

1. Apply import transformations within @mui packages.

Showcased above, even when importing a component or icon directly, thousands of downstream modules get pulled in. This happens because within @mui itself, barrel files are being utilized for their developer experience. In that case, why not follow the same recommendation that @mui gives, and add these transforms to the build process?

2. Make "modularize imports" work for all @mui packages

In [docs] Modularize Imports for Nextjs, the comment #35457 (comment) requested that the docs don't include @mui/material for import transformations via babel or swc, since there are outliers that cause the transformation to fail.

Instead of backing off here, the work should be put in to fix it. The same barrel files that @mui is using internally for better developer experience is what users of the library need as well. By fixing point 1, this will come for free.

Resources and benchmarks 🔗

Below are the webpack metrics I collected for the applications in the background statement:
mui-default-import-icon-truncated-metrics.txt
mui-path-import-button-metrics.txt
mui-path-import-icon-metrics.txt

[anthonyalayo]

@oliviertassinari this issue isn't specific to icons, please see "Demo App 3":

This problem isn't unique to @mui/icons-material either.

[flaviendelangle]

We could enforce a few rules to improve the amount of dependencies listed.
On of them being to never import from the root of @mui/utils and @mui/system.

[anthonyalayo]

@flaviendelangle that would be great!

[flaviendelangle]

I think Olivier added the icon label because it is the scenario with the most obvious gains.
An icon is a super small component and we are looking through a full package (@mui/system at least).
For regular component, I don't think you should expect the list of files visited to be small, but we can definitely make improvements.

By the way, on the X components (data grid and pickers), we are probably even worse than that.
But the components being big by themselves, it's less notceable.
We can still improve though.

[michaldudak]

Thanks for such a detailed report, @anthonyalayo! I agree with @flaviendelangle, we can start with creating an eslint rule to disallow imports from barrel files. It could be better than introducing a build transform, as anyone reading the source code will see the proper way of importing other modules.

@anthonyalayo, would you be interested in working on this?