Next.js: Fix webpack fsCache not working

[sentience]

Closes #29621

What I did

Next.js overrides certain internal webpack packages (particularly webpack-sources), which are involved with filesystem caching, with its own versions from next/dist/compiled.

For filesystem caching to work, Next.js must be allowed to perform these overrides before webpack is first initialized by @storybook/builder-webpack5. If it is not, the objects to be serialized to disk in the caching process will be instantiated using the original (non-Next.js) modules, but the cache serializers will be created using the Next.js modules. This mismatch between the objects to be cached and the serializers that write the filesystem cache prevents the cache from being written; instead, webpack outputs a warning message to the console for every object that it tries and fails to find a matching serializer for.

This fix works by invoking Next.js to configure webpack in the core hook of @storybook/nextjs/preset, immediately before loading @storybook/builder-webpack5. We don't actually use this configuration that Next.js creates; the actual configuration that will be used in the build is still generated in webpackFinal as before.

fsCache (docs) has a large impact on Storybook build performance. Even in a minimal project with a single story, enabling it reduces hot-cache build time by 66%! This is therefore a very valuable option to be able to enable.

Checklist for Contributors

Testing

@storybook/nextjs does not appear to have a test suite, so I don't see an easy way to test this feature. I've verified that it resolves the issue in my repro project for #29621.

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

1. Clone https://github.com/sentience/storybook-nextjs-webpack-cache-issue-demo
2. Run Storybook (pnpm i, pnpm storybook build) and note the console warnings at the end of the build that indicate the filesystem cache is not working.
3. Upgrade @storybook/nextjs to a canary release with this change.
4. Run Storybook again and note the console warnings are gone, and the build runs in 33% of the time!

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

• Make sure this PR contains one of the labels below:

  Available labels

  • bug: Internal changes that fixes incorrect behavior.
  • maintenance: User-facing maintenance tasks.
  • dependencies: Upgrading (sometimes downgrading) dependencies.
  • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
  • cleanup: Minor cleanup style change. Will not show up in release changelog.
  • documentation: Documentation only changes. Will not show up in release changelog.
  • feature request: Introducing a new feature.
  • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
  • other: Changes that don't fit in the above categories.

🦋 Canary release

This pull request has been released as version 0.0.0-pr-29654-sha-2a02e134. Try it out in a new sandbox by running npx [email protected] sandbox or in an existing project with npx [email protected] upgrade.

More information

Published version	0.0.0-pr-29654-sha-2a02e134
Triggered by	@valentinpalkovic
Repository	sentience/storybook
Branch	fix-nextjs-webpack-fscache
Commit	2a02e134
Datetime	Tue Nov 26 08:12:42 UTC 2024 (1732608762)
Workflow run	12026402956

To request a new release of this pull request, mention the @storybookjs/core team.

core team members can create a new canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=29654

Greptile Summary

This PR fixes webpack's filesystem caching in @storybook/nextjs by ensuring Next.js can properly override webpack internals before Storybook's webpack builder initialization, significantly improving build performance.

• Modified code/frameworks/nextjs/src/preset.ts to load Next.js configuration in the core hook before webpack initialization
• Added early webpack configuration setup to allow Next.js to override internal packages like webpack-sources
• Enables proper filesystem caching functionality, reducing build times by up to 66%
• Maintains existing webpack configuration generation in webpackFinal hook for actual builds

[greptile-apps]

_{1 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile}

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 2a02e13. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 1 target

• nx run-many -t build --parallel=3

---

Sent with 💌 from NxCloud.

[sentience]

@valentinpalkovic May I request a canary release of this? Sorry if you're not the right person to mention; the PR template says to mention @storybookjs/core, but that doesn't seem to be recognised by GitHub.

[valentinpalkovic]

Here we go: https://github.com/storybookjs/storybook/actions/runs/12026402956/job/33525318348

When the job is done, you can see the canary version :)