[Bug]: ElectronApplication.evaluate() is unreliable

[jjeff]

Version

1.49.0

Steps to reproduce

1. Create an Electron Application
2. Package it
3. Launch it with Playwright
4. Run a test similar to the following:

test('stress test: run electronApp.evaluate() over and over', async () => {
  const app = getApp()
  const iterations = 1000
  for (let i = 0; i < iterations; i++) {
    const displays = await app.evaluate(({ screen }) => screen.getAllDisplays())
    expect(displays).toBeTruthy()
    expect(displays.length).toBeGreaterThan(0)
  }
})

Expected behavior

No errors

Actual behavior

When communicating with main or renderer processes, I regularly get errors similar to the following:

• "context or browser has been closed"
• "Promise was collected"
• "Execution context was destroyed, most likely because of a navigation"
• "Cannot read properties of undefined (reading 'getOwnerBrowserWindow')"

These errors seem to happen randomly. And they will also happen with methods like electronApplication.getBrowser(page) or page.evaluate()

Additional context

This seems to have started with the release of Electron 27 and has continued in subsequent versions.

I know that Electron support in Playwright is experimental. And I've been writing some workarounds for the electron-playwright-helpers library. But before I release the workarounds, I'd like to figure out what's happening.

I have yet to create a minimal reproduction that generates these errors nearly as reliably as my (maximal) application.

Environment

System:
    OS: macOS 15.1.1
    CPU: (12) arm64 Apple M2 Max
    Memory: 262.55 MB / 32.00 GB
  Binaries:
    Node: 20.15.0 - ~/.nvm/versions/node/v20.15.0/bin/node
    Yarn: 1.22.18 - /usr/local/bin/yarn
    npm: 7.24.2 - ~/Code/missioncontrol/node_modules/.bin/npm
  IDEs:
    VSCode: 1.95.3 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: 1.49.0 => 1.49.0 
    playwright: 1.49.0 => 1.49.0

[mxschmitt]

We would like to help you - would it be possible to share a minimal repro?