Docker image downloads pnpm and prisma at startup

[leopold-gravier]

Description

I tried to setup a splitpro server behind a strong firewall that did not let Splitpro make any outgoing request.
The prisma migration failed because corepack downloads pnpm which then downloads prisma (and its dependencies). All of this is done at runtime, each time the container is started.
This is an issue for performance reasons, and it also prevents to run split-pro with limited network access.

Steps to reproduce

See that prisma is installed at runtime

1. docker run ossapps/splitpro:v1.4.7
2. The logs show that pnpm downloads and installs some packages
   Progress: resolved 53, reused 0, downloaded 53, added 53, done
.../prisma@6.13.0/node_modules/prisma preinstall$ node scripts/preinstall-entry.js
3. (Since no database was configured in this minimal exemple, prisma will fail. The main point here is that prisma had to be downloaded at runtime)

See that pnpm is installed at runtime

1. docker run --network none ossapps/splitpro:v1.4.7
2. It fails to run the prisma migration with the following error:
   Error when performing the request to https://registry.npmjs.org/pnpm/-/pnpm-10.11.0.tgz; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting

Expected behavior

The docker image should contain all the dependencies it needs to run.
Thus it should not download them at startup.
And this should allow it to run in an air-gapped environment, or behind a strong firewall.

App version

Splitpro 1.4.7 docker image (current latest)

Additional context

Full logs with network access

+ echo 'Deploying prisma migrations'
+ pnpx prisma@6 migrate deploy --schema ./prisma/schema.prisma
Deploying prisma migrations
Progress: resolved 1, reused 0, downloaded 0, added 0
Progress: resolved 2, reused 0, downloaded 1, added 0
Progress: resolved 11, reused 0, downloaded 6, added 0
Progress: resolved 22, reused 0, downloaded 14, added 0
Progress: resolved 47, reused 0, downloaded 34, added 0
Progress: resolved 50, reused 0, downloaded 48, added 0
Progress: resolved 52, reused 0, downloaded 48, added 0
Packages: +53
+++++++++++++++++++++++++++++++++++++++++++++++++++++
Progress: resolved 53, reused 0, downloaded 52, added 51
Progress: resolved 53, reused 0, downloaded 52, added 52
Progress: resolved 53, reused 0, downloaded 53, added 52
Progress: resolved 53, reused 0, downloaded 53, added 53, done
.../prisma@6.13.0/node_modules/prisma preinstall$ node scripts/preinstall-entry.js
.../prisma@6.13.0/node_modules/prisma preinstall: Done
warn The configuration property `package.json#prisma` is deprecated and will be removed in Prisma 7. Please migrate to a Prisma config file (e.g., `prisma.config.ts`).
For more information, see: https://pris.ly/prisma-config

Prisma schema loaded from prisma/schema.prisma
Error: Prisma schema validation - (get-config wasm)
Error code: P1012
error: Environment variable not found: DATABASE_URL.
  -->  prisma/schema.prisma:15
   |
14 |     // https://www.prisma.io/docs/reference/api-reference/prisma-schema-reference#string
15 |     url      = env("DATABASE_URL")
   |

Validation Error Count: 1
[Context: getConfig]

Prisma CLI Version : 6.13.0
+ echo 'Starting web server'
+ node server.js
Starting web server
  ▲ Next.js 14.2.29
  - Local:        http://0a8aa2e4b1ad:3000
  - Network:      http://172.17.0.2:3000

 ✓ Starting...
Registering instrumentation
Validating auth env
 ✓ Ready in 1226ms

Full logs without network access

+ echo 'Deploying prisma migrations'
+ pnpx prisma@6 migrate deploy --schema ./prisma/schema.prisma
Deploying prisma migrations
/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22051
    throw new Error(
          ^

Error: Error when performing the request to https://registry.npmjs.org/pnpm/-/pnpm-10.11.0.tgz; for troubleshooting help, see https://github.com/nodejs/corepack#troubleshooting
    at fetch (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22051:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async fetchUrlStream (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22081:20)
    ... 4 lines matching cause stack trace ...
    at async Object.runMain (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:23660:7) {
  [cause]: TypeError: fetch failed
      at node:internal/deps/undici/undici:13510:13
      at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
      at async fetch (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22045:16)
      at async fetchUrlStream (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22081:20)
      at async download (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22208:18)
      at async installVersion (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22300:55)
      at async Engine.ensurePackageManager (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22851:32)
      at async Engine.executePackageManagerRequest (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:22962:25)
      at async Object.runMain (/usr/local/lib/node_modules/corepack/dist/lib/corepack.cjs:23660:7) {
    [cause]: Error: getaddrinfo EAI_AGAIN registry.npmjs.org
        at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:122:26) {
      errno: -3001,
      code: 'EAI_AGAIN',
      syscall: 'getaddrinfo',
      hostname: 'registry.npmjs.org'
    }
  }
}

Node.js v22.16.0
Starting web server
+ echo 'Starting web server'
+ node server.js
 ⨯ Failed to start server
Error: getaddrinfo EAI_AGAIN 1a87bff38f6b
    at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:122:26) {
  errno: -3001,
  code: 'EAI_AGAIN',
  syscall: 'getaddrinfo',
  hostname: '1a87bff38f6b'
}

[leopold-gravier]

I see in the Dockerfile that there was an attempt to copy prisma from the base image to the release image, however running pnpm list on a fresh release image gives an empty output, so it does not recognise that any package is installed.

[krokosik]

Hmm yes, you're right. We do however copy prisma into the final Docker stage. Can you try if you can replace the pnpx call with calling the binary directly? If that works then we can probably also get rid of pnpm itself and reduce the image size a bit.

[leopold-gravier]

The binary fails to load @prisma/engines. Actually the prisma engines lib is not in the package.json, maybe that's the root cause?

# /app/node_modules/.bin/prisma --help
node:internal/modules/cjs/loader:1404
  throw err;
  ^

Error: Cannot find module '@prisma/engines'

[leopold-gravier]

I am struggling a lot to install prisma in the production container, without installing every production dependency of split-pro.

Prisma is not packaged by next.js because it is not a dependency of the typescript source code. And I checked but Prisma has no support for js API for migrations, it must be done through their CLI.
I tryied to manually copy prisma from the base image to the release image in the Dockerfile. But it is very tedious, as it means doing all the work of the package manager for Prisma, which has a bunch of dependencies. I could not get it to work and it would require a lot of maintenance, as it is specific to one version of Prisma.

From here I see 3 solutions, each with their pros and cons.

1. Use pnpm to install Prisma on the release image
   We could tell pnpm to install Prisma on the release image. It's not too complicated to setup, I have a proof of concpet working. However it creates some dependency-duplication, as some dependencies are included by both the Next.js standalone and pnpm. It makes for a 500MB image (instead of the previous 300MB without prisma)
2. Copy Prisma from the base image to the release image
   It would involve manually copying Prisma from the base image to the release image in the Dockerfile. As I said before, it involves a lot of tedious work and maintenance. I hope that it can produce a slimmer image than 500MB, but Prisma is quite bulky anyways so I'm not even sure that it would save much disk-space.
3. Use another migration backend for production
   There are other SQL migration tools that are better suited for production. It would require some migration effort, but it would produce the cleanest architecture I think.
   One good candidate would be Atlas, which has a lightweight runtime (it can come in a 20MB docker image). It can be installed with pnpm and exposes a js API. So if we call it from the source code, maybe Next.js could package it automatically into the standalone. And is has some compatibily with a Prisma development environment. I think Prisma can format the migrations to an Atlas format.

Please let me know what you think about these solutions, and if you have any other idea.
I'd rather go for the first first solution because it looks like the easiest to setup.

[krokosik]

I would go for option number 1 or 2. Where image size is concerned, option 2 would probably be better as we do not install pnpm this way and only get the built primsa client AFAIK. When I tried to do it with the current setup, I kept getting an error about missing prisma engines. I suspect this is caused by pnpm hoisting, disabling this caused the install time to increase, but I don't mind, since it's cached anyway (not in CI, but can be added).

I would encourage you to try working around it, but if you deem it not worth the effort, option 1 sounds good to me as well.

See this Dockerfile for reference: prisma/prisma#24885 (comment)

[krokosik]

I also see an issue. For some reason the final image has 200MB of node modules, including those that should be bundled in next standalone.

[krokosik]

@leopold-gravier you can try the v2.0.0-beta.7 release and let me know if it works okay for you

[leopold-gravier]

Thanks for your work! It looks promessing.
I tested it by creating a new instance with ans dithout internet access, and the prisma migration starts even with no internet access, however the initial migration failed regardless. Here is an extract of the log:

Migration 20251119183616_switch_to_balance_view will be executed
prisma:error Raw query failed. Code: `42601`. Message: `ERROR: cannot insert multiple commands into a prepared statement`
[Same error repeated 8 times]
prisma:error 
Invalid `prisma.$executeRawUnsafe()` invocation:


Raw query failed. Code: `42601`. Message: `ERROR: cannot insert multiple commands into a prepared statement`
prisma:error Raw query failed. Code: `42601`. Message: `ERROR: cannot insert multiple commands into a prepared statement`
Failed to prepare server [Error [PrismaClientKnownRequestError]: An error occurred while loading instrumentation hook: 
Invalid `prisma.$executeRawUnsafe()` invocation:

And the full log here: splitpro.log

[krokosik]

Ah, thanks for checking that, a tiny error slipped in. Can you try the beta.8 tag? By the way I forgot to wanr you about updating from v1, be sure to make a backup of your DB.