[BUG] all npm commands hang after 1 week?

[mfshao]

Background: we are using quay.io to build docker images for our applications. We have some npm ... commands in the startup script when a container has been brought up.

Recently after upgrading to npm 7, we start to notice that some of our containers are stuck at booting up, more specifically they stuck at the first attempt to run any npm ... commands in that startup script.

After further invistigation we found that rebuild the images in quay seems to be able to get rid of the problem, but just temporarily. After 7 days, the same image will get stuck again at the first attempt to run any npm ... commands

Our older images, those who were still using npm 6, does not have any similar issues

So basically right now we are scratching our heads trying to figure out why is this happening. Any insight on this matter is appreciated

Current Behavior:

Run an example startup script when docker container is spinning up

#!/bin/bash

node --version
npm config list --loglevel verbose <--- will hang forever at here
node --version
npm version

Output

v14.16.0
npm verb cli [
npm verb cli '/usr/bin/node',
npm verb cli '/usr/bin/npm',
npm verb cli 'config',
npm verb cli 'list',
npm verb cli '--loglevel',
npm verb cli 'verbose'
npm verb cli ]
npm info using npm@7.9.0
npm info using node@v14.16.0
npm timing config:load:defaults Completed in 4ms
npm timing config:load:file:/usr/lib/node_modules/npm/npmrc Completed in 2ms
npm timing config:load:builtin Completed in 2ms
npm timing config:load:cli Completed in 3ms
npm timing config:load:env Completed in 2ms
npm timing config:load:file:/data-portal/.npmrc Completed in 0ms
npm timing config:load:project Completed in 1ms
npm timing config:load:file:/root/.npmrc Completed in 1ms
npm timing config:load:user Completed in 1ms
npm timing config:load:file:/usr/etc/npmrc Completed in 1ms
npm timing config:load:global Completed in 2ms
npm timing config:load:cafile Completed in 0ms
npm timing config:load:validate Completed in 0ms
npm timing config:load:setUserAgent Completed in 1ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 17ms
npm verb npm-session 2de562b7ad69b05f
npm timing npm:load Completed in 30ms. <--- stuck at here forever

Expected Behavior:

npm commands should not hang

Steps To Reproduce:

Really not sure if it is easy to reproduce by you because on our end this issue will only happen on images that has been build 1 week before or older. But please let me know if there is anything I can help with you in order to verify this issue.

Environment:

• OS: Ubuntu 20.04
• Node: 14.16.0
• npm: 7.9.0

[isaacs]

The fact that it's hanging after 7 days is a very strong indicator that it has to do with the update notifier. (Ie, the bit where npm checks for a new version of npm, and nags you to upgrade.)

As a workaround, you can try running with npm_config_update_notifier=false in the environment (or update-notifier=false in a .npmrc file, or --no-update-notifier on the cli, but I figure since it's a CI env, the environment is easiest to modify.)

But..... why is that request hanging? That's very strange.

[mfshao]

Thanks for that info, that's helpful!

Is there any logic changes for the update notifier in npm 7? Because we are not seeing this issue in our images using npm 6

Also can I ask which domain is this update notifier hitting? Is it going to try fetching registry.npmjs.org? We do have some proxies for outbound traffics, so if we know where it is going to hit, we can check if it is blocked by our proxies

[nlf]

the update notifier was replaced entirely for npm 7, and yes it does fetch from registry.npmjs.org, specifically it fetches https://registry.npmjs.org/npm (or whatever your default registry is)

[mfshao]

Interesting, that domain already exists in our proxy's whitelist 🤔

Oh and a bit of more details, we also tried if we get a shell of the running container which stuck in the middle of the script and run npm ... directly from the shell, it will work fine. I don't know if this will cause the update notifier to behave differently?

[nlf]

that's very interesting.. i wonder if we're doing something strange when we don't have a tty. we'll look into it further and see what we can find

[ReneWerner87]

we also had problems with the update check

because our company uses a private registry, which is used when building

in the live system we use the "npm run" command to start the server
which runs the update check on command, but the private registry is not accessible from the live system

until the time i found this article we had no idea why it requests the registry and hangs on the "run" command

our fix now is to disable this update check in the .npmrc, however this was completely unexpected from our side^^

-> we have noticed this by the way, since we switched from npm 6 to 7 | nodejs 14 to16

[wraithgar]

This was fixed by #3348 and published in v7.16.0