`ln: node-v14.18.2.tar.gz: File exists` error after failed installation

[jfly]

There's probably a more elegant way of reproducing this, but here's how I did it:

# Make sure there's no working compiler available
$ ln -s /usr/bin/false gcc
$ ln -s /usr/bin/false clang

# Now try to install by compiling from source
# Note that the current directory (with it's broken gcc and clang) is first in the PATH.
$ PATH="$PWD:$PATH" ASDF_NODEJS_FORCE_COMPILE=1 asdf install nodejs 14.18.1
Trying to update node-build... ok

WARNING: node-v14.18.1 is in LTS Maintenance mode and nearing its end of life.
It only receives *critical* security updates, *critical* bug fixes and documentation updates.

Installing node-v14.18.1...

BUILD FAILED (Arch rolling using node-build 4.9.69)

Inspect or clean up the working tree at /home/jeremy/.asdf/downloads/nodejs/14.18.1
Results logged to /tmp/node-build.20220216020150.176007.log

Last 10 log lines:
~/.asdf/downloads/nodejs/14.18.1 ~/tmp/demoing
~/.asdf/downloads/nodejs/14.18.1/node-v14.18.1 ~/.asdf/downloads/nodejs/14.18.1 ~/tmp/demoing
Node.js configure: Found Python 3.9.6...
WARNING: C compiler (CC=gcc, 0.0.0) too old, need gcc 4.2 or clang 3.2
WARNING: Could not recognize `gas`:
ERROR: Did not find a new enough assembler, install one or build with
       --openssl-no-asm.
       Please refer to BUILDING.md

Now there's this asfd downloads directory:

$ ls -alh /Users/jeremyfleischman/.asdf/downloads/nodejs/14.18.2
total 0
drwxr-xr-x   3 jeremyfleischman  staff    96B Feb 16 01:30 .
drwxr-xr-x   4 jeremyfleischman  staff   128B Feb 16 01:30 ..
drwxr-xr-x  33 jeremyfleischman  staff   1.0K Nov 30 04:05 node-v14.18.2

Now try to install that version of nodejs again:

$ PATH="$PWD:$PATH" ASDF_NODEJS_FORCE_COMPILE=1 asdf install nodejs 14.18.1
Trying to update node-build... ok
Downloading node-v14.18.2.tar.gz...
-> https://nodejs.org/dist/v14.18.2/node-v14.18.2.tar.gz
-> https://nodejs.org/dist/v14.18.2/node-v14.18.2.tar.gz

BUILD FAILED (OS X 12.2 using node-build 4.9.66-10-g77de5c76)

Inspect or clean up the working tree at /Users/jeremyfleischman/.asdf/downloads/nodejs/14.18.2
Results logged to /var/folders/79/_76sx6gn5d58bzngw1q_pxk80000gp/T/node-build.20220216013051.38646.log

Last 10 log lines:
~/.asdf/downloads/nodejs/14.18.2 ~
ln: node-v14.18.2.tar.gz: File exists
ln: node-v14.18.2.tar.gz: File exists

Eep! This failed in a different way: node-build is crashing when trying to ln some files after downloading the node source code (see the relevant code here). That doesn't work because there's actually already a node-v14.18.2 directory leftover from the previous failed build.

Further adding to the mystery, we're now left with a downloads folder that looks like this:

$ ls -alh /Users/jeremyfleischman/.asdf/downloads/nodejs/14.18.2
total 127104
drwxr-xr-x   4 jeremyfleischman  staff   128B Feb 16 01:30 .
drwxr-xr-x   4 jeremyfleischman  staff   128B Feb 16 01:30 ..
drwxr-xr-x  33 jeremyfleischman  staff   1.0K Nov 30 04:05 node-v14.18.2
-rw-r--r--   1 jeremyfleischman  staff    61M Feb 16 01:30 node-v14.18.2.tar.gz

Which means that the next build will actually pass node-build's reuse_existing_tarball check, and fail with the original symptom.

tl;dr: If you have a nodejs build fail once for a legitimate reason, subsequent installs will alternate between that legitimate build failure and this confusing ln error.

I spent a while digging into this. It's super convoluted. Here's my attempt to explain it:

1. The first time we attempt to install: bin/install passes in
   $ASDF_DOWNLOAD_PATH as a NODE_BUILD_CACHE_PATH. However, asdf hasn't
   actually created that $ASDF_DOWNLOAD_PATH directory yet (that only happens
   if your plugin defines a bin/download
   script,
   and this plugin does not*). Later on, node-build discards the given
   NODE_BUILD_CACHE_PATH because it doesn't
   exist.
   Then it goes on to create the given
   NODE_BUILD_BUILD_PATH
   (which happens to also be $ASDF_DOWNLOAD_PATH), and later extracts the source
   code to there and deletes the original tar.gz
   file.
   This way, we end up with a $ASDF_DOWNLOAD_PATH/node-v14.18.2 folder, but
   no $ASDF_DOWNLOAD_PATH/node-v14.18.2.tar.gz file.

2. The second time we attempt to install, a lot of the same things happen, but
   now NODE_BUILD_CACHE_PATH does exist. However, there's no tar.gz file, so
   we blow past
   reuse_existing_tarball
   and go on to crash in this line of
   download_tarball.

IMO, there are 2 bugs here:

1. We incorrectly assume that $ASDF_DOWNLOAD_PATH actually exists (or maybe,
   we incorrectly use it when it's designated only for "newer" asdf plugins that
   define a bin/download script?) Either way, I'm pretty sure we're doing
   something wrong here.
2. We set the same directory for both NODE_BUILD_BUILD_PATH and
   NODE_BUILD_CACHE_PATH. node-build looks like it was built with the assumption
   that if these are specified, they're not pointing at the same directory.
   I've filed this bug upstream: Should using the same directory for NODE_BUILD_CACHE_PATH and NODE_BUILD_BUILD_PATH work? nodenv/node-build#731 just to check if I'm understanding this correctly.

*NOTE: It sounds like asdf is going to change in the future to require that plugins define a bin/download script. From https://asdf-vm.com/plugins/create.html#environment-variables:

All plugins must include this script, and eventually support for legacy plugins will be removed.

I'm not super clear on how this plugin could be changed to have an explicit download phase. I think we'd need to tweak node-build to provide some mechanism to download without trying to install?