[Bug] Docker build creates yarn error with esbuild

[exowanderer]

In the main branch, the automated yarn installation and build through docker build is flagging the following error

=> ERROR [17/18] RUN cd frontend && yarn install && yarn build
11.17 error /workspace/frontend/node_modules/esbuild: Command failed.
11.17 Error: Command failed: /usr/bin/node /workspace/frontend/node_modules/esbuild/bin/esbuild --version
11.17 <ref *1> Error: spawnSync /workspace/frontend/node_modules/@esbuild/linux-x64/bin/esbuild ETXTBSY
11.17 error: [Circular *1],

ERROR: failed to solve: process "/bin/sh -c cd frontend && yarn install && yarn build" did not complete successfully: exit code: 1

Here is the full error traceback from docker build

 => [16/18] RUN npm install -g yarn                                                                                                                                                                           6.9s 
 => ERROR [17/18] RUN cd frontend && yarn install && yarn build                                                                                                                                              11.3s 
------                                                                                                                                                                                                             
 > [17/18] RUN cd frontend && yarn install && yarn build:                                                                                                                                                          
0.770 yarn install v1.22.21                                                                                                                                                                                        
0.824 [1/4] Resolving packages...                                                                                                                                                                                  
0.917 [2/4] Fetching packages...                                                                                                                                                                                   
0.923 warning Pattern ["string-width@^4.1.0"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-string-width-cjs-4.2.3-269c7117d27b05ad2e536830a8ec895ef9c6d010-integrity/node_modules/string-width-cjs" as pattern ["string-width-cjs@npm:string-width@^4.2.0"]. This could result in non-deterministic behavior, skipping.
7.783 [3/4] Linking dependencies...
11.05 [4/4] Building fresh packages...
11.17 error /workspace/frontend/node_modules/esbuild: Command failed.
11.17 Exit code: 1
11.17 Command: node install.js
11.17 Arguments: 
11.17 Directory: /workspace/frontend/node_modules/esbuild
11.17 Output:
11.17 node:internal/errors:932
11.17   const err = new Error(message);
11.17               ^
11.17 
11.17 Error: Command failed: /usr/bin/node /workspace/frontend/node_modules/esbuild/bin/esbuild --version
11.17 node:child_process:929
11.17     throw err;
11.17     ^
11.17 
11.17 <ref *1> Error: spawnSync /workspace/frontend/node_modules/@esbuild/linux-x64/bin/esbuild ETXTBSY
11.17     at Object.spawnSync (node:internal/child_process:1124:20)
11.17     at spawnSync (node:child_process:876:24)
11.17     at Object.execFileSync (node:child_process:919:15)
11.17     at Object.<anonymous> (/workspace/frontend/node_modules/esbuild/bin/esbuild:221:28)
11.17     at Module._compile (node:internal/modules/cjs/loader:1376:14)
11.17     at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
11.17     at Module.load (node:internal/modules/cjs/loader:1207:32)
11.17     at Module._load (node:internal/modules/cjs/loader:1023:12)
11.17     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12)
11.17     at node:internal/main/run_main_module:28:49 {
11.17   errno: -26,
11.17   code: 'ETXTBSY',
11.17   syscall: 'spawnSync /workspace/frontend/node_modules/@esbuild/linux-x64/bin/esbuild',
11.17   path: '/workspace/frontend/node_modules/@esbuild/linux-x64/bin/esbuild',
11.17   spawnargs: [ '--version' ],
11.17   error: [Circular *1],
11.17   status: null,
11.17   signal: null,
11.17   output: null,
11.17   pid: 0,
11.17   stdout: null,
11.17   stderr: null
11.17 }
11.17 
11.17 Node.js v20.11.0
11.17 
11.17     at checkExecSyncError (node:child_process:890:11)
11.17     at Object.execFileSync (node:child_process:926:15)
11.17     at validateBinaryVersion (/workspace/frontend/node_modules/esbuild/install.js:99:28)
11.17     at /workspace/frontend/node_modules/esbuild/install.js:284:5 {
11.17   status: 1,
11.17   signal: null,
11.17   output: [
11.17     null,
11.17     Buffer(0) [Uint8Array] [],
11.17     Buffer(1157) [Uint8Array] [
11.17       110, 111, 100, 101,  58,  99, 104, 105, 108, 100,  95, 112,
11.17       114, 111,  99, 101, 115, 115,  58,  57,  50,  57,  10,  32,
11.17        32,  32,  32, 116, 104, 114, 111, 119,  32, 101, 114, 114,
11.17        59,  10,  32,  32,  32,  32,  94,  10,  10,  60, 114, 101,
11.17       102,  32,  42,  49,  62,  32,  69, 114, 114, 111, 114,  58,
11.17        32, 115, 112,  97, 119, 110,  83, 121, 110,  99,  32,  47,
11.17       119, 111, 114, 107, 115, 112,  97,  99, 101,  47, 102, 114,
11.17       111, 110, 116, 101, 110, 100,  47, 110, 111, 100, 101,  95,
11.17       109, 111, 100, 117,
11.17       ... 1057 more items
11.17     ]
11.17   ],
11.17   pid: 153,
11.17   stdout: Buffer(0) [Uint8Array] [],
11.17   stderr: Buffer(1157) [Uint8Array] [
11.17     110, 111, 100, 101,  58,  99, 104, 105, 108, 100,  95, 112,
11.17     114, 111,  99, 101, 115, 115,  58,  57,  50,  57,  10,  32,
11.17      32,  32,  32, 116, 104, 114, 111, 119,  32, 101, 114, 114,
11.17      59,  10,  32,  32,  32,  32,  94,  10,  10,  60, 114, 101,
11.17     102,  32,  42,  49,  62,  32,  69, 114, 114, 111, 114,  58,
11.17      32, 115, 112,  97, 119, 110,  83, 121, 110,  99,  32,  47,
11.17     119, 111, 114, 107, 115, 112,  97,  99, 101,  47, 102, 114,
11.17     111, 110, 116, 101, 110, 100,  47, 110, 111, 100, 101,  95,
11.17     109, 111, 100, 117,
11.17     ... 1057 more items
11.17   ]
11.17 }
11.17 
11.17 Node.js v20.11.0
11.17 info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
------
Dockerfile:61
--------------------
  59 |     
  60 |     # Install frontend dependencies and build it for production (into the frontend/dist folder)
  61 | >>> RUN cd frontend && yarn install && yarn build
  62 |     
  63 |     # Container start script
--------------------
ERROR: failed to solve: process "/bin/sh -c cd frontend && yarn install && yarn build" did not complete successfully: exit code: 1

[exowanderer]

The [search engine of your choice], is pointing me at articles about

1. Adding the node_modules directory to our .dockerignore. But we do not have a node_modules directory or a .dockerignore.

2. Another suggestion was to replace yarn with npm or npm-ci.

3. And what is "npm-run-all2"? in the package.json file? I can only see references to "npm-run-all". Is "npm-run-all2" and new version?

[rti]

Sorry to hear that main is broken for you.

Do you have a frontend/node_modules folder? Our current Dockerfile does indeed not take care here, just implicitly copies it into the container with the COPY --chmod=755 frontend frontend in line 48, after that, yarn install line 61 will try to work off of that. If this was in a broken state, it could be the reason.

What you could try:

• either delete the node_modules folder in frontend and run docker build . from the project root again
• or create a .dockerignore in the root of the project and put frontend/node_modules in there to not copy it into the container during docker build
• or delete frontend/node_modules and really exchange yarn for npm: changing line 61 to RUN cd frontend && npm install && npm run build should be enough
  • Note: npm ci might not work, because we do not have a package.lock at the moment, only a yarn.lock

[andrewtavis]

I'd say it makes sense to try the first suggestions before we exchange yarn for npm as we should all make the switch if we're going to do it.

[exowanderer]

Thank you for the suggestions.

1. I never had a node_modules directory in base ./ or in ,/frontend/. Should I create blank frontend/node_modules dir?
2. I had already added node_modules to our .dockerignore, but not frontend/node_modules. I just added frontend/node_modules to the .dockerignore file and the behaviour was nearly the same. I had a new error at first, then the same error.
3. After the previous two cases did not "fix the bug" (the bug could be me), I also changed yarn to npm as you suggested. The result was a very similar error, with the word yarn replaced in the error message for npm.

May I confirm that if you download the gbnc/main branch to a fresh diretory or laptop, that the docker indeed builds correctly? I am imagining that you have files in your directories from building this code that are helping you avoid the bug I am finding.

I did grab a fresh directory, cloned the repo, purged docker from my system (except the docker binary), and tried to build the Docker. Unfortunately, in the fresh cloned directoy, I still have the same yarn error

Btw. Here is the docker build command that I am using, from our previous iterations, in case somethig is funky here:
DOCKER_BUILDKIT=1 docker build --no-cache -t gbnc .

Here is the npm error from step 3 above:

 => ERROR [16/17] RUN cd frontend && npm install && npm build                                                                                                                                                30.5s 
------                                                                                                                                                                                                             
 > [16/17] RUN cd frontend && npm install && npm build:                                                                                                                                                            
29.83 npm notice                                                                                                                                                                                                   
29.83 npm notice New minor version of npm available! 10.2.4 -> 10.4.0
29.83 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.4.0>
29.83 npm notice Run `npm install -g npm@10.4.0` to update!
29.83 npm notice 
29.83 npm ERR! code 1
29.83 npm ERR! path /workspace/frontend/node_modules/esbuild
29.83 npm ERR! command failed
29.83 npm ERR! command sh -c node install.js
29.83 npm ERR! node:internal/errors:563
29.83 npm ERR!     ErrorCaptureStackTrace(err);
29.83 npm ERR!     ^
29.83 npm ERR! 
29.83 npm ERR! <ref *1> Error: spawnSync /workspace/frontend/node_modules/esbuild/bin/esbuild ETXTBSY
29.83 npm ERR!     at Object.spawnSync (node:internal/child_process:1124:20)
29.83 npm ERR!     at spawnSync (node:child_process:876:24)
29.83 npm ERR!     at Object.execFileSync (node:child_process:919:15)
29.83 npm ERR!     at validateBinaryVersion (/workspace/frontend/node_modules/esbuild/install.js:99:28)
29.83 npm ERR!     at /workspace/frontend/node_modules/esbuild/install.js:286:5 {
29.83 npm ERR!   errno: -26,
29.83 npm ERR!   code: 'ETXTBSY',
29.83 npm ERR!   syscall: 'spawnSync /workspace/frontend/node_modules/esbuild/bin/esbuild',
29.83 npm ERR!   path: '/workspace/frontend/node_modules/esbuild/bin/esbuild',
29.83 npm ERR!   spawnargs: [ '--version' ],
29.83 npm ERR!   error: [Circular *1],
29.83 npm ERR!   status: null,
29.84 npm ERR!   signal: null,
29.84 npm ERR!   output: null,
29.84 npm ERR!   pid: 0,
29.84 npm ERR!   stdout: null,
29.84 npm ERR!   stderr: null
29.84 npm ERR! }
29.84 npm ERR! 
29.84 npm ERR! Node.js v20.11.0
29.84 
29.84 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-01-30T19_12_37_268Z-debug-0.log
------
Dockerfile:61
--------------------
  59 |     
  60 |     # Install frontend dependencies and build it for production (into the frontend/dist folder)
  61 | >>> RUN cd frontend && npm install && npm build
  62 |     
  63 |     # Container start script
--------------------
ERROR: failed to solve: process "/bin/sh -c cd frontend && npm install && npm build" did not complete successfully: exit code: 1

[exowanderer]

[Blurry Update] I added "esbuild": "^0.14.3" under devDependencies in our package.json file. StackOverflow suggested this in case we wanted to build esbuild globally "just to get rid of the problem".

It did not get rid of the problem inside of Docker. But: I was able to install nodejs and yarn (npm) in a new conda env. Then I ran yarn install and yarn build in the frontend directory.

The result is that the nodejs outside Docker seems to build correctly on my system (not in Docker), because then I run bash start.sh and get a Python error, and not a nodejs error.

[andrewtavis]

I'm thinking a quick call tomorrow might help with all of this so we can compare setups? I'm gonna hammer out the rest of the frontend now, and we can have a quick sync to hook it all together so it's working for all of us? My afternoon is fairly free :)

[rti]

Thanks for all the details!

It could be this one: libuv/libuv#4141, as reported here nodejs/docker-node#1912

Apparently node 18 reverted the libuv update causing it here nodejs/node#50036

So it could be worth a try to downgrade our nodejs to 18 in the Dockerfile line 52. Or update your linux kernel beyond 6.1 🫣

Tomorrow before 13:30 works for me.

[exowanderer]

Close in #19