Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:

- uses: actions/setup-node@v6
with:
node-version: 18
node-version: 24
cache: 'npm'

- name: Install dependencies
Expand Down
11 changes: 5 additions & 6 deletions DevDockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (c) Microsoft Corporation and others. Licensed under the MIT license.
# SPDX-License-Identifier: MIT

FROM node:18-bullseye
FROM node:24-trixie
ENV APPDIR=/opt/service

# Set environment variables from build arguments
Expand All @@ -22,9 +22,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends --no-install-su

# Scancode
ARG SCANCODE_VERSION="32.4.1"
RUN pip3 install --upgrade pip setuptools wheel && \
curl -Os https://raw.githubusercontent.com/nexB/scancode-toolkit/v$SCANCODE_VERSION/requirements.txt && \
pip3 install --constraint requirements.txt scancode-toolkit==$SCANCODE_VERSION && \
RUN curl -Os https://raw.githubusercontent.com/nexB/scancode-toolkit/v$SCANCODE_VERSION/requirements.txt && \
pip3 install --break-system-packages --constraint requirements.txt scancode-toolkit==$SCANCODE_VERSION && \
rm requirements.txt && \
scancode-reindex-licenses && \
scancode --version
Expand All @@ -41,8 +40,8 @@ RUN gem install nokogiri:1.16.0 --no-document && \
gem install licensee:9.16.1 --no-document

# REUSE
RUN pip3 install setuptools
RUN pip3 install reuse==3.0.1
RUN pip3 install --break-system-packages setuptools
RUN pip3 install --break-system-packages reuse==3.0.1

RUN git config --global --add safe.directory '*'

Expand Down
11 changes: 5 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (c) Microsoft Corporation and others. Licensed under the MIT license.
# SPDX-License-Identifier: MIT

FROM node:18-bullseye
FROM node:24-trixie
ENV APPDIR=/opt/service

# Set environment variables from build arguments
Expand All @@ -20,9 +20,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends --no-install-su

# Scancode
ARG SCANCODE_VERSION="32.4.1"
RUN pip3 install --upgrade pip setuptools wheel && \
curl -Os https://raw.githubusercontent.com/nexB/scancode-toolkit/v$SCANCODE_VERSION/requirements.txt && \
pip3 install --constraint requirements.txt scancode-toolkit==$SCANCODE_VERSION && \
RUN curl -Os https://raw.githubusercontent.com/nexB/scancode-toolkit/v$SCANCODE_VERSION/requirements.txt && \
pip3 install --break-system-packages --constraint requirements.txt scancode-toolkit==$SCANCODE_VERSION && \
rm requirements.txt && \
scancode-reindex-licenses && \
scancode --version
Expand All @@ -39,8 +38,8 @@ RUN gem install nokogiri:1.16.0 --no-document && \
gem install licensee:9.16.1 --no-document

# REUSE
RUN pip3 install setuptools
RUN pip3 install reuse==3.0.1
RUN pip3 install --break-system-packages setuptools
RUN pip3 install --break-system-packages reuse==3.0.1

# Crawler config
ENV CRAWLER_DEADLETTER_PROVIDER=cd(azblob)
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ Due to an issue with ScanCode's install configuration on Windows, you may need t

## Setup for running tests

If you are just interested in running the tests for the crawler then you need to be using Node v18.20.8 on your local system to get `npm install` and the tests to run safely. This will be fixed as we upgrade the dependencies but for now this is the best solution. If you are using [nvm](https://github.com/nvm-sh/nvm) you can follow the instructions below, using the lowest working node version, to get things setup to run tests.
If you are just interested in running the tests for the crawler then you need to be using Node v24 on your local system to get `npm install` and the tests to run safely. This will be fixed as we upgrade the dependencies but for now this is the best solution. If you are using [nvm](https://github.com/nvm-sh/nvm) you can follow the instructions below, using the lowest working node version, to get things setup to run tests.

1. Clone this repo
2. cd into the `crawler` directory
3. run `nvm install v18.20.8; nvm use v18.20.8` to instal the correct node version
3. run `nvm install v24; nvm use v24` to install the correct node version
4. run `npm run test` and profit

## Queuing work with the crawler
Expand Down
Loading