Unable to build SCWB on Windows 10

[johnmhoran]

I discovered in recent days that I am unable to build the current version of SCWB on my Windows 10 laptop using Git Bash.

This process still works when I check out the latest tag -- v3.1.1 from the latest release, in September 2019 -- and build with Python 2.7.15.

• run npm install
• run ./node_modules/.bin/electron-rebuild
• run npm start -- SCWB launched, looked good, displayed some data just fine
• close the SCWB 3.1.1 instance
• run /c/Python27/python.exe build.py
• navigate to the /dist/ folder, double click ScanCode-Workbench.exe and no errors -- SCWB 3.1.1 opens, runs fine, looks great.

With the current code, the 1st 3 steps work with both Python 2.7.15 and Python3 (3.8.3). No error during the build process, but when I try to run the built .exe I get a JavaScript error: Cannot find module '../dist/src/main'.

Not sure what is meant by Cannot find module '../dist/src/main' but can confirm there is no such folder. Same result when extracting the .zip and running the .exe found there. I also get an error running npm test.

Perhaps I'm using the wrong version of npm, or node.js (but they look like they meet the RTD Windows requirements -- https://scancode-workbench.readthedocs.io/en/develop/contribute/building.html#windows)? (I have npm 5.2.0 and node 12.16.3.) Seems like some conflict was introduced after the 3.1.1 tag was created.

[FrySlim]

No error during the build process, but when I try to run the built .exe I get a JavaScript error: Cannot find module '../dist/src/main'.

Not sure what is meant by Cannot find module '../dist/src/main' but can confirm there is no such folder.

I got the same behaviour with Python 3.9.10 building SCWB 3.1.1 today.
Running just 'npm start' works and the SCWB opens up and everything works (except Exporting any JSON/conclusion file).
I am using npm version 8.3.1 and node version 16.14.0.

Best Regards
FrySlim

[pombredanne]

There is some work in progress, but this in the HEAD of the develop branch, not in the tagged release yet.
Have you tried this too?

[FrySlim]

TL;DR:
Luckily your release 'dist' subdirectory content in ScanCode-Workbench-windows-x64-3.1.1.zip works out of the box with starting "ScanCode-Workbench.exe". And all features work so far.
But I was not able to build the master branch source code (I assume, this is the same content as in source code download https://github.com/nexB/scancode-workbench/archive/refs/tags/v3.1.1.zip )
on my own with current development tools (see below the line).

Regards
FrySlim

E:\sctk>git clone --branch master --single-branch https://github.com/nexB/scancode-workbench.git
Cloning into 'scancode-workbench'...
remote: Enumerating objects: 4081, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 4081 (delta 0), reused 9 (delta 0), pack-reused 4072
Receiving objects: 100% (4081/4081), 11.09 MiB | 15.62 MiB/s, done.
Resolving deltas: 100% (2754/2754), done.

E:\sctk>cd scancode-workbench

E:\sctk\scancode-workbench>git branch -a

• master
  remotes/origin/master

E:\sctk\scancode-workbench>npm install

from [..]debug-0.log:
[..]
2910 timing command:install Completed in 47756ms
2911 verbose stack Error: command failed
2911 verbose stack at ChildProcess. (C:\Program Files\nodejs\node_modules\npm\node_modules@npmcli\promise-spawn\index.js:64:27)
2911 verbose stack at ChildProcess.emit (node:events:520:28)
2911 verbose stack at maybeClose (node:internal/child_process:1092:16)
2911 verbose stack at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)
2912 verbose pkgid [email protected]
2913 verbose cwd E:\sctk\scancode-workbench
2914 verbose Windows_NT 10.0.14393
2915 verbose argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install"
2916 verbose node v16.14.0
2917 verbose npm v8.3.1
2918 error code 1
2919 error path E:\sctk\scancode-workbench\node_modules\sqlite3
2920 error command failed
2921 error command C:\Windows\system32\cmd.exe /d /s /c node-pre-gyp install --fallback-to-build
2922 error Die Projekte in dieser Projektmappe werden nacheinander erstellt. Um eine parallele Erstellung zu erm�glichen, m�ssen Sie den Schalter "-m" hinzuf�gen.
2922 error C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(439,5): error MSB8020: Die Buildtools f�r Visual Studio 2015 (Plattformtoolset = "v140") wurden nicht gefunden. Installieren Sie zum Erstellen mithilfe der v140-Buildtools die Visual Studio 2015-Buildtools. Alternativ k�nnen Sie ein Upgrade auf die aktuellen Visual Studio-Tools durchf�hren, indem Sie das Men� "Projekt" ausw�hlen oder mit der rechten Maustaste auf die Projektmappe klicken und dann "Projektmappe neu zuweisen" ausw�hlen. [E:\sctk\scancode-workbench\node_modules\sqlite3\build\deps\action_before_build.vcxproj]
2922 error Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=E:\sctk\scancode-workbench\node_modules\sqlite3\lib\binding\node-v93-win32-x64\node_sqlite3.node --module_name=node_sqlite3 --module_path=E:\sctk\scancode-workbench\node_modules\sqlite3\lib\binding\node-v93-win32-x64 --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v93' (1)
2923 error node-pre-gyp info it worked if it ends with ok
2923 error node-pre-gyp info using [email protected]
2923 error node-pre-gyp info using [email protected] | win32 | x64
2923 error node-pre-gyp WARN Using request for node-pre-gyp https download
2923 error node-pre-gyp info check checked for "E:\sctk\scancode-workbench\node_modules\sqlite3\lib\binding\node-v93-win32-x64\node_sqlite3.node" (not found)
2923 error node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.1.0/node-v93-win32-x64.tar.gz
2923 error node-pre-gyp http 403 https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.1.0/node-v93-win32-x64.tar.gz
2923 error node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.1.0/node-v93-win32-x64.tar.gz
2923 error node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v93 ABI, unknown) (falling back to source compile with node-gyp)
2923 error node-pre-gyp http 403 status code downloading tarball https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.1.0/node-v93-win32-x64.tar.gz

Then I started Visual Studio Installer und added the single component:
MSVC v140 - VS2015 C++-Buildtools (v14.00)

Afterwards, 'npm install' runs through with WARNINGS only. Although with vulnerabilities which I could not fix immediately all without breaking changes ("npm audit fix").

The 'npm start' does start the SCWB, but I was not able to Open/load a json file, because the FileDialog windows does not appear.

The 'python build.py' itself does not work to build the 'dist' folder by myself with current build tools (neither in git bash console nor normal windows console):
E:\sctk\scancode-workbench>python build.py
DEBUG: get_version: failed to get git data, usind default: 3.1.1
Traceback (most recent call last):
File "E:\sctk\scancode-workbench\build.py", line 92, in get_version
tag, distance, commit, dirty = get_git_version()
File "E:\sctk\scancode-workbench\build.py", line 137, in get_git_version
dirty = version.endswith('-dirty')
TypeError: endswith first arg must be bytes or a tuple of bytes, not str

#############################################################
=> BUILDING ScanCode Workbench App release: 3.1.1
platform: Windows-10-10.0.14393-SP0 sys.platform: win32
using NPM bin at: b'E:\sctk\scancode-workbench\node_modules\.bin'
Traceback (most recent call last):
File "E:\sctk\scancode-workbench\build.py", line 318, in
build(osx_sign=args.osx_sign)
File "E:\sctk\scancode-workbench\build.py", line 206, in build
electron_rebuild = os.path.join(npm_bin, 'electron-rebuild')
File "C:\Program Files\Python39\lib\ntpath.py", line 117, in join
genericpath._check_arg_types('join', path, *paths)
File "C:\Program Files\Python39\lib\genericpath.py", line 155, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components

[pombredanne]

in general, I would prefer to avoid supporting building the tool on Windows. We are building it FOR use on Windows alright, but the build scripts are POSIX and are likely to work only on WSL2 (which is Linux and not Windows) and this is not tested.

[mjherzog]

I disagree because the current purpose of SCWB is to view Scan results on your desktop. WSL on Windows is not desktop. When we have something like SCWB as a frontend for SCIO we would not want or need to support Windows directly. We seem to have the Windows build working currently for v4 so I am not sure that this is a priority issue.

[pombredanne]

@mjherzog I am only talking about where we build SCWB, not where we run it. Run-wise we support Linux, Windows and macOS alright. But @johnmhoran also told me that we need to build on Windows for Windows anyway, so both of our points are likely moot.

[mjherzog]

Requiring WSL for the build on Windows is fine if we can make it work.

[OmkarPh]

The current scancode workbench should work fine on Windows (WSL is not mandatory), Linux, and Mac (both x86 & arm64) since we're already using them in Github actions release CI.

You can follow these steps:
https://scancode-workbench.readthedocs.io/en/develop/contribute/building.html#clone-install-build-and-run