Keeping Firefox up-to-date

[vringar]

In #1090 I said:

Please be aware however, that the Firefox version used by OpenWPM is severely out of date and before running any large scale crawls you should probably take the steps described under docs/Release-Checklist to look closer to normal web traffic

as we are currently on FF123 when FF127 has been released for multiple weeks and user statistics show that roughly 70% of the population updates to a new version of FF within 4 weeks.

I also pointed out that I do not have the energy or focus to spare to keep up with this update schedule.

The first step on this page outlines how to get a new version of Firefox integrated into OpenWPM.

@palmeida just to clarify another point from our discussion in the issue: I'm not asking you to take over as a maintainer, what I'm asking you to do is to open a PR with the updated version, so that we can at least close the 4 version gap one time.
If you feel like doing this more regularly then I'd really appreciate that but that's not my expectation.
I've just not been a user of this tool in the last years and everybody seemed happy to use it but rarely did anyone contribute back.

[palmeida]

@vringar Yes, I think it makes sense to start with something simple like a single PR updating the version and then I can think about doing it regularly.

I tried following the instructions in the release page. Step 1, updating Firefox, was easy and it worked, I got FIREFOX_128_0_RELEASE and ran it both on Docker and locally. Then I tried the next step and had some issues.

Step 2, running update.sh, initially failed because repin.sh uses the --force switch from mamba, which had already been removed from another script in OpenWPM. I removed it from repin.sh and got a new error:

CondaValueError: prefix already exists: /home/palmeida/miniforge3/envs/openwpm

Then I tried running just repin.sh and got a different error:

EnvironmentFileNotFound: '/home/palmeida/Projects/fare-audit/OpenWPM/environment-unpinned.yaml' file not found

Should I have that file? Do I create it from environment.yaml? Or is there something else I'm doing wrong?

[vringar]

Thanks for fixing the update script and for learning how to update everything!
Unfortunately, repin.sh currently has to be executed in the scripts directory.
If you want, you can fix it to be runnable from the root of the repo instead. I never invested the time but believe it is a good change to make (the relevant files are repin.sh prune-environment.py and update.sh)

[palmeida]

I see, I didn't think to check the scripts folder for the missing files. But that means the original issue was the real problem:

CondaValueError: prefix already exists: /home/palmeida/miniforge3/envs/openwpm

I thought it was weird I had to do that, but now I deleted the openwpm environment before running update.sh and it worked. My git status is this:

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   Extension/package-lock.json
	modified:   environment.yaml
	modified:   package-lock.json
	modified:   scripts/install-firefox.sh
	modified:   scripts/repin.sh

This seems reasonable, looking at the commit for the latest Firefox upgrade, when it was updated to 123. I still haven't changed the VERSION, which I guess would be 0.29.0 and the CHANGELOG, which I would copy from that commit and do something like bump to FF 128. Does that sound good?

[vringar]

That sounds great!
I'm unfortunately not as familiar as I'd like to be with the current state of the Mama/conda ecosystem. My understanding was that MambaForge was supposed to be a distribution of the mamba tool chain that is fully compatible with previous micromamba distribution but that doesn't seem to be the case.
I guess we can add deleting the environment to the update script as we'll recreate the eng anyway

[palmeida]

I created the pull request. I haven't done this in a while, I hope I didn't miss anything obvious.

Yes, deleting the environment sounds reasonable. If this is all there is to it, I don't think it will be a problem to do it regularly.