Release version 3.0 #654

Jaifroid · 2020-09-30T20:00:28Z

Jaifroid · 2020-09-30T20:06:23Z

@ikreymer it may be of interest to you and your replay project that Kiwix JS now supports ZSTD-encoded ZIM files, and it will be in release 3.0 we are preparing for this issue. Support is already merged into the master branch. Given that you used our implementation of XZ for your project, you might now want to add ZSTD decoding.

Jaifroid · 2020-09-30T20:08:35Z

@ernesst Are you able to test our master branch for support on Ubuntu Touch? We are preparing release 3.0.

ernesst · 2020-10-01T19:50:49Z

Sure, I'll do it by Monday, if not please do not hesitate to ping me.

Is this version relevant for the test : https://download.kiwix.org/nightly/2020-10-01/kiwix-ubuntu-touch-2.9commit-0c53010.click ?

Jaifroid · 2020-10-03T09:03:41Z

@ernesst Thank you! Yes, that package corresponds to 0c53010. Monday would be fine to let us know if there are any issues or not. You might like to test it with a ZSTD-encoded ZIM, though don't feel you have to. Two ZIM files are available: https://tmp.kiwix.org/wikipedia_bm_all_2020-07.zstd.zim (~30MB) and https://tmp.kiwix.org/wikipedia_en_medicine_2020-09_zstd.zim (~4GB).

mossroy · 2020-10-03T15:32:01Z

I've done a few tests on Firefox 81.0 (through extension or not), Chromium 85.0 (through extension or not), and Firefox OS 2.5
The new zstd ZIM files work on all these platforms, and I did not see any regression on various other types of ZIM files.

Jaifroid · 2020-10-03T15:45:35Z

Thanks @mossroy . I think the CHANGELOG will only have one thing: "Support for reading ZIM archives compressed with the zstandard compression algorithm". I'll do a PR for it nevertheless, so we can add in the release date before merging.

mossroy · 2020-10-03T15:54:09Z

@kelson42 : if you agree, I could give @Jaifroid the keys and info that would let him be able to do some (or all) of the steps I do on releases.
It is at least :

a private SSH key to be able to upload to the server behind https://download.kiwix.org/nightly/
the private key used by Travis to read the secret files
the Mozilla API key used to validate and sign the Firefox nightly builds

I could show him the manual steps I do on releases (it's not complicated), so that he might be more autonomous in the future, if necessary.

NB : if you do the release, we might call version 3.0 "the Jaifroid release", as you literally did all the work ;-)

Jaifroid · 2020-10-03T15:57:26Z

He he, but it's always a collaboration. (I could not have done zstddec wrapper if I didn't have the xz wrapper model to follow closely...).

Happy to learn how to do those things if it's OK with @kelson42.

mossroy · 2020-10-03T16:11:02Z

What can already be done together is to rename the github milestones, and move the remaining issues.
Here is the procedure :

in https://github.com/kiwix/kiwix-js/milestones , rename the milestones. Usually, when it's not a major version, I simply increment the second digit of all milestones (starting for the highest ones to avoid problems). In the current case (major version), 2.9 must be renamed 3.0, 2.10 -> 3.2, 2.11 -> 3.3
create a new milestone (3.1, here) for everything that has not been fixed in 3.0
select all the open issues in 3.0 (except the one for the release) : here, it is in https://github.com/kiwix/kiwix-js/milestone/23 , and use the github feature to set them to milestone 3.1
set an approximate due date to 3.0

That should be enough for this step

Jaifroid · 2020-10-03T16:30:11Z

Thanks for explanation. I'll do it this evening.
EDIT: Done.

ernesst · 2020-10-03T18:21:22Z

Tested on ubuntu touch with ZSTD-encoded ZIM, works fine ! Thanks

…

On Sat, 3 Oct 2020 02:03:52 -0700, Jaifroid wrote: @ernesst Thank you! Yes, that package corresponds to 0c53010. Monday would be fine to let us know if there are any issues or not. You might like to test it with a ZSTD-encoded ZIM, though don't feel you have to. Two ZIM files are available: https://tmp.kiwix.org/wikipedia_bm_all_2020-07.zstd.zim (~30MB) and https://tmp.kiwix.org/wikipedia_en_medicine_2020-09_zstd.zim (~4GB).

-- Sent using Dekko from my Ubuntu device

Jaifroid · 2020-10-04T08:00:35Z

Thank you @ernesst!

Jaifroid · 2020-10-04T08:08:01Z

@mossroy I could create the tag now, after adding the date to CHANGELOG and merging. Are you OK for me to do that today?

mossroy · 2020-10-04T09:01:42Z

I won't be available all the time today, but yes : you can create the tag today.
I suggest to create a release through the github UI : https://github.com/kiwix/kiwix-js/releases (with a changelog), that will also create the tag.
It's highly probable that it won't trigger the relevant build on Travis (because it did not for the last 2 releases, at least).

But you should be able to create the release packages (you need the files I sent you with an encrypted email), by running the following command in the scripts directory :
./create_all_packages.sh -t -v 3.0.0

I only tested with bash on Ubuntu. It might work with WSL.
I suggest you should test first a "dry-run", by adding -d command-line argument to the command (it will generate the packages, without uploading them, or communicating with Mozilla servers)

Jaifroid · 2020-10-04T10:10:08Z

@mossroy I've created a draft release, but before I publish it and create the tag, should I update the version numbers in package.json and manifest.webapp or is that something that creating the tags should trigger? Any other file with a version number that you're aware of that might need manually updating?
EDIT: index.html also has a version number - should I update manually?

mossroy · 2020-10-04T10:18:23Z

Creating the tag does not update the version numbers. create_all_packages.sh updates them on-the-fly before generating the packages (and without commiting anything), and we manually update them after the release for next release.
Try a dry run of create_all_packages.sh. You should be able to check by looking into the generated packages.

Jaifroid · 2020-10-04T11:14:19Z

I successfully ran the dry run on WSL (Ubuntu flavour) for all packages except for the Ubuntu touch build. All the other packages are packaged and zipped up, but with the .click package I get these messages:

Packaging .click application for Ubuntu Touch, version 3.0.0
scripts/package_ubuntu_touch_app.sh: line 28: click: command not found
mv: cannot stat 'kiwix*.click': No such file or directory
Skipping uploading the files, because it's a dryrun test

I tried sudo apt install click, but get "Unable to locate package click". I am offered:

Command 'click' not found, did you mean:
  command 'klick' from deb klick
Try: sudo apt install <deb name>

mossroy · 2020-10-04T13:30:35Z

Try to install the deb package from https://packages.ubuntu.com/xenial-updates/click
As you're probably using x86_64 CPU architecture, it should be from https://packages.ubuntu.com/xenial-updates/amd64/click/download

mossroy · 2020-10-04T14:44:40Z

In fact, when Canonical dropped Ubuntu Touch, it also dropped the click package. So it is not available any more in recent versions of Ubuntu.
But the .deb file of earlier versions of Ubuntu (xenial, here) still works, at least on Ubuntu 18.04. You can download the .deb file somewherer, and try to install it with sudo dpkg -i click_0.4.43+16.04.20170613-0ubuntu1_amd64.deb

Jaifroid · 2020-10-04T16:38:47Z

Thanks for the leads, @mossroy I'll look for it later this evening.

Jaifroid · 2020-10-05T19:09:26Z

So, I found that package in the Ubuntu archive, and tried to install, but I get:

dpkg: dependency problems prevent configuration of click-dev:
 click-dev depends on python3-click-package (= 0.4.43+16.04.20170613-0ubuntu1); however:
  Package python3-click-package is not installed

I have found an Ubuntu packages page for python3-click-pakage here, but I'm afraid I don't know which of the packages to download or how to install.

While solving this, I'll try to go ahead with the other packgages, i.e. I'll try a non-dry-run now.

mossroy · 2020-10-05T19:18:52Z

Wait, it would be safer to do all of them in one go.
Because the script slightly modifies some files between each package, so the order is important

Jaifroid · 2020-10-05T19:20:05Z

OK, I haven't started. Will wait.

mossroy · 2020-10-05T19:21:52Z

I have found the .deb packages I had put aside for that. There are 5 files. You were on the right track but I'll send them to you by e-mail to simplify

Jaifroid · 2020-10-08T06:22:03Z

With @mossroy's help, I've now been able to build the unsigned binaries and upload them to download.kiwix.org. 😊

I can take care of uploading/signing in the Edge Store.

EDIT: Version 3.0.0 files have been uploaded to: https://download.kiwix.org/nightly/2020-10-08/

Jaifroid · 2020-10-08T06:51:37Z

3.0.0 now in review in Edge Extension Store. The page for the Edge Extension is:

https://microsoftedge.microsoft.com/addons/detail/jlepddlenlljlnnhjinfaciabanbnjbp
(currently showing 2.8.0 until review is passed).

mossroy · 2020-10-08T19:38:56Z

It's now available on Firefox and Ubuntu Touch

Jaifroid · 2020-10-08T19:46:28Z

Great, @kelson42 are you able to upload the Chrome version? I've uploaded to the Edge store.

mossroy · 2020-10-09T10:02:20Z

@Jaifroid : to prepare the source code for v3.1, you can do the same as what had been done for the last release. See commit e126ee8

kelson42 · 2020-10-09T12:44:02Z

I have uploaded the ZIP, but the web admin of chrome extension tells me this is version 2.9. Might that be that the manifest file for chrome extension is wrong?

mossroy · 2020-10-09T12:49:02Z

Can you check you uploaded https://download.kiwix.org/nightly/2020-10-08/kiwix-chrome-unsigned-extension-3.0.0.zip ? I verified : the right version is in the manifest.
Maybe the web interface is still showing v2.9 until v3.0 is approved?

Jaifroid · 2020-10-09T12:49:19Z

Hmm, manifest looks OK:

    "manifest_version": 2,
    "name": "Kiwix",
    "version": "3.0.0",

    "description": "Kiwix : offline Wikipedia reader",

However, the Edge extension (which is the same file) has been stuck for two days "in review", when it should take a few hours.

kelson42 · 2020-10-09T13:10:54Z

Maybe the web interface is still showing v2.9 until v3.0 is approved?

AFAIK latest version is 2.8.0.

kelson42 · 2020-10-10T07:49:34Z

Available on chrome store but labeled 2.9, see https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfcioahhk?hl=en

Jaifroid · 2020-10-10T07:53:56Z

Hmm, I wonder if this is the wrong package. It shows this in the top corner:

Jaifroid · 2020-10-10T07:54:41Z

@Jaifroid : to prepare the source code for v3.1, you can do the same as what had been done for the last release. See commit e126ee8

OK @mossroy will do now.

mossroy · 2020-10-10T08:06:53Z

@kelson42 : I'm pretty sure you did not upload the file I mentioned above, but the nightly build in the same directory.
It's not a big issue because the version number (2.9) is lower, and the code is the same.
Can you upload https://download.kiwix.org/nightly/2020-10-08/kiwix-chrome-unsigned-extension-3.0.0.zip ?

kelson42 · 2020-10-11T06:20:44Z

You are right. Problem is now fixed in Chrome extensions store. Will upload the file to download.kiwix.org ASAP.

Jaifroid · 2020-10-12T19:40:25Z

This has now finally passed to 3.0 in the Edge Store as well, so I think everything is updated. If @kelson42 has uploaded the binaries to download.kiwix.org, then we can complete the last steps here, publicize and close this issue.

kelson42 · 2020-10-13T08:01:42Z

@Jaifroid new Kiwix JS 3.0.0 packages uploaded to Firefox/Chrome stores and on download.kiwix.org

Jaifroid · 2020-10-17T11:40:04Z

@mossroy How do you update https://kiwix.github.io/kiwix-js/www/index.html ? On KJSW, every time I commit to master the kiwix.github.io implementation is automatically updated, but here it seems to be a manual process. I notice it is still on version 2.8.

Jaifroid · 2020-10-17T11:53:00Z

I have added a reddit post about Kiwix JS version 3.0. @kelson42 can you publicize it where you normally do?

kelson42 · 2020-10-18T12:50:04Z

@Jaifroid Done https://twitter.com/KiwixOffline/status/1317810034592669699

Jaifroid · 2020-10-20T09:10:27Z

I figured out which branch GitHub Pages was running from and have updated it to current master. I did try setting it to the last release, but it then still shows the wrong version number, which is maybe a slight issue with the workflow for releases. It would make sense for the release commit to have the correct version number.

If it should be done differently, we can always commit separately to the gh-pages branch. But I'm closing this issue now as complete.

mossroy · 2020-10-21T11:18:50Z

To update https://kiwix.github.io/kiwix-js/www/index.html , I usually use scripts/push_to_gh_pages.sh to update gh-pages branch.
I agree it's not great regarding the version number.
There is room for improvements on this release process.

Jaifroid · 2020-10-21T12:06:39Z

OK, thanks, I wasn't aware of that script!

Jaifroid added the information label Sep 30, 2020

Jaifroid assigned Jaifroid and mossroy Sep 30, 2020

Jaifroid added this to the v3.0 milestone Oct 3, 2020

Jaifroid closed this as completed Oct 20, 2020

Release version 3.0 #654

Release version 3.0 #654

Comments

Jaifroid commented Sep 30, 2020 • edited Loading

Jaifroid commented Sep 30, 2020

Jaifroid commented Sep 30, 2020

ernesst commented Oct 1, 2020

Jaifroid commented Oct 3, 2020

mossroy commented Oct 3, 2020 • edited Loading

Jaifroid commented Oct 3, 2020

mossroy commented Oct 3, 2020

Jaifroid commented Oct 3, 2020

mossroy commented Oct 3, 2020

Jaifroid commented Oct 3, 2020 • edited Loading

ernesst commented Oct 3, 2020 via email

Jaifroid commented Oct 4, 2020

Jaifroid commented Oct 4, 2020

mossroy commented Oct 4, 2020

Jaifroid commented Oct 4, 2020 • edited Loading

mossroy commented Oct 4, 2020

Jaifroid commented Oct 4, 2020

mossroy commented Oct 4, 2020

mossroy commented Oct 4, 2020

Jaifroid commented Oct 4, 2020

Jaifroid commented Oct 5, 2020 • edited Loading

mossroy commented Oct 5, 2020 • edited Loading

Jaifroid commented Oct 5, 2020

mossroy commented Oct 5, 2020

Jaifroid commented Oct 8, 2020 • edited Loading

Jaifroid commented Oct 8, 2020

mossroy commented Oct 8, 2020

Jaifroid commented Oct 8, 2020

mossroy commented Oct 9, 2020

kelson42 commented Oct 9, 2020

mossroy commented Oct 9, 2020

Jaifroid commented Oct 9, 2020

kelson42 commented Oct 9, 2020

kelson42 commented Oct 10, 2020

Jaifroid commented Oct 10, 2020

Jaifroid commented Oct 10, 2020

mossroy commented Oct 10, 2020

kelson42 commented Oct 11, 2020

Jaifroid commented Oct 12, 2020

kelson42 commented Oct 13, 2020

Jaifroid commented Oct 17, 2020

Jaifroid commented Oct 17, 2020

kelson42 commented Oct 18, 2020

Jaifroid commented Oct 20, 2020

mossroy commented Oct 21, 2020

Jaifroid commented Oct 21, 2020

Jaifroid commented Sep 30, 2020 •

edited

Loading

mossroy commented Oct 3, 2020 •

edited

Loading

Jaifroid commented Oct 3, 2020 •

edited

Loading

Jaifroid commented Oct 4, 2020 •

edited

Loading

Jaifroid commented Oct 5, 2020 •

edited

Loading

mossroy commented Oct 5, 2020 •

edited

Loading

Jaifroid commented Oct 8, 2020 •

edited

Loading