Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript API for toc #1237

Closed
wants to merge 13 commits into from
Closed

Javascript API for toc #1237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
a3eb4a5
three toc api completed
bakshiutkarsha Jul 26, 2020
af6b245
gitignore updated
bakshiutkarsha Jul 31, 2020
08a3080
suggested changes applied
bakshiutkarsha Aug 8, 2020
66591ac
path chnged
bakshiutkarsha Aug 24, 2020
361823d
file path changes
bakshiutkarsha Aug 25, 2020
20c5472
file path for bundled file set
bakshiutkarsha Aug 25, 2020
eda3f1a
file path for bundled file set
bakshiutkarsha Aug 25, 2020
ad802b8
exporting zim object
bakshiutkarsha Aug 25, 2020
f5e8d99
exporting zim obj
bakshiutkarsha Aug 25, 2020
b99de3e
file location change in project
bakshiutkarsha Aug 29, 2020
40c434e
zim object exposed globally
bakshiutkarsha Sep 2, 2020
1c94d3b
master rebased
bakshiutkarsha Sep 2, 2020
c7c6068
package lock changed
bakshiutkarsha Sep 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .babelrc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"presets": [
["@babel/preset-env", {
"useBuiltIns": "entry",
"corejs": 3
}]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bakshiutkarsha Looking at the Babel usage guide, I think the issue with missing polyfills might be because you haven't specified any browsers in this setup file. E.g. the setup has these entries (NB don't use these, they need to be modified for our needs):

  "presets": [
    [
      "@babel/env",
      {
        "targets": {
          "edge": "17",
          "firefox": "60",
          "chrome": "67",
          "safari": "11.1",
        },
        "useBuiltIns": "usage",
        "corejs": "3.6.4",
      }
    ]
  ]

As a minimum we need to support:

"edge": "15",
"firefox": "52",
"ie": "11"

I'm not sure what we support for Chrome (e.g. minimum Android version) and Safari for iOS app), but with IE11, most other things are probably covered.

Copy link
Contributor Author

@bakshiutkarsha bakshiutkarsha Sep 9, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Jaifroid I have done that in package.json here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. So any thoughts on why it's not working?

Two random guesses:

  1. (I don't really know babel config, so correct me if I'm wrong): maybe adding the browser targets in package.json only affects the environment in which mwoffliner runs, i.e. node.js (I assume)? Whereas we need a config that will load corejs polyfills according to the environment in which -/_api/toc.js is running, i.e. the detection logic needs to run inside toc.js. That may be completely wrong...
  2. You're using corejs 3 with babel-preset-env. I wonder if the breaking change mentioned in Update to core-js@3 babel/babel#7646 affects this setup:

Updating @babel/polyfill to core-js@3 will break @babel/preset-env. @babel/preset-env with useBuiltIns transforms import of @babel/polyfill to import of required core-js modules. Most users don't add core-js dependency directly. In case of a minor update, even if @babel/polyfill dependency will be updated with @babel/preset-env, in the top level of node_modules we will have core-js@2, not core-js@3 because a serious part of dependencies depends on packages with core-js@2 in dependencies.

A not great alternative would be to provide our own closest function if Element.closest is undefined:

        // Create a closest function alternative because IE11 and others do not support closest
        function closest (ele, s) {
            var cele = ele;
            var cmatches = Element.prototype.matches || Element.prototype.msMatchesSelector || Element.prototype.webkitMatchesSelector;
            do {
                if (cmatches.call(cele, s)) return cele;
                cele = cele.parentElement || cele.parentNode;
            } while (cele !== null && cele.nodeType === 1);
            return null;
        };

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bakshiutkarsha Can you help here or have you left this PR to someone else?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can close this. Will implement the changes suggested by @Jaifroid .

]
}
4 changes: 3 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,6 @@ npm-debug.log
.env
.nyc_output
coverage
mwo-test-*
mwo-test-*
dist/*
res/toc.js
18,975 changes: 12,438 additions & 6,537 deletions package-lock.json
View file
Open in desktop

Large diffs are not rendered by default.

21 changes: 17 additions & 4 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
},
"scripts": {
"example-wiki": "docker run -p 8080:80 -v $(pwd)/test/mediawiki:/var/www/data -it openzim/mediawiki",
"mwoffliner": "node ./lib/cli.js",
"mwoffliner": "node ./lib/cli.js && npm run webpack-build",
"tape": "tape -r tsconfig-paths/register -r ts-node/register",
"test:e2e": "npm run tape -- test/e2e/*.test.ts | tap-spec",
"test:unit": "npm run tape -- test/unit/*.test.ts | tap-spec",
Expand All @@ -25,7 +25,9 @@
"redis:start": "docker run -p 6379:6379 --name redis-mwoffliner -d redis",
"redis:kill": "docker rm -f redis-mwoffliner || :",
"redis": "npm run redis:kill && npm run redis:start",
"start": "./node_modules/.bin/ts-node ./src/cli.ts"
"start": "./node_modules/.bin/ts-node ./src/cli.ts",
"develop": "webpack --mode development --watch",
"webpack-build": "webpack --mode production"
},
"nyc": {
"extension": [
Expand Down Expand Up @@ -54,6 +56,7 @@
},
"dependencies": {
"@openzim/libzim": "2.4.2",
"@babel/polyfill": "^7.10.4",
"@types/async": "^3.2.3",
"@types/backoff": "^2.5.1",
"@types/bluebird": "^3.5.32",
Expand All @@ -74,6 +77,7 @@
"aws-sdk": "^2.777.0",
"axios": "^0.20.0",
"backoff": "^2.5.0",
"core-js": "^3.6.5",
"country-language": "^0.1.7",
"deepmerge": "^4.2.2",
"details-element-polyfill": "^2.4.0",
Expand Down Expand Up @@ -114,13 +118,17 @@
"zim"
],
"devDependencies": {
"@babel/core": "^7.11.0",
"@babel/preset-env": "^7.11.0",
"@babel/register": "^7.10.5",
"@percy/puppeteer": "^1.1.0",
"@types/blue-tape": "^0.1.33",
"@types/jest": "^26.0.15",
"@types/redis": "^2.8.28",
"@types/redis-mock": "^0.17.0",
"@types/tape-promise": "^4.0.1",
"@types/tmp": "^0.2.0",
"babel-loader": "^8.1.0",
"blue-tape": "^1.0.0",
"dotenv": "^8.2.0",
"eslint": "^7.11.0",
Expand All @@ -134,9 +142,14 @@
"tmp": "^0.2.1",
"ts-jest": "^26.4.1",
"ts-node": "^9.0.0",
"tsconfig-paths": "^3.9.0"
"tsconfig-paths": "^3.9.0",
"webpack": "^4.44.1",
"webpack-cli": "^3.3.12"
},
"jest": {
"preset": "ts-jest/presets/js-with-ts"
}
},
"browserslist": [
"> .05%"
]
}
1 change: 1 addition & 0 deletions res/toc.js
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions src/config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ const config = {
jsResources: ['script', 'masonry.min', 'article_list_home', 'images_loaded.min',
'../node_modules/details-element-polyfill/dist/details-element-polyfill'],

apiResources:['toc'],

// JS/CSS resources to be imported from MediaWiki
mw: {
css: [
Expand All @@ -73,6 +75,7 @@ const config = {
javascript: 'j',
styleModules: 'css_modules',
jsModules: 'js_modules',
api: '_api'
},

// Output templates (TODO: Needs more docs)
Expand Down
15 changes: 15 additions & 0 deletions src/util/misc.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,20 @@ export function saveStaticFiles(config: Config, zimCreator: ZimCreator) {
logger.warn(`Could not create ${js} file : ${error}`);
}
});

const apiPromises = config.output.apiResources.map(async (api) => {
try {
const apiConst = await readFilePromise(pathParser.resolve(__dirname, `../../res/${api}.js`));
const article = new ZimArticle({ url: apiPath(config, api), data: apiConst, ns: '-' });
zimCreator.addArticle(article);
} catch (error) {
logger.warn(`Could not create ${api} file : ${error}`);
}
});
return Promise.all([
...cssPromises,
...jsPromises,
...apiPromises
]);
}

Expand All @@ -179,6 +190,10 @@ export function jsPath({ output: { dirs } }: Config, js: string) {
const path = (isNodeModule(js)) ? normalizeModule(js) : js;
return [dirs.javascript, `${dirs.jsModules}/${path.replace(/(\.js)?$/, '')}.js`].join('/');
}
export function apiPath({ output: { dirs } }: Config, api: string) {
const path = (isNodeModule(api)) ? normalizeModule(api) : api;
return [dirs.api, `${path.replace(/(\.js)?$/, '')}.js`].join('/');
}
export function genHeaderCSSLink(config: Config, css: string, articleId: string, classList = '') {
const resourceNamespace = '-';
const slashesInUrl = articleId.split('/').length - 1;
Expand Down
26 changes: 26 additions & 0 deletions webpack.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
const path = require('path')
const webpack = require('webpack')

module.exports = {
entry: './zim/api/toc.js',
output: {
filename: 'toc.js',
path: path.resolve(__dirname, './res/')
},
module: {
rules: [{
test: /\.js$/,
exclude: /node_modules/,
use: {
loader: 'babel-loader',
}
}]
},
plugins: [
new webpack.ProvidePlugin({
'$': 'jquery',
'jQuery': 'jquery',
'window.jQuery': 'jquery'
})
],
}
45 changes: 45 additions & 0 deletions zim/api/toc.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import "core-js/stable";
import "regenerator-runtime/runtime";

let zim = {toc: {}};

function getSectionList(){
let sectionList = [];
for (i = 0; i < document.querySelectorAll('h1, h2, h3, h4, h5, h6').length; i++) {
let headerObject = document.querySelectorAll('h1, h2, h3, h4, h5, h6')[i];
if (headerObject.id === "") {
headerObject.id = "documentparserid" + i;
}
sectionList.push(headerObject);
}
return sectionList;
}

zim.toc.hasTableOfContent = function() {
return document.querySelectorAll('h1, h2').length > 0 ? true : false;
}

zim.toc.getSections = function() {
let respArrOfSections = [];
const sectionList = getSectionList()
sectionList.forEach(section => {
respArrOfSections.push({
"toc_level": section.tagName,
"section_id": section.id,
"section_name": section.innerText,
})
});
return respArrOfSections;
}

zim.toc.scrollToSection = function(index){
const sectionIdElem = getSectionList()[index];
sectionIdElem.closest('details').setAttribute('open', '');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bakshiutkarsha I think this line has to be changed to:

const closestClosed = sectionIdElem.closest('details:not([open])');
if (closestClosed) closestClosed.setAttribute('open', '');

The reason is that often h3 headings are open in MediaWiki ZIMS, while the parent h2 is closed, so we have to search for the nearest ancestor that is closed and open that one.

location.href = `#${sectionIdElem.id}`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bakshiutkarsha This can be changed to:

sectionIdElem.scrollIntoView();

}

window.zim = zim;

export default {
zim: zim
}