Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop refactoring #428

Merged
67 commits merged into from
Dec 28, 2020
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
eac015d
Extract `version.js` into different classes and extend tests
oltionchampari Sep 4, 2020
3fd9eac
Write tests for `pip.js` and refactor into a class method
oltionchampari Sep 4, 2020
2beb41c
Write tests for `repo.js` and refactor into a Util class
oltionchampari Sep 4, 2020
a448c9b
Merge branch 'develop' into develop_refactoring
oltionchampari Sep 10, 2020
5d202f5
Refactor SpawnUtils
oltionchampari Sep 10, 2020
19ead1e
Remove list-plugins file
oltionchampari Sep 10, 2020
9666dd8
Extend tests and extract functions into util clases
oltionchampari Sep 11, 2020
ab4ae9b
Extract mkdir function
oltionchampari Sep 11, 2020
e160827
Create `WorkspaceUtils`
oltionchampari Sep 11, 2020
9bae930
Refactor useDevVersion
oltionchampari Sep 11, 2020
d6e60ca
Fix spawnOrAbort bug
oltionchampari Sep 15, 2020
3b13e28
Extract yo method
oltionchampari Sep 15, 2020
8f5e84c
Remove unused import
oltionchampari Sep 15, 2020
03a4106
Fix useDevVersion bug
oltionchampari Sep 15, 2020
841235f
Refactor clone repo
oltionchampari Sep 15, 2020
28273c5
Commenting
oltionchampari Sep 15, 2020
df2f068
Refactor resolveNeighbours
oltionchampari Sep 15, 2020
c708b6d
Update function calls
oltionchampari Sep 15, 2020
16e04aa
Test check-node-version generator
oltionchampari Sep 15, 2020
894da0f
Minor change
oltionchampari Sep 15, 2020
e7cfd95
Remove unused import
oltionchampari Sep 15, 2020
1c245eb
Test clone-repo generator
oltionchampari Sep 15, 2020
18d9212
Extending tests
oltionchampari Sep 16, 2020
61268a9
Code cleanup
oltionchampari Sep 16, 2020
fcf51bd
Further refactoring
oltionchampari Sep 16, 2020
eeb8090
Refactor setup-workspace generator and extend tests
oltionchampari Sep 16, 2020
bb27b9c
Extend tests-fix bugs
oltionchampari Sep 16, 2020
b5c4c23
Refactor- extend tests
oltionchampari Sep 17, 2020
3ffe83f
Add comments
oltionchampari Sep 17, 2020
20c7c0f
Remove `chalk.default()` since it does not exist
oltionchampari Sep 23, 2020
10517bc
Run npm install sync
oltionchampari Sep 23, 2020
bbda376
Fix spawn function impl
oltionchampari Sep 23, 2020
6963e17
Rename function `spawn` to `spawnSync`
oltionchampari Sep 23, 2020
4a4a3df
Extend test of the init-lib sub-generator
oltionchampari Sep 23, 2020
4fab422
Extract and test function `toLibraryAliasMap`
oltionchampari Sep 23, 2020
5559968
Extract and test function `toLibraryExternals`
oltionchampari Sep 23, 2020
ff09962
Change text color
oltionchampari Sep 24, 2020
c5d32ab
Remove jenkins file from product templates
oltionchampari Sep 24, 2020
b7bf824
Extract and test function `buildPossibleAdditionalPlugins`
oltionchampari Sep 24, 2020
661bf0b
Add test for the init-product generator
oltionchampari Sep 24, 2020
c3a9484
Further refactoring
oltionchampari Sep 24, 2020
f8615de
Extract and test `stringifyAble` function
oltionchampari Sep 24, 2020
716baa6
Extract and test `toJSONFromText` function
oltionchampari Sep 25, 2020
9608dd8
Create BasePhoveaGenerator class
oltionchampari Sep 25, 2020
3c8edda
Use import instead of System.import in phovea.ts
oltionchampari Sep 25, 2020
bb1b219
Add tests for the add-extension generator
oltionchampari Sep 25, 2020
4ca798f
Further test refactoring
oltionchampari Sep 25, 2020
f50ec5c
Refactor tests
oltionchampari Sep 25, 2020
db6be89
Change folder structure of the tests
oltionchampari Sep 25, 2020
8714ae4
FIx path in tests
oltionchampari Sep 25, 2020
4627664
Use single instead of double quotes
oltionchampari Oct 30, 2020
e59fa08
Use enum for basetypes
oltionchampari Oct 30, 2020
f22f93d
Format
oltionchampari Oct 30, 2020
e459909
Do not log error twice
oltionchampari Nov 26, 2020
0d3305a
Extend tests
oltionchampari Nov 26, 2020
6f35028
Minor fix
oltionchampari Nov 26, 2020
5835196
Fix test
oltionchampari Nov 26, 2020
9772d5d
Remove cwd from config
oltionchampari Nov 26, 2020
819c6f8
add space
Nov 30, 2020
aa0ddb8
Merge branch 'develop_refactoring' of github.com:phovea/generator-pho…
Nov 30, 2020
36bd182
Copy templates in correct subdir
oltionchampari Nov 30, 2020
a1e2b95
Ignore emty dir when copying templates
oltionchampari Nov 30, 2020
c319308
Extend test
oltionchampari Nov 30, 2020
cb230f9
Transform dir name to lowercase
oltionchampari Nov 30, 2020
8eb906f
Fix cwd bug
oltionchampari Dec 1, 2020
585a58a
Remove buildInfo.js from web plugins
oltionchampari Dec 2, 2020
3e46731
Add base directory to package.json files
oltionchampari Dec 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions base/BaseInitHybridGenerator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
'use strict';
const BaseInitPluginGenerator = require("./BaseInitPluginGenerator");

class BaseInitHybridGenerator extends BaseInitPluginGenerator {

constructor(args, options) {
super(args, options, 'hybrid');
}

initializing() {
// since just last in the hierarchy used, need to do super calls
super.initializing();
}

default() {
return super.default();
}

writing() {
return super.writing();
}
}

module.exports = BaseInitHybridGenerator;
93 changes: 93 additions & 0 deletions base/BaseInitPluginGenerator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
'use strict';
const path = require('path');
const fs = require('fs-extra');
const chalk = require('chalk');
const GeneratorUtils = require('../utils/GeneratorUtils');
const BasePhoveaGenerator = require('../base/BasePhoveaGenerator');

class BaseInitPluginGenerator extends BasePhoveaGenerator {

constructor(args, options, basetype) {
This conversation was marked as resolved.
Show resolved Hide resolved
super(args, options);
this.type = path.basename(path.dirname(this.resolved)).substring(5); // init-web ... web
this.basetype = basetype || 'web';
// Make options available
this.option('skipInstall');
this.option('noSamples');
this.option('useDefaults');
this.cwd = '';
}

initializing() {
if (this._isInvalidWorkspace()) {
throw new Error(chalk.red('Execution failed, because a ".yo-rc.json" and ".yo-rc-workspace.json" file was found. If this directory is a workspace, please remove the ".yo-rc.json" file and try again.\n'));
}

this.composeWith(['phovea:_check-own-version', 'phovea:check-node-version']);

this.config.defaults({
type: this.type
});
}

_isWorkspace() {
return fs.existsSync(this.destinationPath('.yo-rc-workspace.json'));
}

_hasConfigFile() {
return fs.existsSync(this.destinationPath('.yo-rc.json'));
}

/**
* If there is both a `.yo-rc-workspace.json` and `.yo-rc.json` file in the current directory
* the workspace is invalid and the generator cannot function properly.
*/
_isInvalidWorkspace() {
return this._isWorkspace() && this._hasConfigFile();
}

/**
* Create a subdirectory in the current directory.
* Initialize the property cwd.
* @param {string} dir Directory name.
*/
_createSubDir(dir) {
if (this._isWorkspace() && this.cwd !== dir + '/') {
this.cwd = dir + '/';
GeneratorUtils.mkdir(dir);
}
}

readmeAddon() {
const f = this.templatePath('README.partial.md');
if (fs.existsSync(f)) {
return this.fs.read(f);
}
return '';
}

default() {
this.composeWith('phovea:_init-' + this.basetype, {
options: Object.assign({
readme: this.readmeAddon() + (this.options.readme ? `\n\n${this.options.readme}` : '')
}, this.options),
isWorkspace: this._isWorkspace() // inform the sub generator that the cwd is the workspace to avoid reading prompt default values from the workspace package.json
});
}

writing() {
const config = this.config.getAll();
this._createSubDir(config.cwd || config.name);
if (fs.existsSync(this.templatePath('package.tmpl.json'))) {
this._patchPackageJSON(config, null, null, this.cwd);
}
if (fs.existsSync(this.templatePath('_gitignore'))) {
this.fs.copy(this.templatePath('_gitignore'), this.destinationPath(this.cwd + '.gitignore'));
}

this._writeTemplates(config, !this.options.noSamples, this.cwd);

}
}

module.exports = BaseInitPluginGenerator;
25 changes: 25 additions & 0 deletions base/BaseInitServerGenerator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
'use strict';

const BaseInitPluginGenerator = require('./BaseInitPluginGenerator');

class BaseInitServerGenerator extends BaseInitPluginGenerator {

constructor(args, options) {
super(args, options, 'python');
}

initializing() {
// since just last in the hierarchy used, need to do super calls
super.initializing();
}

default() {
return super.default();
}

writing() {
return super.writing();
}
}

module.exports = BaseInitServerGenerator;
97 changes: 97 additions & 0 deletions base/BasePhoveaGenerator.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
'use strict';
const Generator = require('yeoman-generator');
const {merge, template} = require('lodash');
const path = require('path');
const glob = require('glob').sync;
const fs = require('fs-extra');
const GeneratorUtils = require('../utils/GeneratorUtils');

class BasePhoveaGenerator extends Generator {

/**
* Modify package.json by passing the configuration.
*
* @param {object} config Current configuration
* @param {*} unset
* @param {*} extra
* @param {*} replaceExtra
* @param {string} cwd The directory from which the generator is being called, i.e., `tdp_core/`.
* If cwd is provided than the `package.json` is going to be written to that subdirectory, otherwise to the current directory.
*/
_patchPackageJSON(config, unset, extra, replaceExtra, cwd = '') {
const pkg = this.fs.readJSON(this.destinationPath(cwd + 'package.json'), {});
let pkgPatch;
if (fs.existsSync(this.templatePath('package.tmpl.json'))) {
pkgPatch = JSON.parse(template(this.fs.read(this.templatePath('package.tmpl.json')))(config));
} else {
pkgPatch = {};
}
merge(pkg, pkgPatch);
if (replaceExtra && extra) {
Object.assign(pkg, extra);
} else {
merge(pkg, extra || {});
}

(unset || []).forEach((d) => delete pkg[d]);

this.fs.writeJSON(this.destinationPath(cwd + 'package.json'), pkg);
}

/**
* Copies the template files to the current directory or to a subdirectory if `cwd` is provided.
* @param {object} config Current configuration
* @param {*} withSamples
* @param {string} cwd The directory from which the generator is being called, i.e., `tdp_core/`.
* If `cwd` is provided than the `package.json` is going to be written to that subdirectory, otherwise to the current directory.
*/
_writeTemplates(config, withSamples, cwd = '') {
const includeDot = {
globOptions: {
dot: true
}
};

const pattern = GeneratorUtils.stringifyAble(config);

const copyTpl = (base, dbase, initialize_once) => {
// see https://github.com/SBoudrias/mem-fs-editor/issues/25
// copyTpl doesn't support glob options
const f = glob(base + '/**/*', {
dot: true
});
f.forEach((fi) => {
const rel = path.relative(base, fi);
if (!initialize_once || !fs.existsSync(this.destinationPath(cwd + dbase + rel))) {
this.fs.copyTpl(fi, this.destinationPath(cwd + dbase + rel), pattern);
}
});
};

const copy = (prefix) => {
if (fs.existsSync(this.templatePath(prefix + 'plain'))) {
this.fs.copy(this.templatePath(prefix + 'plain/**/*'), this.destinationPath(cwd), includeDot);
}

const plainTemplatePath = this.templatePath(prefix + 'plain_initialize_once');
if (fs.existsSync(plainTemplatePath)) {
copyTpl(plainTemplatePath, '', true);
}

copyTpl(this.templatePath(prefix + 'processed'), '', false);

if (config.name) {
if (fs.existsSync(this.templatePath(prefix + 'pluginname_plain'))) {
this.fs.copy(this.templatePath(prefix + 'pluginname_plain/**/*'), this.destinationPath(cwd + config.name.toLowerCase() + '/'), includeDot);
}
copyTpl(this.templatePath(prefix + 'pluginname_processed'), cwd + config.name.toLowerCase() + '/', false);
}
};
copy('');
if (withSamples) {
copy('sample_');
}
}
}

module.exports = BasePhoveaGenerator;
10 changes: 5 additions & 5 deletions generators/_init-hybrid/index.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
'use strict';
const Base = require('../../utils').Base;
const known = () => require('../../utils/known');
const {toLibraryAliasMap, toLibraryExternals} = require('../_init-web');
const BaseInitPluginGenerator = require('../../base/BaseInitPluginGenerator');
const RepoUtils = require('../../utils/RepoUtils');

class Generator extends Base {
class Generator extends BaseInitPluginGenerator {

initializing() {
this.config.defaults({
Expand Down Expand Up @@ -31,8 +31,8 @@ class Generator extends Base {
this.config.set('modules', props.modules);
this.config.set('libraries', props.libraries);
}
this.config.set('libraryAliases', toLibraryAliasMap.call(this, this.config.get('modules'), this.config.get('libraries')));
this.config.set('libraryExternals', toLibraryExternals.call(this, this.config.get('modules'), this.config.get('libraries')));
this.config.set('libraryAliases', RepoUtils.toLibraryAliasMap(this.config.get('modules'), this.config.get('libraries')));
this.config.set('libraryExternals', RepoUtils.toLibraryExternals(this.config.get('modules'), this.config.get('libraries')));
});
}

Expand Down
28 changes: 15 additions & 13 deletions generators/_init-python/index.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
'use strict';
const _ = require('lodash');
const Base = require('yeoman-generator');
const {writeTemplates, patchPackageJSON, stringifyAble, useDevVersion} = require('../../utils');
const {parseRequirements} = require('../../utils/pip');
const PipUtils = require('../../utils/PipUtils');
const NpmUtils = require('../../utils/NpmUtils');
const GeneratorUtils = require('../../utils/GeneratorUtils');
const fs = require('fs');
const BasePhoveaGenerator = require('../../base/BasePhoveaGenerator');

const known = () => require('../../utils/known');

class Generator extends Base {
class Generator extends BasePhoveaGenerator {
constructor(args, options) {
super(args, options);

Expand Down Expand Up @@ -66,8 +67,8 @@ class Generator extends Base {
}

_generateDependencies(useDevelopDependencies, cwd) {
let requirements = parseRequirements(this.fs.read(this.destinationPath(cwd + 'requirements.txt'), {defaults: ''}));
const dockerPackages = parseRequirements(this.fs.read(this.destinationPath(cwd + 'docker_packages.txt'), {defaults: ''}));
let requirements = PipUtils.parseRequirements(this.fs.read(this.destinationPath(cwd + 'requirements.txt'), {defaults: ''}));
const dockerPackages = PipUtils.parseRequirements(this.fs.read(this.destinationPath(cwd + 'docker_packages.txt'), {defaults: ''}));

const concat = (p) => Object.keys(p).map((pi) => pi + p[pi]);
// merge dependencies
Expand All @@ -78,9 +79,9 @@ class Generator extends Base {
modules.filter(known().plugin.isTypeServer).forEach((m) => {
const p = known().plugin.byName(m);

// avoid having a requirement twice in two different formats that occurs when in the requirements.txt a requirement is written
// in the format -e git+https://github.com/phovea/[email protected]#egg=phovea_server
// and the incoming format is phovea_server>=5.0.1,<6.0.0
// avoid having a requirement twice in two different formats that occurs when in the requirements.txt a requirement is written
// in the format -e git+https://github.com/phovea/[email protected]#egg=phovea_server
// and the incoming format is phovea_server>=5.0.1,<6.0.0
if (!useDevelopDependencies) {
const devRequirement = Object.keys(p.develop.requirements)[0];
const masterRequirment = Object.keys(p.requirements)[0];
Expand All @@ -106,17 +107,18 @@ class Generator extends Base {
writing() {
const config = this.config.getAll();
this.cwd = this.options.isWorkspace ? (config.cwd || config.name) + '/' : '';
const deps = this._generateDependencies(useDevVersion.call(this, this.cwd), this.cwd);
const {version} = this.fs.readJSON(this.destinationPath(this.cwd + 'package.json'), {version: '1.0.0'});
const deps = this._generateDependencies(NpmUtils.useDevVersion(version), this.cwd);

patchPackageJSON.call(this, config, ['devDependencies'], null, null, this.cwd);
writeTemplates.call(this, config, !this.options.noSamples, this.cwd);
this._patchPackageJSON(config, ['devDependencies'], null, null, this.cwd);
this._writeTemplates.call(this, config, !this.options.noSamples, this.cwd);

this.fs.write(this.destinationPath(this.cwd + 'requirements.txt'), deps.requirements.join('\n'));
this.fs.write(this.destinationPath(this.cwd + 'docker_packages.txt'), deps.dockerPackages.join('\n'));

// don't overwrite existing registry file
if (!fs.existsSync(this.destinationPath(this.cwd + config.name.toLowerCase() + '/__init__.py'))) {
this.fs.copyTpl(this.templatePath('__init__.tmpl.py'), this.destinationPath(this.cwd + config.name.toLowerCase() + '/__init__.py'), stringifyAble(config));
this.fs.copyTpl(this.templatePath('__init__.tmpl.py'), this.destinationPath(this.cwd + config.name.toLowerCase() + '/__init__.py'), GeneratorUtils.stringifyAble(config));
}
this.fs.copy(this.templatePath('_gitignore'), this.destinationPath(this.cwd + '.gitignore'));
this.fs.copy(this.templatePath('docs_gitignore'), this.destinationPath(this.cwd + 'docs/.gitignore'));
Expand Down
Loading