Package management for Meteor-agnostic packages in Meteor, using jspm.

[trusktr]

Hi @guybedford There's some discussion at MeteorCommunity/discussions#1 on preventing from having to create wrapper packages for meteor-agnostic libraries. Nothing technical yet, just ideas. Meteor is so nice, but since it uses it's own package management system, every meteor-agnostic package has to be wrapped in a Meteor package then shipped on atmospherejs.com. I'm still learning about Meteor myself, so I'm not ready to try to propose a working solution, but I have a feeling jspm might be the tool to use for this. I just wanted to open this issue to have an official place for discussion if needed.

[subesokun]

I've never worked with Meteor but it looks interesting and seems to have with Atmosphere a large package repository. One argument of them against jspm seems to be that the registry is too small. But actually this registry isn't mandatory in case the package doesn't need some special package overwrites. But if there is a Meteor endpoint for jspm which correctly translates the package.js into a package.json they don't need such package overwrites and could access all their packages maintained under Atmosphere just via a simple:

jspm install meteor:accounts-password

# Will then automatically fetch the dependencies from the Atmosphere registry
underscore
meteor
npm-bcrypt
...

or

# install iron:router
jspm install meteor:iron/router

So there is actually no worry about the jspm registry as they could still use their own registry.

Having this Meteor endpoint for jspm would also mean that they don't have to write a wrapper in order to support jspm. And to create local aliases for their packages within the package.js they could just do a

// package.js
api.use('iron=iron:[email protected]');  // looks more like jspm syntax
// or if they prefer
api.use('iron:[email protected]').as('iron'); 

// some.js
// and import iron in their js files with
import iron;

// or with the normalized name
import meteor:iron/layout

But as I've never used Meteor I can't say what actually happens behind the scenes of these api calls. But the big benefit for them when using jspm would be of course that they can use SystemJS right away and don't have to write their own ES6 compatible System.import wrapper (which is not an easy task at all) or some none ES6 spec compatible wrapper as Bower.require('famous');.
With SystemJS they're open to the world and can include any AMD, CJS or ES6 module... and in case Meteor has their own module format they could add a plugin to SystemJS to support it.

.. so far my quick brainstorming in the morning :)

[subesokun]

In Meteor it seems they're defining the order of how files shall be loaded in the package.js via the api.add_files() calls so they have kind of own module format. But this is no problem if they add a Metor plugin to SystemJS which just imports the files in the correct order as defined in the package.js. In future it might be probably better two use also for the packages itself the ES6 module format.

The nice thing about such plugin would be that they can use packages formatted as ES6, AMD, or CJS together or with their own modules and then step by step update the packages to consists only out of ES6 modules if they wish to.

And the api.export() calls won't be also a problem with jspm / SystemJS as it has an export mechanism as well. So the plugin could add then an export command to the transpiled meteor module.

Edit: Actually in case of Meteor a build step within the endpoint just as for npm packages would make more sense. So upon installing the packages the endpoint will merge all files in the order defined by the add_files calls and adds the correct export commands at the end of this file. This way also no plugin is required for SystemJS.

[trusktr]

@subesokun Thanks for the interesting take! This would make jspm a meteor replacement for package management, so the jspm command would be used instead of the meteor command to install packages, which would make it easy for Meteor packages to be loaded outside of Meteor. This would be great because non-meteor projects could potentially import meteor packages from Atmosphere. What I'm wanting to do is actually a little different:

There's a problem with Meteor where if someone wants to use jQuery in their Meteor project, then they have to create a "wrapper" package for jQuery and put it on Atmosphere. This means that for every non-meteor package (e.g. jQuery, Underscore, Bootstrap, etc) a new package in Atmosphere needs to be created to allow that external library to be imported into the Meteor app using the api.* methods inside of package.js, etc.

This is highly inefficient because there are thousands of meteor-agnostic packages that would need to be wrapped up and put on Atmosphere, polluting Atmosphere with a bunch of wrapper packages.

What I think would be nice is a new package on Atmosphere that can import third party meteor-agnostic libraries into your meteor app, from a single configuration file that the package will read. Behind the scenes, the package might use jspm to retrieve the packages and build them so that they work inside the Meteor app. To make things easy at first, the package could take advantage of the fact that the libraries it's loading are meteor-agnostic, so it would somehow (I'm researching on it right now) load the libraries before all of the meteor-specific libraries.

So basically Meteor users would do something like

meteor add username:meteor-agnostic-loader

then the user would create a configuration file, probably using standard Meteor Package.* and api.* calls, to load the libraries. The meteor-agnostic-loader would probably create some type of extension to the Package API so that jspm will be used to fetch the libraries behind the scenes, load them, and build them.

[guybedford]

@trusktr thanks so much for looking into this stuff, it is appreciated, and great that you are opening up these discussions. Note though that these are hard problems to tackle and to make real progress most likely requires larger support.

[guybedford]

Closing for now as there is nothing active happening here, but a Meteor endpoint would be very interesting to explore at some point in future (@crisptrutski).