Discuss workaround for multi-module loose app limitations

[scottkurz]

Hi @doteo78, I wanted to follow-up on your comment here: #1712 (comment)

Though you didn't specifically mention Liberty Tools or dev mode, I imagine that in running your apps in VS Code that you were looking to use these Liberty developer features.

I can describe how you might workaround the limitations in the "loose application" multi-module deployment that we recognized in the #1712 issue.

I thought it might be easier to separate the discussion out from that issue and move it here.

[doteo78]

Hi @scottkurz!! Yes, I was using dev mode from Liberty Tools in VS Code. If you suggest a workaround I will give it a try :-)

[scottkurz]

Using dev mode without "loose application" support

So, the workaround at a high-level is to disable the "loose application" support (the plugin default) and do a full repackage and redeploy each time you make an application update.

While this, unfortunately, requires you to lose some of the nicest parts of the Liberty "dev mode" function and do this yourself manually, it hopefully provides enough value to still be worth using.

But this workaround is only possible once you know the right commands.

Though dev mode is designed to handle most cases with a simple mvn liberty:dev, going outside that requires some expertise. So, while this is a bit lengthy to read through, hopefully it will be worth the effort, and will be enough such that you can apply it to your app.

Example

To illustrate, let's look at the multi-module guide sample:
https://github.com/OpenLiberty/guide-maven-multimodules

Say I want to use this pattern and deploy an EAR containing a WAR and a JAR, with the WAR leveraging "filtering" in the web.xml.

One-time setup

I just mention this for completeness. Some multi-module usages will fail without the dependencies installed in the local .m2 repo so you might need to do an initial install. (You're probably well past this step though.) This should NOT be something you have to do every time you launch.

mvn install

Launch dev mode with loose apps disabled

So you can still compile and deploy your app to a plugin-managed Liberty installation, and launch the runtime with a simple command. You just need to modify it a bit to first package the EAR, while disabling loose applications:

mvn package; mvn liberty:dev -DlooseApplication=false

It should launch normally, and you can exercise (e.g. test) the app. (Note there are two invocations of 'mvn' here, not one).

Repackage and redeploy manually after app changes

After making a change, (to your Java source, to the web.xml, to one of the properties that provides the value, via Maven "filters", for your target web.xml, etc.), you now have an extra step to do (vs. normal "dev mode" usage).

Using a separate terminal (session), re-package and re-deploy your app

mvn package -DskipTests; mvn liberty:deploy -DlooseApplication=false -pl ear

Let's break this down.

First, we need to do a full (multi-module) mvn package to rebuild the entire EAR. Next, we need to do a deploy, however we don't want to run the 'deploy' goal for every module, only for the module that contains the Liberty server deployment, in this case the 'ear' module. So in addition to disabling loose applications -DlooseApplication=false, we also add the -pl ear arguments to avoid running deploy in the other modules where it would cause an error.

Because the first command is multi-module and the second is not, these need to be two Maven invocations, then, not one.

If this seems complicated, well, yes, it kind of is, and it's not obvious to think to do this. However, having learned the right command, it can be simple enough to perform the application update. Simply scroll back through your command history, and re-enter this single (multi-part) command line.

Dev mode automatically handles other changes: server.xml, pom.xml(*), etc.

You can still benefit from dev mode auto-updating your server config (server.xml, bootstrap.properties, server.env), installing newly-configured features (from server.xml).

It will handle pom.xml updates as well. I put the asterisk(*) above to note that while dev mode will handle pom.xml updates that relate to your server config (which would be in the 'ear' module in this example), it will not necessarily pick up, e.g the 'war' module changes. I only mention this in case this is where your filtered web.xml property values are coming from; in that case you'd still have to repackage and redeploy the app to pick up those changes.

Test

You would probably not want to run with -DhotTests=true since the tests will be run before your deployment is updated. So hit <Enter> to run tests after re-deploy completes (or whenever).

Looking back

So dev mode provides:

• simple "all-in-one" install without having to bind goals to phases in your POM
• rapid iterative updates of app & config changes
• easy iterative test execution

We can see that we do still manage to achieve most of this, though it's a bit less convenient to have to run an extra command to redeploy.

Without the "loose application" support you have to actually repackage your archive and redeploy it, since you're no longer able to use a "logical" archive that's defined with pointers to the app artifacts laid out in your other project directories, like target/classes (because we don't handle filtering completely yet).

But note ,it's not especially more time-consuming, for smaller apps at least. If you are running in an IDE with auto build enabled, your classes will already be compiled, and by repackage, you're just doing a rezip basically (with a large enough archive this could be an issue still though).

OK, @doteo78, please let me know what you think.

[doteo78]

Thanks for the update!! I'll try your workaround as soon as I get back from my 2-week holidays.

[doteo78]

I've had the opportunity to test the solution you proposed. In the "Repackage and redeploy" section, you mention looseApplication=true. I assume this was a mistake and that you actually meant looseApplication=false, just as when launching the application.

The solution works, but the process is too slow for our scenario. Our applications are quite complex and include numerous modules such as JARs, WARs, EJBs, etc. They are deployed in the production environment on a WebSphere Full server. The reason we use Liberty locally is precisely to speed up development tasks. Currently, we use Eclipse with Liberty's WTP connector, and we're evaluating alternatives like Visual Studio Code and the liberty-maven-plugin because some developers are experiencing issues with Eclipse. The scenario you're proposing is, as of today, clearly slower than what we already have.

In any case, I really appreciate the effort, and we are always open to trying new alternatives that may arise.

[scottkurz]

I assume this was a mistake and that you actually meant looseApplication=false
Yes, thank you for correcting that. (I updated my post).

Thank you for taking the time to try all that and share some feedback. I plan to give this some more thought and respond when I have a chance.

[scottkurz]

@doteo78, just reflecting on this a bit more..

I'm realizing this workaround could help if:

• compile is essentially already done by the IDE
• AND package is fast.

That's true because, stepping through the rest of the sequence: deploy should be quick, (just copying over a file), and app restart would be the same whether deploying via "loose" app or the full archive.

However, if you have significant content, then having to zip that up then basically prevents this from being a nice alternative. (E.g. I'm just looking at a WAR with decent size web resources and it can take a minute just to zip up this one WAR on my machine.)

After your last response, I wondered for a moment if maybe there might be some "duplicate" compilation that might somehow be able to be optimized away... but even if so, zipping up a lot of content will make this undesirable. I'm guessing that's probably what you found.

Other tools

As far as all alternatives, since all of our new "Liberty Tools" IDE support delegate to the Liberty Maven/Gradle plugins... you'd find the same core issue in any of the Liberty Tools Eclipse, IDEA, VSCode extensions.

That said, I'm glad at least that the WTP-based Eclipse approach has been helpful. Is it correct to conclude that it's handling the web.xml filtering correctly while also providing a useful "loose app" deployment?

And if you'd like to reach out re: whatever problems your developers are having one forum would be: https://github.com/OpenLiberty/open-liberty-tools

Thanks again.

[doteo78]

Exactly @scottkurz, packaging the EAR takes too much time and yes, the WTP-based Eclipse approach handles the web.xml filtering just fine while also providing a "loose app" deployment.

By the way, the problems we are having in Eclipse are related to bugs introduced by the installation of the WebSphere Application Server V8.5 Developer Tools.

Thank you so much!