-
Notifications
You must be signed in to change notification settings - Fork 77
Getting started
- Installing and learning Gradle
- Starting a new project
- Using the new project wizard
- Configuring the application
- Adding resources to the application
- Using other ml-gradle tasks
To get started using ml-gradle, you should first install Gradle, as ml-gradle is a Gradle plugin.
Beginning with the ml-gradle 4.3.0 release, the plugin supports Gradle versions 6.4 up to the latest 7.x release. MarkLogic recommends using the latest Gradle 7.x version. For earlier ml-gradle versions, use Gradle versions 6.0 up to the latest 6.x release.
Next, if you've never used Gradle, consider at least skimming through some of the following Gradle docs to get a little familiar with it:
- Gradle overview
- Using Gradle from the command line
- Gradle build script basics
- The Gradle build environment and how properties are used
- How to write custom Gradle tasks
Finally, you'll need MarkLogic installed somewhere. You should have at least MarkLogic 8.0-4 installed, as the Manage API that ml-gradle depends on reached a fairly stable state in that version. Ideally, you can install the latest version of 9.x or 8.x. MarkLogic doesn't have to be on the same computer that you installed Gradle on, but that's of course easier for getting started. Below, you'll see how to point at a different host. Unless otherwise stated, this tutorial assumes you have installed MarkLogic on localhost.
To start a new project, Gradle provides an init plugin that can stub out a variety of projects. But for this tutorial, we're going to start from scratch.
First, let's make a new directory for our MarkLogic-based application (equivalent to a "project") which we'll name "petstore" and cd into it:
mkdir petstore
cd petstore
Next, use your favorite text editor to create a file named build.gradle with the following Groovy (why Groovy?) code in it (spaces/tabs/newlines do not matter, you can format this however you wish, but what's below is customary for Groovy code):
build.gradle
plugins {
id "com.marklogic.ml-gradle" version "4.5.0"
}
Because ml-gradle applies a number of defaults for configuration items like host, port, and the admin username and password, we already have a working build file that can create a new REST API-based application. To see that in action, run the following command (this assumes your MarkLogic admin password is "admin" and that port 8003 is open - if neither is true, it will fail, and you can see how to change those below):
gradle mlDeploy
You should see some logging like this:
:mlDeleteModuleTimestampsFile
:mlPrepareRestApiDependencies
:mlDeployApp
:mlPostDeploy UP-TO-DATE
:mlDeploy
BUILD SUCCESSFUL
Total time: 22.518 secs
So what did that actually do? To find out, go to the Admin app on your MarkLogic instance, and you'll see the following resources:
- An app server named "my-app" on port 8003
- Databases named "my-app-content" and "my-app-modules"
- 3 forests for my-app-content and 1 for my-app-modules
The names of these resources are set via the "mlAppName" property, which is listed in the Property Reference. This property has a default value of "my-app"; you can set it to anything you'd like.
You can check out your new REST API server by going to http://localhost:8003 .
All of these resources were created via the MarkLogic Management API. We'll soon see how to customize these resources and create anything else we'd like to create.
If you don't want ml-gradle to create a REST API server by default (which also includes creating a content database and a modules database), just set the following property in gradle.properties:
mlNoRestServer=true
One issue with what we did above though is that we have little insight into how ml-gradle talks to the Management API. To solve that, we can turn on info-level logging when we run a Gradle task:
gradle -i mlDeploy
You can run that again, though mlDeploy won't create anything new because the app server, databases, and forests already exist. But you'll see a lot of logging like this:
Executing command [com.marklogic.appdeployer.command.restapis.DeployRestApiServersCommand] with sort order [200]
Could not find REST API file at D:\workspace\temp\myapp\src\main\ml-config\rest-api.json, will use default payload
Checking for existence of REST API with name: my-app
Sending XML GET request as user 'admin' to path: /v1/rest-apis?format=xml
REST API server already exists with name: my-app
Finished executing command [com.marklogic.appdeployer.command.restapis.DeployRestApiServersCommand]
Via this logging, you can see that mlDeploy runs a series of commands, each of which attempts to deploy one or more instances of a particular resource type. This commands aren't defined in ml-gradle though - they're instead part of ml-app-deployer, a Java library that is both a Java Management API client and a command-driven deployment framework. In fact, there's really not a lot of code in ml-gradle - most of the interesting code is in ml-app-deployer, and ml-gradle provides a thin layer of Gradle-specific code around it. To learn more about this, see the Wiki page on How ml-gradle works.
Gradle also supports debug-level logging, which is very verbose, as you'd expect:
gradle -d mlDeploy
Typically though, info-level logging is sufficient for debugging errors that pop up, such as errors in a payload file. ml-gradle will show exactly what REST call it made when the error occurred, which is a very useful starting point for debugging.
Having deployed a default application, we can easily undeploy it, and we'll do so with info-level logging:
gradle -i mlUndeploy
This does what you'd expect - the app server, database, and forests are all removed in the correct order. Undeploying can take a little time, as each time an app server is removed, ml-gradle has to wait for MarkLogic to restart.
Starting in ml-gradle 2.7.0, you'll still need to stub out a build.gradle file that applies the ml-gradle plugin, but once you've done that, you can run "mlNewProject" to use a wizard approach for setting up a new project:
gradle mlNewProject
(If you are trying ml-gradle 4.5.0, you will need to do gradle -PmlUsername= -PmlPassword= mlNewProject due to a bug that will be fixed in ml-gradle 4.5.1).
The wizard will ask you several questions to help stub out a new project, which optionally includes running the "mlScaffold" task to generate some basic resources like a database file and user/role files.
The below sections for configuring the application and adding resources are still relevant for moving past what the wizard generates for, but with ml-gradle 2.7.0, using the wizard will typically be the best way to start a new project.
Of course we'll almost never use the default configuration. That configuration may not have worked for you because your admin password isn't "admin", or because port 8003 is taken already.
To change the configuration, we'll adjust the properties that ml-gradle reads in. With Gradle, properties are commonly stored in the gradle.properties file that's in the project root directory along with build.gradle. Every property that ml-gradle looks for is prefixed with "ml" to help avoid overlap with other Gradle plugins. So let's create a gradle.properties file with the most common properties to adjust right away:
gradle.properties
mlAppName=petstore
mlHost=localhost
mlRestPort=8010
mlUsername=admin
mlPassword=changeme
Now let's run mlDeploy (assuming you undeployed the application from the first part of this tutorial) with info-level logging:
gradle -i mlDeploy
First, we'll see some useful logging near the start of the task:
Initializing ml-gradle
Admin host: localhost
Admin username: admin
App name: petstore
App host: localhost
App REST port: 8010
REST admin username: admin
Manage host: localhost
Manage username: admin
Any properties in gradle.properties that ml-gradle reads in, except for passwords, will be logged at the info-level so you can quickly confirm them. One thing to note from the logging above is that ml-gradle supports multiple MarkLogic user accounts for different tasks - see Configuring security for more information.
And next, we'll see that we now have the following resources:
- An app-server named "petstore" on port 8010
- And databases of petstore-content and petstore-modules, with associated forests
See the Property reference for a complete list of properties that ml-gradle supports. A goal of ml-gradle is to allow for as much configuration/customization as possible via properties. If you have an idea for a new property to simplify configuration, please create a new issue for it.
One of the benefits of using Gradle is there's a whole ecosystem of Gradle plugins, and thus ml-gradle doesn't provide specific support for multiple environments, because there's already another Gradle plugin for this - the aptly named Gradle properties plugin. It supports property files for each environment that you need to deploy to, and it's definitely the way to go for this requirement.
See Configuring ml-gradle for more information on environment-based properties.
Generally, a resource is created and added to an ml-gradle application in one of two ways:
- (Usually much easier) Find an example in the example projects and copy that to your own project.
- Create the resource manually via the Admin UI, and then go to host:8002/manage/v2, navigate to the created resource, click on its properties, and copy them as JSON or XML to a new file in the appropriate directory.
ml-gradle uses a directory structure for storing resource files, so your first question will be - in which directory do I store a particular resource file? The Project layout reference should answer that, along with the example projects.
As for what to put in a resource file, see the Resource reference and Configuring resources for more information, including how to define your own tokens for substituting values in a file.
See How modules are loaded for information on the different child directories of "src/main/ml-modules", which is the default location for application module files.
And be sure to read about Watching for module changes, which should greatly ease development by automatically loading modules as you create and modify them.
To quickly generate a set of configuration files for a new project, run the scaffolding task:
gradle mlScaffold
This will generate a directory structure containing several configuration files - one for a content database, a REST API server, an application role, an application user, and more.
New in ml-gradle 2.7.0 - see https://github.com/marklogic-community/ml-gradle/wiki/Exporting-resources for more information.
After creating a new project and deploying the application, a common next step is to load some data. See the guide on how to use Content Pump with Gradle for assistance with this.
mlDeploy is just one of dozens of tasks that ml-gradle provides. Gradle plugins typically add one or more tasks to your Gradle build file; to see all the tasks added by ml-gradle, just run:
gradle tasks
Gradle will group all the tasks together, and you can see many such groups specific to MarkLogic and ml-gradle. Each task has a description with it. You can see things a bit more cleanly by running the built-in Gradle "help" task, specifying a task to get help for:
gradle help --task mlCreateResource
And see the Task reference for a complete list of tasks provided by ml-gradle.