Application modules and configuration

An application has "modules" and "config". Modules are everything that's loaded into an application's modules database; config is everything else.

Modules are defined into the categories defined by the REST API - assets, options, services (or "resources"), and transforms. Modules can be loaded via the REST API or via mlcp - though it is preferable to load options, services, and transforms via the REST API, as the REST API will determine the URI, collections, and permissions for each module, and it's best not to duplicate that in an mlcp script.

One possible area of confusion for modules is that the REST Client API that loads options, services, and transforms has a URL pattern starting with "/v1/config". Some of the resources under this pattern are truly configuration - indexes, namespaces, and properties. But options (query), services (resources), and transforms are all code.

Config is then everything that is not loaded into the modules database. With ML8, ideally all config can be handled via the REST Management API. From a code project perspective, it thus makes sense to separate modules and config into two separate directories.

Structure of the modules and config directories for any project

The modules directory can have the following default structure, which is based on ML8's samplestack application (these should be configurable within a build tool):

• ext = assets
• options = query options
• services = services (resources)
• transforms

The config directory can have the following default structure, which is based on ML8's REST Management API (these should be configurable within a build tool):

• app-servers
• clusters
• cpf
• databases
• flexrep
• forests
• groups
• hosts
• partitions
• scheduled-tasks
• security, with subdirectories of roles, users, amps, privileges, certificate authorities, certificate revocation lists, certificate templates, certificates, external-security, protected collections
• sql

Directory structure for a Java project

For a Java project that uses Maven conventions, the modules and config directories should go under src/main.

An argument can be made for config to be under a directory other than "main", as the config will not be needed an runtime, only at build time. The Maven layout - defines a "src/main/config" directory, though this seems unused.

To distinguish these directories from other application directories - e.g. src/main/java or src/main/webapp - an "ml-" prefix can be used. So we would have:

• src/main/ml-modules
• src/main/ml-config

These should be defaults and configurable within a build tool.

Implementation notes

For each "category" of endpoints as defined by the Management API documentation, we can have a Java Service class that is responsible for invoking each of the endpoints. Each Service class will depend on an instance of a Spring RestTemplate that is configured to handle authentication. Each Service class should also depend on a base URL - something of the form http://localhost:8002 - so that the class doesn't have to worry about how to connect to an ML host, it just focuses on the Management endpoints that it knows about.

Each Service class will also depend on a File path that identifies the configuration directory. The Service class will then have knowledge of where to find the configuration files within that directory specific to the endpoints it knows about.

For flexibility, the Service class should expose public methods that accept a String as an input for the JSON/XML to be sent to an endpoint so that clients aren't bound to providing files on disk.

Implementation example

The scheduled tasks endpoints - http://docs.marklogic.com/REST/management/scheduled-tasks - can be handled by a ScheduledTasksService. It would have a constructor of:

• public ScheduledTasksService(RestTemplate restTemplate, String baseUrl);

It would then have methods such as:

• public void createTasks(File configDir, String groupId);

The createTasks method would look in configDir/scheduled-tasks for JSON or XML files. For each file, it would make a POST request to http://docs.marklogic.com/REST/POST/manage/v2/tasks to create a scheduled task, using the groupId as a parameter and sending the format parameter with a value based on the input file type.