Home

Purpose

This page is aimed at developers. It does not tell you how to install or use ids.server but how to test any modifications you may make. The build and test relies upon maven - so there is a pom.xml in the top directory. You may choose to develop with an IDE; both Eclipse and NetBeans work with maven however no artefacts from these IDEs are stored in GitHub.

WARNING

Running the IDS tests includes deleting the contents of various folders used by the IDS and also deleting all of the data in the ICAT used by the tests. It is, therefore, very important that the ICAT and IDS installation directories are set up purely to run these tests. DO NOT configure the tests to run on an ICAT that you may already have set up unless you are happy for all the data to be deleted!

What to do

Install an icat.server configured as described at https://github.com/icatproject/icat.server/wiki

Following the ICAT Server tutorial, you will need to complete the steps up to and including installing the ICAT server component. You will need a Linux machine (possibly Vagrant as in the tutorial), a JDK, MariaDB and Payara installed, the authn.db authenticator (not authn.simple as per the ICAT tutorial), the icat.lucene and icat.server components. The tests need some user accounts to be setup in the authn.db authenticator. For your convenience, a SQL script creating the required entries in the authn.db database is provided in the file src/test/resources/add-test-accounts.sql.

Install an ids.storage_test. (see section below for more details)

This is a variant of the ids.storage_file which has never been released as it is only used in development. It reads from a file ~/reliability which contains a number between 0.0 and 1.0 giving the probability that a requested operation will succeed. If it is zero then all calls will generate IOExceptions. The reliability file is modified as needed by the tests. In practice it is only set to 0 or 1 as tests with indeterminate results are difficult to write.

Install an ids.server (this will be done automatically when you follow the instructions in the sections further down in order to run the IDS tests)

When you run maven it will attempt to deploy to glassfish. For this to work you must have asadmin in your path and have one domain configured to run on port 4848. It is deployed with --force=true so that it should overwrite anything currently deployed. Tests are for three different configurations: one which is single level, two which is two level with the archive stored as datasets and twodf where the archive is stored as datafiles. The system is reconfigured as necessary when the tests are run by writing files to a dedicated install directory (src/test/install) and running setup from that folder. The property files that are used are derived from files in src/test/resources.

Rather than running mvn install you may prefer to run mvn install -DskipTests and then run the tests from your IDE in which case you should ensure that serverUrl, javax.net.ssl.trustStore, luceneUrl and containerHome are set as system properties. When running from maven, the properties are taken from the ~/.m2/settings.xml as environment variables and passed into the tests as system properties.

Vagrant build and test instructions

These instructions follow on from the Vagrant installation tutorial and from the ICAT server build and test instructions

Overview

Assuming ICAT server and authn.db are installed as per the instructions above, it is now necessary to install an IDS storage plugin. After that it will be possible to do a mvn install on the main IDS server code to compile, deploy and test it.

Note that there are two ICAT user accounts required for the IDS server tests to run. These are configured in the src/test/resources/test.properties file (of the IDS server component) and by default have the username/passwords root/password and guest/guess. So if you haven't already set up any users in the DB authenicator that you installed, you need to connect to the database specified in the authn.db setup.properties file and add a row for each of these users.

Also, before going any further, ensure that any previous versions of IDS server are not deployed by using the undeploy command e.g.

asadmin undeploy ids.server-1.9.0

Installing ids.storage_test

For IDS server, the ids.storage_test storage plugin must be installed as it enables the tests to run correctly. First, the repository must be cloned from GitHub.

cd ~/code
git clone https://github.com/icatproject/ids.storage_test.git
cd ids.storage_test

Next, we need to compile the package and extract the files to our install directory.

mvn package
cd ~/install
unzip ~/code/ids.storage_test/target/ids.storage_test-1.0.0-SNAPSHOT-distro.zip
cd ids.storage_test

Finally, we add a setup.properties file

#Glassfish
secure = true
container = Glassfish
home = /home/glassfish/payara41
port = 4848

and then we install the plugin

./setup -vv install

Maven settings

If you followed the ICAT server compilation instructions, then your Maven settings will already have most of the configuration needed, except for the new <testHome> property which specifies where the tests will expect the data they use during testing to be stored under.

If you didn't follow the ICAT server instructions, create ~/.m2/settings.xml and put following in it, making sure that the urls are correct and that the paths are mapped correctly for your install. The example below should work if you have followed the tutorial instructions.

<settings>
   <profiles>
      <profile>
         <activation>
             <activeByDefault>true</activeByDefault>
         </activation>
         <properties>
             <serverUrl>https://localhost.localdomain:8181</serverUrl>
             <luceneUrl>https://localhost.localdomain:8181</luceneUrl>
             <javax.net.ssl.trustStore>/home/glassfish/payara41/glassfish/domains/domain1/config/cacerts.jks</javax.net.ssl.trustStore>
             <containerHome>/home/glassfish/payara41</containerHome>
             <testHome>/home/glassfish/test</testHome>
         </properties>
      </profile>
   </profiles>
</settings>

Setting up the test folder structure

The tests need a few folders to exist in the testHome directory for the tests to run correctly, the rest of the files and folders will be generated by the tests themselves.

mkdir -p ~/test/data/ids/main
mkdir -p ~/test/data/ids/archive
mkdir -p ~/test/data/ids/cache

Build and run tests for IDS server

Everything should now be set up to be able to build the IDS server and run the associated tests.

If you don't already have a copy of the IDS server code that you are ready to build and test then the following commands will get the latest copy:

cd ~/code
git clone https://github.com/icatproject/ids.server.git
cd ids.server

Using the command below will build, deploy and test the source code.

mvn install

IDS server has three sets of integrations tests, during which IDS server will be undeployed and redeployed with the needed settings. one tests using just main storage (single level storage - hence the name), two tests using an archive and twodf tests using an archive and using datafiles as the storageUnit. You may want to just deploy IDS server and not want to run the integration tests every time, this can be done by supplying the -DskipTests flag when running mvn install.

mvn install -DskipTests