Skip to content

v5.5.0

Latest
Latest
Compare
Choose a tag to compare
@swaminathanvasanth swaminathanvasanth released this 03 Apr 10:44
· 15 commits to master since this release
v5.5.0
cb6e304
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Version Summary

Version Number : v5.5.0 [tag: v5.5.0]
Date: 03-April-2024

Projects / Repos Included in this release: IUDX File Server, IUDX Deployment and installation, IUDX's Documentation

Release Summary :

India Urban Data Exchange (IUDX) | File Server v5.5.0 is released with enhanced features.

Highlights:

  • Updated Software Client libraries
  • Updated Auditing Flow
  • Updated Software Testing
    • Updated Postman Collection
    • Included REST Assured based integration testing
    • Improved test coverage
  • Software License:

Test Reports:

Please find the release test details and reports here

API Docs

The api docs can be found here.

How to use the Release:

External Dependencies Installation

The File Server connects with various external dependencies namely

  • ELK stack
  • PostgreSQL
  • RabbitMQ
  • ImmuDB

Find the installations of the above along with the configurations to modify the database url, port and associated credentials in the appropriate sections here

Get Started

Docker based

  1. Install docker and docker-compose (one time setup)
  2. Create following docker volumes (one time setup) using the commands: 
    # Creates fs-volume
docker volume create fs-volume
  3. Build the images 
     ./docker/build.sh
  4. There are following two ways of setting/deploying the file server using docker-compose:
    1. Non-Clustered setup with all verticles running in a single container:
      • This needs no hazelcast, zookeeper, the deployment can be done on non-swarm too and suitable for development environment.
      • This makes use of iudx/fs-dev:latest image and config-dev.json present at secrets/all-verticles-configs/config-dev.json 
        # Command to bring up the non-clustered file-server container
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
# Command to bring down the non-clustered file-server container
docker-compose -f docker-compose.yml -f docker-compose.dev.yml down
    2. Clustered setup with all verticles running in a single container:
      • This needs following things:
        • Docker swarm and overlay network having a name 'overlay-net'. Refer here
        • Zookeeper running in 'overlay-net' named overlay network. Refer here
      • This makes use of iudx/fs-depl:latest image and config-depl.json present at secrets/all-verticles-configs/config-depl.json 
        # Command to bring up the clustered one file-server container
docker-compose -f docker-compose.yml -f docker-compose.depl.yml up -d
# Command to bring down the clustered one file-server container
docker-compose -f docker-compose.yml -f docker-compose.depl.yml down

Note

  1. If you want to try out or do temporary things, such as
    • use different config file than the standard two
    • Binding the ports of clustered file-server container to host, etc.

      Please use this technique of overriding/merging compose files i.e. using non-git versioned docker-compose.temp.yml file and do not modify the git-versioned files.
  2. Modify the git versioned compose files ONLY when the configuration is needed by all (or its for CI - can preferably name it as docker-compose.ci.yml) and commit and push to the repo.

Maven based

  1. Install jdk 11 and maven
  2. Use the maven exec plugin based starter to start the server 
    mvn clean compile exec:java@file-server

Note

  1. Privileged access maybe required to bring up the http server at port 80.
  2. Maven based setup by default uses secrets/all-verticles-configs/config-dev.json and is non-clustered setup of verticles. Also it cannot take values from .file-server.env file and so the default values apply.
  3. If you want to use a different named config called config-x.json, need to place it at secrets/all-verticles-configs/config-x.json and use following command to bring it up: 
    mvn clean compile  exec:java@file-server -Dconfig-dev.file=config-x.json

JAR based

  1. Install java 11 and maven
  2. Set Environment variables
export FS_URL=https://<fs-domain-name>
export LOG_LEVEL=INFO
  1. Use maven to package the application as a JAR
    mvn clean package -Dmaven.test.skip=true
  2. 2 JAR files would be generated in the target/ directory
    • iudx.file.server-cluster-0.0.1-SNAPSHOT-fat.jar - clustered vert.x containing micrometer metrics
    • iudx.file.server-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Running the clustered JAR

Note: The clustered JAR requires Zookeeper to be installed. Refer here to learn more about how to set up Zookeeper. Additionally, the zookeepers key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.

The JAR requires 3 runtime arguments when running:

  • --config/-c : path to the config file
  • --hostname/-i : the hostname for clustering
  • --modules/-m : comma separated list of module names to deploy

e.g. java -jar ./fatjar.jar --host $(hostname) -c configs/config.json -m iudx.file.server.apiserver.FileServerVerticle,iudx.file.server.authenticator.AuthenticationVerticle ,iudx.file.server.database.elasticdb.DatabaseVerticle,iudx.file.server.auditing.AuditingVerticle

Use the --help/-h argument for more information. You may additionally append an FS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export FS_JAVA_OPTS="-Xmx4096m"
$ java $FS_JAVA_OPTS -jar target/iudx.file.server-cluster-0.0.1-SNAPSHOT-fat.jar ...

Running the non-clustered JAR

The JAR requires 1 runtime argument when running:

  • --config/-c : path to the config file

e.g. java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.file.server-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json

Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.

e.g.

$ export FS_JAVA_OPTS="-Xmx1024m"
$ java $FS_JAVA_OPTS -jar target/iudx.file.server-dev-0.0.1-SNAPSHOT-fat.jar ...

License

With this release, all IUDX Softwares will be distributed under Apache 2.0 License

What's Changed

New Contributors

Full Changelog: v5.0.0...v5.5.0

Contributors

sushanthakumar, karun-singh, and DivyaSreeMunagavalasa
Assets 3
Loading