Release 1.15.0 (#642)

* issue-586: fix missing '/' in the studyURL (cherry picked from commit c84df3a) * Update version to 1.13.1 (#590) * 1.14.0-QA (#595) * New version: 1.14.0 * ARACHNE-2541. Fixed selection of deleted datasources * ARACHNE-2541. Reverted Access Denied log * ARACHNE-2541. Fixed selection of deleted datasources * Moved Java migration mechanism to commons (#548) * ARACHNE-2563. Changed order of elements in achilles report (#556) * fixes failed tests (#558) * Fixes Manual datanode creation (#559) * ARACHNE-2552 - Correct text in email to admin (#554) (#564) (cherry picked from commit 89fb264) * ARACHNE-2494 - Soft deleting for ds' tenants (cherry picked from commit 6f7ea00) * ARACHNE-2550 Datanode: DS health check fails (prod env) (cherry picked from commit 211c7e9) * ARACHNE-2564. Added new reg form fields [back_merge] (#568) * fix user reg test * ARACHNE-2591. Fixed failing import of Cohort entity [back_merge] (#576) * ARACHNE-2618. Added default locale equal to US (#580) * Arachne-2598. Added vocabulary_database_schema for PLP * ARACHNE-2610. Replaced string message with constant (#581) * ARACHNE-2607. Added sorting for study participants (#584) * issue-586: fix missing '/' in the studyURL * Update SqlRender version * Adds import of new Estimation and Prediction analyses (#585) * ARACHNE-593. Allow listing and managing manual Node's users from Central (#594) * Implements new CC import (#592) * QA-1.14 * Preserve Analysis sorting order during Study - StudyDTO conversion (#597) (cherry picked from commit a63045e) * Removed duplication of @EnableGlobalMethodSecurity (#599) * QA 1.15.0 (#630) * Study type mandatory - validation message * Disable Heracles analysis type due to dysfunctional Packrat (#634) * Fixes summary info parsing for IRs with multiple target/outcome cohorts (#636) * Removes packrat from IR (#632) * user's workspace should not be visible for inviting experts (#638) * Workspace should not be visible for inviting experts * Workspace should not be visible for inviting experts - code review comments * Hides Pathway analysis type
OHDSI · Oct 11, 2019 · ada003b · ada003b
1 parent 28791fe
commit ada003b
Show file tree

Hide file tree

Showing 80 changed files with 1,024 additions and 2,155 deletions.
diff --git a/.gitignore b/.gitignore
@@ -117,4 +117,9 @@ hs_err_pid*
 **/resources/public
 
 # build dir
-target
+target
+
+### Developer's personal properties ###
+**/resources/application*-dev-*.properties
+**/resources/application*-dev-*.yaml
+**/resources/application*-dev-*.yml
diff --git a/README.md b/README.md
@@ -1 +1,93 @@
-# ArachneCentralAPI
+#Arachne Community Edition build and run manual
+
+##Prerequisites
+For building and run the Arachne please install following applications:
+- [Apache Maven 3](https://maven.apache.org/download.cgi)
+- [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html)
+- [LibreOffice 6](https://www.libreoffice.org/download/download/)  - for running the ArachneCentralAPI only
+- [Apache Solr 7](http://lucene.apache.org/solr/downloads.html)
+- [Postgres DBMS 9.6+](https://www.postgresql.org/download/windows/)
+
+####Prepare databases: 
+Please create ohdsi user and 2 databases: arachne_portal and datanode. That can achieved by running following command in psql console:
+```   
+create role ohdsi with LOGIN password 'ohdsi';
+create database arachne_portal owner ohdsi;
+create database datanode owner ohdsi;
+```
+
+##Getting sources
+Arachne network consists of two applications – Datanode and CentralApi. Sources are located in the github repositories. Please checkout: 
+[ArachneCentralAPI](https://github.com/OHDSI/ArachneCentralAPI)
+[ArachneNodeAPI](https://github.com/OHDSI/ArachneNodeAPI) 
+The latest released version can be found in the master branch.
+
+##Solr Configuration
+Download solr 7 binaries.
+Solr configuration is stored in ArachneCentralAPI/solr-config. Please run command to create and configure cores:  
+```
+solr start -c && \
+    solr create_collection -c users -n arachne-config && \
+    solr create_collection -c data-sources -n arachne-config && \
+    solr create_collection -c studies -n arachne-config && \
+    solr create_collection -c analyses -n arachne-config && \
+    solr create_collection -c analysis-files -n arachne-config && \
+    solr create_collection -c papers -n arachne-config && \
+    solr create_collection -c paper-protocols -n arachne-config && \
+    solr create_collection -c paper-files -n arachne-config && \
+    solr create_collection -c submissions -n arachne-config && \
+    solr create_collection -c insights -n arachne-config && \
+    solr create_collection -c result-files -n arachne-config && \
+    solr create_collection -c study-files -n arachne-config && \
+    solr zk upconfig -n arachne-config -d /$PATHTO/ArachneCentralAPI/solr_config -z localhost:9983 && \
+    solr stop -all
+```
+Start solr application by running following command in the terminal:
+```
+solr start -c && \
+    solr zk upconfig -n arachne-config -d /$PATHTO/ArachneCentralAPI/solr_config -z localhost:9983 
+```
+Solr console should be available at: http://localhost:8983/solr
+
+
+
+####Build ArachneCentralAPI and ArachneNodeApi
+Arachne application property files contains few configuration profiles. For this manual we use DEV. Please review available options in the: 
+- ArachneCentralAPI/src/main/resources
+- ArachneNodeAPI/src/main/resources
+
+1. Open command prompt terminal in the ArachneCentralAPI/ folder and run:
+mvn clean package -DskipTests -DskipDocker -P dev
+
+2. Open command prompt terminal in the ArachneNodeAPI/ folder and run:
+mvn clean package -DskipTests -DskipDocker -P dev
+
+Two artifacts should be created: ArachneCentralAPI/target/portal-exec.jar and ArachneNodeAPI/target/datanode-exec.jar which are spring-boot fat jars and contains all the required dependencies.
+
+####Start ArachneCentralAPI
+Create folder and grant RW access: mkdir -p /var/arachne/files/jcr/workspaces. 
+Arachne applications expect jasypt encryption for passwords. Please generate values using e.g.:
+[jasypt online encoder](https://www.devglan.com/online-tools/jasypt-online-encryption-decryption)
+
+
+Start application using following command: 
+```
+java -jar ArachneCentralAPI/target/portal-exec.jar --office.home=/usr/lib/libreoffice/ --jasypt.encryptor.password=dummy "--spring.datasource.password=ENC(3b0hKjcVNZjGGLwd85Q+tw==)" "--spring.mail.password=ENC(O8Of4J1ejz9r7tZo05CS/Q==)" --portal.urlWhiteList=https://localhost:8080
+```
+spring.mail.password and spring.mail.username contains dummy values in this example, please replace them with your settings, otherwise send mail functionality will not work. Please encrypt email and database passwords with the same jasypt password. You can do it via [online](https://www.devglan.com/online-tools/jasypt-online-encryption-decryption)  or via [jasypt cli](http://www.jasypt.org/cli.html)  
+
+ArachneCentralAPI should be available at: https://localhost:8080
+
+####Start ArachneNodeAPI
+
+Arachne DataNodeAPI application at start registers in ArachneCentralAPI. In current scenario it should be running on https://localhost:8080
+
+```
+java -jar target/datanode-exec.jar --datanode.arachneCentral.host=https://localhost --datanode.arachneCentral.port=8080 --jasypt.encryptor.password=dummy "--spring.datasource.password=ENC(3b0hKjcVNZjGGLwd85Q+tw==)" "--spring.mail.password=ENC(O8Of4J1ejz9r7tZo05CS/Q==)" --spring.datasource.url=jdbc:postgresql://localhost:5432/datanode
+```
+ArachneNodeAPI should be available at: https://localhost:8880
+
+
+Supply your mail sender configuration parameters otherwise emails will not work.
+
+You may override any configuration parameter using  “--name=value” spring boot notation.
diff --git a/pom.xml b/pom.xml
@@ -6,35 +6,35 @@
 
     <artifactId>portal</artifactId>
     <groupId>com.odysseusinc.arachne</groupId>
-    <version>1.14.0</version>
+    <version>1.15.0</version>
     <packaging>jar</packaging>
 
     <parent>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-parent</artifactId>
-        <version>1.5.9.RELEASE</version>
+        <version>1.5.20.RELEASE</version>
         <relativePath/>
     </parent>
 
     <properties>
-        <spring.boot.version>1.5.4.RELEASE</spring.boot.version>
+        <spring.boot.version>1.5.20.RELEASE</spring.boot.version>
         <spring.security.acl.version>4.2.1.RELEASE</spring.security.acl.version>
         <start-class>com.odysseusinc.arachne.portal.PortalStarter</start-class>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <java.version>1.8</java.version>
-        <jjwt.version>0.6.0</jjwt.version>
+        <jjwt.version>0.10.5</jjwt.version>
         <ehcache.version>1.3.0</ehcache.version>
         <postgresql.version>42.2.1</postgresql.version>
         <gson.version>2.7</gson.version>
-        <commons.lang3.version>3.5</commons.lang3.version>
+        <commons.lang3.version>3.8.1</commons.lang3.version>
         <hikaricp.version>2.6.3</hikaricp.version>
         <build.number>${BUILD_NUMBER}</build.number>
         <build.id>${BUILD_TIMESTAMP}</build.id>
         <dockerImagePrefix>hub.arachnenetwork.com</dockerImagePrefix>
         <jodConverter.version>4.1.0</jodConverter.version>
         <hibernate.version>5.2.12.Final</hibernate.version>
         <clamav.version>2.0.1</clamav.version>
-        <springRetryVersion>1.2.1.RELEASE</springRetryVersion>
+        <springRetryVersion>1.2.4.RELEASE</springRetryVersion>
     </properties>
 
     <profiles>
@@ -111,6 +111,11 @@
                 <artifactId>flyway-core</artifactId>
                 <version>4.2.0</version>
             </dependency>
+            <dependency>
+                <groupId>org.ohdsi</groupId>
+                <artifactId>authenticator</artifactId>
+                <version>0.0.1-QA</version>
+            </dependency>
         </dependencies>
     </dependencyManagement>
 
@@ -121,6 +126,11 @@
             <version>${project.version}</version>
             <type>jar</type>
         </dependency>
+        <dependency>
+            <groupId>com.odysseusinc.arachne</groupId>
+            <artifactId>arachne-common-utils</artifactId>
+            <version>${project.version}</version>
+        </dependency>
         <dependency>
             <groupId>com.odysseusinc.arachne</groupId>
             <artifactId>arachne-no-handler-found-exception-util</artifactId>
@@ -189,11 +199,7 @@
             <artifactId>spring-security-test</artifactId>
             <version>${spring-security.version}</version>
         </dependency>
-        <!--<dependency>-->
-            <!--<groupId>net.sf.ehcache</groupId>-->
-            <!--<artifactId>ehcache</artifactId>-->
-            <!--<version>${ehcache.version}</version>-->
-        <!--</dependency>-->
+
         <dependency>
             <groupId>org.postgresql</groupId>
             <artifactId>postgresql</artifactId>
@@ -211,8 +217,20 @@
         </dependency>
         <dependency>
             <groupId>io.jsonwebtoken</groupId>
-            <artifactId>jjwt</artifactId>
+            <artifactId>jjwt-api</artifactId>
+            <version>${jjwt.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>io.jsonwebtoken</groupId>
+            <artifactId>jjwt-impl</artifactId>
             <version>${jjwt.version}</version>
+            <scope>runtime</scope>
+        </dependency>
+        <dependency>
+            <groupId>io.jsonwebtoken</groupId>
+            <artifactId>jjwt-jackson</artifactId>
+            <version>${jjwt.version}</version>
+            <scope>runtime</scope>
         </dependency>
         <dependency>
             <groupId>org.imgscalr</groupId>
@@ -233,7 +251,7 @@
         <dependency>
             <groupId>net.lingala.zip4j</groupId>
             <artifactId>zip4j</artifactId>
-            <version>1.3.2</version>
+            <version>2.1.1</version>
         </dependency>
         <dependency>
             <groupId>edu.vt.middleware</groupId>
@@ -356,6 +374,10 @@
             <artifactId>jericho-html</artifactId>
             <version>3.4</version>
         </dependency>
+        <dependency>
+            <groupId>org.ohdsi</groupId>
+            <artifactId>authenticator</artifactId>
+        </dependency>
     </dependencies>
 
     <build>
@@ -505,6 +527,10 @@
                 <directory>src/test/resources</directory>
                 <filtering>true</filtering>
             </testResource>
+            <testResource>
+                <directory>src/test/resources-binary</directory>
+                <filtering>false</filtering>
+            </testResource>
         </testResources>
         <pluginManagement>
             <plugins>

diff --git a/src/main/java/com/odysseusinc/arachne/portal/PortalStarter.java b/src/main/java/com/odysseusinc/arachne/portal/PortalStarter.java
@@ -34,7 +34,7 @@
 import org.springframework.scheduling.annotation.EnableScheduling;
 
 @Configuration
-@ComponentScan(basePackages = {"com.odysseusinc.arachne.*"})
+@ComponentScan(basePackages = {"com.odysseusinc.arachne.*", "org.ohdsi.authenticator.*"})
 @EnableAutoConfiguration
 @EnableScheduling
 @EnableJpaRepositories(

diff --git a/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/AnalysisController.java b/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/AnalysisController.java
@@ -64,16 +64,6 @@ public class AnalysisController extends BaseAnalysisController<Analysis, Analysi
     public static final String RUN_IR_ANALYSIS_FILE_NAME = "run_ir_analysis.r";
     private static final String RUN_CC_REPORTS_FILE_NAME = "run_cc_reports.R";
     private static final String CC_SQLS_DIR = "sql/cc";
-    private static final String CIRCE_JAR = "circe-1.2.2-SNAPSHOT.jar";
-    private static final String CIRCE_JAR_RES = "circe-1.2.2-SNAPSHOT.jar.res";
-    private static final String COMMONS_IO_JAR = "commons-io-2.6.jar";
-    private static final String COMMONS_IO_JAR_RES = "commons-io-2.6.jar.res";
-    private static final String COMMONS_LANG_JAR = "commons-lang3-3.7.jar";
-    private static final String COMMONS_LANG_JAR_RES = "commons-lang3-3.7.jar.res";
-    private static final String JACKSON_JAR = "jackson-annotations-2.9.2.jar";
-    private static final String JACKSON_JAR_RES = "jackson-annotations-2.9.2.jar.res";
-    private static final String JARS_IR_PATH = "jars/ir/";
-    public static final String IR_RESOURCES_PATH = "r/ir/";
 
     static {
         ANALISYS_MIMETYPE_MAP.put(CommonAnalysisType.COHORT, CommonFileUtils.TYPE_COHORT_SQL);
@@ -125,41 +115,6 @@ protected void attachPredictionFiles(List<MultipartFile> files) throws IOExcepti
                 readResource("r/" + RUN_PLP_ANALYSIS_FILE_NAME)));
     }
 
-    protected void attachIncidenceRatesFiles(List<MultipartFile> files) throws IOException {
-
-        files.add(new MockMultipartFile(RUN_IR_ANALYSIS_FILE_NAME, RUN_IR_ANALYSIS_FILE_NAME, null,
-                readResource("r/" + RUN_IR_ANALYSIS_FILE_NAME)));
-        files.add(new MockMultipartFile(CIRCE_JAR, CIRCE_JAR, null,
-                readResource(JARS_IR_PATH + CIRCE_JAR_RES)));
-        files.add(new MockMultipartFile(COMMONS_IO_JAR, COMMONS_IO_JAR, null,
-                readResource(JARS_IR_PATH + COMMONS_IO_JAR_RES)));
-        files.add(new MockMultipartFile(COMMONS_LANG_JAR, COMMONS_LANG_JAR, null,
-                readResource(JARS_IR_PATH + COMMONS_LANG_JAR_RES)));
-        files.add(new MockMultipartFile(JACKSON_JAR, JACKSON_JAR, null,
-                readResource(JARS_IR_PATH + JACKSON_JAR_RES)));
-        files.add(new MockMultipartFile("additionalCriteria.sql", "additionalCriteria.sql", null,
-                readResource(IR_RESOURCES_PATH  + "additionalCriteria.sql")));
-        files.add(new MockMultipartFile("analysis_summary.sql", "analysis_summary.sql", null,
-                readResource(IR_RESOURCES_PATH  + "analysis_summary.sql")));
-        files.add(new MockMultipartFile("delete_strata.sql", "delete_strata.sql", null,
-                readResource(IR_RESOURCES_PATH  + "delete_strata.sql")));
-        files.add(new MockMultipartFile("groupQuery.sql", "groupQuery.sql", null,
-                readResource(IR_RESOURCES_PATH  + "groupQuery.sql")));
-        files.add(new MockMultipartFile("ir_analysis_query_builder.r", "ir_analysis_query_builder.r", null,
-                readResource(IR_RESOURCES_PATH  + "ir_analysis_query_builder.r")));
-        files.add(new MockMultipartFile("ir_dist.sql", "ir_dist.sql", null,
-                readResource(IR_RESOURCES_PATH  + "ir_dist.sql")));
-        files.add(new MockMultipartFile("performAnalysis.sql", "performAnalysis.sql", null,
-                readResource(IR_RESOURCES_PATH  + "performAnalysis.sql")));
-        files.add(new MockMultipartFile("strata.sql", "strata.sql", null,
-                readResource(IR_RESOURCES_PATH  + "strata.sql")));
-        files.add(new MockMultipartFile("strata_rules.sql", "strata_rules.sql", null,
-                readResource(IR_RESOURCES_PATH  + "strata_rules.sql")));
-        files.add(new MockMultipartFile("strata_stats.sql", "strata_stats.sql", null,
-                readResource(IR_RESOURCES_PATH + "strata_stats.sql")));
-    }
-
-
     @Override
     protected void attachCohortHeraclesFiles(List<MultipartFile> files) throws IOException, URISyntaxException {
 

diff --git a/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/AuthenticationController.java b/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/AuthenticationController.java
@@ -22,12 +22,12 @@
 
 package com.odysseusinc.arachne.portal.api.v1.controller;
 
-import com.odysseusinc.arachne.portal.security.TokenUtils;
 import com.odysseusinc.arachne.portal.security.passwordvalidator.ArachnePasswordValidator;
 import com.odysseusinc.arachne.portal.service.LoginAttemptService;
 import com.odysseusinc.arachne.portal.service.PasswordResetService;
 import com.odysseusinc.arachne.portal.service.ProfessionalTypeService;
 import com.odysseusinc.arachne.portal.service.UserService;
+import org.ohdsi.authenticator.service.Authenticator;
 import org.springframework.security.authentication.AuthenticationManager;
 import org.springframework.security.core.userdetails.UserDetailsService;
 import org.springframework.web.bind.annotation.RestController;
@@ -36,7 +36,7 @@
 public class AuthenticationController extends BaseAuthenticationController {
 
     public AuthenticationController(AuthenticationManager authenticationManager,
-                                    TokenUtils tokenUtils,
+                                    Authenticator authenticator,
                                     UserService userService,
                                     UserDetailsService userDetailsService,
                                     PasswordResetService passwordResetService,
@@ -45,9 +45,8 @@ public AuthenticationController(AuthenticationManager authenticationManager,
                                     LoginAttemptService loginAttemptService) {
 
         super(authenticationManager,
-                tokenUtils,
+                authenticator,
                 userService,
-                userDetailsService,
                 passwordResetService,
                 passwordValidator,
                 professionalTypeService,

diff --git a/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/BaseAchillesController.java b/src/main/java/com/odysseusinc/arachne/portal/api/v1/controller/BaseAchillesController.java
@@ -129,6 +129,7 @@ public JsonResult<List<CharacterizationDTO>> list(@PathVariable("id") Long datas
         return result;
     }
 
+    @ApiOperation("Retrieve Achilles reports for the specified datasource")
     @RequestMapping(value = "datasource/{id}/reports", method = RequestMethod.GET)
     public JsonResult<List<AchillesReportDTO>> reports(
             @PathVariable("id") Long datasourceId) throws NotExistException {