Merge pull request #68 from o19s/standalone

Changing from a plugin to a standalone CLI application

.github/workflows/build.yml

@@ -7,18 +7,16 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macOS-latest, windows-latest]
-        jdk: [ 11, 17, 21 ]
+        jdk: [ 21 ]
         experimental: [false]
     steps:
       - uses: actions/checkout@v4
+        with:
+          lfs: true
       - name: Set up JDK ${{ matrix.jdk }}
         uses: actions/setup-java@v4
         with:
-          distribution: temurin
+          distribution: adopt
           java-version: ${{ matrix.jdk }}
-          cache: gradle
-      - name: Assemble target plugin
-        uses: gradle/gradle-build-action@v3
-        with:
-          cache-disabled: true
-          arguments: -Dtests.security.manager=false assemble
+      - name: Build
+        run: mvn --batch-mode test

.gitignore

@@ -29,3 +29,17 @@ gradle-app.setting
 .project
 .classpath
 
+# Ignore Gradle project-specific cache directory
+.gradle
+
+# Ignore Gradle build output directory
+build
+
+# intellij files
+.idea/
+*.iml
+*.ipr
+*.iws
+build-idea/
+out/
+

README.md

@@ -1,64 +1,11 @@
-# Search Evaluation Framework
+# OpenSearch Evaluation Framework
 
-This repository contains the search quality evaluation framework as described in the [RFC](https://github.com/opensearch-project/OpenSearch/issues/15354).
+This is an application to perform search evaluation.
 
-Note: Some of the data files in this repository are tracked by `git lfs`.
+## Building
 
-## Repository Contents
-
-* `data` - The data directory contains scripts for creating random UBI queries and events for purposes of development and testing.
-* `opensearch-search-quality-evaluation-plugin` - An OpenSearch plugin that extends the OpenSearch Scheduler plugin that provides the ability to generate scheduled (and on-demand) implicit judgments from UBI data.
-* `opensearch-search-quality-implicit-judgments` - A standalone Java application to generate implicit judgments from indexed UBI data.
-
-## OpenSearch Search Quality Evaluation Plugin
-
-This is an OpenSearch plugin that extends the OpenSearch Scheduler plugin that provides the ability to generate scheduled (and on-demand) implicit judgments from UBI data.
-
-To use the plugin:
-
-```
-./gradlew build
-cd opensearch-search-quality-evaluation-plugin
-docker compose build
-docker compose up
-```
-
-To create a schedule to generate implicit judgments:
+Build the project from the top-level directory to build all projects.
 
 ```
-curl -s -X POST "http://localhost:9200/_plugins/search_quality_eval/schedule?id=1&click_model=coec&job_name=test&interval=60" | jq
-```
-
-See the created job:
-
-```
-curl -s http://localhost:9200/search_quality_eval_scheduled_jobs/_search | jq
-```
-
-To run an on-demand job without scheduling:
-
+mvn clean install
 ```
-curl -X POST "http://localhost:9200/_plugins/search_quality_eval/judgments?click_model=coec&max_rank=20" | jq
-```
-
-To see the job runs:
-
-```
-curl -X POST "http://localhost:9200/search_quality_eval_completed_jobs/_search" | jq
-```
-
-See the first 10 judgments:
-
-```
-curl -s http://localhost:9200/judgments/_search | jq
-```
-
-## OpenSearch Search Quality Implicit Judgments
-
-This is a standalone Java application to generate implicit judgments from indexed UBI data. It runs outside OpenSearch and queries the UBI indexes to get the data for calculating the implicit judgments.
-
-To run it, run the `org.opensearch.eval.App` class. This will connect to OpenSearch running on `localhost:9200`. It expects the `ubi_events` and `ubi_queries` indexes to exist and be populated.
-
-## License
-
-This code is licensed under the Apache 2.0 License. See [LICENSE.txt](LICENSE.txt).

data/esci/index.sh

@@ -7,4 +7,4 @@ echo "Initializing UBI..."
 curl -s -X POST "http://localhost:9200/_plugins/ubi/initialize"
 
 echo "Indexing queries and events..."
-curl -s -T "http://localhost:9200/_bulk?pretty" -H "Content-Type: application/x-ndjson" --data-binary @ubi_queries_events.ndjson
+curl  -X POST 'http://localhost:9200/index-name/_bulk?pretty' --data-binary @ubi_queries_events.ndjson -H "Content-Type: application/x-ndjson"