KSQL-12422: Added Debug Capability and Updated ReadMe (#10414)

* Added Debug Capability
* Updated ReadMe

.gitignore

@@ -38,3 +38,8 @@ junit*/
 .tern-port
 ui/
 /ksqldb-functional-tests/src/test/resources/**/scratch.json
+
+# Local Development Files
+docker-compose.local.yml
+config/ksql-server.local.properties
+.env.local

CONTRIBUTING.md

@@ -46,10 +46,16 @@ There is likely a similar option available for other IDEs.
 
 ### Building and running ksqlDB locally
 
-To build and run ksqlDB locally, run the following commands:
+To build ksqlDB locally, run the following commands:
 
 ```shell
 $ ./mvnw clean package -DskipTests
+```
+
+To run ksqldb locally, you can start the ksqlDB server and CLI with the following commands. Ensure you have dependent
+services like Kafka and Schema Registry running before starting ksqlDB.
+
+```shell
 $ ./bin/ksql-server-start -daemon config/ksql-server.properties
 $ ./bin/ksql
 ```

README.md

@@ -8,7 +8,7 @@ ksqlDB is a database for building stream processing applications on top of Apach
 
 * **[Streams](https://docs.ksqldb.io/en/latest/concepts/collections/streams/) and [tables](https://docs.ksqldb.io/en/latest/concepts/collections/tables/)** - Create relations with schemas over your Apache Kafka topic data
 * **[Materialized views](https://docs.ksqldb.io/en/latest/concepts/materialized-views/)** - Define real-time, incrementally updated materialized views over streams using SQL
-* **[Push queries](https://docs.ksqldb.io/en/latest/concepts/queries/push/)**- Continuous queries that push incremental results to clients in real time
+* **[Push queries](https://docs.ksqldb.io/en/latest/concepts/queries/push/)** - Continuous queries that push incremental results to clients in real time
 * **[Pull queries](https://docs.ksqldb.io/en/latest/concepts/queries/pull/)** - Query materialized views on demand, much like with a traditional database
 * **[Connect](https://docs.ksqldb.io/en/latest/concepts/connectors)** - Integrate with any [Kafka Connect](https://docs.confluent.io/current/connect/index.html) data source or sink, entirely from within ksqlDB
 

bin/ksql-run-class

@@ -160,12 +160,30 @@ if [ "x$HELP" = "xtrue" ]; then
   exit 1
 fi
 
+# Set Debug options if enabled
+if [ "x$KSQL_DEBUG" != "x" ]; then
+    # Use default ports
+    DEFAULT_JAVA_DEBUG_PORT="5005"
+    if [ -z "$KSQL_JAVA_DEBUG_PORT" ]; then
+        KSQL_JAVA_DEBUG_PORT="$DEFAULT_JAVA_DEBUG_PORT"
+    fi
+    # Use the defaults if KSQL_JAVA_DEBUG_OPTS was not set
+    DEFAULT_JAVA_DEBUG_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=${DEBUG_SUSPEND_FLAG:-n},address=$KSQL_JAVA_DEBUG_PORT"
+    if [ -z "$KSQL_JAVA_DEBUG_OPTS" ]; then
+        KSQL_JAVA_DEBUG_OPTS="$DEFAULT_JAVA_DEBUG_OPTS"
+    fi
+    echo "Enabling Java debug options: $KSQL_JAVA_DEBUG_OPTS"
+fi
+
 OPTIONS=($KSQL_HEAP_OPTS)
 OPTIONS+=($KSQL_JVM_PERFORMANCE_OPTS)
 OPTIONS+=($KSQL_JMX_OPTS)
 OPTIONS+=($KSQL_LOG4J_OPTS)
 OPTIONS+=($KSQL_OPTS)
 OPTIONS+=($KSQL_GC_LOG_OPTS)
+OPTIONS+=($KSQL_JAVA_DEBUG_OPTS)
+
+echo "Debug Options: $KSQL_JAVA_DEBUG_OPTS"
 
 # Launch mode
 if [ "x$DAEMON_MODE" = "xtrue" ]; then

docker-compose.yml

@@ -1,4 +1,4 @@
-# Docker compose from bringing up a local ksqlDB cluster and dependencies.
+# Docker compose for bringing up a local ksqlDB cluster and dependencies.
 #
 # By default, the cluster has two ksqlDB servers. You can scale the number of ksqlDB nodes in the
 # cluster by using the docker `--scale` command line arg.
@@ -9,7 +9,7 @@
 # or a 1 node cluster run:
 # > docker-compose up --scale additional-ksqldb-server=0
 #
-# The default is one `primary-ksqldb-server` and one `additional-ksqdb-server`. The only
+# The default is one `primary-ksqldb-server` and one `additional-ksqldb-server`. The only
 # difference is that the primary node has a well-known port exposed so clients can connect, where
 # as the additional nodes use auto-port assignment so that ports don't clash.
 #

ksqldb-docker/README.md

@@ -7,33 +7,33 @@ Module for building ksqlDB docker images.
 To build a new image with local changes:
 
 1. Ensure you're logged in to docker:
-    ```
-    > docker login
+    ```shell
+   $ docker login
     ```
 
-1. Build docker images from local changes.
-    ```
-    > mvn clean
-    > mvn -Pdocker package --settings maven-settings.xml -DskipTests -Dspotbugs.skip -Dcheckstyle.skip  -Ddockerfile.skip=false -Dskip.docker.build=false -Ddocker.upstream-tag=latest-ubi8 -Ddocker.tag=local.build  -Ddocker.upstream-registry=''
+2. Build docker images from local changes.
+    ```shell
+   $ mvn clean
+   $ mvn -Pdocker package --settings maven-settings.xml -DskipTests -Dspotbugs.skip -Dcheckstyle.skip  -Ddockerfile.skip=false -Dskip.docker.build=false -Ddocker.upstream-tag=latest-ubi8 -Ddocker.tag=local.build  -Ddocker.upstream-registry=''
     ```
    Change `docker.upstream-tag` if you want to depend on anything other than the latest master upstream, e.g. 5.4.x-latest.
 
-1. Check the image was built:
-    ```
-    > docker image ls | grep local.build
+3. Check the image was built:
+    ```shell
+    $ docker image ls | grep local.build
     ```
     You should see the new image listed. For example:
 
     ```
     placeholder/confluentinc/ksqldb-docker       local.build   94210cd14384   About an hour ago   716MB
     ```
 
-1. To use with the `docker-compose.yml` file in the root of the project, run the following additional commands to create
+4. To use with the `docker-compose.yml` file in the root of the project, run the following additional commands to create
    copies of the image with the required `ksqldb-server` and `ksqldb-cli` image names:
 
-   ```
-   > docker tag placeholder/confluentinc/ksqldb-docker:local.build placeholder/confluentinc/ksqldb-server:local.build
-   > docker tag placeholder/confluentinc/ksqldb-docker:local.build placeholder/confluentinc/ksqldb-cli:local.build
+   ```shell
+   $ docker tag placeholder/confluentinc/ksqldb-docker:local.build placeholder/confluentinc/ksqldb-server:local.build
+   $ docker tag placeholder/confluentinc/ksqldb-docker:local.build placeholder/confluentinc/ksqldb-cli:local.build
    ```
 
    To use the images uncomment the properties at the bottom of the `.env` file in the root of the project and run