Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logging documentation and provide example Logback config for logging in JSON format #2933

Merged
merged 3 commits into from
Aug 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
docs/
scripts/
src/
!src/main/docker/logback*.xml
target/
!target/*.jar
/*.md
4 changes: 4 additions & 0 deletions docs/_docs/getting-started/deploy-docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,10 @@ services:
# - ALPINE_CORS_ALLOW_CREDENTIALS=true
# - ALPINE_CORS_MAX_AGE=3600
#
# Optional logging configuration
# - LOGGING_LEVEL=INFO
# - LOGGING_CONFIG_PATH=logback.xml
#
# Optional metrics properties
# - ALPINE_METRICS_ENABLED=true
# - ALPINE_METRICS_AUTH_USERNAME=
Expand Down
65 changes: 65 additions & 0 deletions docs/_docs/getting-started/monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,66 @@ acquired and used successfully. The check spans both connection pools (see [Conn
}
```

### Logging

Logging of the API server is configured via [Logback]. All distributions of the API server ship with
a [default Logback configuration]. It defines the following behavior:

1. Log messages from the embedded Jetty server to:
* `$HOME/.dependency-track/server.<NUMBER>.log`
2. Log messages from Dependency-Track and the underlying Alpine framework to:
* `$HOME/.dependency-track/dependency-track.<NUMBER>.log`
* Standard Output
3. Log security-related messages to:
* `$HOME/.dependency-track/dependency-track-audit.<NUMBER>.log`
* Standard Output
4. For log files:
* Create a new log file once the current one exceeds 10MB in size
* Retain a history of up to 9 files per log before overwriting them
5. Output logs in a human-friendly format

> For containerized deployments, `$HOME` will refer to the `/data` directory.

#### Custom Logging Configuration

When operating Dependency-Track in container-centric environments, where logs are typically forwarded
from containers' standard output to a centralized log aggregator (e.g. ElasticSearch, OpenSearch, Splunk),
it is desirable to disable logging to disk, and even change the output to a more machine-readable format.

Starting with Dependency-Track v4.9.0, it is possible to provide a custom Logback configuration,
and configure JSON as output format (powered by [logstash-logback-encoder]).

An example configuration file for JSON logging to standard output ([`logback-json.xml`]) is included
in the API server container image, and can be enabled using the `LOGGING_CONFIG_PATH` environment variable:

```shell
# (Other configuration options omitted for brevity)
docker run -it --rm \
-e "LOGGING_CONFIG_PATH=logback-json.xml" \
dependencytrack/apiserver:latest
```

Refer to the [logstash-logback-encoder documentation] for advanced customization details.

In order to use a truly custom configuration file, it has to be mounted into the container, e.g.:

```shell
# (Other configuration options omitted for brevity)
docker run -it --rm \
-v "./path/to/logback-custom.xml:/etc/dtrack/logback-custom.xml:ro" \
-e "LOGGING_CONFIG_PATH=/etc/dtrack/logback-custom.xml" \
dependencytrack/apiserver:latest
```

For non-containerized distributions of the API server, a custom configuration file may be provided
via the `logback.configurationFile` JVM property:

```shell
# (Other configuration options omitted for brevity)
java -Dlogback.configurationFile=/path/to/logback-custom.xml \
-jar dependency-track-apiserver.jar
```

### Metrics

The API server can be configured to expose system metrics via the Prometheus [text-based exposition format].
Expand Down Expand Up @@ -275,11 +335,16 @@ An [example dashboard] is provided as a quickstart. Refer to the [Grafana docume
[community integrations]: {{ site.baseurl }}{% link _docs/integrations/community-integrations.md %}
[Configuration]: {{ site.baseurl }}{% link _docs/getting-started/configuration.md %}
[Connection Pooling]: {{ site.baseurl }}{% link _docs/getting-started/database-support.md %}#connection-pooling
[default Logback configuration]: https://github.com/DependencyTrack/dependency-track/blob/master/src/main/docker/logback.xml
[dependency-track-exporter]: https://github.com/jetstack/dependency-track-exporter
[example dashboard]: {{ site.baseurl }}/files/grafana-dashboard.json
[executors]: https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/ThreadPoolExecutor.html
[Grafana]: https://grafana.com/
[Grafana documentation]: https://grafana.com/docs/grafana/latest/dashboards/export-import/#import-dashboard
[Logback]: https://logback.qos.ch/
[`logback-json.xml`]: https://github.com/DependencyTrack/dependency-track/blob/master/src/main/docker/logback-json.xml
[logstash-logback-encoder]: https://github.com/logfellow/logstash-logback-encoder
[logstash-logback-encoder documentation]: https://github.com/logfellow/logstash-logback-encoder/tree/logstash-logback-encoder-7.3#loggingevent-fields
[MicroProfile Health]: https://download.eclipse.org/microprofile/microprofile-health-3.1/microprofile-health-spec-3.1.html
[MicroProfile Health REST interfaces specifications]: https://download.eclipse.org/microprofile/microprofile-health-3.1/microprofile-health-spec-3.1.html#_appendix_a_rest_interfaces_specifications
[Prometheus]: https://prometheus.io/
Expand Down
12 changes: 9 additions & 3 deletions src/main/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ ENV TZ=Etc/UTC \
HOME=${DATA_DIR} \
# Default notification publisher templates override environment variables
DEFAULT_TEMPLATES_OVERRIDE_ENABLED=false \
DEFAULT_TEMPLATES_OVERRIDE_BASE_DIRECTORY=${DATA_DIR}
DEFAULT_TEMPLATES_OVERRIDE_BASE_DIRECTORY=${DATA_DIR} \
LOGGING_CONFIG_PATH="logback.xml"

# Create the directories where the WAR will be deployed to (${APP_DIR}) and Dependency-Track will store its data (${DATA_DIR})
# Create a user and assign home directory to a ${DATA_DIR}
Expand All @@ -52,7 +53,7 @@ RUN mkdir -p ${APP_DIR} ${DATA_DIR} \
COPY --from=jre-build /opt/java/openjdk $JAVA_HOME

# Copy the compiled WAR to the application directory created above
COPY ./target/${WAR_FILENAME} ${APP_DIR}
COPY ./target/${WAR_FILENAME} ./src/main/docker/logback-json.xml ${APP_DIR}

# Specify the user to run as (in numeric format for compatibility with Kubernetes/OpenShift's SCC)
USER ${UID}
Expand All @@ -61,7 +62,12 @@ USER ${UID}
WORKDIR ${APP_DIR}

# Launch Dependency-Track
CMD exec java ${JAVA_OPTIONS} ${EXTRA_JAVA_OPTIONS} --add-opens java.base/java.util.concurrent=ALL-UNNAMED -DdependencyTrack.logging.level=${LOGGING_LEVEL} -jar ${WAR_FILENAME} -context ${CONTEXT}
CMD exec java ${JAVA_OPTIONS} ${EXTRA_JAVA_OPTIONS} \
--add-opens java.base/java.util.concurrent=ALL-UNNAMED \
-Dlogback.configurationFile=${LOGGING_CONFIG_PATH} \
-DdependencyTrack.logging.level=${LOGGING_LEVEL} \
-jar ${WAR_FILENAME} \
-context ${CONTEXT}

# Specify which port Dependency-Track listens on
EXPOSE 8080
Expand Down
4 changes: 4 additions & 0 deletions src/main/docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ services:
# - ALPINE_CORS_ALLOW_CREDENTIALS=true
# - ALPINE_CORS_MAX_AGE=3600
#
# Optional logging configuration
# - LOGGING_LEVEL=INFO
# - LOGGING_CONFIG_PATH=logback.xml
#
# Optional metrics properties
# - ALPINE_METRICS_ENABLED=true
# - ALPINE_METRICS_AUTH_USERNAME=
Expand Down
22 changes: 22 additions & 0 deletions src/main/docker/logback-json.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
<appender name="JSON_STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

<logger name="alpine" level="${dependencyTrack.logging.level:-INFO}" additivity="false">
<appender-ref ref="JSON_STDOUT" />
</logger>

<logger name="org.dependencytrack" level="${dependencyTrack.logging.level:-INFO}" additivity="false">
<appender-ref ref="JSON_STDOUT" />
</logger>

<logger name="org.eclipse.jetty" level="${dependencyTrack.logging.level:-INFO}" additivity="false">
<appender-ref ref="JSON_STDOUT" />
</logger>

<root level="WARN">
<appender-ref ref="JSON_STDOUT" />
</root>
</configuration>