[JMSPublishFilters] Do not filter messages during broker shutdown

[eolivelli]

Motivation:

The JMSPublishFilters enqueue the task to filter messages in the topicOrderedExecutor threadpool, that is beeing shutdown here.

The Broker interceptors are stopped before that point, here and closing the interceptor also closes the Classloader that loaded the .nar file.

So there is a possibility for the Broker Interceptor code to be executed after the classloader is closed, resulting in errors like this:

24-05-06T01:15:33,486+0000 [broker-topic-workers-OrderedExecutor-7-0] ERROR com.datastax.oss.pulsar.jms.selectors.JMSFilter - Error while processing entry java.lang.NoClassDefFoundError: com/datastax/oss/pulsar/jms/selectors/SelectorSupport$2
java.lang.NoClassDefFoundError: com/datastax/oss/pulsar/jms/selectors/SelectorSupport$2
        at com.datastax.oss.pulsar.jms.selectors.SelectorSupport.matches(SelectorSupport.java:85) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSFilter.matches(JMSFilter.java:507) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSFilter.filterEntry(JMSFilter.java:254) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSPublishFilters.lambda$messageProduced$0(JMSPublishFilters.java:153) ~[?:?]
        at org.apache.bookkeeper.common.util.SingleThreadExecutor.safeRunTask(SingleThreadExecutor.java:137) ~[org.apache.bookkeeper-bookkeeper-common-4.16.5-fx-796523d3ea.jar:4.16.5-fx-796523d3ea]
        at org.apache.bookkeeper.common.util.SingleThreadExecutor.run(SingleThreadExecutor.java:107) ~[org.apache.bookkeeper-bookkeeper-common-4.16.5-fx-796523d3ea.jar:4.16.5-fx-796523d3ea]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.108.Final.jar:4.1.108.Final]
        at java.lang.Thread.run(Thread.java:842) ~[?:?]
Caused by: java.lang.ClassNotFoundException: com.datastax.oss.pulsar.jms.selectors.SelectorSupport$2
        ... 8 more
2024-05-06T01:15:28,683+0000 [broker-topic-workers-OrderedExecutor-3-0] ERROR com.datastax.oss.pulsar.jms.selectors.JMSFilter - Error while processing entry java.lang.NoClassDefFoundError: com/datastax/oss/pulsar/jms/selectors/SelectorSupport$2
java.lang.NoClassDefFoundError: com/datastax/oss/pulsar/jms/selectors/SelectorSupport$2
        at com.datastax.oss.pulsar.jms.selectors.SelectorSupport.matches(SelectorSupport.java:85) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSFilter.matches(JMSFilter.java:507) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSFilter.filterEntry(JMSFilter.java:254) ~[?:?]
        at com.datastax.oss.pulsar.jms.selectors.JMSPublishFilters.lambda$messageProduced$0(JMSPublishFilters.java:153) ~[?:?]
        at org.apache.bookkeeper.common.util.SingleThreadExecutor.safeRunTask(SingleThreadExecutor.java:137) ~[org.apache.bookkeeper-bookkeeper-common-4.16.5]
        at org.apache.bookkeeper.common.util.SingleThreadExecutor.run(SingleThreadExecutor.java:107) ~[org.apache.bookkeeper-bookkeeper-common-4.16.5]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.108.Final.jar:4.1.108.Final]
        at java.lang.Thread.run(Thread.java:842) ~[?:?]
Caused by: java.lang.ClassNotFoundException: com.datastax.oss.pulsar.jms.selectors.SelectorSupport$2
        ... 8 more

Solution:
Intercept the closing of the broker and stop applying filters, in any case, even if the interceptor would have tried to persist a acknoledgment for a message, that would have been probably lost, so this is not a big deal.

Alternative solution (rejected):
Execute the filtering logic in the same thread as the onPublish callback: this would slow down too much the write path.

Alternative solution (rejected):
Keep a reference to all the pending "filtering actions" and wait for all of them during the broker shutdown. As they are expensive operations, that would slow down the shutdown of the broker, but we want the broker to shutdown as quickly as possible.