Feature/logisland 532 #553
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…BFS, using the DB spark context
… databricks environment (no more useless spark dependency when outside of databricks)
…rk streaming dependencies
…xed a bug where read position configiguration property cannot be anything else than string
…rkaround to class not found for org.apache.bahir.utils.Logging when using AzureEventHubsStructuredStreamProviderService
… the plugin registry is not filled as the current thread context classloader is a repl.ExecutorClassLoader, so class of plugins cannot be loaded
…ersion of azure event hub spark structured stream source/sink
…ving some debug traces and adding some info logs
… spark standalone cluster in client mode, not tested cluster mode. Also to be improved
… user password without https is supported and validated
… client connection created by the real service himself
…by supporting input/output avro schema configuration properties
…us now usable in structured streams
…ons of iptogeo service and processor dependencies (jackson/hadoop)
…th bahir underlying dependency that uses scala 2.12 when using 2.4.0 version (don't know why)
… Also attempting to align bumped jackson version everywhere where possible and also use the jackson.version property
… was using 2.4 engine, this made fail the structured stream travis integration test
oalam
approved these changes
Apr 9, 2020
Comment on lines
+560
to
+569
| ////////////////////////////////////// | ||
| // Azure event hubs options | ||
| ////////////////////////////////////// | ||
|
|
||
| val EVENTHUBS_NAMESPACE: PropertyDescriptor = new PropertyDescriptor.Builder() | ||
| .name("eventhubs.nameSpace") | ||
| .description("EventHubs namespace.") | ||
| .addValidator(StandardValidators.NON_EMPTY_VALIDATOR) | ||
| .required(true) | ||
| .build |
There was a problem hiding this comment.
could be placed in a more event hub specific package.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This branch includes many features to support the new azure/databricks/eventhub environment: