Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update spark version to 2.3.0 #3

Merged
merged 1 commit into from
Dec 6, 2018

Conversation

elmiko
Copy link
Contributor

@elmiko elmiko commented Aug 6, 2018

the radanalytics.io tooling is now deploying 2.3.0 as the default
version. this change will ensure that the example continues to work with
the upstream changes.

the radanalytics.io tooling is now deploying 2.3.0 as the default
version. this change will ensure that the example continues to work with
the upstream changes.
@elmiko
Copy link
Contributor Author

elmiko commented Aug 6, 2018

we would like to update the spark version for the radanalytics tooling, but when we try to redeploy the application with this change we are seeing the following output:

spark-submit --class com.redhat.iot.spark.AMQPTemperature --master spark://mycluster:7077 --conf spark.driver.host=sparkdriver-headless /opt/app-root/src/amqp-spark-driver-1.0-SNAPSHOT.jar artemis.zroubali.svc 5672
18/08/06 13:58:19 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
	at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:330)
	at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:263)
	at java.util.jar.JarVerifier.processEntry(JarVerifier.java:318)
	at java.util.jar.JarVerifier.update(JarVerifier.java:230)
	at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
	at java.util.jar.JarFile.getInputStream(JarFile.java:450)
	at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:977)
	at sun.misc.Resource.cachedInputStream(Resource.java:77)
	at sun.misc.Resource.getByteBuffer(Resource.java:160)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:454)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at org.apache.spark.util.Utils$.classForName(Utils.scala:235)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:836)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
18/08/06 13:58:20 INFO ShutdownHookManager: Shutdown hook called

@elmiko
Copy link
Contributor Author

elmiko commented Aug 6, 2018

this pr addresses issue #2

@ppatierno
Copy link
Member

@elmiko sorry to be really late on this! Did you try the demo with the new Spark version? If it works fine, I can merge this one.

@elmiko
Copy link
Contributor Author

elmiko commented Dec 6, 2018

hi @ppatierno , yes i did test with that new pr. we could probably even bump that to 2.3.1, but i know this works for now.

@ppatierno
Copy link
Member

@elmiko ok so let me just merge this one for now, thanks!

Copy link
Member

@ppatierno ppatierno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@ppatierno ppatierno merged commit 4f1a35f into redhat-iot:master Dec 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants