You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When two instances of hlink try to load data from the same database, a huge error is printed by Spark. One of the important lines printed looks like
ERROR XSDB6: Another instance of Derby may have already booted the database <database_path>
With Spark 2, I was able to catch the Python exception that's thrown and print a more useful error message, but the huge JVM stack trace was still printed. In Spark 3, I think that we can set the spark.sql.pyspark.jvmStacktrace.enabled configuration option to prevent Spark from printing these stack traces. This may have an effect on other places where we get large JVM stack traces too.
The text was updated successfully, but these errors were encountered:
When two instances of hlink try to load data from the same database, a huge error is printed by Spark. One of the important lines printed looks like
With Spark 2, I was able to catch the Python exception that's thrown and print a more useful error message, but the huge JVM stack trace was still printed. In Spark 3, I think that we can set the
spark.sql.pyspark.jvmStacktrace.enabled
configuration option to prevent Spark from printing these stack traces. This may have an effect on other places where we get large JVM stack traces too.The text was updated successfully, but these errors were encountered: