Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

presto的hbase依赖目录只需要编译出来的那一个jar包么 #8

Open
scienceyang opened this issue Jun 28, 2019 · 10 comments
Open

Comments

@scienceyang
Copy link

presto版本0.221,我将编译出来的jar包放入plugin/hbase目录下,配置好hbase相关信息后,运行show tables;是正常的,但是运行select的时候select name from demo;就报错
2019-06-28T17:59:05.865+0800 DEBUG query-execution-12 com.facebook.presto.execution.QueryStateMachine Query 20190628_095905_00002_x8g6u is FAILED
2019-06-28T17:59:05.865+0800 DEBUG Query-20190628_095905_00002_x8g6u-257 com.facebook.presto.execution.QueryStateMachine Query 20190628_095905_00002_x8g6u failed
java.lang.AbstractMethodError
at com.facebook.presto.split.SplitManager.getSplits(SplitManager.java:87)
at com.facebook.presto.split.CloseableSplitSourceProvider.getSplits(CloseableSplitSourceProvider.java:51)
at com.facebook.presto.sql.planner.SplitSourceFactory$Visitor.lambda$visitTableScan$0(SplitSourceFactory.java:126)
at com.facebook.presto.sql.planner.SplitSourceFactory$Visitor.visitTableScan(SplitSourceFactory.java:131)
at com.facebook.presto.sql.planner.SplitSourceFactory$Visitor.visitTableScan(SplitSourceFactory.java:102)
at com.facebook.presto.sql.planner.plan.TableScanNode.accept(TableScanNode.java:185)
at com.facebook.presto.sql.planner.SplitSourceFactory$Visitor.visitAggregation(SplitSourceFactory.java:222)
at com.facebook.presto.sql.planner.SplitSourceFactory$Visitor.visitAggregation(SplitSourceFactory.java:102)
at com.facebook.presto.sql.planner.plan.AggregationNode.accept(AggregationNode.java:199)
at com.facebook.presto.sql.planner.plan.InternalPlanNode.accept(InternalPlanNode.java:31)
at com.facebook.presto.sql.planner.SplitSourceFactory.createSplitSources(SplitSourceFactory.java:84)
at com.facebook.presto.execution.scheduler.SqlQueryScheduler.createStreamingLinkedStages(SqlQueryScheduler.java:411)
at com.facebook.presto.execution.scheduler.SqlQueryScheduler.createStreamingLinkedStages(SqlQueryScheduler.java:506)
at com.facebook.presto.execution.scheduler.SqlQueryScheduler.createStages(SqlQueryScheduler.java:327)
at com.facebook.presto.execution.scheduler.SqlQueryScheduler.(SqlQueryScheduler.java:207)
at com.facebook.presto.execution.scheduler.SqlQueryScheduler.createSqlQueryScheduler(SqlQueryScheduler.java:154)
at com.facebook.presto.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:490)
at com.facebook.presto.execution.SqlQueryExecution.startExecution(SqlQueryExecution.java:359)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
除了编译出的那两包还需要其他的jar包么

@Crossoverrr
Copy link
Contributor

需要把connector依赖的jar包都放到plugin/hbase目录下
也可以合起来打成一个比较大的jar包放到这个目录下,合起来打包的命令是:
mvn -U clean package

@scienceyang
Copy link
Author

然而并不知道还依赖哪些jar。。

@scienceyang
Copy link
Author

现在查询存在这种情况
image
一个select * from course就一直卡在这里,debug日志也没有报错

@Crossoverrr
Copy link
Contributor

看一下worker有异常日志吗?

@scienceyang
Copy link
Author

知道啥问题了,zookeeper地址错了。。。打扰了

@Crossoverrr
Copy link
Contributor

OK,那我先close了啊。再有问题再打开

@0ZhangJc0
Copy link

你好,我运行mvn -U clean package之后报同样的错误。
prestodb:0.228
connector:0.221

@Crossoverrr Crossoverrr reopened this Aug 5, 2020
@Crossoverrr
Copy link
Contributor

我觉得这个问题应该是presto版本的问题,我是基于prestosql-315版本进行研发的,而我看你的日志应该是prestodb的,两者是不兼容的。如果你要基于prestodb来使用,可以考虑尝试dev_prestodb-0.221_0.1.2分支的代码。

@0ZhangJc0
Copy link

我觉得这个问题应该是presto版本的问题,我是基于prestosql-315版本进行研发的,而我看你的日志应该是prestodb的,两者是不兼容的。如果你要基于prestodb来使用,可以考虑尝试dev_prestodb-0.221_0.1.2分支的代码。

嗯嗯。应该是版本不兼容的问题。我是使用dev_prestodb-0.221_0.1.2分支的代码生成的connector,但是prestodb-0.228比prestodb-0.221的ConnectorPageSource接口多了getCompletedPositions()这个函数,所以才会发生上述问题

@0ZhangJc0
Copy link

忘记贴报错信息了

2020-08-05T08:21:25.549Z ERROR remote-task-callback-28 com.facebook.presto.execution.StageExecutionStateMachine Stage execution 20200805_082125_00012_p5bbm.1.0 failed java.lang.AbstractMethodError: Method com/analysys/presto/connector/hbase/query/HBaseUpdatablePageSource.getCompletedPositions()J is abstract at com.analysys.presto.connector.hbase.query.HBaseUpdatablePageSource.getCompletedPositions(HBaseUpdatablePageSource.java) at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:287) at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:236) at com.facebook.presto.operator.Driver.processInternal(Driver.java:379) at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283) at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675) at com.facebook.presto.operator.Driver.processFor(Driver.java:276) at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077) at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162) at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483) at com.facebook.presto.$gen.Presto_0_228_SNAPSHOT_6b48d04____20200805_074502_1.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants