QEP takes forever to start

[ate47]

Part of the endpoint? (leave empty if you don't know)

• Backend (hdt-qs-backend)
• Frontend (hdt-qs-frontend)
• Benchmark (hdt-qs-benchmark)
• Other

Description of the issue

I'm trying to start QEP with 12G and a WD-all HDT, it is on this line for 20min

"main" #1 prio=5 os_prio=0 cpu=696640.62ms elapsed=1038.81s tid=0x00000248f745dde0 nid=0x3fd8 runnable  [0x0000006e346fb000]
   java.lang.Thread.State: RUNNABLE
        at [...]
        at org.eclipse.rdf4j.common.io.NioFile.readLong(NioFile.java:341)
        at com.the_qa_company.qendpoint.utils.BitArrayDisk.initWordsArray(BitArrayDisk.java:142)
        at com.the_qa_company.qendpoint.utils.BitArrayDisk.<init>(BitArrayDisk.java:95)
        at com.the_qa_company.qendpoint.utils.BitArrayDisk.<init>(BitArrayDisk.java:71)
        at com.the_qa_company.qendpoint.store.EndpointStore.initDeleteArray(EndpointStore.java:286)
        at com.the_qa_company.qendpoint.store.EndpointStore.<init>(EndpointStore.java:210)
        at com.the_qa_company.qendpoint.compiler.CompiledSail.<init>(CompiledSail.java:137)
        at com.the_qa_company.qendpoint.compiler.CompiledSail$CompiledSailCompiler.compile(CompiledSail.java:415)
        at com.the_qa_company.qendpoint.controller.Sparql.initializeEndpointStore(Sparql.java:435)
        at com.the_qa_company.qendpoint.controller.Sparql.init(Sparql.java:300)
        at com.the_qa_company.qendpoint.controller.Sparql.runClient(Sparql.java:326)

Full logs:

2023-02-07 11:25:02.886  INFO 25672 --- [           main] c.the_qa_company.qendpoint.Application   : Starting Application using Java 17.0.2 on Not-a-mac-AW with PID 25672 (C:\Users\wilat\workspace\qEndpoint\qendpoint-1.7.5-exec.jar started by wilat in C:\Users\wilat\workspace\qEndpoint)
2023-02-07 11:25:02.887  INFO 25672 --- [           main] c.the_qa_company.qendpoint.Application   : No active profile set, falling back to 1 default profile: "default"
2023-02-07 11:25:04.982  INFO 25672 --- [           main] c.t.qendpoint.controller.Sparql          : move old file to .\qendpoint\logs\old\logs.output_20230207-112504
2023-02-07 11:25:04.982  INFO 25672 --- [           main] c.t.qendpoint.controller.Sparql          : using applicationDirectory: .\qendpoint
2023-02-07 11:25:04.982  INFO 25672 --- [           main] c.t.qendpoint.controller.Sparql          : locationHdt:                C:\Users\wilat\workspace\qEndpoint\.\qendpoint\hdt-store\
2023-02-07 11:25:04.982  INFO 25672 --- [           main] c.t.qendpoint.controller.Sparql          : locationNative:             C:\Users\wilat\workspace\qEndpoint\.\qendpoint\native-store\
2023-02-07 11:25:05.505  INFO 25672 --- [           main] c.t.qendpoint.store.EndpointStore        : CHECK IF A PREVIOUS MERGE WAS STOPPED
2023-02-07 11:52:41.354  INFO 25672 --- [           main] c.t.qendpoint.compiler.CompiledSail      : Threshold for triples in Native RDF store: 1000000 triples
2023-02-07 11:52:41.355  INFO 25672 --- [           main] c.t.qendpoint.compiler.CompiledSail      : Using storage mode http://the-qa-company.com/modelcompiler/endpointStoreStorage, optimized: true.
2023-02-07 11:52:42.091  INFO 25672 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 1234 (http)
2023-02-07 11:52:42.126  INFO 25672 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2023-02-07 11:52:42.127  INFO 25672 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.68]
2023-02-07 11:52:42.312  INFO 25672 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2023-02-07 11:52:42.312  INFO 25672 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1659322 ms
2023-02-07 11:52:42.788  WARN 25672 --- [           main] .s.s.UserDetailsServiceAutoConfiguration :

Using generated security password: 9d144efc-58ec-41af-921f-8a2aaa32fa6a

This generated password is for development use only. Your security configuration must be updated before running your application in production.

2023-02-07 11:52:43.070  INFO 25672 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page: class path resource [static/index.html]
2023-02-07 11:52:43.717  INFO 25672 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 13 endpoint(s) beneath base path '/actuator'
2023-02-07 11:52:43.746  INFO 25672 --- [           main] o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@5183d589, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@511d5d04, org.springframework.security.web.context.SecurityContextPersistenceFilter@710b30ef, org.springframework.security.web.header.HeaderWriterFilter@5f574cc2, org.springframework.web.filter.CorsFilter@682c1e93, org.springframework.security.web.authentication.logout.LogoutFilter@3ecedf21, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@28b576a9, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@2f6bbeb0, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@27068a50, org.springframework.security.web.session.SessionManagementFilter@60222fd8, org.springframework.security.web.access.ExceptionTranslationFilter@5c09d180, org.springframework.security.web.access.intercept.AuthorizationFilter@125c082e]
2023-02-07 11:52:44.245  INFO 25672 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 1234 (http) with context path ''
2023-02-07 11:52:44.278  INFO 25672 --- [           main] c.the_qa_company.qendpoint.Application   : Started Application in 1662.402 seconds (JVM running for 1663.684)
+--------------------------------------------------+
|   qEndpoint started at: http://localhost:1234/   |
+--------------------------------------------------+

Excepted behavior

Mapping the HDT takes ~30s, so 5min max

Obtained behavior

No response

How to reproduce

Start the QEP with the WD-all dataset: https://qanswer-svc4.univ-st-etienne.fr/

Endpoint version

1.7.5

Do I want to contribute to fix it?

Maybe

Something else?

No response