Question： how to parse the codes parallel?

[hnwyllmm]

I have a large set of code and it take a looooong time to check duplicate.
So I have two question.

1. how to parse the code parallel?
2. how can I get the progress of the parsing?

Thanks very much!

[tsaglam]

The submission comparison is always run in parallel if you use the most recent version of JPlag. What can take time is the clustering calculations. Could you try --cluster-skip? And could you tell me how many submissions you are comparing?

[hnwyllmm]

I have 20 submissions in total and there are 1800+ files in each submission.
Only 1 CPU is in use:

[tsaglam]

Okay, I see. The problem is that JPlag checks submissions in parallel, meaning pairs of submissions are compared at the same time. For one pair, the files are compared sequentially but with some optimizations. Also: Parsing the files is not done in parallel. What is the use case you have there? 1800 files per submission are quite an unusual setting for JPlag. What version of JPlag are you using?

[hnwyllmm]

JPlag's version is 4.0.0. The project with 1800 files is a very big project (https://github.com/oceanbase/oceanbase). There is a database competition based on oceanbase and I want to check duplicate after the competition.
I got the result now and JPlag print Total time for comparing submissions: 8 h 6 min 33 sec.
As you said, this use case is not a normal case so please ignore this question.
I'm so appreciate your work!

[tsaglam]

Ah, interesting! I will make note of that issue, maybe we will find an easy way to optimize the performance regarding that in future.

[tsaglam]

Just one idea, giving the JVM more memory with -Xmx could help, but that is just a guess.

[hnwyllmm]

Thank your advice. The job was done. I have a machine with large memory (50G+), so I think the default parameter is good enough.
I hope more scenarios can make JPlag better!