Single parse errors ignores the entire submission

[libre-man]

If a single file in the submission has a parse error the entire file is ignored. This makes the detection quite vulnerable as students simply need to hand in one invalid file.

[tsaglam]

See this comment.
Process-wise, it is probably best anyways to expect your students to submit code that is valid/compiles.

[wllhf]

Yes, it is and I understand the problem. But this way a student can avoid to be detected by just adding a file that cannot be parsed. Furthermore it would be nice to generate a list of submissions that were not included as looking through the console output if you have hundreds of submission of which dozens aren't parseable is quite tedious.

[tsaglam]

The CLI output will list all invalid files after parsing and before comparison:

JPlag initialized
ERROR: nothing to parse for submission "Empty
5 submissions parsed successfully!
3 parser errors!
0 too short submissions!

The following errors occured:
[A - broken] Demo3\A - broken\Structure.java:7: error: ';' expected
        int hash_length = -1
                            ^
[A - broken] Demo3\A - broken\Token.java:44: error: illegal start of expression
        public static int numberOfTokens() {
        ^
[D - broken] Demo3\D - broken\TokenConstants.java:4: error: = expected
  final static int FILE_END  0;

This will help you to see all invalid submissions at one. Then you can fix them and run them again if you want to accept invalid submissions from students. However, you can always solve that problem by requiring students to submit functioning code.

[wllhf]

Thanks for your time. Can you imagine how this output looks like if you teach an introduction to programming course for non cs students with >1k participants (independent of what you require them to do)? ;) But I see, I'll script a workaround for myself. Thanks anyways!

[tsaglam]

In that case, I would use the -d option that copies all erroneous submissions in an error folder. Sadly, this feature has a bug in v3.0.0, but on the master branch, this is already fixed (you could build your own jar if you want).
We also deal with large numbers of students; however, in the system where students submit their code, part of the (automated) checks for grading is checking if the code compiles. Code that doesn't is rejected. This is why we don't have this particular issue.

It is just one of the caveats of JPlag itself that it needs a valid token structure to function. One could theoretically try to build a prototypical frontend that does not stop when a problem with the code is detected, but that would probably mean that it can rely on existing APIs like ANTLR or the javax.tools.JavaCompiler.

[wllhf]

I'll do something similar, however we wouldn't find plagiarism in code that is rejected this way. Thanks again.