Analyze summary output should distinquish between failed, not started and other error

[kain88-de]

Currently, we show ? if there is no benchmark data available yet for a job. It would be nice if we have symbols show a more granular error message. We could use

• - awaiting the result
• ? job finished but no benchmark time found

Are any other error states possible?

[mimischi]

So we could actually do three states:

• (empty) not yet started
• - awaiting result
• ? job finished, but no performance printed

But how would we distinguish between these states? Currently (I think) we parse the log file and search for the given output. If the file is not present, or the performance line is missing we print ?.

Now if a job has not started, there is no log file and have a first state. When a job has started, but not finished yet, we have a log file, this is state number two. How do we distinguish between a state where the benchmark is currently running and one where the benchmark job has crashed? Both have a log file, without any performance information.

In theory we could grab the job id that the queuing system prints to stdout and use qstat | grep $JOBID to figure out whether the job is still running. This way we could distinguish between all states.