Document queue polling behavior better

[hategan]

It may be surprising to users that when a job running on a LRM completes, PSI/J may not immediately pick up the status update because the queue status is only queried at certain intervals (by default 30 seconds). This should probably be made more clear where users are likely to read.

See, e.g., #358

[j-woz]

I think the enum JobState could capture this. Maybe split NEW into "not yet submitted" and "not yet polled"?

[hategan]

I think the enum JobState could capture this. Maybe split NEW into "not yet submitted" and "not yet polled"?

Not sure. There's an initial polling delay of two seconds. So, after 5 seconds, it has been polled once.

[andre-merzky]

I think the enum JobState could capture this. Maybe split NEW into "not yet submitted" and "not yet polled"?

For this to be useful and consistent, wouldn't one need to do that for every state? Like, the system may have the job FAILED while psij thinks it's still RUNNING because the update has not been pulled just yet. In other words: the locally exposed state is a cached value, and we know that cache invalidation is indeed one of the hard problems. I would agree that relying on clear documentation is more viable.

[j-woz]

Yes, the user should know that the state is old, but that is different from the state of a job that has never been submitted or polled. Right now, "not yet polled" returns None, but I think that documented enum values for these cases would be more clear.

[hategan]

Yes, the user should know that the state is old

If PSI/J knows that there is a new state, there is no point in reporting an old state and marking it as out of date. You can simply report the new state instead.

Right now, "not yet polled" returns None

That's not quite the case. wait() specifically returns None when you have a timeout parameter and it times out. And that's independent of whether the job was polled or not.

The polling state (whether a job was polled or and when) is not meant to be conveyed to the user. There is only an upper bound on how old the state can be from PSI/J's perspective. But there is no absolute upper bound. The queuing system may have arbitrary delays, and the queue querying may introduce arbitrary delays.