Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KPI calculation again: need to be fully transparent about that #320

Open
bansp opened this issue Sep 29, 2024 · 1 comment
Open

KPI calculation again: need to be fully transparent about that #320

bansp opened this issue Sep 29, 2024 · 1 comment
Labels
hub Meant as target for individual action tickets. Please don't assign 'hub' tickets. SIS:centres centre-oriented part of the SIS
Milestone

Comments

@bansp
Copy link
Member

bansp commented Sep 29, 2024

This is a new issue, so that we can let #273 rest in peace after the last adjustment mentioned there.

The context for this issue is the progress report for CLARIN, where I have just had to perform calculations by hand, while we should be able to present them in the SIS.

Obviously, one obstacle to that is the level of curation on the part of centres -- but that is being worked on, and we need to give it time, while being open about it and mentioning the issue at various meetings (such as the upcoming CAC), and, fundamentally, we need to actively support the TCC is their now cyclic task of getting centres to use the SIS. Until that is fully handled, we need a crutch or a sort, a way to influence the KPI calculation so that it gets as precise as we can make it, given the information lag wrt CLARIN registry, individual centre decisions, etc.

  1. Crutch no. 1 we need an attribute or element that will indicate whether the centre has published its own recommendations, even if it doesn't share them. Perhaps an element that points to the relevant page.
  2. Crutch no. 2 -- an attribute, I guess, that marks the fact that while the centre doesn't publish their own recommendations, they skirt around the certification requirements (in the case of B-centres), by pointing somewhere (effectively, pointing at the sky) and saying "we want what it says there", even if it says something largely off-topic if not harmful (like advocating the LMF standard from 2008, which is withdrawn by now). Just an attribute, because I'd rather not propagate that information, merely recording the fact that the centre performs that manoeuvre.

Information for both 1 and 2 is updated semi-regularly in issue #14 .

  1. We need an explicit calculation or perhaps several, on the KPI page. It looks like there are two possible bases for the calculation: the number of B-centres and the number of depositing centres. Secondly, we can count centres that (a) curate their info in the SIS plus those that post their info locally (one calculation, for the two bases), and then count these plus the centres that are pointing at the sky (we can call that a reporting KPI, in contrast to the real KPI, except we probably don't want to do that explicitly, sadly). That calculation is also going to hold for two bases.
  2. Finally, for housekeeping, we should indicate all the values in the centre statistics page, and visualise at least crutch no. 1 on the individual centre pages.

Now let's have a list of dependent tasks:

@bansp bansp added SIS:centres centre-oriented part of the SIS hub Meant as target for individual action tickets. Please don't assign 'hub' tickets. labels Sep 29, 2024
@bansp bansp added this to the SIS v. 2.9.0 milestone Sep 29, 2024
@bansp
Copy link
Member Author

bansp commented Oct 7, 2024

It seems to me that only crutch no. 1 should be implemented. Centres that only send the user elsewhere simply do not publish their own requirements. And thus they do not count for the KPI.

I would implement crutch no. 1 as an element with URI content (single URI, we're not out to document a multitude of self-published recommendations by a single centre). The KPI calculation would then use the curated centres + the uncurated self-publishing ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hub Meant as target for individual action tickets. Please don't assign 'hub' tickets. SIS:centres centre-oriented part of the SIS
Projects
None yet
Development

No branches or pull requests

1 participant