Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ctexamined_detected request: generic categorical terms #8419

Open
mkoo opened this issue Jan 10, 2025 · 8 comments
Open

ctexamined_detected request: generic categorical terms #8419

mkoo opened this issue Jan 10, 2025 · 8 comments
Labels
Function-CodeTables Priority-Normal (Not urgent) Normal because this needs to get done but not immediately.

Comments

@mkoo
Copy link
Member

mkoo commented Jan 10, 2025

We have a new table definition for
https://arctos.database.museum/info/ctDocumentation.cfm?table=ctexamined_detected

Examination for or detection of a condition or organism associated with the cataloged object. These may include natural or introduced toxins, environmental factors, parasites, pathogens, disease causative agents, or pathologies. Any entry should include the method (in the Method) and date of examination (in Date) and determiner (Determiner). Remark should be used to provide further details. Method or publication citations are encouraged in the Method. Use the most specific term available or if not available, use the general category with details in the Remark field (e.g., if "bacteria: Escherichia coli" does not exist, use the broad term "bacteria" and define in the Remark).

(@mkoo @campmlc came up with this but please let us know if you have improvements to suggest)

To allow for all entries and broader use of this table, we will need more generic categorical terms. We expect that in the future if some of these broader terms are used a lot (eg lots of examinations for E. coli) then we can 'elevated' to a specific ct term of bacteria: E. coli

For now we need:

  • bacteria
  • fungi

@dustymc can you please provide a frequency count for the terms used in table=ctexamined_detected? Mariel suggests that some of the little used terms can be eliminated (eg "ectoparasite: mite or tick") in favor of just ectoparasite (and the details in Methods in Remarks) Thx

@mkoo mkoo added Priority-Normal (Not urgent) Normal because this needs to get done but not immediately. Function-CodeTables labels Jan 10, 2025
@dustymc
Copy link
Contributor

dustymc commented Jan 10, 2025

      ?column?       |       examined_detected        | count  
---------------------+--------------------------------+--------
 ctexamined_detected | ectoparasite: flea             |    508
 ctexamined_detected | isotopic signature             |    446
 ctexamined_detected | virus: Orthohantavirus         |  43080
 ctexamined_detected | pathology: skeletal            |      3
 ctexamined_detected | virus: SARS-CoV-2              |     32
 ctexamined_detected | bacteria: Leptospira           |      0
 ctexamined_detected | ectoparasite: mite             |    303
 ctexamined_detected | PFAS                           |    305
 ctexamined_detected | ectoparasite: tick             |    276
 ctexamined_detected | virus: Borealpox               |    110
 ctexamined_detected | parasite                       |  44061
 ctexamined_detected | ectoparasite                   | 108720
 ctexamined_detected | microbiome                     |      0
 ctexamined_detected | ectoparasite: fly              |    124
 ctexamined_detected | ectoparasite: louse            |    178
 ctexamined_detected | virus: EHDV                    |      0
 ctexamined_detected | fungi: Ophidiomyces            |      0
 ctexamined_detected | fungi: Coccidioides            |    406
 ctexamined_detected | prion: Chronic Wasting Disease |      0
 ctexamined_detected | pathology                      |      7
 ctexamined_detected | ectoparasite: mite or tick     |     11
 ctexamined_detected | virus: Rabies                  |    897
 ctexamined_detected | endoparasite                   |  93631

@mkoo
Copy link
Member Author

mkoo commented Jan 10, 2025

Great! thx, let's see....

My proposal:

  • remove bacteria: Leptospira
  • add bacteria
  • combine pathology and pathology: skeletal to the generic pathology (I know Mariel mentioned a zoo need for pathology: skeletal but that can still be handled in the metadata of methods and remarks
  • ectoparasite: mite or tick Mariel mentioned that these records could simply use ectoparasite with metadata.
  • prion: Chronic Wasting Disease has ZERO usage-- it was added in 2023. Is anyone going to use this? no? then let's remove and we can add it as a value in the future. This applies to the other ones with zero usage (virus: EHDV , fungi: Ophidiomyces, microbiome). If you have data, then send a csv and let's get on it-- we can help! If not, let's purge and add when you have data as part of the request (actually that may be a good thing to consider in the future!) Remember for virus: EHDV if it's removed you can still use virus in the future to capture any EHDV tests.

If anything maybe this will spur some csv files that we can load for these unused terms! Happy to talk again @campmlc

@Jegelewicz
Copy link
Member

prion: Chronic Wasting Disease has ZERO usage-- it was added in 2023. Is anyone going to use this? n

Was added for NCWRC - https://github.com/ArctosDB/data-migration/issues/889#issuecomment-1494703891

If they aren't going to migrate data and nobody else cares, then remove it.

@campmlc
Copy link

campmlc commented Jan 10, 2025

MSB has data for these: virus: EHDV , fungi: Ophidiomyces, microbiome
The fact that no records have been added yet is a function of staff time and resources, not need. Please do not remove.
Can we get a csv of which collections are using this and tag the relevant parties so they know what is happening?

@mkoo
Copy link
Member Author

mkoo commented Jan 10, 2025

MSB has data for these: virus: EHDV , fungi: Ophidiomyces, microbiome The fact that no records have been added yet is a function of staff time and resources, not need. Please do not remove. Can we get a csv of which collections are using this and tag the relevant parties so they know what is happening?

There's no CSV possible to send because there's no collections using these values-- if you have data, send it. If not, lets remove until there's data ready to load.

I understand that there's need but it's hypothetical without data. I know MVZ too has 'microbiome' possible records as we have loans that show such usage BUT we dont have the time or motivation to add that attribute at this time so I advocate removing until there is both. As discussed, we do need to draw the line somewhere and a year+ with no action seems reasonable to me. Again, If it's just sitting around ready to load but no staff time, then just send it for us to load.

@campmlc
Copy link

campmlc commented Jan 10, 2025

Sorry, I meant can we get a csv of collections that are using terms that do have associated records? That wasn't in the above list. I know at least some are MSB, and I'd like to make sure our CMs are informed.
And I think that once we provide notice, we should allow for some time (3-6 months?) for people who may have them to get them loaded. I am waiting on a student who is in the process right now of generating a spreadsheet for Ophidiomyces.

@mkoo
Copy link
Member Author

mkoo commented Jan 11, 2025

Sorry, I meant can we get a csv of collections that are using terms that do have associated records? That wasn't in the above list. I know at least some are MSB, and I'd like to make sure our CMs are informed. And I think that once we provide notice, we should allow for some time (3-6 months?) for people who may have them to get them loaded. I am waiting on a student who is in the process right now of generating a spreadsheet for Ophidiomyces.

I am confused-- how can we identify the collections if there are no records associated with them? We only have the requests.

virus:EDHV May 2023 #6118
fungi: Ophidiomyces Nov 2024 https://github.com/ArctosDB/code-table-work/issues/88
microbiome May 2023 #6269
prion: Chronic Wasting Disease May 2023 #6116 (The collection that requested this NCWRC is no longer active in Arctos and is in stasis so I think it's fine to remove and revisit as needed in the future)

If you mean the low count terms-- @dustymc can you give us unique collections for:
| ectoparasite: mite or tick | 11 |
| pathology: skeletal | 3 |
| pathology | 7 |

Thanks, MK
(edited with better formatting!)

@dustymc
Copy link
Contributor

dustymc commented Jan 13, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Function-CodeTables Priority-Normal (Not urgent) Normal because this needs to get done but not immediately.
Projects
None yet
Development

No branches or pull requests

4 participants