Skip to content

Latest commit

 

History

History
39 lines (26 loc) · 3.57 KB

paper_text.md

File metadata and controls

39 lines (26 loc) · 3.57 KB

Software makes science better, but is it research? Arguments for a research agenda in scientific software work.

Better software makes for better science, yet software work is questioned as scientific contribution. This is a clear tension in the lives of scientists writing and using software. In this presentation I describe different arguments made for why software work is research. I draw on interviews from empirical studies of scientific software work, discussions at workshops, as well as participation in peer review, promotion and tenure, and grant review panels. I present arguments that seem to work well, arguments that seem to work poorly, and sketch potential arguments rarely made. I hope to provoke discussion, reflection, and to encourage better writing about the contribution of scientific software work.

How to pursue a research agenda in building software for scientists.

How to bring out the intellectual contribution of work that provides software that underlies science, or How to avoid being called "mere plumbing" or "trivial software development".

What works?

  1. Develop novel algorithms or statistical methods realized in the software. Here the contribution is clear, the algorithms stand apart from their particular implementation.

  2. Identify and evaluate the generalizable data or runtime software techniques instantiated in the software (e.g., a new design pattern, a new data structure, new architectural move.)

    • sociotech version (architecture of participation)

  3. Identify and evaluate the generalizable approaches in surrounding areas (i.e., not software development, not domain science):

    • requirements analysis approaches (e.g., new and better ways to learn about how scientists do their work).
    • interface approaches (e.g., new and better ways to suggest using ontology terms rather than free form text) (also knowledge representation literature)
    • community building approaches (e.g., novel architectures of participation or that provide better incentives for integration)
    • software development methodologies (e.g., theory and challenges for introducing agile into a lab, project as case study)
    • introducing change in scientific communities through software (hypothesize that the system will create change, measure whether it does).

Each of those approaches would need to highlight an epistemology and evidence. For empirical approaches that would require data collection and analysis and a sampling approach. These might best be approached by interdisciplinary teams.

What generally doesn't work:

  • test and evaluate a specific piece of software (to see if it works); not generalizable.
  • argue entirely on broader impacts (by saying that the work makes science as a whole better). Rightly or wrongly being asked to show a "research agenda" means an individual research agenda, not being part of a team research approach.

Things that perhaps should work but are often questioned:

  • Argue for quicker science (just as algorithm) by showing that you have made a new technique available to a domain. (good impacts, but "intellectually trivial", missing epistemology and evidence.)
    • This mistakes reconstructed logic for logic in use, the path to a contribution is not published about. connect to discussion of craft knowledge vs intellectualism.
    • could potentially connect this to teaching (e.g., writing and improving online resources or creating tutorials, often done in tech practitioner conference circuit.)

Possible data to support these assertions:

- Could code software papers?
- And/or SI2 abstracts (and OCI/OAC CAREER grants).