Process documentation: Engineering team "Research Spikes"

[MeltyBot]

Migrated from GitLab: https://gitlab.com/meltano/handbook/-/issues/62

Originally created by @aaronsteers on 2022-02-09 22:02:35

---

Cc Engineering team @tayloramurphy, @edgarrmondragon, @pandemicsyn

TODO: We ant to explore how and if we should support "Spikes" in our flow - and specifically "Research Spikes".

Challenges with "Spikes":

• Difficulty in time constraining the spike. Given the nature of ambiguity being an inherent driver of wanting a "spike", the same uncertainty also makes it hard to guess how much time it will take to resolve the ambiguity. The very nature of spikes is that they have high uncertainty.
• Often hard to define "Definition of Done". The same spike could take 2 hours, 6 hours, or 12 hours, depending on how it is approached, who's tackling it, how much documentation we want as output, and what level of certainty we expect in the output.
• Learnings from the spike may or may not be portable/usable to other engineers in the team.
• A "second opinion" may not be possible without doubling the effort.
• The cost of running, coordinating, and feeding back learnings from the spike may be more expensive than handling inline with the dev work.

Alternatives:

• Avoiding the word "Spike" and calling these instead Research Assignment issues or Spec Definition issues. The subtle difference in description also carries a more clear "definition of done".
• Building in smaller code increments (with perhaps a non-shipped code output in an MR) instead of dedicated Research Spikes (which may otherwise have no code or product output).
• In any issue weighted 8 or 12, build in an expectation for 'uncertainty', along with scheduling upfront a synchronous or asynchronous mid-delivery spec check or spec alignment discussion.
• For issues with high uncertainty, build a practice of documenting "assumptions" upfront and then setting expectations with Product that dev efforts may be aborted, paused, or punted if assumptions are proven incorrect, paired with a clear communication when we think that might be the case.

[MeltyBot]

View 3 previous comments from the original issue on GitLab