Feedback on draft proposal for "Upstream" work

[aprilmj]

This is a proposal for how we might make upstream leadership more visible and more shared. This proposal is a bit more experimental than the other board and process proposals (#817 and #819) and attempts to address a range of different types of work.

Questions for feedback

1. What's missing or glaringly wrong about what I've captured?
2. What other alternatives, if any, should we consider?
3. Is this safe to try, and if so, how do we know it's succeeded or failed?
4. Please either make changes to this top level comment to change wording, but add new comments with bigger questions - so we have a history of the discussion.

Why is this important?

We want to act as a more effective team, both in developing 2i2c's product and service and in contributing to the wider world of knowledge community infrastructure (trying out the language from the value proposition). If we can see all the work happening in a given period, we'll be clearer about our true capacity for work, make better decisions about how we use our capabilities, and be able to contribute and grow more intentionally.

Which kinds of work need a "something else" process or board?
When we attempted to refine the collection of boards currently used to track work, several similar points came up as needing "something else" - meaning they were not visible in our current boards and processes (from the March Initiative Board refinement meeting):

• Catalyst Board is something else
• Board for 2i2c to provide leadership upstream? ???
• JupyterHealth Board
• JupyterGIS
• We need a tool for cross-functional coordination at the mid-level (ie. between a subset of areas)
• VEDA board
• HHMI
• Openscapes
• CryoCloud
• Pythia
• NASA ScienceCore

@colliand added some more items above.

The common elements across all these points are - they involve external teams (and may have their own project boards where work is tracked), they result in tasks that need to be completed by Engineering and Partnerships team members, and they often are not visible to the entire 2i2c organization.

@colliand adding. These external engagements share: 2i2c providing thought/science/technical/collaboration leadership; 2i2c serving as "glue" to bring otherwise separated groups together; 2i2c not controlling the agenda of the collaboration; 2i2c exposure to scope creep without cost recovery;

The proposal: external and upstream leadership initiatives

We already have a source of truth (as we improve it) for non-reactive work: the Initiatives Board. We also have a process proposed for planning Engineering or Operations (including Partnerships) tasks within a sprint. And we have project boards we share with external collaborators. Creating more boards might make this worse; instead, let's add more process to the boards we have.

Track every significant upstream/project leadership effort as an initiative
This will give us a container for identifying and planning all work we do for each effort (whether it's and independent project or something else). All the process outlined on #817 applies to these initiatives as well.

Add a functional area for "Upstream Leadership"
The initiatives board (and all board views)can easily be grouped by functional areas like People, Product, Engineering, etc today. Adding an area would help us see everything we're doing upstream in one place without creating another team, board and process.

Tasks from Upstream Leadership initiatives flow to Engineering and Operations sprint boards
We already have a process for retrospecting and planning fortnightly sprints in these teams, so planning tasks as part of those teams' sprints gives us the greatest visibility, flexibility and collaboration across all the work that impacts the teams.

A note about assignments
Upstream work appears to need cross-team collaboration often. Tracking the work in only one team's sprint board does pose some risk - what if the other team doesn't see the board? One way we could reduce that risk is by using assignments in GitHub liberally. Add people who need to be consulted as well as the people responsible for driving action. Remove yourself from the issue when your part is done.

[choldgraf]

A few quick thoughts:

• I like the idea that we re-use the concept of "initiatives" in the context of upstream work. This both means we don't have to keep track of a different kind of work, and it also means we can re-use the same system for this work that we use for other non-reactive work.

• I think it's fine if the other boards still exist, and we can use those boards in place of the [tasklist] objects that we often use in initiatives. Upstream boards become glorified tasklists since they require a bit more metadata and flow-control. (maybe we decide something similar for other initiatives that are particularly complex)

• I think it'd be helpful to have an example of what the "flow" of work might look like.

  For example: for Catalyst, would we just have a single, ongoing initiative in our initiatives board, and this initiative would cross-link to the Catalyst project board? In that case, when teams are doing their sprint planning, they'd need to then cross-reference the catalyst board and decide if there's something there that they need to pull into that week's sprint. Once it is in a sprint board, it is now "tracked as in-progress work". Until it is tracked in a sprint board, we assume 2i2c's team has no short-term plan to do it. Does that sound right?

[choldgraf]

Just noting that I have a PR to add some language about the boards stuff below, but it doesn't quite address this issue. If we want to fold this one in there too I'm happy to think through language. Just giving the FYI for now:

• Add initiatives and delivery board description #821

[aprilmj]

I'd like to hear from @yuvipanda and @colliand on this one - it feels like something we should try, where our solution will need refinement once we try it.

The way you captured the workflow in your comment is also what I imagined, @choldgraf

For example: for Catalyst, would we just have a single, ongoing initiative in our initiatives board, and this initiative would cross-link to the Catalyst project board? In that case, when teams are doing their sprint planning, they'd need to then cross-reference the catalyst board and decide if there's something there that they need to pull into that week's sprint. Once it is in a sprint board, it is now "tracked as in-progress work". Until it is tracked in a sprint board, we assume 2i2c's team has no short-term plan to do it. Does that sound right?

[haroldcampbell]

@choldgraf regarding this question/example

• For example: for Catalyst, would we just have a single, ongoing initiative in our initiatives board, and this initiative would cross-link to the Catalyst project board? In that case, when teams are doing their sprint planning, they'd need to then cross-reference the catalyst board and decide if there's something there that they need to pull into that week's sprint. Once it is in a sprint board, it is now "tracked as in-progress work". Until it is tracked in a sprint board, we assume 2i2c's team has no short-term plan to do it. Does that sound right?

We are currently doing something similar with the Partnership/Operations board and the Engineering. Work originating from the Catalyst board (e.g. czi-catalystproject/132) is added to the Partnership board (e.g. meta/838). Sub-issues are created that then get tracked on the Engineering board (e.g. infrastructure/3740). The (minor) challenge is that the Catalyst folks don't always have visibility into our private repos when we create sub-issues.

[aprilmj]

What @haroldcampbell mentioned is exactly what I had in mind, just extended to other projects and upstream work, in addition to Catalyst.

[choldgraf]

IMO, this feels safe to try. What are our next steps here?

[aprilmj]

I agree, this feels ok to try. I believe these are the next steps. Should I create a card for this, or just add it to #828?

1. Add the new initiatives to the initiatives board (the list in the top level comment here, to start)
2. Incorporate them into our next round of initiative-level planning (requires someone knowledgeable to do some refinement/prep of the initiative)
3. From that planning, add delivery-level tasks to the next round of Engineering & Operations sprint planning

Do we need to add anything to the Team Compass, or have we already sufficiently captured the different planning flight levels with the PR @choldgraf did last week?

[choldgraf]

How about creating a new issue with the list of projects we want to track, and adding the issue as a sub-task within the "initiatives revamp" initiative (how many times can I type initiative in one comment?).

• Define the Initiatives board and the Delivery boards (operations + engineering) purpose, structure, and process #816

And closing out this issue with a little docs PR to our inititaive board docs that explains what we do with upstream projects. I don't think we need anything super specific, but maybe a sentence or two in a ## What about upstream and external projects? section. Does that make sense?

https://compass.2i2c.org/cross-functional/workflow/#initiatives-board

[haroldcampbell]

@aprilmj may I ask you to run the next initiative-level planning meeting?

[aprilmj]

@aprilmj may I ask you to run the next initiative-level planning meeting?

@haroldcampbell if you mean the one on April 9, sure (and let's make time to coordinate in advance).