Discover collaborating orgs via contributor activity feeds

[andrew]

For issues and pull requests that are opened by non-core non-collab folk, we can look in their personal activity feeds for PushEvent events to see which orgs that user has commit access to, and so is likely to be a core contributor of.

We have the contributors table now so we can store the etag of the users latest activity poll to keep api requests to a minimum, we can also store the events in the existing events table.

This can then be used to generate some kind of weekly report of suggested orgs as potential new collabs, perhaps even speculatively indexing some of their repositories to look for dependencies on internal packages.

[andrew]

Did a quick spike of this using recent activity from the top 5% external contributors to ipfs projects (who've opened issues or pull requests), there are definitely some orgs that are using ipfs in there, also some related software orgs like bazil, d3, npm, golang that get used a lot by ipfs projects, so it's tending to find both up and downstream projects when we really are just looking for downstream (things that use ipfs rather that things that ipfs uses).

Interestingly there was no cross-over between those top 5% orgs, which suggests that there is often one person within an org that is handling contributions and bug reporting to ipfs:

• https://github.com/mkg20001
  • https://github.com/ssd-solar
  • https://github.com/mercode-org
  • https://github.com/ZeroNerds
  • https://github.com/ZeroNetJS
• https://github.com/noffle
  • https://github.com/cabal-club
  • https://github.com/digidem
  • https://github.com/kappa-db
• https://github.com/rht
  • https://github.com/ox-inet-resilience
• https://github.com/kumavis
  • https://github.com/LavaMoat
• https://github.com/keks
  • https://github.com/scuttlebutt-eu
  • https://github.com/ssbc
  • https://github.com/cryptoscope
• https://github.com/JustinDrake
  • https://github.com/vdfresearch
• https://github.com/cryptix
  • https://github.com/cryptoscope
  • https://github.com/planetary-social
• https://github.com/mildred
  • https://github.com/cantaverde
• https://github.com/mitra42
  • https://github.com/Safe2COVIDApp
• https://github.com/AuHau
  • https://github.com/rsksmart
• https://github.com/nginnever
  • https://github.com/finalitylabs
• https://github.com/mitar
  • https://github.com/tozd
  • https://github.com/wlanslovenija
  • https://github.com/meteor-vue
• https://github.com/Nashatyrev
  • https://github.com/PegaSysEng
• https://github.com/bluelovers
  • https://github.com/demonovel
• https://github.com/mcollina
  • https://github.com/fastify
  • https://github.com/pinojs
• https://github.com/SidHarder
  • https://github.com/YPII
• https://github.com/mappum
  • https://github.com/nomic-io
• https://github.com/nothingismagick
  • https://github.com/tauri-apps
  • https://github.com/webview
  • https://github.com/iotaledger
• https://github.com/wraithgar
  • https://github.com/npm
  • https://github.com/npm-wharf
• https://github.com/NukeManDan
  • https://github.com/Cryptorado-Community
• https://github.com/thisconnect
  • https://github.com/digitalbitbox
• https://github.com/machawk1
  • https://github.com/metadata-research
• https://github.com/tdiesler
  • https://github.com/wildfly-extras
  • https://github.com/jboss-fuse
• https://github.com/celeduc
  • https://github.com/Caelumlabs
  • https://github.com/lorena-ssi
• https://github.com/mxinden
  • https://github.com/tikv
• https://github.com/geoah
  • https://github.com/nimona
• https://github.com/kyledrake
  • https://github.com/neocities
  • https://github.com/dorkbotpdx
• https://github.com/Mikerah
  • https://github.com/hashcloak
• https://github.com/aphelionz
  • https://github.com/rs-ipfs
  • https://github.com/hackforthesea
  • https://github.com/MRH-io
  • https://github.com/opencollective
• https://github.com/PascalPrecht
  • https://github.com/thoughtram
  • https://github.com/rebase-book
• https://github.com/jezhiggins
  • https://github.com/cyclestreets
  • https://github.com/wmfs
• https://github.com/eminence
  • https://github.com/overviewer
• https://github.com/koalalorenzo
  • https://github.com/Qm64-tech
• https://github.com/dokterbob
  • https://github.com/ipfs-search
  • https://github.com/dsrp
• https://github.com/DemiMarie-parity
  • https://github.com/runtimeverification
• https://github.com/larsks
  • https://github.com/CCI-MOC
• https://github.com/vasa-develop
  • https://github.com/simpleaswater
• https://github.com/AgeManning
  • https://github.com/sigp
  • https://github.com/rust-ethereum
• https://github.com/aeddi
  • https://github.com/berty
• https://github.com/ntninja
  • https://github.com/flathub
• https://github.com/richsilv
  • https://github.com/reinfer
• https://github.com/ya7ya
  • https://github.com/livepeer
• https://github.com/cpacia
  • https://github.com/gcash
• https://github.com/dvc94ch
  • https://github.com/ipfs-rust
  • https://github.com/rust-windowing
  • https://github.com/sunshine-protocol
• https://github.com/tv42
  • https://github.com/bazil
• https://github.com/hosh
  • https://github.com/matsuri-rb
  • https://github.com/LegendsOfLearning
• https://github.com/AliabbasMerchant
  • https://github.com/CommunityOfCoders
• https://github.com/cannium
  • https://github.com/journeymidnight
• https://github.com/kevinsimper
  • https://github.com/greenmobility
  • https://github.com/copenhagenjs
• https://github.com/kpp
  • https://github.com/tox-rs
• https://github.com/pcowgill
  • https://github.com/getuni
  • https://github.com/tasitlabs
• https://github.com/whereswaldon
  • https://github.com/arborchat
  • https://github.com/IBM
• https://github.com/Frijol
  • https://github.com/edgi-govdata-archiving
• https://github.com/aknuds1
  • https://github.com/grafana
• https://github.com/gnunicorn
  • https://github.com/RadikalSternchen
• https://github.com/Luzifer
  • https://github.com/luzifer-docker
  • https://github.com/luzifer-aur
  • https://github.com/nohelloinfo
• https://github.com/koushiro
  • https://github.com/chainx-org
  • https://github.com/PolkaX
• https://github.com/samparsky
  • https://github.com/AdExNetwork
• https://github.com/prestonvanloon
  • https://github.com/prysmaticlabs
• https://github.com/albrow
  • https://github.com/0xProject
• https://github.com/enricomarino
  • https://github.com/pantareijs
  • https://github.com/desidera
  • https://github.com/ipws
• https://github.com/fulldecent
  • https://github.com/chain76org
• https://github.com/negamaxi
  • https://github.com/jeditools
• https://github.com/npfoss
  • https://github.com/sipb
  • https://github.com/p2p-cdn
  • https://github.com/battlecode
• https://github.com/timthelion
  • https://github.com/auto-mat
  • https://github.com/subuser-security
• https://github.com/anarcat
  • https://github.com/linkchecker
• https://github.com/gpestana
  • https://github.com/hashmatter
• https://github.com/NoraCodes
  • https://github.com/beloitcollegecomputerscience
  • https://github.com/rust-native-ui
• https://github.com/phritz
  • https://github.com/rocicorp
• https://github.com/sbuss
  • https://github.com/LiberalTomorrow
  • https://github.com/golang
  • https://github.com/sfcpc
• https://github.com/tobowers
  • https://github.com/quorumcontrol
• https://github.com/upperwal
  • https://github.com/pravahio
• https://github.com/Wondertan
  • https://github.com/proximax-storage
• https://github.com/zaibon
  • https://github.com/threefoldtech
  • https://github.com/threefoldfoundation
• https://github.com/chpio
  • https://github.com/vdom-rs
• https://github.com/crackcomm
  • https://github.com/parchiff
  • https://github.com/rootchain
  • https://github.com/ipfnio
• https://github.com/Fil
  • https://github.com/d3
  • https://github.com/visionscarto
• https://github.com/godcong
  • https://github.com/glvd
  • https://github.com/portmapping
  • https://github.com/javscrape
• https://github.com/interfect
  • https://github.com/NovakDistributed
• https://github.com/Mikaela
  • https://github.com/Piraattinuoret
  • https://github.com/privacytoolsIO
• https://github.com/thomaseizinger
  • https://github.com/comit-network
  • https://github.com/coblox
  • https://github.com/testcontainers
• https://github.com/bcomnes
  • https://github.com/gumcast
  • https://github.com/little-core-labs
  • https://github.com/hypermodules
• https://github.com/ChihChengLiang
  • https://github.com/appliedzkp
• https://github.com/elopio
  • https://github.com/JaquerEspeis
  • https://github.com/OpenZeppelin

[andrew]

Further filtering by these orgs but ones that have mentions of "ipfs" in their codebase works well, 40% of possible orgs include at least one result and 20% had over ten results:

Organization	GitHub code search for "ipfs" result count
livepeer	1407
proximax-storage	281
berty	267
quorumcontrol	249
glvd	183
ipfs-search	146
jboss-fuse	92
AdExNetwork	78
rs-ipfs	66
simpleaswater	65
IBM	52
rsksmart	49
iotaledger	40
ipfs-rust	37
PolkaX	33
pravahio	32
sunshine-protocol	26
p2p-cdn	24
0xProject	23
neocities	20
wildfly-extras	19
hashmatter	19
NovakDistributed	17
prysmaticlabs	16
ssbc	12
Cryptorado-Community	11
threefoldfoundation	11
YPII	10
gcash	10
rootchain	10
JaquerEspeis	10
ZeroNetJS	9
threefoldtech	9
Caelumlabs	8
lorena-ssi	8
MRH-io	8
PegaSysEng	6
demonovel	6
OpenZeppelin	6
runtimeverification	4
mercode-org	3
opencollective	3
ipws	3
comit-network	3
nomic-io	2
tauri-apps	2
npm	2
dsrp	2
tasitlabs	2
edgi-govdata-archiving	2
rocicorp	2
golang	2
LavaMoat	1
hackforthesea	1
Qm64-tech	1
sigp	1
arborchat	1
auto-mat	1
subuser-security	1
appliedzkp	1
ssd-solar	0
ZeroNerds	0
cabal-club	0
digidem	0
kappa-db	0
ox-inet-resilience	0
scuttlebutt-eu	0
cryptoscope	0
vdfresearch	0
planetary-social	0
cantaverde	0
Safe2COVIDApp	0
finalitylabs	0
tozd	0
wlanslovenija	0
meteor-vue	0
fastify	0
pinojs	0
webview	0
npm-wharf	0
digitalbitbox	0
metadata-research	0
tikv	0
nimona	0
dorkbotpdx	0
hashcloak	0
thoughtram	0
rebase-book	0
cyclestreets	0
wmfs	0
overviewer	0
CCI-MOC	0
rust-ethereum	0
flathub	0
reinfer	0
rust-windowing	0
bazil	0
matsuri-rb	0
LegendsOfLearning	0
CommunityOfCoders	0
journeymidnight	0
greenmobility	0
copenhagenjs	0
tox-rs	0
getuni	0
grafana	0
RadikalSternchen	0
luzifer-docker	0
luzifer-aur	0
nohelloinfo	0
chainx-org	0
pantareijs	0
desidera	0
chain76org	0
jeditools	0
sipb	0
battlecode	0
linkchecker	0
beloitcollegecomputerscience	0
rust-native-ui	0
LiberalTomorrow	0
sfcpc	0
vdom-rs	0
parchiff	0
ipfnio	0
d3	0
visionscarto	0
portmapping	0
javscrape	0
Piraattinuoret	0
coblox	0
testcontainers	0
gumcast	0
little-core-labs	0
hypermodules	0

[andrew]

Here's the output of a similar query for libp2p, finding lots of the expected collabs and doing a good job of filtering out false positives:

Organization	GitHub code search for "libp2p" result count
ipfs	1816
filecoin-project	385
status-im	212
PegaSysEng	128
ethereum	122
ZeroNetJS	45
MetaMask	29
drand	12
open-services	7
protocol	6
ipfs-cluster	2
mntnr	1
hashcloak	1
openethereum	1
SearchAndRescue2	0
async-email	0
rpgp	0
async-rs	0
deltachat	0
RustCrypto	0
ssd-solar	0
mercode-org	0
ZeroNerds	0
sustainers	0
foss-responders	0
IndigoUnited	0
LavaMoat	0
traveltomadeira	0
vdfresearch	0
tikv	0
cyclestreets	0
wmfs	0

[andrew]

One note here for both ipfs and libp2p: collabs that had already been imported into those dashboards where filtered out from suggestions, incase anyone was worried that those collabs had been missed.

[andrew]

Filecoin didn't find many non-protocol orgs as potential collabs.

Organization	GitHub code search for "filecoin" result count
ipld	322
ipfs	135
ProtoSchool	25
protocol	8
ipfs-shipyard	8
libp2p	7
drand	4
multiformats	1
rocicorp	0
quarantenalive	0
ipfs-rust	0
soramitsu	0
celo-org	0

[autonome]

This is fantastic, thanks for pulling on this thread. I'd like to have this running daily, tracking absolute number over time for all three projects (including existing collabs), and pushing to Influx, Grafana (or somewhere). Additionally I'd like to have a daily digest email that covers all three projects and shows additions and removals so that we can act within 24hrs to onboard potential new users and also to know about *departures* (is after the fact... but also might be the soonest possible for us to know). Additionally additionally, I'd like this to run hourly and post notifications to #collabs on Slack about additions/removals.

…

On Tue, Jun 16, 2020 at 11:25 AM Andrew Nesbitt ***@***.***> wrote: Filecoin didn't find many non-protocol orgs as potential collabs. Organization GitHub code search for "filecoin" result count ipld <https://github.com/ipld> 322 ipfs <https://github.com/ipfs> 135 ProtoSchool <https://github.com/ProtoSchool> 25 protocol <https://github.com/protocol> 8 ipfs-shipyard <https://github.com/ipfs-shipyard> 8 libp2p <https://github.com/libp2p> 7 drand <https://github.com/drand> 4 multiformats <https://github.com/multiformats> 1 rocicorp <https://github.com/rocicorp> 0 quarantenalive <https://github.com/quarantenalive> 0 ipfs-rust <https://github.com/ipfs-rust> 0 soramitsu <https://github.com/soramitsu> 0 celo-org <https://github.com/celo-org> 0 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMHN7YTIJN4DCWA35EIVTRW6FGLANCNFSM4N6O562A> .