[Feature] Use anonymous statistics on KSP version and mod version to allow visibility on "what is working for how many"

[mgalyean]

Currently many CKAN users sort by "downloaded" to get a rough idea of how many people are successfully using a particular mod. But there is no indication of how many of those users are using the same KSP version, or what version of the mod they are using.

The idea would be, and it isn't fully formed, to somehow allow the user to see how many users with a set up (KSP version + mods ideally, but definitely KSP version) similar to the user have installed a given mod and what version of that mod those similar users mostly install.

This would allow CKAN to become a self-updating clearinghouse of ballpark clues on what configurations work.

This would allow users to get beyond the game of getting the game to work and actually play the game more often.

This could also potentially allow mod developers a way to know if a certain subgroup defined by KSP version and other installed mods are recently uninstalling their mod outside the norm so they can more quickly respond to version and combination specific bugs, or alternatively, when fielding a bug report from a user with a specific KSP version and mod combo they could tell at a glance if that is a commonly installed combo for other users or not and so triage the issue better

[HebaruSan]

Sounds a lot like #1566 and #1797. Not sure if a duplicate.

[mgalyean]

It does look a lot like them. Maybe not duplicate but ideas in all could probably be gathered into one issue to centralize the general concept. Basically data mine CKAN. And all the ways that could be done probably needs a fence around it to start with so it isn't too big an ask to break ground on. So maybe triage the ideas in the three issues into one by utility and if is low-hanging fruit code-wise?

[HebaruSan]

gathered into one issue to centralize the general concept

@mgalyean if you want to do that analysis and centralize all the info in this issue, I'd be happy to close those older ones.

[mgalyean]

I would be happy to do that, but the "low-hanging-fruit" criteria will not be so obvious to me so if active maintainers aren't shy with feedback about what would be doable then rounding up and synopsizing the rest is on my plate

[mgalyean]

Combining this issue with #1566 and #1797

I probably missed some things in the other issues but think I caught the gist of them as I understand them. It is not my intention to leave anything out so if I did, tell me what needs to be in here, or what doesn't need to be in here

Main tasks overall (rough, overkill and yet incomplete)

Data gathering (brain dump laundry list here; not prioritized or triaged).
User opt-in, multilevel opt-in, not all-or-nothing opt-in
Truly anonymous, (but an opt-in to allow contacts from devs or other users based on your game configuration. Could be a brokered contact, where others never see your direct contact info, but they can get a simple contact request to you via CKAN)
Versions: KSP, mods, OS, window mgr, hardware, mono version, .net version blah blah
Track actual parts/modules used in active sfs files to highlight parts mods that are truly being used rather than merely remaining installed
Unobtrusive (don't hold up gameplay simply because there is no internet connect to report back on for example; big peeve of many. Save stats and upload later)
A mod like 'exception detector' that centralizes log analysis with regards to mod error messages
Terse/compressed data structures to keep bandwidth requirements minimal.
Thoughtful timing and frequency of data collection. A game should not be delayed, slowed, nor interrupted and the user's time is valuable and respected.

Server side storage
A decent multiuser database with real transactions; postgreSQL, Oracle, etc because the nature of access-creep will outgrow anything less. Especially if KSP players will be running reports via a web interface or similar on the database. Though I'd just imagined the stats being obliquely available via the CKAN client along the grain of how one can currently sort and filter, not necessarily gushingly available via a report engine web interface or similar.

Reporting (same caveats, laundry list + kitchen sink at this point)
User driven queries (from within CKAN client or a separate web interface? (web would require webpage/server to have access to user data while client already has it at hand)
"how many set ups like mine are working?" For self-checking viability of user's mod combo and version expectations
"what version of mod x works with my current set up if any based on other installs elsewhere?"
"what changes of similar configurations that also changed in my set up caused the bug symptoms I'm seeing?"
"what are popular mods based on actual usage, not just installs?" look at actual part/module usage in active sfs?
"what mod and version of mod does this version of this part come from?"
"if i want to run version x.xx of a given mod, what else must be in place?"
"what other mods might i want to look into based on popularity among others who are using a similar set of mods?"
Mod dev driven querys
"what configurations of KSP, mod, os, hardware are having issues with my mod?"
"what recent events correlate with my mod's behavior changing?" KSP version change, metadata version, other mod update, OS update etc.
Mod adopter driven queries
"what mods are highly popular based on actual usage, but have an increasing problem rate as KSP and other version increase, and a decreasing maintenance rate (i.e. appears to be lagging in support)" The idea being to detect mods that at risk of causing exponential problems if not revisited soon. I'm winging it here

Bug Tracking/Reporting (multiple angles, passive and active problem detection)
Passively inferring general user problems statistically by abandonment rate of usage of mod rather than an explicit report from user. Usage inferred by actual part/module usage in active sfs files, not merely existence of install in GameData (though that mere existence would direct what parts/mods to look for in sfs files)
Only analyze recent copies of sfs files in read-only mode to not just prevent harm to user game data, but to give users confidence that no harm is possible. Never copy a currently open game file, use copy of most recent Backup instead if game is running etc.
Active bug reporting interface for players where they can multi select some mods and flag that combo they are having an issue, or alternatively, that the combo is golden and works well. Ways to attach log files, screen shots, etc. Probably a separate bugtracking web app would be indicated but there'd need to be a gateway for said app to have access to user and CKAN installation stats database(s)

All reporting needs to be on actual active play time using the mods. Many people have installs and sfs files that haven't been used in months which could seriously skew data so focus needs to be where player gaming focus is. So some mechanism of detecting when a game is launched and where to find the active sfs file and its backups are that would likely be OS dependent. If users don't opt in for this level of tracking then perhaps a weaker version based on the most active CKAN managed instance and the installed files could be used instead. But the sfs file would be where the real info is with regards to mod/partmodules actually in use. Maybe users only get access to others' info at their own opt-in level. So if not at the full opt-in, you get less than fully focused/informed reports based on similarly lower opt-in level data from others? ouch. the wrinkles hurt.

I wanted to get this these issues merged while thinking about them and apologize if it looks rushed. It was, in fact, rushed