How to limit increased maintenance cost caused by premature translations of beta kata

[hobovsky]

One of common problems occurring during beta phase of many kata are maintenance problems caused by translations added to freshly published beta kata. Usually, the flow is like this:

1. A new kata is published and goes through a couple of reviews.
2. Users of various experience levels smell the fresh meat and publish translation to their favorite language(s), often quite esoteric ones, like Haskell or Rust.
3. Authors are happy about the fact that their new kata is so great that it attracts translations, and approve whatever is thrown at them.
4. An issue which affects all existing translations gets reported.
5. The kata becomes difficult to fix, because authors do not know languages, translators are not easily available, or there's no good agreement how to fix a general issue across languages of different paradigms or type systems.

I am looking for answers to following questions:

• Is this, in your opinion, a significant issue currently affecting the system?
• If yes, do you have any idea how to improve the situation?

[hobovsky]

My idea: introduce a state of a kata which makes it non-eligible for accepting translations.

Criteria would be to be defined, and there's a couple of possibilities:

• Wait some amount of time, like X days, or at least hours - as today translations can pop up a couple of quarters after publication of a kata, even some minimal limit would help to settle the endorphines down before translations appear. However, it's not my favorite idea as I am not sure whether the age of a beta kata reflects its quality in any way.
• Allow translations only for kata with X solutions - this way we'd hope that beta kata with a reasonable amount of solutions are stable and of acceptable quality. This could work, the question is, how large should X be. One potential drawback are kata originally published in a language which does not guarantee many solutions.
• Allow translations only on approved kata - IMO not a perfect idea because approval is not really a good sign of the stability of a kata. A kata can be well curated while still at beta, just as it can get prematurely approved and have issues reported shortly after.
• I cannot find any good application for other metrics, like amount of open issues, satisfaction rating, rank, etc.

One possible complaint I can see is that marking kata as ineligible for translations will lock out translators who definitely know what they are doing and are unlikely to introduce issues. I think this argument does not really apply, as the issues which might require fixing might be related to a general idea, like input format, handling of floating point inputs, etc. and skills of a translator do not change how stable a kata is.
How this could be too restrictive, on the other hand, is limiting experienced authors. There are authors who do publishing correctly right from the start, or are able to handle reported issues in all (most of) translations. Maybe some notion of a "renowned author" could be used (like it currently is while determining eligibility of a kata for approval), or just accept this fact as a trade-off.

[ejini6969]

• Yes, it is an issue. Not only it makes maintenance more troublesome, but the overall satisfaction rating of the kata will be dropped since the solver who solves in specific languages (not the original language) might find it odd either on the implementation, output format, paradigms etc.

• Based on Hob's comment above, the first two recommendations can be difficult to establish a threshold especially the difficulty of the kata increases (in which the time as well as solution required to gather will increase exponentially too). Hence, the most optimal way is to completely deny external translation proposal to a kata (as in no one other than the author can translate the kata) until approval comes in. The author has the right to do so since it is his kata and as long as one is knowledgeable enough in the language one ought to translate in, it should not be a problem! If in any case the translated language has been reported with several issues, the author can take action on either to fix the translation or completely removed it, which can guarantee that their kata will not receive continuous downvotes. Hence, a win-win solution for content-maintainers as well as the author!

[hobovsky]

Limiting the criteria for eligibility just to beta kata (i.e. translations can appear after a kata is published) can cause problems similar to the original issue. Approval of a kata is a point when it gains a lot of exposure, attracts new solvers, and there's a high potential for someone spotting new problems with it. Ideally, this should not happen and kata which leave beta should (and today usually are) well.curated and stable, but its still possible that a kata leaves a beta in not a really good shape, gets translations, and becomes difficult to fix.
I wonder if a mechanism of eligibility should maybe span on whole lifetime of a kata, not depending on its beta state.

[ejini6969]

I can't seemed to understand how one defines a kata as in a premature state (as stated in your previous post), since every kata is discrete enough from one another and basically it is extremely rare to find fully non-issue-existant katas on CW even nowadays! (even though a kata has been said to have 100% test coverage within the scope of kata description, some random users will still propose to add specific fixed test cases in the future since many of them are using IDE, hence will add their own test cases and potentially exploit a bug in their own / other's solutions)

I, too, believed that the job of content maintainers (mods and menders) are not to propose translations to beta katas for testing purposes due to (i. Providing a translation requires higher level of knowledge compared to reviewing, hence things can go wrong easily!
ii. Content maintainers become the main driving force in translating since when they propose one to.new.beta kata, and find that all tests passed, hence they propose it to author. Author immediately approves since they saw it's proposed by mod and assumes nothing can go wrong which can become a difficult hole to patch along the way. It also steals away the chance for other language enthusiast to translate into their favourable languages (especially js since 90% of katas alr have js enabled)
iii. Difficulty in defining a threshold for consistency among languages (mostly talking about performance versions, since not so experienced maintainers can falsely set up a threshold range of 1e21 for instance for inputs based on their solution which might not be the most efficient one. Upon approval, some users then come up with a more efficient solution which can work for inputs.beyond 1e32, hence they found the language's rank is.inconsistent with the original.one, thus more.issues and confusions!)

Instead, content maintainers should only fix existing issues of approved katas, review betas, provide constructive feedback, propose different solutions to find workarounds for edge cases. As for translating.to beta katas, it should be a voluntary process due to issues mentioned above!

(for translation part, i was referring to.comment.by B4B below, but didnt want to write my comment twice, hence i write them all here)

[Blind4Basics]

I agree it would be easier if translations weren't possible during beta. Tho, there is a potential drawback to this: kata authored in "esoteric" languages couldn't be translated in popular ones (JS/python), and may never be approved, so.

[hobovsky]

This is something what I'd see as one of tasks for menders/the "content team": to actively animate approval of good betas, by asking for, or providing, solutions, translations, votes, etc. I did it before, I added a python and JS translation to a nice, Swift only kata, and even purple N-queens were C++-only and caught attention after translating to JS.

[Blind4Basics]

yeah, that could do it too. But the mender group would need to gain in variety (practiced languages-wise)

[Kacarott]

While disallowing translations in beta would mostly make the beta process better, I also have the same concern as B4B, that kata published in anything besides the Python/JS can take incredibly long to get enough solves for approval, if ever.

Perhaps a workaround would be to prohibit translations on beta kata, until they have gone 30 days without an issue (could be consecutive days, or cumulative). This way, older kata which have stabilised can still be translated to more popular languages.

[hobovsky]

Another point is how exactly new translations should be forbidden:

• is it not possible to create a translation at all?
• is it possible to start a translation and keep it as a draft, but not publish it?
• is it possible to publish a translation, but not approve it?

[awesomeAD1]

• is it not possible to create a translation at all?
• is it possible to start a translation and keep it as a draft, but not publish it?
• is it possible to publish a translation, but not approve it?

There is no point starting a translation only to keep it as a draft. By the time you are allowed to publish it according to above parameters, it will likely be out of date. Same goes for publishing without approval.
If a user truly can't possibly wait to start translating, then they can do what I do: create a local file and write a translation there (of best quality thanks to the help of an IDE...), save it to disk, and copy it to codewars once the time is right.

[hobovsky]

Solutions based on a kata becoming at some point eligible for translations still suffer from a problem akin to the thundering herd phenomenon: translations still pile up, in one place or another, and then appear in a potentially large bunch, causing conflicts, and the queue can get clogged quite easily.

Maybe another direction would be worth considering: reduce amount of pending translations. For example, there can be only one, or two, or X, pending translations at a time.

Advantages are:

• translations do not pile up and don't get in a way each of another: no more merge conflicts!
• income of translations is slower, having smaller potential to throw kata off-balance.

Cons:

• very restrictive compared to what we have now. But is it a bad thing?
• can potentially interfere with bug fixed and edits introduced via forks.

[Blind4Basics]

can potentially interfere with bug fixed and edits introduced via forks.

I guess kazk can differentiate forks from regular translations? (since forks can be done on existing languages only. As a matter of fact, that might be a good occasion to forbid bug fixes through translations?)

[ejini6969]

I think this will have more cons than pros, Sorry to say, but when users who have strong liking on a kata found out that it had reached maximum pending translations available and had to.wait. Either they just gave up translating or they did it then saved it in their own storage, and eventually they forgot about.it,in which both resulting in a deficit lost of quality potential.content.to community!

The best way to.resolve this is to.make all CW kata language-agnostic except.for extra notes about different input range or output type expected across.different languages. But I'm afraid we had a long way to go as some users still insisted on having independent language description within kata....
BTW, I rmb there was a mention about kata description being not.part.of the forming procedure some time agoo or moving it.to an external.panel to.prevent conflicts. For that, i think it could be a potential direction to.head on.in the future!