RoboCup@home, site & wiki: stronger linking to GitHub

[moriarty]

Common complaint heard was the new participants went to the RoboCup@Home site, and then to the wiki for information... but GitHub is where the action and discussions happen, and this information isn't always ported back to the wiki.

Stronger linking to GitHub. Perhaps maintenance plans for the wiki should be discussed.

[kyordhel]

Or to a most robust platform.

I think the Wiki is the best option for a compromised league, but the reference to Github must be highlighted.
Wiki

• All previous rulebooks
• Goals, milestoness and data for the years to come (redundant)
• History of team participation, including:
  • TDP
  • Link to team's website
  • Link to qualification video
  • Videos during the competition
  • Open source code listing
  • Rankings
• Previous rankings (winners of @Home)
• Maintained by the league itself

However this doesn't happen, the OC should organize and validate the information first (there is too many, outdated and hard to find), specially for the teams that doesn't exist any more. We lack of volunteers.

GitHub

• Contributions to last rulebook
• Current rulebook discussion
• Milestones and data for the years to come (redundant)
• Helper applications for the competition (e.g. generators)
  As it is right now

[LoyVanBeek]

The wiki and website do need a thorough update and cleanup.

The wiki is hosted by RTWH Aachen if I recall correctly and getting edit access was difficult or impossible.

@kyordhel what do you mean by 'compromised league'?

[moriarty]

The wiki is hosted by RTWH Aachen if I recall correctly and getting edit access was difficult or impossible.

@timn do you have the contact info for the maintainers of that wiki?
Actually... I was about to tag you and then decided to poke around the wiki first.
From this query: https://robocup.rwth-aachen.de/athomewiki/index.php?title=Special:RecentChanges&days=300&limit=250 it looks like you've created all the recent users 😃

From the list of all pages, there isn't much on the wiki: 12 pages.

And from expanding the recent changes back to the beginning of 2016, we see that not many of the pages are active.

[dirkholz]

We should completely get rid of the old wiki

[kyordhel]

@LoyVanBeek a tricky word and false friend for Spanish. I was aiming for engaged, committed, or involved. Teams who have taken the compromise.

@ALL
In former years it was mandatory to update the wiki for qualify, something most teams did more for obligation than with any interest. The information is disorganised and hard to find, so I removed this requirement from the call.

I think a wiki (or similar platform) is important. Each team should have a page listing all videos, TDP, rankings, committee members, and contributions. Teams must be searchable by name but also by ranking. In the same sense, awards and rulebooks must be listed here, as well as videos of the tests and finals.

Again, we need volunteers to set up this. Is quite some work (a time consuming one) and I don't want it to be left abandoned before is usable by the teams. As OC chair I can coordinate efforts but I won't do it myself.

[LoyVanBeek]

I 100% agree a wiki is very useful. Some stuff like best practices, which teams uses which software for what etc should be on there.

[dirkholz]

Useful yes, but not the old one

[tkelestemur]

Why don't we use GitHub wiki? It is markdown based meaning there is no-need for accessing any admin panel (I don't know how old wiki is maintained but I'm guessing it has a admin panel to edit content) and also wiki will be on the same platform with RuleBook.

FYI, I am willing to help maintaining wiki either old or a new one but I am also in favor creating a new wiki.

[LoyVanBeek]

An added benefit of GitHub wiki is that it's also backed by git, so you can clone a local copy.

I does not make sense (to me) to put the 'new' wiki in the Rulebook repository though, but a new repo is created easily.

[tkelestemur]

Agreed. It should be a new repo called 'wiki'.

[LoyVanBeek]

@dirkholz @kyordhel @balkce @moriarty @swachsmu Is this the way to go?

[kyordhel]

Big favour:

When replying by email, please remove the old text.

The GitHub wiki is not powerful enough for this task, I think. It lacks of organization tools like publishing pages in folders, or generate automated content pages. It is OK for document software, no but not necessarily for our needs, as it wouldn't be blogger.

A repo for the wiki would work if we talk about maintaining Joomla content or so. For other purposes...

[balkce]

Somebody mentioned Slack in another issue. As I understand it, that platform also has some archiving functionalities. Could that work for this purpose? If not, the options are getting slimmer.

[tkelestemur]

@kyordhel (sorry about the email, didn't know) I agree that GitHub wiki is simple and lack of some features but I think this is making it more easy-to-edit and easy-to-navigate/read. If these features are must, then a Content Management System is needed. I would recommend either Wordpress or Jekyll (can be hosted on github for free).

But I still think GitHub wiki would be enough for us to host everything related to teams.

@balkce Slack is mainly a instant messaging platform. Although you can integrate other web services, I don't think it is flexible enough to be a wiki site.

[LoyVanBeek]

@kyordhel do we have any auto-generated content on the 'old' wiki?

@balkce Slack is mainly chat.

[nickswalker]

Another voice for GitHub wikis. The experience is better because Markdown is easier to use than Mediawiki, and tying to an existing account makes the barrier to contribution quite low.

I do agree that the lack of hierarchical organization is frustrating. We have gotten around this by having what are essentially table of contents pages:

Nettlesome because they have to be updated manually, but they work well for us because we don't add new pages often. The alternative is to make users dig through the all-encompassing pages navigation on the right, or find the search, which isn't obvious to most.

[LoyVanBeek]

The current wiki is 12 pages only, with only the Teams page being quite long. This one page warrants a split-up, but then still it's a small wiki.

[timn]

We have already reached out to Luca to move the wiki elsewhere a while ago, at the time thinking about the new RoboCup-driven infrastructure. We are just lacking resources to maintain the wiki further and currently it's always me approving new account requests.

We're happy to provide data of the Wiki if this is of any help. We're also happy to setup a forwarding from the old to the new wiki for some time until things have settled. We would like to take down this wiki instance rather sooner than later. Let me know in what way we can help to help with a migration.

[kyordhel]

AFAIK there is nothing automated in the former wiki. Regarding the platform, all I expect is that the result is easy to browse, specially when people requires data like previous participation of team X, rankings and winners of pasta years, publications, videos, etc. I don't care if the content is static or not, but what I do care is: - How teams will keep the information updated - How Teams/OC/TC can quickly access data - How to prevent someone to (unintentionally) destroy/alter verified information It's clear to me that the former wiki didn't work, that setting a Joomla won't be any better (not sure about Wordpress) and that GitHub has its advantages, but I don't see which platform addresses this two points.

[LoyVanBeek]

Some good points/questions @kyordhel, thnx.
On GitHub, optionally anyone with a GitHub account can edit pages, but we can turn this option on/off if we want.
We might create a GitHub team of only the team leaders and give them access specifically for the wiki only.

GitHub wikis are also git repos themselves and going back to earlier revisions and comparing them is entirely possible: https://help.github.com/articles/viewing-a-wiki-s-history-of-changes/

[moriarty]

It looks like the problem with any wiki platform is restricting edit permissions to only some sections, and keeping spammers away (but that's an issue everyone has: ros-infrastructure/roswiki#139) GitHub wikis are git repos, which also means that evil commands which mess with git history are possible... so the Organisation Team for team leaders and then trust them, as @LoyVanBeek suggests sounds the best.

Do we know if git push -f is restricted to only the project owner, or can collaborators also mess with the history? I just made a wiki in a repo and then was able to rebase and push -f removing history...

There is also the GitHub pages option, but that would mean managing more pull requests, but allows for fancier static only sites (not that we use them)... But then it'd be possible to have finer grain control and allow team leaders to also maintain those pull requests. Also, if a new repo is going to be made called wiki, then it's also possible to do both but I don't know how well that UI blends back and forth.

@timn the old data may not be of much help, but it won't hurt to have, could you give me access? I opened a request. It'll probably be easiest just to recreate the 12 pages... because media wiki syntax isn't markdown.

We should completely get rid of the old wiki

👍

[nickswalker]

Force pushes are allowed for all contributors on all of a repo's branches by default. They provide a protected branch feature to turn this off. There does not appear to be a similar feature to protect a wiki. 🙁

[LoyVanBeek]

I've been thinking about github pages with pull requests from teams as well. This also gives us the ability (not obligatory) to do some quality checks, eg. if a team submits info but is missing some stuff, we notice that before it merges.

[tkelestemur]

Or we can have .md files in repo and a index readme that way pull requests can be used.

[moriarty]

Or we can have .md files in repo and a index readme that way pull requests can be used.

That's basically what a GitHub pages site is, just a bunch of md files, but you can run them through a template engine to make them look pretty...
Here is a very good example: https://github.com/ev3dev/ev3dev.github.io which makes this site http://www.ev3dev.org

If you look in at this simple md page: https://github.com/ev3dev/ev3dev.github.io/blob/master/projects/_posts/2017-02-05-ROS-and-wiimote.md
It becomes this: http://www.ev3dev.org/projects/2017/02/05/ROS-and-wiimote/
and is showcased on the the project page here: http://www.ev3dev.org/projects/

That project has some dedicated contributors making that magic work. And is much more than what we discussed is required, but it shows how github pages can grow.

[kyordhel]

I don't like this because it looks like I'm just pointing out cons for everything, but this needs to be spoken. 1. What's the disk quota of the GitHub wiki/pages? It must be enough for maintaining there the TDP of all applicants (new and past). 2. Thinking about ease the qualification procedure. Teams upload via a form the info to the server and the TDP and Qualification information is stored there for the OC (and other teams) to see. Later on the OC can check which teams attended to the competition, and add their scores in each test, making the score publicly available in real time. 3. Good history of @Home for TC It would be nice that here we can also store all rulebooks, lists of names, objects, categories, arena layouts, and so on (I think I have some in my HDD somewhere). This will make easier for future TC to move the difficulty bar for teams over the year, as they will be able to see the arena layouts, lists of names, objects, categories, and locations, as well as given commands and solved tasks from 2017. 4. Security I would grant some permissions to edit and contribute to veteran teams, something that newbies would require to earn (kinda pull request). This has more to do with the expertise in @Home (what is relevant and what is not) and English proficiency. To me, this stinks like a custom system, but maybe any of you know of a platform/CMS that can handle it. I think that restricting the new "wiki" to just a list of TDP and names will lead us to the same situation we have now.

[LoyVanBeek]

It's good to be critical.

1. Size limits: 1GB repo size, 100MB file size. I don't know the average TDP PDF size though for comparison.
2. Instead of a form, submit a PR before some deadline? It's a bit of a hack, but it can work. Running a server for this alone is a PITA as well I suppose. I don't run it so I can't tell now.

[tkelestemur]

@moriarty I proposed using a static website generator (turns md files to html) like Jekyll in earlier comments. I am using Jekyll for my personal website and I'm pretty happy. There are plenty of free elegant themes. Only downside is that people need to fork, locally edit, push their github account, and open pull request for changes. That's why I said using github wiki or stored md files are easier.

[moriarty]

@tkelestemur 👍 yes I saw your earlier proposal and agree it's good, but the GitHub pages approach just simplifies a few of the steps. I would go Jekyll before going with a CMS system.

@LoyVanBeek a random sample shows they're ~4Mb each.

@kyordhel I agree with @LoyVanBeek on the PITA and don't think this warrants a custom CMS solution- look at the usage of the old wiki...

But the original thread has gotten way off topic: what I meant by stronger linking was: just plastering something like:

THIS WIKI IS PROBABLY OUT OF DATE:

  1. If you're looking for rules, go check GitHub.
  2. If you're looking for software: check Google, ROS wikis and Google Scholar, get creative
  3. The only remaining unsolved problem is the Team list. which a simple github wiki or github pages could sort out.

And just to pass on the info, GitHub customer service did reply and unfortunately the permissions on the wiki is "all or nothing"

Hey Alex,

I'm afraid there isn't a way to restrict permissions for a wiki for certain pages or sections.
It's pretty much an all or nothing type thing but I can certainly pass your request onto the team to consider.
What I can say is that it's normally possible to restore any version of wiki if someone deletes a page or force pushes to it.
The wiki is a git repository so each change is marked with a commit SHA and can be reverted back to previous commits.
I know this isn't exactly what you're looking for but hopefully its helpful somewhat!

Cheers

[LoyVanBeek]

So, we seem to have (presented here) some options:

GitHub wiki

Pro:

1. Quickly editable by all team members, low barrier of contribution
2. It's a git repo and thus locally editable
3. 1 basic OK-ish theme

Con:

1. Editable by anyone, easy to mess up
2. Some size limit (but reasonable)

GitHub repo with plain markdown pages

Pro:

1. Content is modified by pull requests, so reviewed by us
2. Anyone can get GitHub/e-mail notifications on changes
3. Fine-grained access control
4. Can be transitioned into GitHub pages with Jekyll rendered HTML in future

Con:

1. Requires review work
2. Going through pull requests makes updating and previewing not as fast as a wiki
3. No pretty themes
4. Some size limit (but reasonable)

GitHub pages with Jekyll rendered HTML

Pro:

1. Content is modified by pull requests, so reviewed by us
2. Anyone can get GitHub/e-mail notifications on changes
3. Fine-grained access control
4. Pretty themes available

Con:

1. Requires review work
2. Going through pull requests makes updating and previewing not as fast as a wiki
3. Some size limit (but reasonable)

Custom CMS

Pro:

1. Forms etc are possible
2. Generated content
3. Under our full control

Con:

1. Someone needs to manage it

Current wiki

Pro:

1. Everyone knows it already (fixed by a redirect or notice)
2. Content is already there (but with 12 pages easily transferred)

Cons:

1. Need someone to manage it
2. Outdated

[tkelestemur]

On this note, what if we have 1-2 volunteer who will collect TDPs, videos, etc.. from teams and only these maintainers will be editing content of the wiki. It might help slow editing/pull request problem. I assume that it will be faster to keep the wiki updated.

[LoyVanBeek]

Collecting and reviewing TDPs is the OC's responsibility, but with 31 qualified teams (not to mention the amount of applications), editing the wiki/pages for all of these by 1-2 volunteers is quite the job. Reviewing content made by teams itself is much easier and more distributed.

As for the GitHub wiki's: does anyone know if 'people messing up' is a common problem witht hem? I've never seen a GitHub wiki get borked by external folks.

[LoyVanBeek]

So, what's it gonna be, boys, yes or no?

[balkce]

If I understand this correctly, the options boil down to:

• Needing someone to review everything that is posted (which can be 1 or 2 people).
• Needing someone to manage a site (which requires more people).
• Letting the team members do everything, running the risk of a mess up (which requires no people but is risky).

I say we go for the first option. It does not sound like fun if teams start messing with each other's posts.

[kyordhel]

I would say "no go" because I'm not willing to do it and I haven't read someone who strongly commits to the task and maintain the info during this year (at least).

I hope you prove me wrong.

[balkce]

That's always the problem, isn't it?

Good point. Too optimistic of me.

Then I guess the third option would be only option. Lets hope that the teams don't eat themselves.

[LoyVanBeek]

So, GitHub wiki's it is then? I've never seen this go wrong to be honest. I suspect people will not mess up: this didn't happen in the current wiki and since edits are linked to GitHub accounts, I don't expect it with this as well.

We need to do this though: https://help.github.com/articles/changing-access-permissions-for-wikis/

[moriarty]

Yes, and @LoyVanBeek that's a nice link, I went into the docs a bit more and I think with:

• https://help.github.com/articles/adding-outside-collaborators-to-repositories-in-your-organization/
• https://help.github.com/articles/repository-permission-levels-for-an-organization/

We do gain some access control, outside collaborators can be given read-only access to a repository, and this gives them write access to the wiki.

[lmmartine]

The Github wiki is a good option, but I think mostly of the teams will not have the concern of adding and updating this information. Unless this is required for the application of RoboCup.
I want to offer me to recollecting the data of the teams and manage to do a first approach.

[LoyVanBeek]

Thank you @lmmartine. Teams used to be required to enter some info into the wiki wnd we can require this again with a lower barrier for it.
I'll switch this on in the coming week.

[LoyVanBeek]

Allright, there we go: https://github.com/RoboCupAtHome/AtHomeCommunityWiki

I added @lmmartine as a first collaborator with Read acces to the code which gives you edit rights to the wiki.

I copied the pages of ye olde wiki 1-on-1 with some edits to make some things current but not everything. The software page is very very out of date...

[timn]

Can I disable the old wiki and forward to the new one already?

[LoyVanBeek]

For me it's fine, but lets wait a couple days for others to check and see.

[swachsmu]

Looks fine to me. Thanks to Loy for adding this content. It is a good starting point. Best regards Sven

[LoyVanBeek]

Hi @timn Disabling the old wiki and forwarding to the new at https://github.com/RoboCupAtHome/AtHomeCommunityWiki is OK. There already have been some small edits to the new one.

[timn]

I have setup the redirect.

[LoyVanBeek]

@timn Awesome, thnx for helping us out over the past years!

[moriarty]

This looks like it can be closed, except the link on:
http://wiki.robocup.org/@Home_League

Points to:
http://robocup.rwth-aachen.de/athomewiki/index.php/Main_Page

Which redirects (thanks @timn) to a "create page index.php" on GitHub, so I created that page with a link to the home page.

The old info is there and an issue is open for users to get access to collaborate RoboCupAtHome/AtHomeCommunityWiki#1

[LoyVanBeek]

@kyordhel @swachsmu Who to contact to make http://wiki.robocup.org/@Home_League point to the GitHub wiki?

[kyordhel]

Not sure if Luca or Dirk have access to that Wiki.

[kyordhel]

Just to close this.

We have moved to github wiki. Yey!
We need volunteers (OC) to update the wiki. Most information is unknown by new members in old teams, but might be available somewhere. In the near future I'll place the policies in the readme.md. So far I've uploaded scores (thanks Luz) and all TDP's I do have.