Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update issue templates #5

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]

**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]

**Additional context**
Add any other context about the problem here.
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
69 changes: 69 additions & 0 deletions .github/ISSUE_TEMPLATE/retrospective.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
name: Retrospective
about: Template to submit a retrospective
title: "[SUBMISSION] Title of the paper"
labels: 'retrospective'
assignees: ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest adding the keys paper_name and paper_url in the template itself, so that we can render them in Liquid tags for the Published Retrospectives Page.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The key tags can also be added which would be a comma separated tag, which will be used later for search

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit confusing, but this section is not keys for Jekyll, they are keys for github's prepolutation of issue's fields. In any case, we should probably not use the issues for retrospectives as discussed in the email thread. I suggest we find some consensus on the email thread and then close this PR is we decided not to go with issues to submit retrospectives. In this case we could also drop the bug and feature templates.


---

## Retrospective for: [*"A Quick Template for Writing Retrospectives"*](https://ml-retrospectives.github.io/retrospectives/accepted_retrospectives/2099/retro-template/)
Paper written by: Alfred Einstein, Mary Curry.

Retrospective written by: Alfred Einstein.


### Paper TL;DR

Give a short description of the original paper. What claims did you make? What were the main results?


### Overall Outlook

Looking back on your paper, what are your thoughts about it now? What's been the impact of the paper in the community (if any)? Give an overview of the things you've learned about your paper since it's been published.


### Opportunities for Improvement

As described on the [how it works](https://ml-retrospectives.github.io/retrospectives/how/) page, there are many things you can talk about in a retrospective. We list three of them below, but you can write about all or none of these.


1. **Flaws or mistakes in the paper’s methodology.**
For example, you discovered a bias in your dataset, or there was accidentally some overlap between validation and test, or there’s a weird bug that you don’t know the cause of and didn’t really figure out. Maybe you didn’t run enough seeds, and when you run more seeds your method isn’t actually that much better. *Basically, any way in which your scientific methodology was flawed or lacking.*

2. **Limitations in the applicability of the work.**
For example, you have reason to believe your approach doesn’t generalize well to other datasets, or your results are not robust to small changes in the experimental set-up. Maybe you tried switching from MLPs to LSTMs and everything broke. Maybe your classification model works better, but is less robust to adversarial attacks. *Basically, any way in which your approach was limited that you didn’t realize or acknowledge at the time.*

3. **Changes in understanding or intuition.**
For example, you have a simpler way to motivate your idea, or your intuition for why your model or approach works has changed. Maybe there are new aspects of your idea that you didn’t explore in the paper, but want to talk about, or new visualizations that improve understanding. These can be accompanied by additional (informal) results. You can also point to new work by others that is relevant to the paper. *Basically, anything that you’ve learned since writing the original paper that would be useful for others to know.*"

This *Opportunities for Improvement* section is a way to combine your thoughts about points 1) and 2). What could have been improved about your paper's methodology? What are some limitations of your work that you've learned since publishing? The more honest you can be about your thoughts, the more the community will benefit.


#### Subsection

This is a subsection where you could describe one of the opportunities for improvement. It's nice to separate each opportunity for improvement by a different subsection, since this makes your retrospective more readable.

For example, I could have written some code like this, which would have made my approach more reproducible:

~~~~
for thing in range(list_of_things):
if thing > other_thing:
print("I'm inside a print statement!")
~~~~


### New Perspectives

This is a section where you can talk about new perspectives that you have on your paper. What have you learned that would be helpful for other people to know? Is your paper's approach worth pursuing in more detail, or was it more of an interesting curiosity? What are interesting directions for future work? Maybe some new papers have come out that have changed your view about what's important about your paper.

If you want to be brief, you might decide to share your perspectives in bullet-point form.

* These are some bullet points.
* They are created using the * or - symbols.
* You can also make nested lists using tabs.


### Other sections

You can write other sections too, if you like! We don't really have anything else to say here. You can write *italics*, **bold**, and ***bold italics*** in Markdown just by using a different number of asterisks. If you want more quick tips on using Markdown, see the page [here](https://en.support.wordpress.com/markdown-quick-reference/).