Merge branch 'develop' into feature/readd-bounty-#185

.dockerignore

@@ -0,0 +1,3 @@
+docker-compose.yml
+Dockerfile
+Jenkinsfile

.gitignore

@@ -23,4 +23,7 @@ profiles.clj
 .idea
 resources/contracts
 node_modules
-.DS_Store
+/config-prod.edn
+/config-dev.edn
+/config-test.edn
+/src/java

CONTRIBUTING.md

@@ -0,0 +1,28 @@
+This document describes process guidelines to be followed when contributing to Status Open Bounty repo.
+
+First, make sure to familiarize yourself with the [README](https://github.com/status-im/open-bounty/blob/develop/README.md) and [Testing](https://github.com/status-im/open-bounty/blob/develop/doc/testing.md) documents in order to setup the project properly.
+
+# Issues
+  - Issues should have type, priority and size (difficulty) assigned via corresponding labels
+  - Issue descriptions should include the following fields:
+    - **Summary**
+    - **Type**
+    - (*Features or enhancements only*) **User story**
+    - (*Bugs only*) **Expected behavior**
+    - (*Bugs only*) **Actual behavior**
+    - **Additional information**
+
+# Pull requests
+  - Branch names should include:
+    - prefixes indicating issue type (`bug`, `feature`, `doc`, `test`)
+    - short description in lisp-case
+    - and include associated issue number
+
+    For instance, `bug/messy-problem-#1234`
+  - Start the title of the PR with [FIX #NNN], where #NNN is the issue number
+  - Always include `Status:` in the PR description to indicate whether PR is `WIP` or `Finished`.
+  - PR description should include the following sections:
+    - **Summary**
+    - **Notes**
+    - **Status**
+  - Merges into `develop` branch should be approved by at least 1 person, into `master` - by 2.

Dockerfile

@@ -0,0 +1,30 @@
+FROM clojure as builder
+
+WORKDIR /tmp
+
+RUN wget -O /usr/local/bin/solc https://github.com/ethereum/solidity/releases/download/v0.4.15/solc-static-linux
+RUN chmod +x /usr/local/bin/solc
+
+RUN wget https://github.com/web3j/web3j/releases/download/v2.3.0/web3j-2.3.0.tar
+RUN tar -xf web3j-2.3.0.tar
+RUN cp -r  web3j-2.3.0/* /usr/local/
+
+
+COPY . /usr/src/app
+WORKDIR /usr/src/app
+
+ENV LEIN_SNAPSHOTS_IN_RELEASE=1
+
+
+RUN lein less once
+RUN lein uberjar
+
+
+FROM clojure
+WORKDIR /root/
+
+COPY --from=builder /usr/src/app/target/uberjar/commiteth.jar .
+
+CMD [""]
+ENTRYPOINT ["/usr/bin/java", "-Duser.timezone=UTC", "-Dconf=config-test.edn", "-jar", "/root/commiteth.jar"]
+

Jenkinsfile

@@ -0,0 +1,35 @@
+#!/usr/bin/env groovy
+
+node('linux') {
+checkout scm
+
+def dockerreponame = "statusim/openbounty-app"
+
+	try {
+		stage('Build & push') {
+
+			GIT_COMMIT_HASH = sh (script: "git rev-parse --short HEAD | tr -d '\n'", returnStdout: true)
+
+			docker.withRegistry('https://index.docker.io/v1/', 'dockerhub-statusvan') {
+				def openbountyApp = docker.build("${dockerreponame}:${env.BUILD_NUMBER}")
+				openbountyApp.push("${env.BRANCH_NAME}")
+		        if (env.BRANCH_NAME == 'develop') {
+		            openbountyApp.push("develop")
+		        } else if (env.BRANCH_NAME == 'master') {
+		            openbountyApp.push("master")
+		        } else {
+		            println "Not named branch have no custom tag"
+		        }
+			}
+
+		}
+
+		stage('Deploy') {
+			build job: 'status-openbounty/openbounty-cluster', parameters: [[$class: 'StringParameterValue', name: 'DEPLOY_ENVIRONMENT', value: "dev"], [$class: 'StringParameterValue', name: 'BRANCH', value: env.BRANCH_NAME]]
+		}
+
+	} catch (e) {
+    // slackSend color: 'bad', message: REPO + ":" + BRANCH_NAME + ' failed to build. ' + env.BUILD_URL
+    throw e
+  }
+}

README.md

@@ -1,44 +1,96 @@
-# Commiteth
+# Status Open Bounty
 
-Allows you to set bounties for Github issues, paid out in Ether.
+Allows you to set bounties for Github issues, paid out in Ether or any ERC-20 token.
 
 More information:
-http://wiki.status.im/proposals/commiteth/
+https://wiki.status.im/Status_Open_Bounty
 
-Live beta version:
+Live production version:
 https://openbounty.status.im
 The `master` branch is automatically deployed here.
 
-
 Live testnet (Ropsten) version:
 https://openbounty.status.im:444
 The `develop` branch is automatically deployed here.
 
 
 ## Prerequisites
 
-You will need [Leiningen](https://github.com/technomancy/leiningen) 2.0 or above installed.
+You will need [Leiningen](https://github.com/technomancy/leiningen) 2.0 or above installed. Also, make sure that you have [wkhtmltoimage](https://wkhtmltopdf.org/downloads.html) available in your PATH. On macOS, it can be installed via `brew cask install wkhtmltopdf`.
 
 ### PostgreSQL
 
 Make sure you install [PostgreSQL](https://www.postgresql.org/) and properly set it up:
 
 ```
-sudo -u postgres psql -c "CREATE USER commiteth WITH PASSWORD 'commiteth';"
-sudo -u postgres createdb commiteth
+psql postgres -c "CREATE USER commiteth WITH PASSWORD 'commiteth';"
+psql postgres -c "CREATE DATABASE commiteth;"
 ```
 
 ### solc
 
 Solidity compiler [0.4.15](https://github.com/ethereum/solidity/releases/tag/v0.4.15) is required and needs to be in $PATH.
+Detailed [installation instructions for various platforms](https://solidity.readthedocs.io/en/develop/installing-solidity.html) can be found in the official Solidity documentation.
+
+```
+brew install https://raw.githubusercontent.com/ethereum/homebrew-ethereum/de1da16f7972a899fc8dd1f3f04299eced6f4312/solidity.rb
+brew pin solidity
+```
 
 ### web3j
 
 Web3j [2.3.0](https://github.com/web3j/web3j/releases/tag/v2.3.0) is required and the command line tools need to be in $PATH.
+Installation instructions for the command line tools can be found in the [Web3j Command Line Tools documentation](https://docs.web3j.io/command_line.html).
+
+```
+brew install https://raw.githubusercontent.com/web3j/homebrew-web3j/881cf369b551a5f2557bd8fb02fa8b7b970256ca/web3j.rb
+brew pin web3j
+```
 
 ## Running
 
-Make sure `env/dev/resources/config.edn` is correctly populated.
+Launch following commands each in its own shell:
+
+```
+lein run
+lein figwheel
+lein less auto
+```
+
+## Application config
+
+Make sure to create `/config-dev.edn` and populate it correctly, which is based on `env/dev/resources/config.edn`. Description of config fields is given below:
+
+Key | Description
+--- | ---
+dev | Currently specifies whether Swagger UI endpoints should be added to routes
+port | HTTP port for the Ring web app
+nrepl-port | nREPL port for development
+jdbc-database-url | PostgreSQL database URL. For instance, URL to local db would be `jdbc:postgresql://localhost/commiteth?user=commiteth&password=commiteth`
+server-address | URL and port of local server that can be resolved from public internet. It will be used as a redirect URI during GitHub OAuth authorization process
+eth-account | Ethereum account ID for the bot
+eth-password | Ethereum account password for the bot
+eth-rpc-url | RPC URL to Ethereum node, e.g. Geth. Either local or remote
+eth-wallet-file | Location of wallet file. If Geth is run with the parameters as given below, it will reside under `$HOME/.ropsten/keystore`
+tokenreg-base-format | Should be set to `:status`
+github-client-id | Related to OAuth. Copied from GitHub account Settings->Developer settings->OAuth Apps
+github-client-secret | Related to OAuth. Copied from GitHub account Settings->Developer settings->OAuth Apps
+github-user | GitHub username for bot account. It is used for posting bounty comments
+github-password | GitHub password for bot account
+webhook-secret | Secret string to be used when creating a GitHub App
+user-whitelist | Set of GitHub user/org IDs to be whitelisted. E.g. `#{"status-im" "your_org"}`
+testnet-token-data | Token data map, useful if there are Geth connectivity problems
+
+## GitHub integration
+Open Bounty uses both OAuth App and GitHub App integration.
+
+### OAuth App
+Follow the steps [here](https://developer.github.com/apps/building-oauth-apps/creating-an-oauth-app/). Specify the value of `:server-address` as "Homepage URL", and `:server-address` + `/callback` as "Authorization callback URL". Be sure to copy Client ID and Client Secret values in the config file.
+
+### GitHub App
+Follow the steps [here](https://developer.github.com/apps/building-github-apps/creating-a-github-app/). Be sure to specify `:server-address` + `/webhook-app` as "Webhook URL", and `:webhook-secret` as "Webhook Secret".
+
+## Running
 
 Lauch a local geth node with the bot account unlocked:
 
@@ -121,6 +173,10 @@ Landing page is static and different CSS and JS due to time constraints.
 
 This copies over necessary artifacts to `resources` dir.
 
+
+### Troubleshooting
+See the [Cookbook](doc/cookbook.md).
+
 ## License
 
 Licensed under the [Affero General Public License v3.0](https://github.com/status-im/commiteth/blob/master/LICENSE.md)

doc/README.md

@@ -0,0 +1,7 @@
+# Table of contents
+
+- [Testing guide](doc/testing.md)
+- [Troubleshooting](doc/cookbook.md)
+- [Deployment flow](doc/deployment_flow.md)
+- [Payout flow](doc/payout_flow.md)
+- [Common sync issues](doc/sync_issues.md)

doc/cookbook.md

@@ -0,0 +1,18 @@
+# Cookbook
+
+Here some common tasks/issues are listed along with their solutions.
+
+## Change config and restart the service
+  - ssh to the host running commiteth
+  - go to `/opt/commiteth` for prod or `/opt/commiteth-test` for test environment.
+  - there edit `config.edn`
+  - restart the service with `sudo service commiteth-test restart`
+
+## Manually add a GitHub repo to SOB app
+Sometimes SOB will not
+  - connect to Postgres instance. Get DB name and username/password from respective `config.edn` (see previous answer). These are set in `:jdbc-database-url` field.
+  - execute the query, e.g.
+```
+insert into repositories(repo_id,user_id,owner,repo,hook_id,state,hook_secret,owner_avatar_url) values(116971984,447328, 'aragon', 'aragon-monthly', 0, 2, '', 'https://avatars1.githubusercontent.com/u/24612534?v=4');
+```
+Note that the value for repo_id field is GitHub's internal repo ID. One way of obtaining it is navigating to `api.github.com/repos/:owner/:repo_name`, e.g. `https://api.github.com/repos/aragon/aragon-monthly`.

doc/deployment_flow.md

@@ -0,0 +1,16 @@
+# Deployment flow
+
+This briefly describes events that occur when an issue is labeled as bounty and a new contract has to be deployed.
+
+1. Issue is labeled
+2. Event is received via GitHub App webhook
+3. Contract is deployed
+4. GitHub issue comment "Deploying contract..." is posted
+5. `transaction_hash` is stored in the `issues` table
+
+The following items execute in scheduler threads that run each minute, so up to 60 sec delay can be expected.
+
+6. `update-issue-contract-address` scheduler thread fetches transaction receipt, updates `contract_address` and updates GitHub comment with a new image and current balance
+7. `deploy-pending-contracts` scheduler thread checks if there are issues that did not have corresponding contracts deployed and attempts to redeploy
+8. `update-balances` scheduler thread checks balances and updates GitHub comment accordingly
+9. `update-contract-internal-balances` scheduler threads updates internal ERC20 token balances for all deployed contracts. This is required by current contract code

doc/payout_flow.md

@@ -0,0 +1,57 @@
+# Payout flow
+
+This describes the sequence of events happening once a PR for an issue with a bounty was merged by repo maintainer.
+
+### Quick info on transaction hashes
+In the sequence described below, several types of hashes are used. SOB checks presence of different hashes on records in the `issues` table to decide which action to take on an issue and associated contract. These hashes are:
+- `transaction_hash`: set when contract is deployed to the labeled issue
+- `execute_hash`: set when PR for an issue with a bounty was merged
+- `confirm_hash`: fetched from receipt from transaction invoked in previous step
+- `payout_hash`: set when payout was confirmed via `Manage Payouts`
+
+The event flow is given below. For the bounty to be paid, each issue has to go through the steps in given order.
+
+### 1. PR closed and merged
+- app receives notification via GitHub App webhook (endpoint: `/webhook-app`)
+- `handle-claim` fn is invoked which will:
+- save PR in the `pull_requests` DB table, where state=1 for merged PRs
+- update issue in the DB with commit SHA, if PR was merged
+
+Afterwards two interleaving sequences of actions come into play: scheduler threads and manual user interaction in the `Manage Payouts` tab.
+
+### 2. `self-sign-bounty` scheduler thread
+- input query name: `pending-bounties`. This selects pending bounties (where `execute_hash` is nil and `commit_sha` is set)
+- execute payout transactions
+- store `execute_hash` and `winner_login` in `issues` DB table
+- update GitHub comment to "Pending maintainer confirmation"
+### 3. `update-confirm-hash` scheduler thread
+- input query name: `pending-payouts`. This selects bounties with `execute_hash` set and no `confirm_hash`
+- fetch `confirm_hash` from transaction receipt
+- store `confirm_hash` in `issues` DB table
+
+### 4. Manage Payouts view
+In order to confirm a payout, following conditions have to be met for an issue:
+- it is merged
+- not paid yet (meaning its `payout_hash` is nil)
+- not being confirmed at the moment (`:confirming?` flag is true)
+  OR
+- already confirmed by a scheduler thread(`confirm_hash` is not nil)
+
+Note that `confirm_hash` issue field and confirmation action in the UI are different things, albeit identically named. In order to confirm a payout from the UI, `confirm_hash` has to be already set by scheduler thread (see above).
+
+Payout confirmation action results in a `:confirm-payout` event. Its handler will
+- use `confirm_hash` to construct transaction payload
+- set `:confirming?` flag to `true`
+- execute `confirmTransaction()` call
+- pass transaction callback to `confirmTransaction()`. Once invoked, the callback will:
+  - get `payout_hash` passed as an argument
+  - dispatch `:save-payout-hash` event. Its handler will:
+    - POST to /api/user/bounty/%s/payout
+      - This will update `payout_hash` in `issues` DB table
+    - if POST is successful, dispatch `:payout-confirmed`
+      - `:payout-confirmed` will update  `:confirmed?` to `true` and remove `:confirming?` flag
+
+### 5. `update-payout-receipt` scheduler thread
+- input query name: `confirmed-payouts`. This selects confirmed payouts (the ones that have `payout_hash` and do not have `payout_receipt` set)
+- store `payout_receipt` in `issues` DB table
+- and update GitHub comment to "Paid to:..."

doc/sync_issues.md

@@ -0,0 +1,11 @@
+# Common sync issues
+
+- Repo rename. Related issue: https://github.com/status-im/open-bounty/issues/219.
+- Transaction callback not executed, hence payout-hash not set.
+- App downtime, multiple GitHub App deliveries failing. These can be afterwards replayed from GitHub App's Advanced tab.
+- Geth issue. Not solvable on app end, Geth restart usually fixes these.
+- Bot out of gas. Relevant issue: https://github.com/status-im/open-bounty/issues/195.
+- Sometimes repos are disabled in DB (state=0), probably only happens to repos that were added earlier through OAuth App.
+- PRs might end up in state=0 (opened) instead of state=1(merged), if PR did not have a proper text ("Fixes #..."). This one needs more investigation and more cases to reproduce.
+- Sometimes contract_address cannot be fetched for a long period of time.
+

doc/testing.md

@@ -10,7 +10,7 @@ For testing you will need:
 * a Github account with administrative access to one or more repositories
 * for approving bounty payouts you will additionally need access to an Ethereum wallet. So far, Mist and [MetaMask](https://metamask.io/) have been used, but anything that provides the web3 javascript interface should work.
 
-The developers can be reached on the `#commiteth` channel in the [Status slack](http://slack.status.im/).
+The developers can be reached on the `#openbounty` channel in the [Status slack](http://slack.status.im/).
 
 ### Signing up
 
@@ -29,13 +29,9 @@ You should now see `Bounties`, `Activity`, `Repositories` and `Manage Payouts` t
 
 ### Creating bounty issues
 
-Before you can create bounties, you need to have administrative access to one or more GitHub repositories. These can be either in the scope of your personal user account or in the scope of a Github orgnazation.
+Before you can create bounties, you need to add Open Bounty GitHub App to your account or repos. Go to https://github.com/apps/status-open-bounty-app-test (or link to another GitHub App you've created for testing, as described in the [README](README.md) and click Install. Specify whether access to all org repos or specific repos is granted. This will install webhooks for SOB in your repos.
 
-* click the `Repositories` tab
-* click on the button `Enable Github Account`
-* If you have 1 or more Organisation repositories then grant Organisation access to each of them by clicking on the button `Grant` 
-* grant Status Open Bounty the needed addtional permissions for managing repository webhooks, adding and modifying comments by clicking on the button `Authorize status-open-bounty`
-* now you should see your repositories on the `Repositories` tab, click `Add` on one. If your account isn't whitelisted you will see instructions how to request an access. If your account is whitelisted then new `bounty` label will become available in the GitHub repository's labels and a new webhook should now exist for the repository.
+* Request for your account to be whitelisted. Contact [Riot](https://chat.status.im) for more information
 * now, add the `bounty` label to a new or an existing issue. This should cause Status Open Bounty to post a new comment for the issue containing an image with text `Deploying contract, please wait`
 * once the contract has been mined, the comment will be updated to contain the bounty contract's address and a QR code
 
@@ -60,6 +56,14 @@ To remove issue from the Bounties list you can close it in GitHub.
 
 ### Reporting bugs
 
-All bugs should be reported as issues in the [CommitETH Github repository](https://github.com/status-im/commiteth/issues).
+All bugs should be reported as issues in the [OpenBounty Github repository](https://github.com/status-im/open-bounty/issues).
 
 Please first check that there is not already a duplicate issue. Issues should contain exact and minimal step-by-step instructions for reproducing the problem.
+
+### Status Open Bounty end-to-end tests
+
+Framework for testing located in: `open-bounty/test/end-to-end`
+
+Full installation and configuring manual: [Status Open Bounty end-to-end tests](https://wiki.status.im/Status_Open_Bounty_end-to-end_tests)
+
+Currently supports local and Jenkins environment running (you can find example of JenkinsFile in `open-bounty/test` )