From cf8c199dd8eeaa855e58dfb4e7092e8f52d87a80 Mon Sep 17 00:00:00 2001 From: ciroye Date: Thu, 9 May 2024 10:56:05 -0500 Subject: [PATCH 1/2] feat: new documentation --- CONTRIBUTING.md | 73 ++++++--------------------------- README.md | 3 +- docs/start.md | 104 ++++++++++++++++-------------------------------- 3 files changed, 49 insertions(+), 131 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4d6aa999a..600d3fe42 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,21 +1,22 @@ # Contributing to dynabench + We want to make contributing to this project as easy and transparent as possible. ## Pull Requests + We actively welcome your pull requests. 1. Fork the repo and create your branch from `main`. 2. If you've added code that should be tested, add tests. 3. If you've changed APIs, update the documentation. -4. Ensure the test suite passes. -5. Make sure your code lints. -6. If you haven't already, complete the Contributor License Agreement ("CLA"). +4. Make sure your code lints. +5. If you haven't already, complete the Contributor License Agreement ("CLA"). ## Coding Style -* In your editor, install the [editorconfig](https://editorconfig.org/) extension which should ensure that you are following the same standards as us. -* Dynabench uses pre-commit hooks to ensure style consistency and prevent common mistakes. Enable it by: +- In your editor, install the [editorconfig](https://editorconfig.org/) extension which should ensure that you are following the same standards as us. +- Dynabench uses pre-commit hooks to ensure style consistency and prevent common mistakes. Enable it by: ```sh pip install pre-commit && pre-commit install @@ -23,12 +24,12 @@ pip install pre-commit && pre-commit install After this pre-commit hooks will be run before every commit. -* Read the [editorconfig](https://github.com/mlcommons/dynabench/blob/main/.editorconfig) file to understand the exact coding style preferences. +- Read the [editorconfig](https://github.com/mlcommons/dynabench/blob/main/.editorconfig) file to understand the exact coding style preferences. -* Ideally, black and isort should be run via pre-commit hooks. -But if for some reason you want to run black and isort separately follow this: +- Ideally, black and isort should be run via pre-commit hooks. + But if for some reason you want to run black and isort separately follow this: -``` bash +```bash pip install black==22.3.0 isort==4.3.21 black ./(mmf|tests|tools)/**/*.py @@ -38,62 +39,14 @@ isort -rc (mmf|tests|tools) Before commit your changes to your branch run the pre-commit hooks and make sure your code pass all the tests. If your code fail a test, fix it and then commit. -__Note:__ If any linter doesn't pass, your pull request is not going to be accepted. - -## Migrations - -We are using [yoyo-migrations](https://ollycope.com/software/yoyo/latest/) tool to do our schema migrations in a systematic manner. -By default, yoyo should run any pending migrations automatically to your database. - -To add or update the database schema, you will have to create a new migration following these steps: - -* `cd api` -* Call `yoyo new ./migrations -m "Message describing your schema change"` -* This will open up an editor with all of the previous dependency migrations already added. -In the `step` call inside the template, you will add two queries, first argument is the query -you actually want to perform and second is the query to rollback your change. -* Add your migration queries, save the file and exit the editor. -* yoyo should create a new migration script for your queries. -* Commit these, create a PR and next time anybody launches there server after the pull, migrations -should get applied automatically. - -An example of adding `api_token` field to table `users` looks like following: - -```py -""" -Add api_token to users -""" - -from yoyo import step - -__depends__ = {} - -steps = [ - step( - "ALTER TABLE users ADD COLUMN api_token VARCHAR(255)", - "ALTER TABLE users DROP COLUMN api_token", - ) -] -``` - -You can read more on yoyo in its [documentation](https://ollycope.com/software/yoyo/latest/). - -> NOTE: Don't do manual CUD queries to database anymore, this can leave yoyo in a weird state - -## Contributor License Agreement ("CLA") -In order to accept your pull request, we need you to submit a CLA. You only need -to do this once to work on any of Facebook's open source projects. - -Complete your CLA here: +**Note:** If any linter doesn't pass, your pull request is not going to be accepted. ## Issues + We use GitHub issues to track public bugs. Please ensure your description is clear and has sufficient instructions to be able to reproduce the issue. -Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe -disclosure of security bugs. In those cases, please go through the process -outlined on that page and do not file a public issue. - ## License + By contributing to dynabench, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree. diff --git a/README.md b/README.md index 1bcc3a4c6..8cdb69ac1 100644 --- a/README.md +++ b/README.md @@ -3,12 +3,11 @@ Dynabench is a research platform for dynamic data collection and benchmarking. Documentation: + 1. [Platform overview](https://github.com/mlcommons/dynabench/blob/main/docs/overview.md) 2. [Developer guide](https://github.com/mlcommons/dynabench/blob/main/docs/start.md) 3. [Contributing guidelines](https://github.com/mlcommons/dynabench/blob/main/CONTRIBUTING.md) 4. [Task owners manual](https://github.com/mlcommons/dynabench/blob/main/docs/owners.md) -5. [Evaluation Server Developer Guide](https://github.com/mlcommons/dynabench/blob/main/docs/evaluation.md) [Possibly Outdated] -6. [Frontend Developer Guide](https://github.com/mlcommons/dynabench/blob/main/docs/frontend.md) [Possibly Outdated] ## License diff --git a/docs/start.md b/docs/start.md index b6bb650c6..c9093241f 100644 --- a/docs/start.md +++ b/docs/start.md @@ -1,96 +1,62 @@ -# Development +## Dynabench Components Overview -Please read our [contributing guidelines](https://github.com/mlcommons/dynabench/blob/main/CONTRIBUTING.md) to understand how to setup your development environment including `pre-commit` hooks. +Dynabench currently consists of three main components: -## Clone the repository +### Old Backend -After forking [facebookresearch/dynabench](https://github.com/mlcommons/dynabench) to your own GitHub account, clone the forked repo using: +This backend is located in the `API` folder. Currently, the main functionality is the login system. To run this backend, follow these steps: -``` -git clone git@github.com:{your_github_username}/dynabench.git -cd dynabench -``` +1. Create a virtual environment (conda or venv can be used) (`python3 -m venv env`). +2. Activate the environment (`source env/bin/activate`). +3. Install the libraries from the `requirements` folder (`python3 -m pip install -r requirements.txt`). +4. Run the command: `python3 server.py dev`. -## Backend +**Note:** Remember to set the environment variables. -The backend is written in Python and requires a locally installed MySQL database. +### New Backend -### Prerequisites +The new backend is located in the `backend` folder. This is the core of the application, following a hexagonal architecture. It is composed of three main folders inside the `app`: -We recommend using Conda to create an environment for the backend and easily managing the dependencies. The following will install all of the dependencies required by the backend server: +- **API:** Endpoints reside in this folder. No business logic should be implemented in these endpoints. +- **Domain:** This folder contains all the logic of the functions, both programming and business. +- **Infrastructure:** Here, calls to the database are made using SQLAlchemy. -``` -conda create -n dev python=3.7 -conda activate dev -pip install -r requirements.txt -``` +To install the new backend, follow the same steps as for the API, with the last command being: -Next, follow these [instructions to install MySQL](database.md). +`python3 -m uvicorn app.main:app --reload` -### Configuration +### Frontend -Set up your SSL certificates, e.g.: +The main frontend is located in the `frontends/web` folder. Inside this folder, you will find a folder called `src/new_front`. About 99% of new changes should be made inside this folder. -``` -mkdir ~/.ssl -cd ~/.ssl -openssl req -newkey rsa:2048 -x509 -new -nodes -keyout local-cert.key -out local-cert.crt -subj /CN=test1 -sha256 -days 365 -addext "extendedKeyUsage = serverAuth" -cat local-cert.key local-cert.crt > local-cert.pem -chmod 600 * -ln -s local-cert.key dynabench.org-key.pem -ln -s local-cert.crt dynabench.org.crt -``` +To install the frontend, follow these steps: -### Setting up the API server +1. Ensure that Node.js and npm are installed. +2. Run `npm i` (you may need to run it with the `--legacy-peer-deps` flag). +3. Run `npm run start`. -Run the installation script to create your configuration files and ensure all outstanding database migrations are marked as completed: +Remember to configure the environment variables. -``` -cd api -python install.py -``` +## DB -The script will ask you a list of questions to fill in the config file (in `api/common/config.py`). The answers will look like as follows: -``` -Please enter your db_name: dynabench -Please enter your db_user: dynabench -Please enter your db_password: {use the password you set in MySQL install instructions} -Please enter your ssl_cert_file_path: {home directory path}/.ssl/dynabench.org.crt -Please enter your ssl_org_pem_file_path: {home directory path}/.ssl/dynabench.org-key.pem -``` +To migrate the database you can use the following link that comes with some artifacts: -### Running the API server +https://models-dynalab.s3.eu-west-3.amazonaws.com/sql/db.sql -Run the server: +## Types of Challenges in Dynabench -``` -cd api -python server.py dev -``` +Dynabench currently features three different types of challenges: -Your API backend should now be running at https://localhost:8081. If you just generated a local and unverified certificate, you may need to tell your browser it's okay to proceed. +### Creation of Examples -## Frontend +Challenges falling under this category include PRISM, Wonders, Help-me, and Nibbler. To host the various interface types from the frontend, we utilize the strategy design pattern. Modifying components not adjustable from the YAML file requires specific actions: -To install and run the frontend, we recommend using [nvm](https://github.com/creationix/nvm) (see [here](https://github.com/nvm-sh/nvm#installing-and-updating) for installation instructions) to manage and install node versions. +1. **Identify the YAML File:** Begin by accessing the YAML of the respective task. -``` -cd frontends/web/ -nvm install node -nvm install-latest-npm -npm install -echo 'REACT_APP_API_HOST = "https://localhost:8081"' >> .env -npm start -``` +2. **Check Context Type:** Determine the type of context. -If you get a warning about SSL certificates, edit the corresponding paths in `package.json`. Your frontend should now be running at https://localhost:3000. +3. **Locate Equivalence in Interface Options:** Navigate to `frontends/web/src/new_front/utils/creation_interface_options.json` and find the equivalence of the component used. -## Docker Compose -Alternatively, you can run the following two commands to setup both backend and frontend locally: +4. **Identify Component in Codebase:** Proceed to `frontends/web/src/new_front/components/CreateSamples/CreateSamples/AnnotationInterfaces/Contexts` and identify the component. -``` -docker-compose build --no-cache -docker-compose up -``` - -Backend will be running on localhost:8081 and Frontend will be running on localhost:3000. +This process ensures efficient modification of components within the Dynabench platform. From e1c4c5f4f7136f0dc68470ce380ab0d9b10e5c21 Mon Sep 17 00:00:00 2001 From: Sara H Date: Tue, 27 Aug 2024 10:51:35 -0500 Subject: [PATCH 2/2] Add somes specifics to the documentation --- docs/start.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/start.md b/docs/start.md index c9093241f..4e24990a7 100644 --- a/docs/start.md +++ b/docs/start.md @@ -11,7 +11,7 @@ This backend is located in the `API` folder. Currently, the main functionality i 3. Install the libraries from the `requirements` folder (`python3 -m pip install -r requirements.txt`). 4. Run the command: `python3 server.py dev`. -**Note:** Remember to set the environment variables. +**Note:** Remember to set the environment variables to run the backend, there is a an example of this file is called .env.example inside the `API` folder. ### New Backend @@ -25,6 +25,8 @@ To install the new backend, follow the same steps as for the API, with the last `python3 -m uvicorn app.main:app --reload` +**Note:** Remember to set the environment variables to run the backend, there is a an example of this file is called .env.example inside the `backend` folder. + ### Frontend The main frontend is located in the `frontends/web` folder. Inside this folder, you will find a folder called `src/new_front`. About 99% of new changes should be made inside this folder. @@ -35,7 +37,7 @@ To install the frontend, follow these steps: 2. Run `npm i` (you may need to run it with the `--legacy-peer-deps` flag). 3. Run `npm run start`. -Remember to configure the environment variables. +**Note:** Remember to configure the environment variables, there is a an example of this file is called .env.example inside the `frontends/web` folder. ## DB