From 3e16f6a73a5db9d5d470354735d869a6a1997e38 Mon Sep 17 00:00:00 2001 From: Sean Lip Date: Mon, 6 Jan 2025 11:23:49 +0800 Subject: [PATCH 1/3] Update wiki --- If-CI-checks-fail-on-your-PR.md | 13 +++++---- Installing-Oppia-(Linux;-Python-3).md | 6 ++-- Troubleshooting.md | 40 +++++++++++++++++++++++++-- 3 files changed, 49 insertions(+), 10 deletions(-) diff --git a/If-CI-checks-fail-on-your-PR.md b/If-CI-checks-fail-on-your-PR.md index 5413229c..fca3ef1f 100644 --- a/If-CI-checks-fail-on-your-PR.md +++ b/If-CI-checks-fail-on-your-PR.md @@ -1,10 +1,9 @@ ## Table of contents * [Introduction](#introduction) -* [Interpreting the CI checks display](#interpreting-the-ci-checks-display) * [Figuring out whether the failure is due to your PR or an existing issue](#figuring-out-whether-the-failure-is-due-to-your-pr-or-an-existing-issue) -* [What to do if the failure is due to your PR](#what-to-do-if-the-failure-is-due-to-your-pr) -* [What to do if the failure is due to an existing issue](#what-to-do-if-the-failure-is-due-to-an-existing-issue) + * [What to do if the failure is due to your PR](#what-to-do-if-the-failure-is-due-to-your-pr) + * [What to do if the failure is due to an existing issue](#what-to-do-if-the-failure-is-due-to-an-existing-issue) * [What to do if CI checks only fail in the merge queue](#what-to-do-if-ci-checks-only-fail-in-the-merge-queue) ## Introduction @@ -40,7 +39,7 @@ Here are some tips for how to determine whether the failing CI check is due to y * If the error isn't related to your PR and there isn't an existing issue for it, you might need to file a new issue for the failure. Before doing this, double-check your PR changes and the error logs to ensure that nothing in your PR could be causing the failure. -## What to do if the failure is due to your PR +### What to do if the failure is due to your PR To debug and fix the failure, always start from the errors in the CI logs. They contain useful information that can help you with debugging. You can also do the following to get more information about what is causing the error: @@ -63,13 +62,13 @@ If your code is wrong, you will need to fix the error just as you would [respond If you are stuck, compile all your findings in a [[debugging doc|Debugging-Docs]] and open a [GitHub Discussion](https://github.com/oppia/oppia/discussions/categories/q-a-debugging-docs) with a link to it. This will make it easier for the Oppia maintainers and other community members to give you suggestions on what to investigate next. -## What to do if the failure is due to an existing issue +### What to do if the failure is due to an existing issue 1. **Document the error.** * If the error message you see matches a known issue on the [issue tracker](https://github.com/oppia/oppia/issues), leave a comment on that known issue that points to the failing check on your PR, to document that it is still happening. This will help the dev workflow team recognize that this issue is serious and increase its priority. - * If the error message you see doesn't match a known issue, but you have confirmed that it's not due to your changes, please file a [CI Flake report](https://github.com/oppia/oppia/issues/new?assignees=&labels=triage+needed%2Cbug&projects=&template=3_ci_error_template.yml&title=%5BFlake%5D%3A+) that documents the error. Additionally, if you can [[trace which PR caused the error|How-to-find-the-commit-which-introduced-a-bug]], please link to it as well so that the Oppia maintainers can [revert it](https://github.com/oppia/oppia/wiki/Revert-and-Regression-Policy) if needed. + * If the error message you see doesn't match a known issue, but you have confirmed that it's not due to your changes, please file a [CI Failure report](https://github.com/oppia/oppia/issues/new?assignees=&labels=triage+needed%2Cbug&projects=&template=3_ci_error_template.yml&title=%5BFlake%5D%3A+) that documents the error. Additionally, if you can [[trace which PR caused the error|How-to-find-the-commit-which-introduced-a-bug]], please link to it as well so that the Oppia maintainers can [revert it](https://github.com/oppia/oppia/wiki/Revert-and-Regression-Policy) if needed. 2. **Restart the failing test on your PR.** @@ -89,6 +88,8 @@ If you are stuck, compile all your findings in a [[debugging doc|Debugging-Docs] * If you don't hear back within 24 hours, please use [GitHub Discussions](https://github.com/oppia/oppia/discussions/categories/q-a-contacting-folks) to request a follow-up. + * If the CI failure persists after 1 restart and continues to block your PR from being merged, please escalate the issue to the dev workflow team on both the issue thread and GitHub Discussions, since this is an indicator that it might start affecting other PRs. + ## What to do if CI checks only fail in the merge queue diff --git a/Installing-Oppia-(Linux;-Python-3).md b/Installing-Oppia-(Linux;-Python-3).md index d8ab8643..0b4c3341 100644 --- a/Installing-Oppia-(Linux;-Python-3).md +++ b/Installing-Oppia-(Linux;-Python-3).md @@ -23,10 +23,10 @@ Oppia relies on a number of programs and third-party libraries. Many of these li sudo apt-get update ``` -2. Make sure you have curl (used to download third-party libraries), setuptools (needed for installing coverage, which checks test coverage for the Python code), git (which allows you to store the source in version control), python-dev (which is used for the numpy installation), python-pip (which is also used for the numpy installation), pyyaml (which is used to parse YAML files) and libbz2-dev (used by Apache Beam): +2. Make sure you have curl (used to download third-party libraries), git (which allows you to store the source in version control), unzip (used for unpacking zip files) and openjdk-11-jre (needed for GCP to work): ``` -sudo apt-get install python3-pip python3-setuptools curl openjdk-8-jre git python3-dev python3-yaml python3-matplotlib unzip libbz2-dev +sudo apt-get install curl git unzip openjdk-11-jre ``` Alternatively, if you are on Debian/Ubuntu, you can use the `install_prerequisites.sh` script to install these. From the oppia directory: @@ -37,6 +37,8 @@ bash scripts/install_prerequisites.sh 3. Install Chrome from [Google's website](https://www.google.com/chrome). You'll need this to run tests. +4. Check that Java version 11+ is installed. You can use the command `java -version` to do this. If you need to change the default binary for the `java` command, use `sudo update-alternatives --config java`. + ## Clone Oppia 1. Create a new, empty folder that will hold your Oppia work. Here, we call the folder `opensource`. diff --git a/Troubleshooting.md b/Troubleshooting.md index 3acbf732..38dd8ba7 100644 --- a/Troubleshooting.md +++ b/Troubleshooting.md @@ -34,7 +34,6 @@ Here are some general troubleshooting tips for Oppia. The platform specific tips - [No Module appengine.api](#no-module-appengineapi) - [ModuleNotFoundError: No module named \_bz2](#modulenotfounderror-no-module-named-_bz2) - [Subprocess.CalledProcessError: Command 'yarn install --pure-lockfile' returned non-zero exit status 1](#subprocesscalledprocesserror-command-yarn-install---pure-lockfile-returned-non-zero-exit-status-1) - - [Mac OS](#mac-os) - [Error: alert\_on\_exit() -\> Iterator\[None\]](#error-alert_on_exit---iteratornone) - [Local datastore data are not being deleted](#local-datastore-data-are-not-being-deleted) @@ -49,6 +48,7 @@ Here are some general troubleshooting tips for Oppia. The platform specific tips - [SSL Verification Issues](#ssl-verification-issues) - [Yarn: ESOCKETTIMEDOUT](#yarn-esockettimedout) - [ERROR: Cannot uninstall {Package-Name-and-version}, RECORD file not found](#error-cannot-uninstall-package-name-and-version-record-file-not-found) + - [Failed to build typed-ast](#failed-to-build-typed-ast) - [Windows](#windows) - [Windows Firewall](#windows-firewall) - [No Such File or Directory /dev/disk/by-id](#no-such-file-or-directory-devdiskby-id) @@ -111,7 +111,7 @@ Then run `pip install future`. ### No Java -If running `which java` on the terminal does not return any output, you do not have java installed. You can install it by running `sudo apt install openjdk-7-jre-headless`. Note that this command might vary for your local machine. +If running `which java` on the terminal does not return any output, you do not have java installed. You can install it by running `sudo apt install openjdk-11-jre`. Note that this command might vary for your local machine. ### Frontend Tests Stuck @@ -309,6 +309,8 @@ Traceback (most recent call last) : Try searching for where protoc is installed (probably in `/opt/homebrew/bin/protoc`) and remove it and then re-run the command. + + ## Linux ### OSError: [Errno 2] No such file or directory @@ -429,6 +431,8 @@ npm install yarn ``` > Note :- If the above solution doesn't work, please go through the following answered discussions and try the solution mentioned there. [oppia/oppia#17425](https://github.com/oppia/oppia/discussions/17425), [oppia/oppia#18428](https://github.com/oppia/oppia/discussions/18428), [oppia/oppia#14814](https://github.com/oppia/oppia/discussions/14814) + + ## Mac OS ### Error: alert_on_exit() -> Iterator[None] @@ -656,6 +660,38 @@ After removing `.direnv`, run `direnv allow` in your active directory (ensure it Next, execute `python -m scripts.start` inside your Oppia directory before running other scripts like `python -m scripts.run_frontend_tests`. The `python -m scripts.start` command automatically reinstalls all necessary python packages required for Oppia's development environment. Running other scripts before this step might result in "Module not found" errors, requiring manual reinstallation of each package using the `pip` command. +### Failed to build typed-ast + +If you see the error `Failed to build typed-ast` with an error along the following lines: + +``` + clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -DOPENSSL_NO_SSL3 -I/opt/homebrew/opt/node@22/include -Iast3/Include -I/Users/hemantayuj/Desktop/open_source/.direnv/python-3.9/include -I/Users/hemantayuj/.pyenv/versions/3.9.20/include/python3.9 -c ast3/Python/ast.c -o build/temp.macosx-14.5-arm64-cpython-39/ast3/Python/ast.o + ast3/Python/ast.c:844:5: warning: code will never be executed [-Wunreachable-code] + abort(); + ^~~~~ + ast3/Python/ast.c:4514:9: error: call to undeclared function '_PyUnicode_DecodeUnicodeEscape'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] + v = _PyUnicode_DecodeUnicodeEscape(s, len, NULL, &first_invalid_escape); + ^ + ast3/Python/ast.c:4514:9: note: did you mean 'PyUnicode_DecodeUnicodeEscape'? + /Users/hemantayuj/.pyenv/versions/3.9.20/include/python3.9/unicodeobject.h:588:23: note: 'PyUnicode_DecodeUnicodeEscape' declared here + PyAPI_FUNC(PyObject*) PyUnicode_DecodeUnicodeEscape( + ^ + ast3/Python/ast.c:4514:7: error: incompatible integer to pointer conversion assigning to 'PyObject *' (aka 'struct _object *') from 'int' [-Wint-conversion] + v = _PyUnicode_DecodeUnicodeEscape(s, len, NULL, &first_invalid_escape); + ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + 1 warning and 2 errors generated. + error: command '/usr/bin/clang' failed with exit code 1 + [end of output] +``` + +This is due to mypy having some dependencies that don't build properly on Mac (see +discussion [here](https://github.com/oppia/oppia/issues/21617)). To fix this +temporarily, remove mypy from requirements_dev.in and try starting your server +again. This will mean that the backend type checks won't work properly (at least +until the issue above is resolved), but it will allow you to at least start the +dev server locally. + + ## Windows ### Windows Firewall From 0fac3e2f3ebc90ce73ff445a7129006d7bdc0632 Mon Sep 17 00:00:00 2001 From: Sean Lip Date: Tue, 7 Jan 2025 22:51:39 +0800 Subject: [PATCH 2/3] Address review comment. --- Troubleshooting.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Troubleshooting.md b/Troubleshooting.md index 38dd8ba7..28dd8791 100644 --- a/Troubleshooting.md +++ b/Troubleshooting.md @@ -113,6 +113,10 @@ Then run `pip install future`. If running `which java` on the terminal does not return any output, you do not have java installed. You can install it by running `sudo apt install openjdk-11-jre`. Note that this command might vary for your local machine. +You can confirm your version of Java using the command `java -version`. It should be 11 or higher. + +If you need to change the default binary for the `java` command, use `sudo update-alternatives --config java`. + ### Frontend Tests Stuck If you're unable to run frontend tests while pushing your changes due to the script getting stuck, please go to "node_modules" directory (located at the same level as that of the root directory) and delete the "webdriverio" directory present inside that folder. From 510dd382e108d661301c338199e508eb6146251e Mon Sep 17 00:00:00 2001 From: Sean Lip Date: Thu, 9 Jan 2025 01:34:09 +0800 Subject: [PATCH 3/3] Addres comments. --- Installing-Oppia-(Linux;-Python-3).md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/Installing-Oppia-(Linux;-Python-3).md b/Installing-Oppia-(Linux;-Python-3).md index 0b4c3341..f9ff5c7e 100644 --- a/Installing-Oppia-(Linux;-Python-3).md +++ b/Installing-Oppia-(Linux;-Python-3).md @@ -17,27 +17,24 @@ Oppia relies on a number of programs and third-party libraries. Many of these libraries are downloaded automatically for you when you first run the `start.py` script provided with Oppia (see step 1 in the next section). However, there are some things that you will need to do beforehand: -1. Update your package list to the latest version by running: +1. Make sure you have curl (used to download third-party libraries), git (which allows you to store the source in version control), unzip (used for unpacking zip files) and openjdk-11-jre (needed for GCP to work): -``` -sudo apt-get update -``` - -2. Make sure you have curl (used to download third-party libraries), git (which allows you to store the source in version control), unzip (used for unpacking zip files) and openjdk-11-jre (needed for GCP to work): ``` +sudo apt-get update sudo apt-get install curl git unzip openjdk-11-jre ``` -Alternatively, if you are on Debian/Ubuntu, you can use the `install_prerequisites.sh` script to install these. From the oppia directory: +Alternatively, if you are on Debian/Ubuntu, you can use the `install_prerequisites.sh` script to install these. From the oppia directory, run: ``` bash scripts/install_prerequisites.sh ``` +2. Verify that Java version 11+ is installed. You can use the command `java -version` to do this. (If you need to change the default binary for the `java` command on your machine, use `sudo update-alternatives --config java`.) + 3. Install Chrome from [Google's website](https://www.google.com/chrome). You'll need this to run tests. -4. Check that Java version 11+ is installed. You can use the command `java -version` to do this. If you need to change the default binary for the `java` command, use `sudo update-alternatives --config java`. ## Clone Oppia