diff --git a/README.md b/README.md index 1a397f2ebb..190f7b67da 100644 --- a/README.md +++ b/README.md @@ -16,20 +16,25 @@ To report an user issue with the **Scripture Forge** application email "issues @ ## Special Thanks To ## -![BrowserStack Logo](https://raw.githubusercontent.com/sillsdev/web-languageforge/master/readme_images/browserstack-logo.png "BrowserStack") +For end-to-end test automation: -For end-to-end test automation. +![BrowserStack Logo](readme_images/browserstack-logo.png "BrowserStack") + +For error reporting: + +[![Bugsnag logo](readme_images/bugsnag-logo.png "Bugsnag")](https://bugsnag.com/blog/bugsnag-loves-open-source) ## Developers ## We use [Gitflow](http://nvie.com/posts/a-successful-git-branching-model/) with two modifications: -* The Gitflow **master** branch is our **live** branch. -* The Gitflow **develop** branch is our **master** branch. All pull requests go against **master**. +- The Gitflow **master** branch is our **live** branch. +- The Gitflow **develop** branch is our **master** branch. All pull requests go against **master**. We merge from **master** to testing (**qa** branch) then ship from **qa** to **live**. ### Builds ### + Status of builds from our continuous integration (CI) [server](https://build.palaso.org): | Site | Master Unit | Master E2E | QA | Live | @@ -48,12 +53,12 @@ Successful builds from our CI server deploy to: PHP code conforms to [PSR-2](http://www.php-fig.org/psr/psr-2/). - * Add `php-cs-fixer` globally installed with *composer* (http://cs.sensiolabs.org/). Here is how to add it to **PhpStorm** (https://hackernoon.com/how-to-configure-phpstorm-to-use-php-cs-fixer-1844991e521f). Use it with the parameters `fix --verbose "$FileDir$/$FileName$"`. +- Add `php-cs-fixer` globally installed with *composer* (http://cs.sensiolabs.org/). Here is how to add it to **PhpStorm** (https://hackernoon.com/how-to-configure-phpstorm-to-use-php-cs-fixer-1844991e521f). Use it with the parameters `fix --verbose "$FileDir$/$FileName$"`. JavaScript code conforms to [AirBNB JS style guide](https://github.com/airbnb/javascript). - * Using PhpStorm with JSCS helps a lot with automating this (see the section below on PhpStorm [Coding Standard and Style](#CodeStyle)). - * We plan to use [Prettier](https://prettier.io/) with pre-commit hook after re-writing the whole repo with Prettier first. +- Using PhpStorm with JSCS helps a lot with automating this (see the section below on PhpStorm [Coding Standard and Style](#CodeStyle)). +- We plan to use [Prettier](https://prettier.io/) with pre-commit hook after re-writing the whole repo with Prettier first. ## TypeScript Integration ## @@ -68,15 +73,17 @@ To this end you'll also want to be familiar with [Upgrading from AngularJS](http We are expecting that TypeScript will help us get things right from the beginning (catching things even as you type) as well as maintenance. We are expecting that it will be an easier transition for the FLEx team and that they will be able to help us with good typing, interfaces and class design. Other useful resources: - - [x] [angularjs-styleguide/typescript at master · toddmotto/angularjs-styleguide](https://github.com/toddmotto/angularjs-styleguide/tree/master/typescript#stateless-components) - - [x] [AngularJS 1.x with TypeScript (or ES6) Best Practices by Martin McWhorter on CodePen](https://codepen.io/martinmcwhorter/post/angularjs-1-x-with-typescript-or-es6-best-practices) - - [x] [What is best practice to create an AngularJS 1.5 component in Typescript? - Stack Overflow](https://stackoverflow.com/questions/35451652/what-is-best-practice-to-create-an-angularjs-1-5-component-in-typescript) - - [x] [Don't Panic: Using ui-router as a Component Router](http://dontpanic.42.nl/2016/07/using-ui-router-as-component-router.html) - - [x] [Lifecycle hooks in Angular 1.5](https://toddmotto.com/angular-1-5-lifecycle-hooks#onchanges) + +- [x] [angularjs-styleguide/typescript at master · toddmotto/angularjs-styleguide](https://github.com/toddmotto/angularjs-styleguide/tree/master/typescript#stateless-components) +- [x] [AngularJS 1.x with TypeScript (or ES6) Best Practices by Martin McWhorter on CodePen](https://codepen.io/martinmcwhorter/post/angularjs-1-x-with-typescript-or-es6-best-practices) +- [x] [What is best practice to create an AngularJS 1.5 component in Typescript? - Stack Overflow](https://stackoverflow.com/questions/35451652/what-is-best-practice-to-create-an-angularjs-1-5-component-in-typescript) +- [x] [Don't Panic: Using ui-router as a Component Router](http://dontpanic.42.nl/2016/07/using-ui-router-as-component-router.html) +- [x] [Lifecycle hooks in Angular 1.5](https://toddmotto.com/angular-1-5-lifecycle-hooks#onchanges) ## Recommended Development Environment ## -### Automatic Install +### Automatic Install ### + The below sections go into detail on how to manually set up the developer environment. As an alternative to these instructions, you can download and run the automatic install script that is known to work on Ubuntu Xenial Native and Ubuntu Xenial on Windows 10 WSL. [Click here for the Automatic Install instructions](#automatic-install-script). @@ -85,6 +92,7 @@ The below sections go into detail on how to manually set up the developer enviro Our recommended development environment for web development is Linux Ubuntu GNOME. Choose either the [Vagrant VM Setup](#VagrantSetup) or the [Local Linux Development Setup](#LocalSetup). Even though the Vagrant VM Setup is definitely easier because it always installs from a clean slate on a new virtual box, we recommend doing development on your local development machine. This approach will make your page loads approximately 50 times faster. In my tests 100 ms (local) vs 5000 ms (Vagrant / Virtualbox). The reason for this is that Virtualbox gives access to the php files via the VirtualBox shared folder feature. This is notoriously slow. ### Ubuntu Xenial on Windows 10 WSL ### + It is also possible to develop on Windows 10 using the Windows Subsystem for Linux (WSL). This allows you to run a Linux distribution, such as Ubuntu, in Windows. The development environment has only been tested on Windows 10 Creators Update (1703). For the full steps on setting up a development environment in Windows 10, see [Windows 10 Setup](#windows-10-setup). ## Automatic Install Script ## @@ -92,39 +100,45 @@ It is also possible to develop on Windows 10 using the Windows Subsystem for Lin This automatic install script assumes that you are starting with a fresh install of Ubuntu Xenial, either native or on Windows 10 WSL. To begin the automatic installation, cd into a source directory (creating one if necessary). On Windows 10 WSL this might look like this (make sure you are in Bash!): -``` + +``` bash mkdir -p /mnt/c/src cd /mnt/c/src ``` Now, download and run the install script: -``` + +``` bash wget -O- https://raw.githubusercontent.com/sillsdev/web-languageforge/master/installer/ubuntu1604Installer.sh | bash - ``` + Expect the install to take 30-60 minutes on a fresh Ubuntu Xenial install, depending upon your internet connection. -### How to Setup/Uninstall/Reinstall Ubuntu on Windows 10 WSL +### How to Setup/Uninstall/Reinstall Ubuntu on Windows 10 WSL ### If you have never installed Ubuntu on Windows 10, [follow the instructions in this guide](https://msdn.microsoft.com/en-us/commandline/wsl/install_guide) to ensure the pre-requisites are installed - If you have already installed Ubuntu on Windows 10 but want to uninstall and re-install fresh open a Windows Command Prompt (Run as Administrator) and type the following: -``` + +``` bash lxrun /uninstall /full ``` + If this is your first time installing Ubuntu Xenial on Windows 10 (or if you just uninstalled in the previous step) type the following: -``` + +``` bash lxrun /install ``` + After Ubuntu Xenial Bash has finished installing, close the Windows Command Prompt and open a Bash prompt (now available in the start menu) ---------------------------------- +------------------------------- -### Vagrant VM Setup +### Vagrant VM Setup ### Clone this repository to your host machine and `vagrant up` the Xenial box. We intentionally postpone provisioning on initial boot so the `Virtualbox guest additions` updates don't interfere with the provisioning process. -``` +``` bash git clone https://github.com/sillsdev/web-languageforge web-languageforge --recurse-submodules cd deploy/xenial vagrant up --no-provision @@ -134,7 +148,7 @@ Once the shell notifies the Virtualbox guest additions have been updated, power Now, vagrant up with provision to install and deploy -``` +``` bash vagrant up --provision ``` @@ -142,84 +156,91 @@ Proceed to [Language Forge Configuration File](#LFConfig) and follow the rest of ------------------------------- -### Local Linux Development Setup +### Local Linux Development Setup ### Start with the Ansible-assisted setup [described here](https://github.com/sillsdev/ops-devbox) to install and configure a basic development environment. +#### Installation and Deployment #### -#### Installation and Deployment After creating your Ansible-assisted setup, clone this repository from your *home* folder... -```` +```` bash mkdir src cd src mkdir xForge cd xForge git clone https://github.com/sillsdev/web-languageforge web-languageforge --recurse-submodules ```` -The `--recurse-submodules` is used to fetch many of the Ansible roles used by the Ansible playbooks in the deploy folder. If you've already cloned the repo without `--recurse-submodules`, run `git submodule update --init --recursive` to pull and initialize them. +The `--recurse-submodules` is used to fetch many of the Ansible roles used by the Ansible playbooks in the deploy folder. If you've already cloned the repo without `--recurse-submodules`, run `git submodule update --init --recursive` to pull and initialize them. If you want to run an independant repo for scriptureforge, clone its repo also... -``` +``` bash git clone https://github.com/sillsdev/web-scriptureforge web-scriptureforge --recurse-submodules ``` Otherwise just create a symbolic link between languageforge and scriptureforge... -``` +``` bash ln -s web-languageforge web-scriptureforge ``` Change the variable *mongo_path: /var/lib/mongodb* in `deploy/vars_palaso.yml`. Set it to a location where MongoDB should store its databases. - - **Vagrant VM Setup**: uncomment line 6 and comment line 5 - - **Local Linux Development Setup**: uncomment line 5 and comment line 6 (or whatever is appropriate on your system, its best to have Mongo store databases on your HDD rather than SSD). Make sure the `mongodb` user has permission to read and write to the path you specify. + +- **Vagrant VM Setup**: uncomment line 6 and comment line 5 +- **Local Linux Development Setup**: uncomment line 5 and comment line 6 (or whatever is appropriate on your system, its best to have Mongo store databases on your HDD rather than SSD). Make sure the `mongodb` user has permission to read and write to the path you specify. Run the following Ansible playbooks to configure Ansible and run both sites. -```` +```` bash cd web-languageforge/deploy ansible-playbook -i hosts playbook_create_config.yml --limit localhost -K ansible-playbook playbook_xenial.yml --limit localhost -K ```` + If you run into an error on the `ssl_config : LetsEncrypt: Install packages` task, run the playbook again and that task should succeed the second time it is run. Install dependencies used to build Sass files and run E2E tests -``` + +``` bash cd web-languageforge npm install gulp sass gulp webpack-lf ``` + or use `gulp webpack-sf` if you are working in **Scripture Forge**. To watch Sass files for changes, run `gulp sass:watch`. The output will also be in a more readable format (rather than compressed as it is with `gulp sass`). You can also pass the `--debug` flag to enable source comments and source maps in comments in the output files. To watch TypeScript files for changes, run `gulp webpack-lf:watch` or `gulp webpack-sf:watch`. This includes a live reload server to refresh the browser on TypeScript changes (browser setup [here](#LiveReloadInstall)). -#### Language Forge Configuration File +#### Language Forge Configuration File #### + Manually edit the Language Forge config file -``` +``` bash sudo gedit /etc/languageforge/conf/sendreceive.conf ``` and modify PhpSourcePath to -``` +``` bash PhpSourcePath = /var/www/virtual/languageforge.org/htdocs ``` ------------------------------- -### Windows 10 Setup +### Windows 10 Setup ### Before setting up a development environment on Windows 10, it is important to understand a couple of things about how WSL works. + 1. You cannot make changes to the Linux filesystem from Windows, but you can make changes to the Windows filesystem from Linux. In practical terms, this means that any files that you want to be able to modify should be stored in the Windows filesystem. For example, Git repos should be cloned to the Windows filesystem. Windows drives are automatically mounted in Linux under the `/mnt` directory. A good practice is to store all Git repos in a directory on Windows, and then create a symbolic link to the directory in your home directory on Linux. 2. WSL is designed to only work with command-line tools. It cannot run GUI applications. This means that any GUI-based code editors or IDEs will need to be run in Windows. 3. Linux processes only run as long as a Bash shell is open. You can start a Bash shell on Ubuntu by running **Bash on Ubuntu on Windows** from the Start menu or by running `bash` from the command prompt. Once you close all Ubuntu Bash shells, all Linux processes will shutdown gracefully. This means that you will have to start any services that you need when you open an Ubuntu Bash shell. -#### Prerequisites +#### Prerequisites #### + The first step is to install Ubuntu 16.04 on Windows. Follow the steps outlined on this [page](https://msdn.microsoft.com/en-us/commandline/wsl/install_guide). Download and install [Git for Windows](https://git-for-windows.github.io/). If you would like a full-fledged GUI for Git, you can install [Git Extensions](https://gitextensions.github.io/). @@ -228,7 +249,7 @@ After, WSL and Git are installed, you will need to perform an Ansible-assisted s To clone **ops-devbox** with Linux line endings, open **Git Bash** in Windows (not **Bash on Ubuntu on Windows**) and run: -``` +``` bash git clone --recurse-submodules --no-checkout https://github.com/sillsdev/ops-devbox cd ops-devbox git config core.autocrlf input @@ -237,11 +258,11 @@ git checkout Once the repo is cloned, you can install Ansible in Ubuntu and run the ansible scripts as outlined in the [**ops-devbox** README](https://github.com/sillsdev/ops-devbox). -#### Installation and Deployment +#### Installation and Deployment #### Follow the steps outlined in the [Installation and Deployment](#installation-and-deployment) section of [Local Linux Development Setup](#local-linux-development-setup). Remember that you will need to clone the **xForge** repos in Windows with Unix line endings. -``` +``` bash git clone --recurse-submodules --no-checkout https://github.com/sillsdev/web-languageforge cd web-languageforge git config core.autocrlf input @@ -252,16 +273,16 @@ The rest of the steps should be performed in an Ubuntu Bash shell. In order to access the local deployment of **xForge** apps, the local host names need to be added to Windows hosts file. The following lines should be added to the hosts file located at `C:\Windows\System32\drivers\etc\hosts`: -``` +``` bash 127.0.0.1 languageforge.local 127.0.0.1 scriptureforge.local ``` -#### Starting Services +#### Starting Services #### All required Linux services must be restarted when you open the first Ubuntu Bash shell. You can start the services required for development with the following commands: -``` +``` bash sudo service postfix start sudo service mongodb start sudo service apache2 start @@ -272,9 +293,10 @@ You can create a shell script that executes these commands in order to make it m ## Installing IDEs and Debugger ## ### Eclipse ### + Install Oracle Java JDK 8 -``` +``` bash sudo add-apt-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java8-installer oracle-java8-set-default @@ -282,7 +304,7 @@ sudo apt-get install oracle-java8-installer oracle-java8-set-default Download the [Eclipse](http://www.eclipse.org/downloads/) installer, extract the tar folder and run the installer. -``` +``` bash tar xvf eclipse-inst-linux64.tar.gz cd eclipse-installer ./eclipse-inst @@ -292,13 +314,13 @@ From the installer, select **Eclipse IDE for PHP Developers** Create a launcher shortcut from your *home* directory -``` +``` bash gedit .local/share/applications/eclipse.desktop ``` Replacing your *USERNAME*, paste the content below and save -``` +``` bash [Desktop Entry] Name=Eclipse Type=Application @@ -319,7 +341,7 @@ Once the MonjaDB plugin is installed, access `MonjaDB` from the Eclipse menu and Download [PhpStorm](https://www.jetbrains.com/phpstorm/download/#section=linux-version), extract the tar file and install. You may need to modify newer version numbers accordingly... -``` +``` bash tar xvf PhpStorm-2016.2.1.tar.gz sudo mv PhpStorm-162.1889.1/ /opt/phpstorm sudo ln -s /opt/phpstorm/bin/phpstorm.sh /usr/local/bin/phpstorm @@ -329,7 +351,7 @@ phpstorm LSDev members can contact their team lead to get the SIL license information. PhpStorm also has an option *Evaluate for free for 30 days*. -#### Coding Standard and Style +#### Coding Standard and Style #### [Download](https://plugins.jetbrains.com/plugin/7294) the PhpStorm plugin for EditorConfig and then install: @@ -367,7 +389,7 @@ Web server root URL: `http://languageforge.local` Ansible will have installed Xdebug, but you still need to manually edit `/etc/php/7.0/apache2/php.ini` and append the following lines: -``` +``` ini [Xdebug] xdebug.remote_enable = 1 xdebug.remote_port = 9000 @@ -382,25 +404,24 @@ Setting *PHP Interpreter* from PhpStorm **File** --> **Settings** --> **Languages & Frameworks** --> **PHP** -From the dropdown to *PHP language level*, select `7` -For *Interpreter*, click "..." to browse, then "+" +From the dropdown to *PHP language level*, select `"7"` +For *Interpreter*, click `"..."` to browse, then `"+"` -``` -Name: PHP 7 -PHP executable: /usr/bin/php -``` +- Name: PHP 7 +- PHP executable: /usr/bin/php Adding *Servers* from PhpStorm **File** --> **Settings** --> **Languages & Frameworks** --> **PHP** --> **Servers** Click the "+" to add the following Name & Hosts: + - default.local - languageforge.local - scriptureforge.local Restart apache2 -``` +``` bash sudo service apache2 restart ``` @@ -410,9 +431,7 @@ Install the [Xdebug helper](https://chrome.google.com/webstore/detail/xdebug-hel Right-click to select **Options** and set **IDE key** -``` -PhpStorm PHPSTORM -``` +- PhpStorm PHPSTORM When it's time to Debug, check that the bug icon is green for **Debug**. @@ -422,7 +441,7 @@ Reference for [Integrating Xdebug with PhpStorm](https://www.jetbrains.com/help/ ### LiveReload ### -#### LiveReload Chrome extension +#### LiveReload Chrome extension #### Install the [LiveReload](https://chrome.google.com/webstore/detail/livereload/jnihajbhpnppcggbcgedagnkighmdlei?hl=en-US) extension. @@ -432,7 +451,7 @@ Then from PhpStorm, click When you want LiveReload running, double-click the **reload** or **build-webpack:watch** Gulp task. Then in the LiveReload chrome extension, click to enable it. A solid dot in the circle means the plugin is connected. Now when an applicable source file is changed and saved, it should trigger an automate page reload in the browser. -### Visual Studio Code +### Visual Studio Code ### Visual Studio Code is a simple, free, cross-platform code editor. You can download VS Code from [here](https://code.visualstudio.com/). @@ -478,7 +497,7 @@ E2E tests live in the `test` folder and have the file extension `.e2e-spec.ts`. From the `web-languageforge` directory -``` +``` bash npm install gulp test-e2e-webdriver_update ``` @@ -487,29 +506,31 @@ gulp test-e2e-webdriver_update From the *web-languageforge* directory, start **webdriver** in one terminal: -```` +```` bash gulp test-e2e-webdriver_standalone ```` Then to run **languageforge** tests in another terminal: -```` +```` bash ./rune2e.sh lf ```` To run **scriptureforge** tests: -``` +``` bash ./rune2e.sh sf ``` (If you get an error messages like `Error: ECONNREFUSED connect ECONNREFUSED 127.0.0.1:4444` you probably forgot to start the **webdriver**, see above) To test a certain test spec, add a parameter `--specs [spec name]`. For example, -``` + +``` bash ./rune2e.sh lf --specs lexicon-new-project ``` -will run the the *lexicon-new-project.e2e-spec.ts* tests on **languageforge**. + +will run the the *lexicon-new-project.e2e-spec.ts* tests on **languageforge**. To add more verbosity during E2E tests, add a parameter `--verbosity true` @@ -523,44 +544,49 @@ Install gulp dependencies by running from the repo root (where): To install the mongodb databases locally, run: -``` +``` bash gulp mongodb-copy-prod-db ``` ## Resetting MongoDB ## If you want to _start over_ with your mongo database, you can use the factory reset script like so (this will delete all data in the mongodb): -```` + +```` bash cd scripts/tools ./FactoryReset.php run ```` + After a fresh factory reset, there is one user. username: admin password: password ## Updating dependencies ## Occasionally developers need to update composer or npm. If something isn't working after a recent code change, try to update the dependencies: -#### Update npm packages #### +### Update npm packages ### In the *root* folder: `npm install` -#### Update composer #### +### Update composer ### In the **src/** folder: `composer install` ## Running the Node Server ## To run the node server to get real time updating... -```` + +```` bash cd src/node sudo node server.js ```` ## Setting up and running the Machine Web Server ## -### Installation and Deployment +### Installation and Deployment ### + From the **web-languageforge** repo root folder... -```` + +```` bash cd .. git clone git@github.com:sillsdev/machine.git cd machine @@ -568,33 +594,38 @@ git checkout -b Translation origin/Translation ```` To deploy the machine server... -```` + +```` bash cd build ./deploy-developer.sh ```` -### Running +### Running ### + To run the machine server... -```` +```` bash ./run-developer.sh ```` Sometimes you may have to remove the `json` file in `/var/lib/languageforge/machine/data/build/` and then restart. -### Suggestion data +### Suggestion data ### + Copy `/var/lib/languageforge/machine/` from live server. Add any project slugs to the `Projects` section of the `json` file in `/var/lib/languageforge/machine/data/engine/`. ## ElasticSearch Data ## + Before putting data into **ElasticSearch**, setup the index with settings and data mappings... -``` + +``` bash ./scripts/server/elasticSearch/setupElasticSearchCATIndex.sh ``` ## Contributors ## -* Elizabeth Koning -* Ethan Clark -* Ziqi Chen -* Ye Joo Oh +- Elizabeth Koning +- Ethan Clark +- Ziqi Chen +- Ye Joo Oh diff --git a/readme_images/bugsnag-logo.png b/readme_images/bugsnag-logo.png new file mode 100644 index 0000000000..1030b17ef7 Binary files /dev/null and b/readme_images/bugsnag-logo.png differ