diff --git a/docs/ww-customization.mdx b/docs/ww-customization.mdx index 73c49d7b..32a74869 100644 --- a/docs/ww-customization.mdx +++ b/docs/ww-customization.mdx @@ -23,12 +23,19 @@ If there are files in your cloned version that are not in the tree below, you do | +-- wordweaver | +-- data | +-- fr +| +-- i18n +| +-- en.json +| +-- fr.json | +-- __init__.py | +-- conjugations.json | +-- models.py | +-- options.json | +-- pronouns.json | +-- verbs.json +| +-- yourlang +| +-- __init__.py +| +-- models.py +| +-- i18n | +-- router | +-- main.py | +-- models.py @@ -48,29 +55,37 @@ When deploying just to a development environment (the default) there is only one This is the main configurable service. It is the `wordweaver` Python package and API. - Every WordWeaver has its data separated into 4 main sections: [Verbs](#verbs), [Pronouns](#pronouns), [Options](#options), and [Conjugations](#conjugations). This data must be in JSON format. To add your data, create a new folder in src/wordweaver/data named after your languages’s ISO 639-3 code. You will need to add 6 files: the described JSON data files, a blank __init__.py file, and a [models](#models) file. The following sections describe the data and models files. +Every WordWeaver has its data separated into 4 main sections: [Verbs](#verbs), [Pronouns](#pronouns), [Options](#options), and [Conjugations](#conjugations). This data must be in JSON format. To add your data, create a new folder in `src/wordweaver/data` named after your languages’s [ISO 639-3 code](https://iso639-3.sil.org/code_tables/639/data). You will need to add 6 files: the described JSON data files, a blank \_\_init\_\_.py file, and a [models](#models) file. The following sections describe the data and models files. -### Models +In order for your data to be used when ran, you must change the variable WWLANG in the file `src/env-backend.env` to your languages’s [ISO 639-3 code](https://iso639-3.sil.org/code_tables/639/data) that you replaced `yourlang` with. For example, to use the french data, it must read as follows: + + WWLANG=fr -Your models.py file is found at `src/wordweaver/data//models.py` and should contain a model for each data file ([Verbs](#verbs), [Pronouns](#pronouns), [Options](#options), and [Conjugations](#conjugations)). If your data does vary from the default that’s been described, you will have to update the associated model. -For example, if you wanted to include a "gender" attribute to your "pronoun" data, you would have to up the pronoun model as follows +__Note__: There is a default folder included in the initial downloaded, called `src/wordweaver/data/yourlang`. This folder contains the default models.py file (exactly as described below) and an empty `__init__.py` file. You may also instead rename this folder after your languages’s ISO 639-3 code and add your JSON data files there. -```typescript {2} -class Pronoun(BaseModel): - ''' Required ''' - gender: str = '' - tag: str = '' -``` +__Note__: There is a french wordweaver included in the initial downloaded, called `src/wordweaver/data/fr`. Feel free to explore this folder for reference or to copy code. -If your data does not vary from the described defaults, the include default models.py file will contain all sufficient information and can be left untouched. +## Language Data +First, we will describe the data structures and files you will need to build. ### Verbs -You must have data related to the verb roots available in your WordWeaver in a single `verbs.json` file. Your verbs should be a JSON array of objects. Minimally, each verb in your verbs array **must** have a unique`tag`, and `classes` keys. +You must have data related to the verb roots available in your WordWeaver in a `verbs.json` file. Your verbs should be a JSON array of objects. Minimally, each verb in your verbs array **must** have a unique `tag` and a `classes` key. + +#### tag + +The `tag` is the unique identifier for that verb. It must also be URL friendly, otherwise known as a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). To be URL friendly, a slug must be restricted to [certain characters](https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid) + +#### classes + +The `classes` key is an array of strings that are CSS class names to apply to that verb. This can be used to distinguish different verb types to the user. For more information, please visit the [WordWeaver UI style customization guide](ww-ui-style#custom-css). Default Definition: + +Source: src/wordweaver/data/yourlang/models.py + ```python class Verb(BaseModel): ''' Required ''' @@ -78,50 +93,105 @@ class Verb(BaseModel): classes: List[str] = [] ``` -#### tag +For example, the verbs JSON file should be of the following form: -The `tag` is the unique identifier for that verb. It must also be URL friendly, otherwise known as a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). To be URL friendly, a slug must be restricted to [certain characters](https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid) +Source: src/wordweaver/data/yourlang/verbs.json -#### classes +```json +[ + { + "tag":"aller", + "en": + { + "tag": "go" + }, + } +] +``` -The `classes` key is an array of strings that are CSS class names to apply to that verb. Please visit the [WordWeaver UI style customization guide](ww-ui-style#custom-css) for more information. ### Pronouns -You must have data related to the pronouns available in your WordWeaver in a single `pronouns.json` file. +You must have data related to pronouns available in your WordWeaver in a `pronouns.json` file. Your pronouns should be a JSON array of objects. Minimally, each pronoun in your pronoun array **must** have a unique`tag` and the JSON file must include both the `tag` and a display form (i.e. what the user will see). The default for the pronouns is to accept only an `agent`. To include a `patient`, it should be included in the pronouns JSON file and the [Conjugations](#conjugations) related models must be updated as well (see section for details). + +:::info +WordWeaver was initially created for Kanyen'kéha, an Iroquoian language. In Iroquoian languages, [semantic](https://en.wikipedia.org/wiki/Theta_role) notions of 'Agent' and "Patient" are more meaningful than purely syntactic notions of 'Subject' and 'Object'. The naming here is vestigial from that, and could be refactored in upcoming versions of wordweaver. +::: + +#### tag + +The `tag` is the unique identifier for that pronoun. It must also be URL friendly, otherwise known as a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). To be URL friendly, a slug must be restricted to [certain characters](https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid) Default Definition: +Source: src/wordweaver/data/yourlang/models.py + ```python class Pronoun(BaseModel): ''' Required ''' tag: str = '' ``` + + +For example, the pronouns JSON file should be of the following form: + + +Source: src/wordweaver/data/yourlang/pronouns.json + +```json +[ + { + "tag":"1SG", + "en": + { + "agent": "I" + }, + } +] +``` + +### Options + +The 'Options' category is a catch-all category for all other types of information associated with choosing a conjugation. You must have data related to the options available in your WordWeaver in a single `options.json` file. Your options should be a JSON array of objects. Minimally, each option in your options array **must** have a unique`tag`. The JSON file must the unique `tag` and a display form (i.e. what the user will see). The default model also includes an optional `type` attribute. #### tag -The `tag` is the unique identifier for that pronoun. It must also be a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). +The `tag` is the unique identifier for that option. It must also be URL friendly, otherwise known as a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). To be URL friendly, a slug must be restricted to [certain characters](https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid). -### Options +#### type -The 'Options' category is a catch-all category for all other types of information associated with choosing a conjugation. You therefore must have all other options related to your data available in a single `options.json` file. +The `type` allows for categorization of options (ex. tense, aspect etc) Default Definition: + +Source: src/wordweaver/data/yourlang/models.py + ```python class Option(BaseModel): ''' Required ''' tag: str = '' type: str = '' ``` + + For example, the options JSON file should be of the following form: + + +Source: src/wordweaver/data/yourlang/options.json + +```json +[ + { + "tag": "indicatif-present", + "type": "indicatif", + "en": + { + "type": "Indicative" + }, + } +] +``` -#### tag - -The `tag` is the unique identifier for that option. It must also be a [slug](https://stackoverflow.com/questions/19335215/what-is-a-slug). - -#### type - -The `type` allows for categorization of options (ex. tense, aspect etc) ### Conjugations @@ -129,13 +199,19 @@ You must have data related to the conjugations available in your WordWeaver in a Each conjugation consists of an `input` and an `output`: + +Source: src/wordweaver/data/yourlang/models.py + ```python class ResponseObject(BaseModel): input: ConjugationInput output: Conjugation ``` -The `input` is an object that includes the tag values for each related `verb`, `pronoun`, and `option`: +The `input` is an object that includes the tag values for each related `verb`, `pronoun`, and `option`. __Note__ that the default ConjugationInput class assumes _only_ an agent is required for any given conjugation. If your language's conjugations may require an `patient`, please be sure to add a `patient` attribute to this model. + + +Source: src/wordweaver/data/yourlang/models.py ```python class ConjugationInput(BaseModel): @@ -144,12 +220,16 @@ class ConjugationInput(BaseModel): agent: str ``` + The `output` is a list of objects containing response morphemes: ```python Conjugation = List[ResponseMorpheme] ``` + +Source: src/wordweaver/data/yourlang/models.py + ```python class ResponseMorpheme(BaseModel): position: Optional[int] @@ -161,7 +241,201 @@ class ResponseMorpheme(BaseModel): Response morphemes are units that can be concatenated into `Tiers`. Tiers are [defined in the WordWeaver UI](#ww-ui-customization#tiers). -### Edit CORS rules +For example, the conjugations JSON file should be of the following form: + + +Source: src/wordweaver/data/yourlang/conjugations.json + +```json +{ + "input": { + "root": "courir", + "option": "indicatif-present", + "agent": "2-pl" + }, + "output": [ + { + "position": 0, + "value": "cour", + "type": [ + "root" + ] + }, + { + "position": 1, + "value": "ez", + "type": [ + "ending" + ] + } + ] +} +``` + +Here, `courir` must be a tag of a verb defined in `verbs.json`. `indicatif-present` must be a tag of an option defined in `options.json` and `2-pl` must be a tag of a pronoun defined in `pronouns.json`. + +If your language data is not available split up into morphemes, instead put the full value in position 0. + +### Models + +Your models.py file is found at `src/wordweaver/data/yourlang/models.py` and should contain a model for each data file ([Verbs](#verbs), [Pronouns](#pronouns), [Options](#options), and [Conjugations](#conjugations)). If your data does vary from the default that’s been described, you will have to update the associated model. + +For example, if you wanted to include a "gender" attribute to your "pronoun" data, you would have to up the pronoun model as follows + + +Source: src/wordweaver/data/yourlang/models.py + +```typescript {2} +class Pronoun(BaseModel): + ''' Required ''' + gender: str = '' + tag: str = '' +``` + +If your data does not vary from the described defaults, the included default models.py file will contain all sufficient information and can be left untouched. + +## Internationalization - Including Other Languages + +Translating your wordweaver into other languages is done in two different places. The current front end in its entirety is translated available in both English and French. +To include translations for the entire website in another language, see [Internationalization](ww-ui-customization.md#i18n). In order to add translations for your language data, follow the steps below. + +:::important +You must have installed wordweaver to use the command line methods needed for this section. To install, first go to your clone `cd ~/wordweaver` and then do an editable installation `pip3 install -e .`. +::: + + +### Create the basic translation files + +First, make sure that `WWLANG` is set to the right language. You can check this by typing `echo $WWLANG` in your terminal. This should produce a value. If not, enter `export WWLANG=yourlang` in your terminal. + +Then, create your translation files by running + +```bash +wordweaver extract-translations yourlang +``` + +If you are wanting to extract translations for multiple languages (including English), then just append them: + +```bash +wordweaver extract-translations en yourlang1 yourlang2 +``` + +This will add parallell translation files at `src/wordweaver/data/yourlang/i18n/en.json`, `src/wordweaver/data/yourlang/i18n/yourlang1.json`, and `src/wordweaver/data/yourlang/i18n/yourlang2.json`. + +These files will be initialized with all of the tags from your language data. + +### Translate files + +You can either translate these files by entering translation values in the json directly. So, the English translation for a French conjugator might look partially like this: + +Source: src/wordweaver/data/fr/i18n/en.json + +```json + +{ + "ww-data": { + "options": { + "items": { + "indicatif-present": "Indicative Present", + "indicatif-imparfait": "Imperfect" + ... + } + } + } +} + +``` + +The corresponding French translation file would like like this: + +Source: src/wordweaver/data/fr/i18n/fr.json + +```json + +{ + "ww-data": { + "options": { + "items": { + "indicatif-present": "Indicatif Présent", + "indicatif-imparfait": "Indicatif Imparfait", + ... + } + } + } +} + +``` + +:::tip +You can also upload your translation files to a translation editing service which means you don't have to edit the json directly and you can get a non-programmer translator working on the project as well. +We recommend [POEditor](https://poeditor.com/) or [BabelEdit](https://www.codeandweb.com/babeledit) +::: + +### Merging translations + +Once your translation files are done, you can merge them back in to your language data files using the following script: + +```bash +wordweaver add-translations en fr yourlang1 +``` + +This will attempt to merge your languages back in to your language data files. By default it will not override values that exist, but you can force that by adding the `--force` flag: + +```bash +wordweaver add-translations en fr yourlang1 +``` + +:::warning +Adding translations with `--force` could erase some of your translations! Do it carefully, and review the changes afterwards. +::: + +Once added, your language data files will be updated with the provided languages. For example, here is what the updated `options.json` file would look like: + +Before `wordweaver add-translations en fr`: +```json +[ + { + "tag": "indicatif-present", + "type": "indicatif" + }, + { + "tag": "indicatif-imparfait", + "type": "indicatif" + } + ... +] +``` + +After `wordweaver add-translations en fr`: + +```json {5,6,7,8,9,10,15,16,17,18,19,20} +[ + { + "tag": "indicatif-present", + "type": "indicatif", + "fr": { + "tag": "Indicatif Pr\u00e9sent" + }, + "en": { + "tag": "Indicative Present" + } + }, + { + "tag": "indicatif-imparfait", + "type": "indicatif", + "fr": { + "tag": "Indicatif Imparfait" + }, + "en": { + "tag": "Imperfect" + } + } + ... +] +``` + + +## Edit CORS rules [Cross-origin resource sharing](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) (CORS) rules are defined in `src/wordweaver/main.py`. You can edit allowable [origins](https://en.wikipedia.org/wiki/Same-origin_policy#:~:text=An%20origin%20is%20defined%20as,that%20page's%20Document%20Object%20Model.), [methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) and [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers) here. @@ -189,10 +463,19 @@ The configuration is defined as a template `volumes/reverse/nginx/conf/wordweave Certbot manages your SSL cert. We recommend using [Let's Encrypt](https://letsencrypt.org/). After securing your server ensuring Docker is available, and cloning your repo onto your server, do the following steps to set up automated renewals of your SSL cert. -1. Change the email and domain in `.env` +Your SSL certification using Let's Encrypt is initialized in the `init-letsencrypt.sh` script in the root of the repo. + +1. Change the email and domain in `.env`. If you are just testing out your deployment, keep `STAGING` set to `1` which will prevent you from hitting [Let's Encrypt Rate Limits](https://letsencrypt.org/docs/rate-limits/). Once you are sure it is up and running though, change `STAGING` to `0` + +```bash +LE_EMAIL=youremail@example.com +DOMAIN=your.website.com +STAGING=1 +``` + 2. On your server, run `init-letsencrypt.sh` to get an SSL cert for your site 3. Spin up your WordWeaver by running `docker-compose -f docker-compose.prod.yml up` :::important -Your DNS records will have to be set up to point to your server +Your DNS records for the DOMAIN will have to be set up to point to your server. Your server host should provide you with information on how to set up either an A record or CNAME record. :::