feat: refactoring for sphinx localisation support

README.md

@@ -1,5 +1,4 @@
-Bypass Censorship Portal
+# Bypass Censorship Portal
-========================
 
 Self-service deployment of censorship circumvention resources.
 
@@ -8,42 +7,44 @@ Self-service deployment of censorship circumvention resources.
 
 ![Home page screenshot](docs/_static/home.png)
 
-Development Setup
+## Development Setup
------------------
 
 For development, it is possible to use SQLite and a single host.
 Clone this repository and navigate to it:
-```
+
+```bash
 git clone https://gitlab.com/guardianproject-ops/bypass-censorship/portal
 cd portal
 ```
+
 Create and activate a Python virtual environment:
 
-```
+```bash
 python3 -m venv venv
 source ./venv/bin/activate
 ```
 
 Install the Python packages needed by the application:
 
-```
+```bash
 pip install -r requirements.txt
 pip install psycopg2-binary
 ```
 
 Set up the database:
 
-```
+```bash
 flask db init
 flask db upgrade
 ```
 
 Before the application can be run, it will need to be configured.
-Run `cp config.yaml.example config.yaml`, then edit `config.yaml` accordingly. Note that at least one cloud provider
+Run `cp config.yaml.example config.yaml`, then edit `config.yaml` accordingly.
+Note that at least one cloud provider
 must be configured for the application to start successfully.
 To run the application:
 
-```
+```bash
 flask run --host=0.0.0.0 --port=5000
 ```
 
@@ -51,13 +52,13 @@ flask run --host=0.0.0.0 --port=5000
 
 To build the new portal frontend:
 
-```
+```bash
 make install-frontend
 ```
 
 To update the build for the new portal frontend:
 
-```
+```bash
 pushd frontend && git pull --ff-only && popd && make install-frontend
 ```
 
@@ -65,27 +66,119 @@ pushd frontend && git pull --ff-only && popd && make install-frontend
 
 The dockerfile supplied will build a container to run the application:
 
-```
+```bash
 docker build -t <image-name> .  
 
-docker|podman run -e WAITRESS_THREADS=4" -e WAITRESS_RUN_HOST="0.0.0.0" -e WAITRESS_RUN_PORT="5000" -e APP="bc" -e APP_BASE="/srv" <image_name>
+docker|podman run -e WAITRESS_THREADS=4" -e WAITRESS_RUN_HOST="0.0.0.0" \
+-e WAITRESS_RUN_PORT="5000" -e APP="bc" -e APP_BASE="/srv" <image_name>
 ```
 
 The following environment variables can be set to configure the container:
 
-- ```WAITRESS_THREADS```: the number of Waitress worker threads for handling requests. Defaults to 4 threads.
+- ```WAITRESS_THREADS```: the number of Waitress worker threads for handling
+  requests. Defaults to 4 threads.
 
-- ```WAITRESS_RUN_HOST```: the network interfaces on which Waitress will listen on inside the container. Defaults to
+- ```WAITRESS_RUN_HOST```: the network interfaces on which Waitress will listen
+  on inside the container. Defaults to
   0.0.0.0 (all).
 
-- ```WAITRESS_RUN_PORT```: the port on which Waitress will serve the application inside the container. Defaults to 5000.
+- ```WAITRESS_RUN_PORT```: the port on which Waitress will serve the application
+  inside the container. Defaults to 5000.
 
 - ```APP```: the app directory name, defaults to ```bc```.
 
-- ```APP_BASE```: the base directory where the app directory will be created. Defaults to ```/srv```.
+- ```APP_BASE```: the base directory where the app directory will be created.
+  Defaults to ```/srv```.
 
-Copyright
+## Docs
----------
+
+The project is built using **Sphinx** and supports localization for multiple
+languages.
+The list of supported languages is defined in the `languages.yaml` file in the
+`docs` directory.
+By default, the project uses **English** as the primary language.
+The theme used for the documentation of this project
+is [Furo](https://pradyunsg.me/furo/).
+
+To add a new language, first edit the `languages.yaml` to add the language code
+and its name in the following format in
+the `languages` section:
+
+```yaml  
+  - code: "ro" # ISO 639-1 two-letter country code
+  - name: "Română" # native language name
+```
+
+Then, to generate translation files for the new language:
+
+```bash
+pip install -r requirements-docs.txt
+cd docs
+make gettext
+sphinx-intl update -p _build/gettext -l <language-code>
+```
+
+Or, in case the documentation has changed, to update translation files for all
+languages defined for the project:
+
+```bash
+make gettext
+for lang in $(awk '/code:/ {print $3}' languages.yaml |  sed s/\"//g ); do
+  sphinx-intl update -p _build/gettext -l $lang
+done
+```
+
+### Development
+
+The following commands will output `html` docs in `_build/html`:
+
+```bash
+cd docs
+make html 
+
+# to build a specific language:
+# make -e SPHINXOPTS="-D language=ro" html 
+```
+
+A script, `docs/build_docs.py` is provided to build the documentation for all
+supported languages, with the output
+placed in a directory called `public`.
+
+The English documentation is located at the root of the directory, while the
+documentation
+for individual languages is placed in separate subdirectories.
+
+For example, the English version will be available at the root as `index.html`,
+and the Romanian and Polish versions
+will be found in the `ro` and `pl` subdirectories, respectively, with their own
+index files.
+
+The theme used by this documentation, Furo, does not have built-in support for
+language configuration.
+To address this, the project extends the theme using Sphinx's templating
+capabilities to display the available
+languages.
+The following block of code takes the languages and passes them to the template
+located in
+`_templates/sidebar/ethical-ads.html`:
+
+```python
+html_context = {
+    'languages': [[lang['code'], lang['name']] for lang in languages]
+}
+```
+
+As the available languages need to be displayed in the sidebar, the template
+`_templates/sidebar/ethical-ads.html`
+overrides one of the default sidebar
+snippets available in
+the [theme](https://pradyunsg.me/furo/customisation/sidebar/.).
+Using the language links currently takes a user to
+`/<language-code>/<current-page>`.
+This assumes that the language-specific documentation is stored in a
+subdirectory, and can be changed in the template.
+
+## Copyright
 
 ![BSD 2-clause](https://img.shields.io/badge/license-BSD%202--clause-blue)
 
@@ -93,20 +186,30 @@ Copyright
 
 Copyright 2021-2022 SR2 Communications Limited.
 
-Redistribution and use in source and binary forms, with or without modification, are permitted provided that the
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the
 following conditions are met:
 
-1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following
    disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
+2. Redistributions in binary form must reproduce the above copyright notice,
-   disclaimer in the documentation and/or other materials provided with the distribution.
+   this list of conditions and the following
+   disclaimer in the documentation and/or other materials provided with the
+   distribution.
 
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ANY EXPRESS OR IMPLIED WARRANTIES,
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+FITNESS FOR A PARTICULAR PURPOSE ARE
-SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
-WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 ---

docs/_templates/sidebar/ethical-ads.html

@@ -0,0 +1,9 @@
+<div class="sidebar-tree">
+    <p class="caption" role="heading"><span class="caption-text">{{ _('Translations') }}</span></p>
+    <ul>
+        {% for language, language_name in languages %}
+            <li class="toctree-l1"><a class="reference internal"
+                                      href="/{{ language }}/{{ pagename }}.html">{{ language_name }}</a></li>
+        {% endfor %}
+    </ul>
+</div>

docs/build_docs.py

@@ -0,0 +1,17 @@
+import subprocess
+import os
+import yaml
+
+with open('languages.yaml', 'r') as file:
+    config = yaml.safe_load(file)
+    languages = config.get('languages', [])
+
+
+def build_doc(language, dest_dir):
+    os.environ['SPHINXOPTS'] = "-D language='{}'".format(language)
+    subprocess.run(f"sphinx-build -b html . {dest_dir}", shell=True)
+
+
+build_doc("en", "../public")
+for language in languages:
+    build_doc(language['code'], f"../public/{language['code']}")

docs/conf.py

@@ -1,17 +1,36 @@
 import builtins
 import os
 import sys
+import yaml
+
 sys.path.insert(0, os.path.abspath('..'))
 
+language = 'en'  # default language
+
+locale_dirs = ['./locale/']
+gettext_compact = False
+
+templates_path = ['./_templates']
+
 builtins.__sphinx_build__ = True
 
+# Define languages and links in html_context
+# this is used with templates in /_templates to extend the default theme,
+# adding languages to the sidebar
+
+with open('languages.yaml', 'r') as file:
+    config = yaml.safe_load(file)
+    languages = config.get('languages', [])
+
+html_context = {
+    'languages': [[lang['code'], lang['name']] for lang in languages]
+}
 # -- Project information -----------------------------------------------------
 
 project = 'Bypass Censorship Portal'
 copyright = '2021-202r SR2 Communications Limited'
 author = 'Bypass Censorship'
 
-
 # -- General configuration ---------------------------------------------------
 
 # Add any Sphinx extension module names here, as strings. They can be
@@ -31,7 +50,6 @@ templates_path = ['_templates']
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
 
-
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for

docs/languages.yaml

@@ -0,0 +1,8 @@
+languages:
+  - code: "en"
+    name: "English"
+  - code: "pl"
+    name: "Polski"
+  - code: "ro"
+    name: "Română"
+

requirements-docs.txt

@@ -1,4 +1,5 @@
 furo
 sphinx
+sphinx-intl
 sphinxcontrib-mermaid
 sphinxcontrib-openapi