majuna/docs/user/proxies.rst

Web Proxies
===========

Background
----------

A web proxy can help users by providing alternate URLs to access censored resources, allowing them to bypass technical censorship and access information that may be otherwise unavailable.
Web proxies work by forwarding requests to the original website, and providing a different URL to access that content.
By accessing the web proxy URL, users can access the same content that would be blocked by censors if accessed through the original website's URL.

Web proxies also use frequently changing URLs to evade censorship, making it more difficult for censors to block access to the content.
This assumption of a limited lifetime is built-in to the system, allowing for automated block detection to trigger the
deployment of new URLs, and for the :doc:`distribution lists <lists>` to allow applications and end-users to discover
new URLs.

Additionally, web proxies can be accessed via a normal web browser, making them easily accessible to users without requiring any special software or technical knowledge.

Simple vs. Smart Proxies
^^^^^^^^^^^^^^^^^^^^^^^^

We currently distinguish between simple and smart proxies.
This distinction will become redundant in the future as we aim to allow for all providers to offer the smart proxy functionality.
Smart proxies differ from simple proxies in that they perform active rewriting of the responses from the original website.
This modifies links to resources to ensure that they are always loaded via the mirror system rather than from the original—blocked—website.

Smart proxies are only supported on AWS right now.

Architecture
------------

.. image:: /_static/proxies/architecture.png
   :width: 800

Before configuring any web origins to create mirrors, you must first configure an origin group.
The group is used to organise larger numbers of web origins so that mirrors can be created for them with a shared configuration.
The shared configuration will include which pools will require mirrors for the origins.

For each configured origin group, a smart proxy instance will be created at each provider where proxies are deployed.
This instance runs the active proxy that will rewrite URLs in responses from the origin webserver to ensure that all resources are loaded via the mirrors.
Some providers, e.g. Fastly, do not require an instance as they already provide the means for URL re-writing in their standard functionality.
No smart proxy instance will be created for providers where it is not necessary.

Web origins represent the original website that has been blocked.

Web Origins
-----------

Rate Limiting
^^^^^^^^^^^^^

CDNs (Content Delivery Networks) can impose rate limiting on websites to ensure that the network resources are
efficiently utilized, to protect the websites from DDoS (Distributed Denial of Service) attacks and to maintain the
quality of service for all the websites using the CDN.

These rate limits will be sized according to the expected rate of requests from an average user, however the proxy
system is a bottleneck that aggregates requests from multiple users and passes these on to the original CDN.
When a single system is used to send a large number of requests to a CDN like this, the CDN may interpret this as a
denial of service (DoS) attack and prevent access to the website.

Deploying mirrors for websites hosted on CDNs will require co-operation from the CDN provider. If you find that mirrors
are producing many "Rate Limited Exceeded" or "Access Denied" errors then you may be suffering from this problem. Ask
your administrator to configure the `Bypass-Rate-Limit-Token` header in the portal and at the CDN to disable the rate
limiting for requests originating via the mirrors.

New Web Origin
^^^^^^^^^^^^^^

To create a new web origin, click "Create new origin" at the top of the list. This will present
you with the new origin form:

.. image:: /_static/origin/new.png
   :width: 800

Domain Name
"""""""""""

Enter the domain name that you want to create a mirror for in this field. This is a required field.

Description
"""""""""""

Enter a brief description of the website that you are creating a mirror for in this field. This is also a required field.

Group
"""""

Select the group that you want the mirror to belong to from the drop-down menu in this field. This is a required field.

Auto-Rotate
"""""""""""

Select this field if you want to enable auto-rotation for the mirror. This means that the mirror will automatically
redeploy with a new domain name if it is detected to be blocked. This field is optional and is enabled by default.

Smart Proxy
"""""""""""

.. note:: This is a transitional option that will be removed once all platforms support smart proxies.

Select this field if the mirror requires a smart proxy to function properly. This is an optional field and is disabled
by default. Enabling this will limit the platforms that the mirror can be deployed to as not all platforms support
smart proxies at this time.

Asset Domain
""""""""""""

.. note:: This is a transitional option that will be removed once all platforms support smart proxies.

Select this field if you want to use mirrors of this domain to host assets for other domains. This is an optional field
and is disabled by default. All of the asset domains within a group will be available for all mirrors using smart
proxies. The smart proxy will rewrite URLs for the asset domains that are included in the source code of the mirror
hosted by a smart proxy.

Static Origins
--------------

Static origins can be used to host static content at a cloud provider, and then have mirrors generated for that content
rather than proxying to an external web server.

Once created a static origin will provide you with details on where and how to upload the web content.

Static origins can also be extended with additional resources:

* `Keanu Convene <https://letsconvene.im/>`_ - A matrix-based chat platform that supports sharing of any kind. Rooms are
  private and encrypted by default. No software is required to join a room other than a web browser, and participants
  can be invited via a link.
* `Clean Insights <https://cleaninsights.org/>`_ - A private measurement platform. It is focused on assisting in
  answering key questions about app usage patterns, and not on enabling invasive surveillance of all user habits.

These resources can be added/removed after the creation of the static origin without having to change the static storage
location or access credentials.

New Static Origin
^^^^^^^^^^^^^^^^^

To create a new static origin, click "Create new static origin" at the top of the list. This will present
you with the new static origin form:

.. image:: /_static/static/new.png
   :width: 800

Description
"""""""""""

Enter a brief description of the static website that you are creating in this field. This is also a required field.

Group
"""""

Select the group that you want the origin to belong to from the drop-down menu in this field. This is a required field.

Auto-Rotate
"""""""""""

Select this field if you want to enable auto-rotation for the mirror. This means that the mirror will automatically
redeploy with a new domain name if it is detected to be blocked. This field is optional and is enabled by default.

Matrix Homeserver
"""""""""""""""""

Select the Matrix homeserver from the drop-down box to enable Keanu Convene on mirrors of this static origin.

Keanu Convene Path
""""""""""""""""""

Enter the subdirectory to present the Keanu Convene application at on the mirror. This defaults to "talk".

Enable Clean Insights
"""""""""""""""""""""

When enabled, a Clean Insights Measurement Proxy endpoint is deployed on the mirror to allow for submission of results
from any of the supported Clean Insights SDKs.

Web Proxies
-----------