diff --git a/docs/_static/origin/new.png b/docs/_static/origin/new.png new file mode 100644 index 0000000..a0dabca Binary files /dev/null and b/docs/_static/origin/new.png differ diff --git a/docs/_static/proxies/architecture.png b/docs/_static/proxies/architecture.png new file mode 100644 index 0000000..5fafcd4 Binary files /dev/null and b/docs/_static/proxies/architecture.png differ diff --git a/docs/user/proxies.rst b/docs/user/proxies.rst index 851a3fe..25d6e98 100644 --- a/docs/user/proxies.rst +++ b/docs/user/proxies.rst @@ -1,29 +1,124 @@ Web Proxies =========== -Web proxies provide alternate URLs to access censored resources. -These can be accessed through a normal web browser. -They use frequently changing URLs to evade censorship. -Some functionality on more complex websites may not work fully. -The URLs will have a limited lifetime as, once discovered, they can be blocked by censors. +Background +---------- +A web proxy can help users by providing alternate URLs to access censored resources, allowing them to bypass technical censorship and access information that may be otherwise unavailable. +Web proxies work by forwarding requests to the original website, and providing a different URL to access that content. +By accessing the web proxy URL, users can access the same content that would be blocked by censors if accessed through the original website's URL. + +Web proxies also use frequently changing URLs to evade censorship, making it more difficult for censors to block access to the content. This assumption of a limited lifetime is built-in to the system, allowing for automated block detection to trigger the deployment of new URLs, and for the :doc:`distribution lists ` to allow applications and end-users to discover new URLs. +Additionally, web proxies can be accessed via a normal web browser, making them easily accessible to users without requiring any special software or technical knowledge. + +Simple vs. Smart Proxies +^^^^^^^^^^^^^^^^^^^^^^^^ + +We currently distinguish between simple and smart proxies. +This distinction will become redundant in the future as we aim to allow for all providers to offer the smart proxy functionality. +Smart proxies differ from simple proxies in that they perform active rewriting of the responses from the original website. +This modifies links to resources to ensure that they are always loaded via the mirror system rather than from the original—blocked—website. + +Smart proxies are only supported on AWS right now. + +Architecture +------------ + +.. image:: /_static/proxies/architecture.png + :width: 800 + +Before configuring any web origins to create mirrors, you must first configure an origin group. +The group is used to organise larger numbers of web origins so that mirrors can be created for them with a shared configuration. +The shared configuration will include which pools will require mirrors for the origins. + +For each configured origin group, a smart proxy instance will be created at each provider where proxies are deployed. +This instance runs the active proxy that will rewrite URLs in responses from the origin webserver to ensure that all resources are loaded via the mirrors. +Some providers, e.g. Fastly, do not require an instance as they already provide the means for URL re-writing in their standard functionality. +No smart proxy instance will be created for providers where it is not necessary. + +Web origins represent the original website that has been blocked. + Web Origins ----------- +Rate Limiting +^^^^^^^^^^^^^ + +CDNs (Content Delivery Networks) can impose rate limiting on websites to ensure that the network resources are +efficiently utilized, to protect the websites from DDoS (Distributed Denial of Service) attacks and to maintain the +quality of service for all the websites using the CDN. + +These rate limits will be sized according to the expected rate of requests from an average user, however the proxy +system is a bottleneck that aggregates requests from multiple users and passes these on to the original CDN. +When a single system is used to send a large number of requests to a CDN like this, the CDN may interpret this as a +denial of service (DoS) attack and prevent access to the website. + +Deploying mirrors for websites hosted on CDNs will require co-operation from the CDN provider. If you find that mirrors +are producing many "Rate Limited Exceeded" or "Access Denied" errors then you may be suffering from this problem. Ask +your administrator to configure the `Bypass-Rate-Limit-Token` header in the portal and at the CDN to disable the rate +limiting for requests originating via the mirrors. + +New Web Origin +^^^^^^^^^^^^^^ + +To create a new web origin, click "Create new origin" at the top of the list. This will present +you with the new origin form: + +.. image:: /_static/origin/new.png + :width: 800 + +Domain Name +""""""""""" + +Enter the domain name that you want to create a mirror for in this field. This is a required field. + +Description +""""""""""" + +Enter a brief description of the website that you are creating a mirror for in this field. This is also a required field. + +Group +""""" + +Select the group that you want the mirror to belong to from the drop-down menu in this field. This is a required field. + +Auto-Rotate +""""""""""" + +Select this field if you want to enable auto-rotation for the mirror. This means that the mirror will automatically +redeploy with a new domain name if it is detected to be blocked. This field is optional and is enabled by default. + +Smart Proxy +""""""""""" + +.. note:: This is a transitional option that will be removed once all platforms support smart proxies. + +Select this field if the mirror requires a smart proxy to function properly. This is an optional field and is disabled +by default. Enabling this will limit the platforms that the mirror can be deployed to as not all platforms support +smart proxies at this time. + +Asset Domain +"""""""""""" + +.. note:: This is a transitional option that will be removed once all platforms support smart proxies. + +Select this field if you want to use mirrors of this domain to host assets for other domains. This is an optional field +and is disabled by default. All of the asset domains within a group will be available for all mirrors using smart +proxies. The smart proxy will rewrite URLs for the asset domains that are included in the source code of the mirror +hosted by a smart proxy. + Static Origins -------------- -These have not yet been implemented. +Static origins can be used to host static content at a cloud provider, and then have mirrors generated for that content +rather than proxying to an external web server. -Simple Proxies --------------- +Once created a static origin will provide you with details on where to upload the web content. -Smart Proxies -------------- +Web Proxies +----------- -Where a simple proxy leads to a broken user experience, it may be necessary to use a smart proxy to mitigate the -brokenness.