87 lines
4.2 KiB
ReStructuredText
87 lines
4.2 KiB
ReStructuredText
Web Mirrors
|
|
===========
|
|
|
|
Background
|
|
----------
|
|
|
|
A web mirror can help users by providing alternate URLs to access censored resources, allowing them to bypass technical censorship and access information that may be otherwise unavailable.
|
|
Web proxies work by forwarding requests to the original website, and providing a different URL to access that content.
|
|
By accessing the web proxy URL, users can access the same content that would be blocked by censors if accessed through the original website's URL.
|
|
|
|
Web mirrors also use frequently changing URLs to evade censorship, making it more difficult for censors to block access to the content.
|
|
This assumption of a limited lifetime is built-in to the system, allowing for automated block detection to trigger the
|
|
deployment of new URLs, and for the :doc:`distribution lists <lists>` to allow applications and end-users to discover
|
|
new URLs.
|
|
|
|
Additionally, web proxies can be accessed via a normal web browser, making them easily accessible to users without requiring any special software or technical knowledge.
|
|
|
|
Architecture
|
|
------------
|
|
|
|
Before configuring any web origins to create mirrors, you must first configure an origin group.
|
|
The group is used to organise larger numbers of web origins so that mirrors can be created for them with a shared configuration.
|
|
The shared configuration will include which pools will require mirrors for the origins.
|
|
|
|
For each configured origin group, a smart proxy instance will be created at each provider where proxies are deployed.
|
|
This instance runs the active proxy that will rewrite URLs in responses from the origin webserver to ensure that all resources are loaded via the mirrors.
|
|
Some providers, e.g. Fastly, do not require an instance as they already provide the means for URL re-writing in their standard functionality.
|
|
|
|
Web origins represent the source website that has been blocked.
|
|
|
|
Web Origins
|
|
-----------
|
|
|
|
Rate Limiting
|
|
^^^^^^^^^^^^^
|
|
|
|
CDNs (Content Delivery Networks) can impose rate limiting on websites to ensure that the network resources are
|
|
efficiently utilized, to protect the websites from DDoS (Distributed Denial of Service) attacks and to maintain the
|
|
quality of service for all the websites using the CDN.
|
|
|
|
These rate limits will be sized according to the expected rate of requests from an average user, however the mirror
|
|
system is a bottleneck that aggregates requests from multiple users and passes these on to the original CDN.
|
|
When a single system is used to send a large number of requests to a CDN like this, the CDN may interpret this as a
|
|
denial of service (DoS) attack and prevent access to the website.
|
|
|
|
Deploying mirrors for websites hosted on CDNs will require co-operation from the CDN provider. If you find that mirrors
|
|
are producing many "Rate Limited Exceeded" or "Access Denied" errors then you may be suffering from this problem. Ask
|
|
your administrator to configure the `Bypass-Rate-Limit-Token` header in the portal and at the CDN to disable the rate
|
|
limiting for requests originating via the mirrors.
|
|
|
|
New Web Origin
|
|
^^^^^^^^^^^^^^
|
|
|
|
To create a new web origin, click "Create new origin" at the top of the list. This will present
|
|
you with the new origin form:
|
|
|
|
.. image:: /_static/origin/new.png
|
|
:width: 800
|
|
|
|
Domain Name
|
|
"""""""""""
|
|
|
|
Enter the domain name that you want to create a mirror for in this field. This is a required field.
|
|
|
|
Description
|
|
"""""""""""
|
|
|
|
Enter a brief description of the website that you are creating a mirror for in this field. This is also a required field.
|
|
|
|
Group
|
|
"""""
|
|
|
|
Select the group that you want the mirror to belong to from the drop-down menu in this field. This is a required field.
|
|
|
|
Auto-Rotate
|
|
"""""""""""
|
|
|
|
Select this field if you want to enable auto-rotation for the mirror. This means that the mirror will automatically
|
|
redeploy with a new domain name if it is detected to be blocked. This field is optional and is enabled by default.
|
|
|
|
Asset Domain
|
|
""""""""""""
|
|
|
|
Select this field if you want to use mirrors of this domain to host assets for other domains. This is an optional field
|
|
and is disabled by default. All of the asset domains within a group will be available for all mirrors using smart
|
|
proxies. The smart proxy will rewrite URLs for the asset domains that are included in the source code of the mirror
|
|
hosted by a smart proxy.
|