scheduled-monitor

A small Flask service that keeps a list of mirror domains and checks, on a schedule, whether they can still be reached from inside a given administrative network domain. Each mirror is tested through two independent vantage points inside that domain — a SOCKS5 proxy and a host reached over SSH — and any mirror that fails is recorded as blocked, with the time it was detected. Downstream tooling can then ask for the mirrors blocked in an administrative network domain since a given time and rotate them out.

How it works

1. Mirrors are pushed into the service with POST /update, each tagged with the origin site it mirrors and the administrative network domains it is intended to serve. They are stored in a local SQLite database.
2. GET /start_timer starts a background thread that wakes every interval seconds and takes a batch of up to 10 unblocked mirrors tagged for CHECKED_COUNTRY.
3. Each mirror is fetched twice:
   • SOCKS5 — the service requests fresh credentials from the proxy's auth endpoint for a randomly chosen port (PROXY_PORT_ROOT + 0–9), then fetches the mirror over socks5h://, so DNS is also resolved inside the domain.
   • SSH — the service logs in to SSH_IP and runs curl -L against the mirror, following redirects, and reads back the final HTTP status code.
4. If either check returns something other than 200, the mirror's blocked_time is set. If both checks fail to run at all (for example the proxy or SSH host is unreachable), the mirror is skipped and left as it was. Every mirror that was tested has its last_checked time updated.
5. GET /blocked returns the mirrors blocked in an administrative network domain, most recent first.

Blocked mirrors are not re-tested once flagged.

Requirements

• Python 3.12 or later
• uv (recommended) for dependency management
• A SOCKS5 proxy inside the administrative network domain being checked, with an HTTP credential endpoint (see Proxy auth endpoint)
• A host inside the same administrative network domain, reachable over SSH with password authentication and with curl installed

Installation

git clone <repository-url> scheduled-monitor
cd scheduled-monitor
uv sync
cp template.env .env

Then edit .env as described below.

Configuration

Settings are read from environment variables, or from a .env file in the working directory.

Variable	Default	Description
DB_PATH	mirrors.db	Path to the SQLite database. Created on first start.
CHECKED_COUNTRY	(empty)	Identifier of the administrative network domain the scheduled checks cover, e.g. IR.
ENABLE_SOCKS	True	Whether the SOCKS settings below are required at start-up.
ENABLE_SSH	True	Whether the SSH settings below are required at start-up.
PROXY_HOST	—	Hostname or IP of the SOCKS5 proxy and its auth endpoint.
PROXY_PORT_ROOT	—	First of ten consecutive SOCKS ports (PROXY_PORT_ROOT to PROXY_PORT_ROOT + 9).
PROXY_AUTH_PORT	—	Port of the proxy's HTTP auth endpoint.
PROXY_AUTH_ENDPOINT	—	Path of the auth endpoint, e.g. /auth.
PROXY_AUTH_KEY	—	Key sent to the auth endpoint as the k query parameter.
SSH_IP	—	Address of the SSH host inside the administrative network domain.
SSH_USERNAME	—	SSH username.
SSH_PASSWORD	—	SSH password. Key-based authentication is not used.

The service refuses to start if a check is enabled but any of its settings are empty. PROXY_PORT_ROOT and PROXY_AUTH_PORT must always be set to integers.

Warning

The SSH client accepts any host key on first connection (AutoAddPolicy). Only point SSH_IP at a host you control and trust the network path to.

Proxy auth endpoint

Before each SOCKS check the service calls:

GET http://{PROXY_HOST}:{PROXY_AUTH_PORT}{PROXY_AUTH_ENDPOINT}?k={PROXY_AUTH_KEY}&port={port}

and expects a 200 response with a JSON body containing the credentials for that port:

{ "socksuser": "user", "sockspass": "pass" }

Running

For development:

uv run flask --app app run

For anything longer-lived, run it under a WSGI server with a single worker process — the scheduler is a thread inside the process, so extra workers would each keep their own timer state. For example:

uv run --with gunicorn gunicorn --workers 1 --threads 4 --bind 127.0.0.1:5000 app:app

The API has no authentication, so bind it to localhost or put it behind a reverse proxy that restricts access.

API

POST /update

Adds mirrors. Mirrors that are already known (matched on mirror_domain) are left unchanged.

curl -X POST http://127.0.0.1:5000/update \
  -H 'Content-Type: application/json' \
  -d '{
    "mappings": {
      "mirror1.example.net": {
        "origin_domain": "www.example.com",
        "valid_from": "2026-09-01T00:00:00+00:00",
        "countries": { "IR": 1, "RU": 1 }
      }
    }
  }'

Field	Required	Description
origin_domain	no	The site this mirror serves.
valid_from	no	ISO 8601 timestamp from which the mirror is valid. Stored, but not currently used to filter checks.
countries	no	Object keyed by administrative network domain identifier; only the keys are used.

Returns Updated.

GET /blocked

Lists mirrors blocked in an administrative network domain.

Parameter	Required	Description
country	yes	Administrative network domain identifier.
time_since	no	ISO 8601 timestamp; only mirrors blocked at or after this time are returned.

curl 'http://127.0.0.1:5000/blocked?country=IR&time_since=2026-09-28T00:00:00%2B00:00'

Returns a JSON array of [id, mirror_domain, origin_domain, blocked_time] rows, newest first.

GET /start_timer

Starts the scheduled checks for the administrative network domain set in CHECKED_COUNTRY.

Parameter	Required	Description
interval	no	Seconds between batches. Defaults to 10.

Only one timer runs at a time; calling this while one is running reports the existing interval. The timer does not survive a restart, so call this again after the service starts.

GET /stop_timer

Stops the scheduled checks, waiting for any batch in progress to finish.

Database

A single mirrors table in SQLite. Timestamps are stored as Unix epoch seconds and the list of administrative network domains as JSON.

Column	Description
id	Primary key.
added	When the mirror was added.
valid_from	Optional validity start, from /update.
last_checked	When the mirror was last tested (0 if never).
mirror_domain	The mirror's domain (unique).
origin_domain	The site it mirrors.
countries	JSON list of administrative network domain identifiers the mirror serves.
blocked_time	When the mirror was first found blocked, or NULL.

Development

uv sync                  # installs the dev group: pytest, ruff, ty
uv run ruff format .
uv run ruff check .
uv run ty check
uv run pytest

Licence

Copyright © 2026 SR2 Communications Limited.

Released under the BSD 2-Clause Licence. See LICENSE for the full text.