AnchorHost.eu/anchorhost-backend
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(domains): migrate user namespaces to wildcard CNAMEs
Build and publish Docker image / build-and-publish (push) Successful in 8s
Details

Browse Source
This commit is contained in:
wfalkowski committed 2026-10-20 18:10:23 +00:00
1 parent 82fa50e9d5
commit 2768f5f97d
6 files changed
+503 -296
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

DOMAIN_SYSTEM_PLAN.md
+53
View File
@@ -0,0 +1,53 @@
# Domain System Plan
## Current user namespace
The domain manager provisions two DNS records in the existing `anchorhost.eu` zone for each username:
- `u-<username>.anchorhost.eu CNAME anchorhost.eu.`
- `*.u-<username>.anchorhost.eu CNAME anchorhost.eu.`
Users claim individual hostnames such as `play.u-<username>.anchorhost.eu` in the application database. The wildcard record makes each single-label hostname resolve without another DNS write. Claims remain necessary for ownership, duplicate prevention, and service linking. DNS resolution alone does not create a Traefik route.
The DNS API refuses to replace a conflicting record at the namespace owner. This matters for accounts that still have an NS delegation from the previous design: an NS delegation and a CNAME cannot coexist at the same name. Those namespaces require an explicit migration.
## Automatic domain for every service
### 1. Allocate a stable service hostname
When a service is created, derive a DNS-safe label from its display name and reserve it under the owner's namespace. Enforce a unique constraint on the normalized full hostname; on collision, add a short stable suffix. Persist the assigned hostname against the service record, and release it only when the service is permanently deleted. Renames should update the display name without silently changing the public hostname.
### 2. Publish a service-ready state
Only publish a hostname after the service, network endpoint, and platform route are ready. Show the assigned address and readiness state in the dashboard. If provisioning fails, retain the reservation for retry and surface a useful error rather than leaving a hostname that resolves to a dead endpoint.
### 3. Route web services
For HTTP/HTTPS services, create a Traefik `IngressRoute` from the assigned `Host()` name to the service's Kubernetes Service and declared HTTP port. Validate ownership and namespace before creating or deleting routes. Provision TLS using a certificate that covers the user namespace; wildcard certificates require DNS-01 with a provider credential, while HTTP-01 cannot issue wildcard certificates. Keep the DNS provider token out of user workloads and API responses.
### 4. Handle game ports by protocol
DNS does not carry a TCP/UDP port. For Minecraft Java, publish an `_minecraft._tcp.<service-host>` SRV record with the allocated external port and point it at the service hostname; Java clients can then connect without users typing the port. Verify client behavior and ensure the SRV target has an address record. Minecraft Bedrock and arbitrary TCP/UDP services need protocol-specific support or a proxy that can multiplex traffic. Traefik TCP SNI routing only applies when clients send usable TLS SNI; it is not a general raw TCP port multiplexer. Until a compatible proxy or a direct port is available, continue showing the allocated port.
### 5. Lifecycle and safety
- Create routes and DNS metadata idempotently, then verify ready Service endpoints before marking a domain active.
- Remove only resources whose labels/annotations and database ownership identify the service being deleted.
- On stop, preserve the hostname but show the service as offline; on delete, remove route and SRV metadata before releasing the name.
- Rate-limit hostname claims and validate every label, hostname length, user ownership, and service ownership server-side.
- Record provisioning failures and cleanup outcomes without logging provider tokens or other secrets.
## Migration from per-user delegated zones
The one-time migration is implemented by `scripts/migrate_delegated_dns.py`. It defaults to a read-only dry run, checks for conflicts, copies child-zone RRsets into the parent zone, removes parent DS/NS delegation records, creates the namespace and wildcard CNAMEs, and updates existing platform claim metadata. It leaves child zones intact as a rollback source. Run it from the backend environment and review the complete dry-run output before passing `--apply`.
Do not remove existing NS or DS records as part of an ordinary domain claim. The migration is idempotent and stops before writing if it finds conflicting records or child-zone apex data that cannot coexist with the required CNAME. After cutover, verify DNSSEC and public resolution before retiring any child zone. Resolver caches may retain old delegation or record data for the previous TTL.
## Acceptance checks before enabling automatic service names
- A new account's base and wildcard names resolve to the platform edge after DNS cache expiry.
- Existing names under a legacy NS delegation are detected and never overwritten.
- A duplicate service label cannot be claimed by another service or user.
- HTTP hostnames reach only the owning service and receive a valid certificate.
- Minecraft Java SRV lookup returns the allocated external port; other protocols retain an explicit port unless a tested proxy supports portless access.
- Service stop, restart, rename, and deletion leave DNS, route, and database state consistent.
background_tasks.py
-47
View File
@@ -3,7 +3,6 @@
from currency_service import fetch_exchange_rates
from database_service import database_connection, dict_from_row
from domain_runtime import domain_manager
from domain_service import cleanup_orphaned_ingressroutes
from kubernetes_service import k8s_manager
import os
@@ -175,52 +174,6 @@ def initialize_sample_data():
else:
logger.info(f"✓ Sample user exists: {existing_user['username']}")
# CREATE USER DOMAIN IN DNS IF IT DOESN'T EXIST
if domain_manager and existing_user:
try:
logger.info(f"Checking/Creating domain for user: u-{existing_user['username']}.{domain_manager.base_domain}")
# Check if user domain exists
user_domain = f"u-{existing_user['username']}.{domain_manager.base_domain}"
domain_exists = domain_manager._check_domain_exists(user_domain)
if domain_exists is None:
logger.error(f"Error checking domain {user_domain}")
elif not domain_exists:
logger.info(f"User domain doesn't exist, creating: {user_domain}")
success = domain_manager._create_domain(user_domain)
if success:
logger.info(f"✓ Created user domain: {user_domain}")
# SETUP DNSSEC DELEGATION for new domain
logger.info("Setting up DNSSEC delegation...")
dnssec_result = domain_manager.setup_dnssec_delegation(existing_user['username'])
if dnssec_result.get('success'):
logger.info(f"✓ DNSSEC delegation configured")
else:
logger.warning(f"⚠ DNSSEC setup issues: {dnssec_result.get('message')}")
# Also create a default subdomain for testing
test_result = domain_manager.create_subdomain(existing_user['username'], 'test')
if test_result.get('success'):
logger.info(f"✓ Created test subdomain for user")
else:
logger.error(f"✗ Failed to create test subdomain: {test_result.get('message')}")
else:
logger.error(f"✗ Failed to create user domain: {user_domain}")
else:
logger.info(f"✓ User domain already exists: {user_domain}")
# ENSURE DNSSEC IS PROPERLY CONFIGURED for existing domain
logger.info("Ensuring DNSSEC is properly configured...")
dnssec_result = domain_manager.setup_dnssec_delegation(existing_user['username'])
if dnssec_result.get('success'):
logger.info(f"✓ DNSSEC properly configured")
else:
logger.warning(f"⚠ DNSSEC configuration issues: {dnssec_result.get('message')}")
except Exception as e:
logger.error(f"Error creating user domain: {e}")
if k8s_manager.connected:
try:
k8s_manager.create_namespace('user-testuser')
domain_manager.py
+145 -152
View File
@@ -6,6 +6,7 @@ import re
import requests
import time
from typing import Dict, List, Optional
from urllib.parse import quote
@@ -453,7 +454,10 @@ class DomainManager:
def create_subdomain(self, username: str, subdomain_name: str) -> Dict:
"""
Create a subdomain for user with proper DNSSEC setup
Claim a hostname below a user's namespace.
DNS is provisioned once per user with a namespace CNAME and wildcard
CNAME in the parent zone. Individual hostnames are application records.
Returns: dict with 'success', 'message', and optionally 'domain_data'
"""
@@ -470,7 +474,10 @@ class DomainManager:
username = username.lower().strip()
subdomain_name = subdomain_name.lower().strip()
# Validate subdomain format
if not self._is_valid_username_label(username):
return {'success': False, 'message': "Username cannot be represented as a DNS name", 'domain_data': None}
# Hostnames are single labels below u-<username>.
if not self._is_valid_subdomain(subdomain_name):
return {
'success': False,
@@ -481,112 +488,24 @@ class DomainManager:
# Construct domains
user_domain = f"u-{username}.{self.base_domain}"
full_subdomain = f"{subdomain_name}.{user_domain}"
namespace = self.ensure_user_namespace(username)
if not namespace['success']:
return {'success': False, 'message': namespace['message'], 'domain_data': None}
logger.info("="*60)
logger.info(f"Creating subdomain: {full_subdomain}")
logger.info(f"User domain: {user_domain}")
logger.info(f"Subdomain: {subdomain_name}")
logger.info(f"Target IP: {self.node_ip}")
logger.info("="*60)
# Step 1: Ensure user's domain exists
logger.info(f"Step 1: Checking if user domain exists: {user_domain}")
user_domain_exists = self._check_domain_exists(user_domain)
logger.info(f"Domain exists: {user_domain_exists}")
if user_domain_exists is None:
return {
'success': False,
'message': f"Error checking domain {user_domain}",
'domain_data': None
}
if not user_domain_exists:
logger.info(f"Step 2: Creating user domain: {user_domain}")
domain_created = self._create_domain(user_domain)
if not domain_created:
return {
'success': False,
'message': f"Failed to create user domain {user_domain}",
'domain_data': None
}
logger.info(f"✓ User domain created: {user_domain}")
# Step 3: Setup DNSSEC delegation for the new domain
logger.info(f"Step 3: Setting up DNSSEC delegation...")
dnssec_result = self.setup_dnssec_delegation(username)
if not dnssec_result.get('success'):
logger.warning(f"⚠ DNSSEC setup had issues: {dnssec_result.get('message')}")
# Continue anyway, as DNSSEC is not critical for basic functionality
else:
logger.info(f"✓ User domain already exists: {user_domain}")
# Check if DNSSEC is properly configured
logger.info("Checking DNSSEC configuration...")
dnssec_result = self.setup_dnssec_delegation(username)
if not dnssec_result.get('success'):
logger.warning(f"⚠ DNSSEC check had issues: {dnssec_result.get('message')}")
# Step 4: Create the A record for the subdomain
logger.info(f"Step 4: Creating A record for subdomain: {subdomain_name}")
logger.info(f"Target: {subdomain_name}.{user_domain} → {self.node_ip}")
url = f"{self.base_url}/domains/{user_domain}/rrsets/"
data = {
"subname": subdomain_name,
"type": "A",
"ttl": 3600,
"records": [self.node_ip]
domain_info = {
'domain': full_subdomain,
'subdomain': subdomain_name,
'username': username,
'record_type': 'CNAME',
'ip_address': self.base_domain,
'ttl': 3600,
'created_at': datetime.now().isoformat()
}
return {
'success': True,
'message': f"Domain {full_subdomain} is ready",
'domain_data': domain_info
}
logger.info(f"POST URL: {url}")
response = requests.post(url, headers=self.headers, json=data, timeout=30)
logger.info(f"Response Status: {response.status_code}")
if response.status_code in [200, 201]:
domain_info = {
'domain': full_subdomain,
'subdomain': subdomain_name,
'username': username,
'record_type': 'A',
'ip_address': self.node_ip,
'ttl': 3600,
'created_at': datetime.now().isoformat()
}
logger.info("="*60)
logger.info(f"✅ SUCCESS: Subdomain created!")
logger.info(f"Domain: {full_subdomain}")
logger.info(f"Points to: {self.node_ip}")
logger.info("="*60)
# Step 5: Create www CNAME record
self._create_www_cname(username, subdomain_name)
return {
'success': True,
'message': f"Subdomain {full_subdomain} created successfully",
'domain_data': domain_info
}
else:
error_msg = f"API error: {response.status_code} - {response.text}"
logger.error("="*60)
logger.error(f"❌ FAILED to create subdomain")
logger.error(f"Status: {response.status_code}")
logger.error(f"Error: {response.text}")
logger.error("="*60)
return {
'success': False,
'message': error_msg,
'domain_data': None
}
except requests.exceptions.RequestException as e:
logger.error(f"Network error creating subdomain: {e}")
@@ -603,30 +522,69 @@ class DomainManager:
'domain_data': None
}
def _create_www_cname(self, username: str, subdomain_name: str):
"""Create www CNAME record for the subdomain"""
try:
user_domain = f"u-{username}.{self.base_domain}"
def ensure_user_namespace(self, username: str) -> Dict:
"""Ensure the base and wildcard CNAME records exist for a user."""
username = (username or '').lower().strip()
if not self._is_valid_username_label(username):
return {'success': False, 'message': "Username cannot be represented as a DNS name"}
# Create www.subdomain CNAME pointing to the subdomain
cname_data = {
"subname": f"www.{subdomain_name}",
user_domain = f"u-{username}.{self.base_domain}"
for owner in (f"u-{username}", f"*.u-{username}"):
if not self._ensure_namespace_cname(owner):
return {
'success': False,
'message': f"Could not safely provision DNS for {user_domain}; check for existing DNS records or a legacy delegation",
}
return {'success': True, 'message': f"DNS namespace is ready for {user_domain}"}
def _ensure_namespace_cname(self, owner: str) -> bool:
"""Create an idempotent CNAME in the configured parent zone."""
try:
path = f"{self.base_url}/domains/{self.base_domain}/rrsets/{quote(owner, safe='.-*')}/CNAME/"
response = requests.get(path, headers=self.headers, timeout=30)
if response.status_code == 200:
records = response.json().get('records', [])
return records == [f"{self.base_domain}."] or records == [self.base_domain]
if response.status_code != 404:
logger.error("Could not inspect DNS record %s (%s)", owner, response.status_code)
return False
# CNAME cannot coexist with other record types at the same owner.
for record_type in ('A', 'AAAA', 'TXT', 'MX', 'NS', 'DS', 'CAA', 'SRV', 'TLSA', 'HTTPS', 'SVCB'):
conflict = requests.get(
f"{self.base_url}/domains/{self.base_domain}/rrsets/{quote(owner, safe='.-*')}/{record_type}/",
headers=self.headers,
timeout=30,
)
if conflict.status_code == 200:
logger.error("Refusing to replace existing %s record at %s", record_type, owner)
return False
if conflict.status_code != 404:
logger.error("Could not inspect %s record at %s (%s)", record_type, owner, conflict.status_code)
return False
payload = {
"subname": owner,
"type": "CNAME",
"ttl": 3600,
"records": [f"{subdomain_name}.{user_domain}."] # Note the trailing dot
"records": [f"{self.base_domain}."]
}
url = f"{self.base_url}/domains/{user_domain}/rrsets/"
response = requests.post(url, headers=self.headers, json=cname_data, timeout=30)
if response.status_code in [200, 201]:
logger.info(f"✓ Created www CNAME for {subdomain_name}.{user_domain}")
else:
logger.warning(f"Failed to create www CNAME: {response.status_code}")
except Exception as e:
logger.error(f"Error creating www CNAME: {e}")
created = requests.post(
f"{self.base_url}/domains/{self.base_domain}/rrsets/",
headers=self.headers,
json=payload,
timeout=30,
)
if created.status_code in (200, 201):
return True
logger.error("Could not create CNAME %s (%s)", owner, created.status_code)
return False
except requests.exceptions.RequestException as exc:
logger.error("DNS API request failed for %s: %s", owner, exc)
return False
except (ValueError, TypeError) as exc:
logger.error("Invalid DNS API response for %s: %s", owner, exc)
return False
def _check_domain_exists(self, domain_name: str) -> Optional[bool]:
"""Check if a domain exists in deSEC"""
@@ -681,9 +639,53 @@ class DomainManager:
username = username.lower().strip()
subdomain_name = subdomain_name.lower().strip()
if not self._is_valid_username_label(username) or not self._is_valid_subdomain(subdomain_name):
return {'success': False, 'message': "Invalid username or subdomain"}
user_domain = f"u-{username}.{self.base_domain}"
full_subdomain = f"{subdomain_name}.{user_domain}"
# New namespaces use a shared wildcard in the parent zone. Do not
# remove shared DNS or stale child-zone data when releasing a claim.
parent_cname = requests.get(
f"{self.base_url}/domains/{self.base_domain}/rrsets/{quote(f'u-{username}', safe='.-')}/CNAME/",
headers=self.headers,
timeout=30,
)
if parent_cname.status_code == 200:
records = parent_cname.json().get('records', [])
if records in ([f"{self.base_domain}."], [self.base_domain]):
for owner in (f"{subdomain_name}.u-{username}", f"www.{subdomain_name}.u-{username}"):
rrsets = requests.get(
f"{self.base_url}/domains/{self.base_domain}/rrsets/",
headers=self.headers,
params={'subname': owner},
timeout=30,
)
if rrsets.status_code != 200:
return {'success': False, 'message': f"Could not inspect DNS records for {owner}"}
for rrset in rrsets.json():
delete_url = (
f"{self.base_url}/domains/{self.base_domain}/rrsets/"
f"{quote(owner, safe='.-*')}/{rrset['type']}/"
)
deleted = requests.delete(delete_url, headers=self.headers, timeout=30)
if deleted.status_code not in (200, 204, 404):
return {'success': False, 'message': f"Could not remove DNS record {owner} {rrset['type']}"}
return {'success': True, 'message': f"Domain claim {full_subdomain} deleted"}
elif parent_cname.status_code != 404:
return {'success': False, 'message': f"Could not inspect parent DNS for {user_domain}"}
# Existing delegated domains retain their old record cleanup until
# their namespace has been migrated to the parent-zone CNAME model.
zone_exists = self._check_domain_exists(user_domain)
if zone_exists is False:
return {
'success': True,
'message': f"Domain claim {full_subdomain} deleted"
}
if zone_exists is None:
return {'success': False, 'message': f"Could not inspect legacy zone {user_domain}"}
logger.info("="*60)
logger.info(f"Deleting subdomain: {full_subdomain}")
logger.info("="*60)
@@ -830,12 +832,20 @@ class DomainManager:
return True
def _is_valid_username_label(self, username: str) -> bool:
"""Validate the username's DNS path while preserving its dot separators."""
labels = f"u-{username}".split('.')
label_pattern = r"[a-z0-9](?:[a-z0-9-]*[a-z0-9])?"
return (
all(len(label) <= 63 and re.fullmatch(label_pattern, label) for label in labels)
and len('.'.join(labels) + f".{self.base_domain}") <= 253
)
def check_domain_availability(self, subdomain_name: str, username: str) -> Dict:
"""Check if a subdomain is available for the user"""
try:
username = username.lower().strip()
subdomain_name = subdomain_name.lower().strip()
user_domain = f"u-{username}.{self.base_domain}"
# First validate the subdomain name
if not self._is_valid_subdomain(subdomain_name):
@@ -844,32 +854,15 @@ class DomainManager:
'message': f"Invalid subdomain name: {subdomain_name}"
}
# Check if user domain exists first
if not self._check_domain_exists(user_domain):
return {
'available': True,
'message': f"Subdomain {subdomain_name} is available"
}
if not self._is_valid_username_label(username):
return {'available': False, 'message': "Username cannot be represented as a DNS name"}
# Check if the specific A record exists
url = f"{self.base_url}/domains/{user_domain}/rrsets/{subdomain_name}/A/"
response = requests.get(url, headers=self.headers, timeout=30)
if response.status_code == 404:
return {
'available': True,
'message': f"Subdomain {subdomain_name} is available"
}
elif response.status_code == 200:
return {
'available': False,
'message': f"Subdomain {subdomain_name} already exists"
}
else:
return {
'available': False,
'message': f"API error: {response.status_code}"
}
# Individual hostnames are claims stored in user_domains. Wildcard
# DNS means DNS existence cannot determine whether a name is free.
return {
'available': True,
'message': f"Subdomain {subdomain_name} is available"
}
except Exception as e:
return {
routes/domains.py
+5 -19
View File
@@ -20,7 +20,7 @@ bp = Blueprint("domains", __name__)
@bp.route('/api/dnssec/fix-all', methods=['POST'])
def fix_all_dnssec_api():
"""Fix DNSSEC delegation for all user domains"""
"""Retired: DNSSEC now belongs to the parent zone, not user namespaces."""
try:
token = get_bearer_token()
user = authenticate_user(token)
@@ -29,16 +29,7 @@ def fix_all_dnssec_api():
if not user or user.get('role') != 'admin':
return jsonify({'error': 'Admin access required'}), 403
if not domain_manager:
return jsonify({'error': 'Domain manager not available'}), 503
# Run the fix
domain_manager.fix_all_dnssec_delegations()
return jsonify({
'success': True,
'message': 'DNSSEC fix process initiated. Check logs for details.'
}), 200
return jsonify({'error': 'Per-user DNSSEC delegation is retired; DNSSEC is managed on the parent zone.'}), 410
except Exception as e:
logger.error(f"Error fixing DNSSEC: {e}")
@@ -46,7 +37,7 @@ def fix_all_dnssec_api():
@bp.route('/api/dnssec/verify/<username>', methods=['GET'])
def verify_dnssec_api(username):
"""Verify DNSSEC delegation for a user"""
"""Retired: DNSSEC is inherited from the parent zone."""
try:
token = get_bearer_token()
user = authenticate_user(token)
@@ -55,12 +46,7 @@ def verify_dnssec_api(username):
if not user or (user.get('role') != 'admin' and user.get('username') != username):
return jsonify({'error': 'Unauthorized'}), 403
if not domain_manager:
return jsonify({'error': 'Domain manager not available'}), 503
result = domain_manager.setup_dnssec_delegation(username)
return jsonify(result), 200 if result.get('success') else 500
return jsonify({'error': 'Per-user DNSSEC delegation is retired; DNSSEC is managed on the parent zone.'}), 410
except Exception as e:
logger.error(f"Error verifying DNSSEC: {e}")
@@ -103,7 +89,7 @@ def sync_domains_api():
domains = get_user_domains(user['id'], True)
return jsonify({
'success': True,
'message': 'Domains synced successfully',
'message': 'DNS namespace verified and wildcard records are ready',
'domains': domains,
'count': len(domains)
}), 200
scripts/migrate_delegated_dns.py
+297
View File
@@ -0,0 +1,297 @@
#!/usr/bin/env python3
"""Move legacy per-user deSEC zones into the parent zone's CNAME wildcard model.
Run from the backend environment. The default is a read-only dry run; pass
``--apply`` to copy records and cut over delegations. Child zones are retained
so their records remain available for rollback.
"""
import argparse
import os
import re
import sys
import time
from typing import Dict, Iterable, List, Tuple
from urllib.parse import quote
import requests
def _load_usernames() -> List[str]:
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from database_service import database_connection
with database_connection() as connection:
cursor = connection.cursor()
cursor.execute(
"""
SELECT DISTINCT u.username
FROM users u
JOIN user_domains d ON d.user_id = u.id
WHERE COALESCE(d.is_custom, FALSE) = FALSE
AND d.full_domain IS NOT NULL
"""
)
return [row['username'] for row in cursor.fetchall()]
def _valid_namespace(username: str, base_domain: str) -> str:
namespace = f"u-{username.strip().lower()}"
labels = namespace.split('.')
pattern = re.compile(r"[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\Z")
if not all(len(label) <= 63 and pattern.fullmatch(label) for label in labels):
raise ValueError(f"Username cannot form a DNS namespace: {username!r}")
if len(f"{namespace}.{base_domain}") > 253:
raise ValueError(f"Username namespace is too long: {username!r}")
return namespace
class DeSec:
def __init__(self, token: str, base_domain: str):
self.base_url = 'https://desec.io/api/v1'
self.base_domain = base_domain.rstrip('.').lower()
self.headers = {'Authorization': f'Token {token}', 'Content-Type': 'application/json'}
self._last_request = 0.0
def request(self, method: str, url: str, **kwargs) -> requests.Response:
for attempt in range(6):
delay = 0.2 - (time.monotonic() - self._last_request)
if delay > 0:
time.sleep(delay)
response = requests.request(method, url, headers=self.headers, timeout=30, **kwargs)
self._last_request = time.monotonic()
if response.status_code != 429:
return response
retry_after = response.headers.get('Retry-After', '2')
try:
wait = min(max(float(retry_after), 1), 30)
except ValueError:
wait = 2
if attempt == 5:
return response
time.sleep(wait)
raise RuntimeError('DNS API retry loop ended unexpectedly')
def list_rrsets(self, zone: str) -> List[Dict]:
url = f'{self.base_url}/domains/{zone}/rrsets/'
rrsets: List[Dict] = []
while url:
response = self.request('GET', url)
if response.status_code != 200:
raise RuntimeError(f'Could not read RRsets for {zone} ({response.status_code})')
result = response.json()
if not isinstance(result, list):
raise RuntimeError(f'Unexpected RRset response for {zone}')
rrsets.extend(result)
url = response.links.get('next', {}).get('url')
return rrsets
def rrset_url(self, subname: str, record_type: str) -> str:
return (
f'{self.base_url}/domains/{self.base_domain}/rrsets/'
f'{quote(subname, safe=".-_*.")}/{record_type}/'
)
def _rrset_records(rrset: Dict) -> Tuple[str, ...]:
return tuple(sorted(str(record).rstrip('.').lower() if rrset.get('type') == 'CNAME' else str(record)
for record in rrset.get('records', [])))
def _relative_owner(zone: str, base_domain: str) -> str:
suffix = f'.{base_domain}'
if not zone.endswith(suffix):
raise ValueError(f'Zone is outside configured parent domain: {zone}')
owner = zone[:-len(suffix)]
_validate_owner(owner)
return owner
def _validate_owner(owner: str) -> None:
labels = owner.split('.')
pattern = re.compile(r"(?:\*|[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)\Z")
if (
not all(len(label) <= 63 and pattern.fullmatch(label) for label in labels)
or ('*' in labels and (labels[0] != '*' or labels.count('*') != 1))
):
raise ValueError(f'Unsupported legacy namespace owner: {owner!r}')
def build_plan(api: DeSec) -> Tuple[List[str], List[Dict], Dict[Tuple[str, str], Dict]]:
zones_response = api.request('GET', f'{api.base_url}/domains/')
if zones_response.status_code != 200:
raise RuntimeError(f'Could not list deSEC zones ({zones_response.status_code})')
zones = zones_response.json()
legacy_zones = sorted(
zone['name'] for zone in zones
if zone.get('name', '').endswith(f'.{api.base_domain}')
and zone.get('name', '').split('.', 1)[0].startswith('u-')
)
parent_rrsets = api.list_rrsets(api.base_domain)
parent_map = {(item.get('subname') or '', item['type']): item for item in parent_rrsets}
namespaces = {_relative_owner(zone, api.base_domain) for zone in legacy_zones}
namespaces.update(
item.get('subname', '') for item in parent_rrsets
if item.get('type') in {'NS', 'DS'} and (item.get('subname') or '').startswith('u-')
)
for username in _load_usernames():
namespaces.add(_valid_namespace(username, api.base_domain))
copies: List[Dict] = []
for zone in legacy_zones:
namespace = _relative_owner(zone, api.base_domain)
for rrset in api.list_rrsets(zone):
child_name = rrset.get('subname') or ''
record_type = rrset['type']
if child_name in {'', '@'} and record_type == 'NS':
continue
if child_name in {'', '@'}:
raise RuntimeError(
f'Cannot preserve apex {record_type} at {zone}; manual DNS migration is required'
)
target_name = f'{child_name}.{namespace}'
_validate_owner(target_name)
key = (target_name, record_type)
existing = parent_map.get(key)
other_types = {
item['type'] for item in parent_rrsets
if item.get('subname', '') == target_name and item['type'] != record_type
}
if record_type == 'CNAME' and other_types:
raise RuntimeError(f'Conflicting parent RRset at {target_name}: {sorted(other_types)}')
if 'CNAME' in other_types:
raise RuntimeError(f'Parent CNAME conflicts with child RRset at {target_name} {record_type}')
if existing and _rrset_records(existing) != _rrset_records(rrset):
raise RuntimeError(f'Conflicting parent RRset at {target_name} {record_type}')
if not existing:
copies.append({
'subname': target_name,
'type': record_type,
'ttl': rrset['ttl'],
'records': rrset.get('records', []),
})
for namespace in sorted(namespaces):
for owner in (namespace, f'*.{namespace}'):
_validate_owner(owner)
current_at_owner = [item for item in parent_rrsets if item.get('subname', '') == owner]
for item in current_at_owner:
if item['type'] in {'NS', 'DS'} and owner == namespace:
continue
if item['type'] == 'CNAME' and _rrset_records(item) == (api.base_domain,):
continue
raise RuntimeError(f'Conflicting parent RRset at {owner} {item["type"]}')
return sorted(namespaces), copies, parent_map
def _delete_rrset(api: DeSec, subname: str, record_type: str) -> None:
response = api.request('DELETE', api.rrset_url(subname, record_type))
if response.status_code not in {200, 204, 404}:
raise RuntimeError(f'Could not remove parent {record_type} at {subname} ({response.status_code})')
def _create_rrset(api: DeSec, payload: Dict) -> None:
response = api.request('POST', f'{api.base_url}/domains/{api.base_domain}/rrsets/', json=payload)
if response.status_code not in {200, 201}:
raise RuntimeError(
f'Could not create parent {payload["type"]} at {payload["subname"]} ({response.status_code})'
)
def apply_plan(api: DeSec, namespaces: Iterable[str], copies: List[Dict],
parent_map: Dict[Tuple[str, str], Dict]) -> None:
# Copy child records and create wildcard records first; old delegations stay
# authoritative until all non-destructive preparation has completed.
for payload in copies:
_create_rrset(api, payload)
print(f'Copied {payload["type"]} {payload["subname"]}')
for namespace in namespaces:
wildcard = f'*.{namespace}'
if (wildcard, 'CNAME') not in parent_map:
_create_rrset(api, {
'subname': wildcard, 'type': 'CNAME', 'ttl': 3600,
'records': [f'{api.base_domain}.'],
})
print(f'Created wildcard CNAME {wildcard}')
for namespace in namespaces:
# Remove DS before NS so a resolver never sees a signed delegation with
# no matching child zone. Preserve exact RRsets for a one-step rollback.
saved = {record_type: parent_map.get((namespace, record_type))
for record_type in ('DS', 'NS')}
try:
for record_type in ('DS', 'NS'):
if saved[record_type]:
_delete_rrset(api, namespace, record_type)
if (namespace, 'CNAME') not in parent_map:
_create_rrset(api, {
'subname': namespace, 'type': 'CNAME', 'ttl': 3600,
'records': [f'{api.base_domain}.'],
})
print(f'Cut over {namespace}')
except Exception:
# The old child zones remain intact. Restore delegation metadata if
# the final CNAME write fails partway through a cutover.
for record_type in ('NS', 'DS'):
rrset = saved[record_type]
if rrset:
_create_rrset(api, {
'subname': namespace,
'type': record_type,
'ttl': rrset['ttl'],
'records': rrset['records'],
})
raise
from database_service import database_connection
with database_connection() as connection:
cursor = connection.cursor()
for namespace in namespaces:
pattern = f'%.{namespace}.{api.base_domain}'
pattern = pattern.replace('!', '!!').replace('%', '!%').replace('_', '!_')
cursor.execute(
"""
UPDATE user_domains
SET record_type = ?, ip_address = ?, ttl = 3600
WHERE COALESCE(is_custom, FALSE) = FALSE
AND lower(full_domain) LIKE ? ESCAPE '!'
""",
('CNAME', api.base_domain, pattern),
)
connection.commit()
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--apply', action='store_true', help='Apply the reviewed DNS migration')
args = parser.parse_args()
token = os.environ.get('DESEC_API_TOKEN')
base_domain = os.environ.get('BASE_DOMAIN')
if not token or not base_domain:
parser.error('DESEC_API_TOKEN and BASE_DOMAIN must be set')
api = DeSec(token, base_domain)
try:
namespaces, copies, parent_map = build_plan(api)
print(f'Namespaces to migrate: {len(namespaces)}')
print(f'Legacy RRsets to copy: {len(copies)}')
print('Namespaces:', ', '.join(namespaces))
print('Record copies:', ', '.join(f'{item["type"]}:{item["subname"]}' for item in copies) or 'none')
print('Mode:', 'APPLY' if args.apply else 'DRY RUN')
if args.apply:
apply_plan(api, namespaces, copies, parent_map)
print('DNS migration completed. Legacy child zones were retained for rollback.')
else:
print('No changes made. Run with --apply to perform the migration.')
except Exception as error:
print(f'Migration stopped: {error}', file=sys.stderr)
return 1
return 0
if __name__ == '__main__':
raise SystemExit(main())
websocket_service.py
+3 -78
View File
@@ -965,65 +965,19 @@ async def handle_delete_domain(client_id, data):
await send_message(client_id, {'type': 'error', 'error': str(e)})
async def handle_list_domains(client_id, data):
"""Handle list domains request - FIXED to separate custom and delegated domains"""
"""Return the user's claimed platform domains from the database."""
client_info = connected_clients.get(client_id)
if not client_info or not client_info['authenticated']:
await send_message(client_id, {'type': 'error', 'error': 'Not authenticated'})
return
user_id = client_info['user_id']
username = client_info['username']
include_serverless = data.get('includeServerless', True)
try:
# Get from database - THIS RETURNS ALL DOMAINS (both delegated and custom)
all_domains = get_user_domains(user_id, include_serverless)
# IMPORTANT: Separate delegated domains from custom domains
delegated_domains = [d for d in all_domains if not d.get('is_custom')]
# Also sync with DNS to ensure consistency for delegated domains only
if domain_manager:
# First ensure the user domain exists
user_domain = f"u-{username}.{domain_manager.base_domain}"
domain_exists = domain_manager._check_domain_exists(user_domain)
if domain_exists is None:
logger.error(f"Error checking domain {user_domain}")
elif not domain_exists:
# Create the user domain if it doesn't exist
logger.info(f"Creating user domain: {user_domain}")
if domain_manager._create_domain(user_domain):
logger.info(f"✓ Created user domain: {user_domain}")
else:
logger.error(f"✗ Failed to create user domain: {user_domain}")
# Now list delegated domains from DNS
result = domain_manager.list_user_domains(username)
if result.get('success'):
# Create a set of existing delegated domains from database
db_domain_set = {d['full_domain'] for d in delegated_domains}
# Add any delegated domains from DNS that aren't in database
for dns_domain in result.get('domains', []):
if dns_domain['full_domain'] not in db_domain_set:
# Add to database
domain_record = {
'user_id': user_id,
'subdomain': dns_domain['subdomain'],
'full_domain': dns_domain['full_domain'],
'record_type': dns_domain['record_type'],
'ip_address': dns_domain['ip_address'],
'ttl': dns_domain.get('ttl', 3600),
'created_at': dns_domain.get('created_at', datetime.now().isoformat())
}
create_domain_record(domain_record)
# Refresh delegated domains list from database
all_domains_refreshed = get_user_domains(user_id, include_serverless)
delegated_domains = [d for d in all_domains_refreshed if not d.get('is_custom')]
# SEND ONLY DELEGATED DOMAINS in the domains_list response
# Custom domains should be fetched via list_custom_domains
await send_message(client_id, {
@@ -1646,41 +1600,12 @@ async def handle_add_credits(client_id, data):
await send_message(client_id, {'type': 'error', 'error': 'Failed to add credits'})
def sync_user_domains_with_dns(user_id: str) -> bool:
"""Sync user domains between database and DNS"""
"""Ensure the user's wildcard namespace exists without creating a child zone."""
try:
user = get_user_by_id(user_id)
if not user or not domain_manager:
return False
username = user['username']
# Get domains from DNS
dns_result = domain_manager.list_user_domains(username)
if not dns_result.get('success'):
return False
dns_domains = {d['full_domain']: d for d in dns_result.get('domains', [])}
# Get domains from database
db_domains = get_user_domains(user_id, True)
db_domain_set = {d['full_domain'] for d in db_domains}
# Add missing DNS domains to database
for dns_domain in dns_result.get('domains', []):
if dns_domain['full_domain'] not in db_domain_set:
domain_record = {
'user_id': user_id,
'subdomain': dns_domain['subdomain'],
'full_domain': dns_domain['full_domain'],
'record_type': dns_domain['record_type'],
'ip_address': dns_domain['ip_address'],
'ttl': dns_domain.get('ttl', 3600),
'created_at': dns_domain.get('created_at', datetime.now().isoformat())
}
create_domain_record(domain_record)
logger.info(f"Added missing domain to DB: {dns_domain['full_domain']}")
return True
return domain_manager.ensure_user_namespace(user['username']).get('success', False)
except Exception as e:
logger.error(f"Error syncing domains: {e}")