Phase 2: Outbound API & HTML Parsing

Overview

POST /api/v1/messages/send is the primary entry point for AI agents and applications to queue outbound email. Each recipient gets its own row and UUID, enabling per-recipient tracking. HTML bodies are automatically enhanced: a 1×1 tracking pixel is injected and all <a href> links are rewritten to route through the tracking gateway before forwarding to their destination. The sender identity is irrevocably bound to the authenticated mailbox — no caller-supplied From, Sender, Reply-To, or Return-Path is ever accepted.

Post-review hardening shipped in this phase:

• Pydantic caps: ≤100 combined recipients, ≤1 MiB html_body/text_body, ≤998-octet subject, empty-list rejection.
• Idempotency-Key header support (MR #8): a (token_jti, idempotency_key) lookup with ≥24 h window prevents duplicate sends on network retries.
• Per-token send_limits and allowed_recipient_domains quotas (MR #9): a leaked token cannot be used as an unlimited spam relay.

Tags & Metadata (opt-in classification)

Both send endpoints accept two optional objects (see architecture_decision_records/2026-07-17__outbound_message_tags_and_metadata.md):

• tags — up to 10 name: value pairs of [A-Za-z0-9_-]{1,256} that classify the message kind ({"type": "welcome", "campaign": "onboarding-v2"}). Tags are echoed on reads and webhook payloads, filterable everywhere messages are listed (repeatable ?tag=name:value, AND semantics, GIN-indexed @> containment), and aggregated by GET /messages/outbound/stats/tags?name=<tag> — per-value sent/failed/bounced/opened/clicked counts, with untagged mail in a single value: null bucket.
• metadata — up to 20 opaque key/value entries for client correlation data (order IDs, user IDs). Echoed on reads and webhooks, never indexed or queried.

Omitting both is the default path: the columns stay NULL, and no behaviour changes. stream remains the compliance axis (unsubscribe, RFC 8058); tags carry no delivery semantics.

Goal

Create the main API endpoint for queuing outbound emails, including automatic parsing and injection of tracking pixels and link rewriting.

Dependencies to add:

• beautifulsoup4
• lxml (for fast parsing)

TDD Acceptance Criteria

1. pytest tests/services/test_html_parser.py::test_injects_pixel_after_body MUST PASS
2. pytest tests/services/test_html_parser.py::test_rewrites_links_with_url_encoding MUST PASS
3. pytest tests/services/test_html_parser.py::test_ignores_mailto_links MUST PASS
4. pytest tests/api/test_outbound_send.py::test_403_missing_messages_send_scope MUST PASS
5. pytest tests/api/test_outbound_send.py::test_202_accepted_and_db_insertion MUST PASS

Technical Specifications

HTML Parsing Service

• Function inject_tracking(html: str, message_uuid: UUID) -> str
• Function rewrite_links(html: str, message_uuid: UUID) -> str
• Use urllib.parse.quote for URL encoding of original URLs.

API Endpoint: POST /api/v1/messages/send

Payload Validation (Pydantic):

• to: List[EmailStr]
• cc: Optional[List[EmailStr]]
• bcc: Optional[List[EmailStr]]
• subject: String
• html_body: String
• text_body: Optional[String]
• attachments: Optional[List[AttachmentSchema]]

Action:

1. Validate messages:send permission in JWT.
2. For each recipient in to, cc, bcc:
   • Generate UUID for the message.
   • Run HTML Parsing Service.
   • Insert row into outbound_messages with status='queued'.
3. Return 202 Accepted with a list of generated message UUIDs.

Input validation & injection prevention (architecture.md §4)

Any identifier that could be interpolated into an SMTP command or mail header — mailbox names, hostnames, subject, recipient addresses — must be validated before use. The key rule: reject strings containing \r or \n (CRLF).

• Pydantic EmailStr handles recipients.
• subject length is capped at 998 octets (RFC 5322); multi-line subjects are rejected by the \r\n check in the Pydantic validator.
• smtp_host, smtp_port, and other operator-supplied mailbox fields are validated at mailboxes:create / mailboxes:update time via the CLI layer.

This mirrors the CRLF-injection prevention applied to legacy IMAP SELECT commands (architecture.md §4) and is enforced by the same CONTRIBUTING.md rule: prefer explicit validation for all user-controlled headers, query params, mailbox names, message IDs, and filenames.

Sender binding invariant (FEEDBACK.md §1.4)

From, Sender, Reply-To, Return-Path, and the SMTP envelope sender are always bound to the authenticated mailbox’s configured email address. The payload has no sender-related field and never will.

• The SendMessageRequest Pydantic model declares model_config = ConfigDict(extra="forbid"). Any unknown key in the request body — from, From, sender, reply_to, return_path, a nested headers bag, anything at all — is a 422 Unprocessable Entity. No sender override is silently ignored.
• The worker’s MIME construction (src/app/workers/smtp_delivery.py, _bind_sender_headers) sets From, Sender, Reply-To, and Return-Path exclusively from mailbox.email. It never reads sender fields from the OutboundMessage row, so even if a future schema change were to let a caller-controlled string land on the row, it could not influence the sender identity.
• The From display name (the human-readable label shown before the address in mail clients) is the one operator-level exception: set mailbox.display_name via mailboxes:update --display-name "Acme Corp". When set, the From header reads Acme Corp <hello@example.com>; when NULL it falls back to the IMAP username. This is a CLI-only, operator-level setting — no token or request payload can influence it.
• Consequence: a stolen messages:send token is scoped to the mailbox it was issued for. It cannot be turned into a spoofing weapon for arbitrary From addresses (no “phish as the CEO” vector).

A future per-token allowed_from override (FEEDBACK.md §1.13, tracked in PLAN.md pre-GA) is the one and only mechanism that can ever relax this; it requires an explicit permission on the token, not a request-body field.