Feature: wait-for-message Long-Poll Primitive

Overview

The wait-for-message long-poll endpoint (GET /api/v1/email/account/{mailbox}/messages:await) enables clients to block synchronously until a new email matching specified criteria arrives in a mailbox.

This is useful for:

• Confirmation workflows that wait for a verification email before proceeding
• Testing/automation scenarios that poll for arrival of system-generated messages
• Reactive user interfaces that update when a specific message lands

API Surface

GET /api/v1/email/account/{mailbox}/messages:await
  ?filter[subject]=...
  &filter[from]=...
  &filter[to]=...
  &since=<ISO8601>
  &timeout=30

Query Parameters

All filters are optional and compose with AND logic (all must match).

Response

Success (200 OK):

• Returns the first matching message using the same response model as GET /api/v1/email/account/{mailbox}/messages (index/show endpoints).
• A single message in the data.collection array.

Timeout (408 Request Timeout):

• Returns an RFC 9457 Problem Detail (application/problem+json).
• Indicates no message matched the filter within the timeout window.

Overloaded (503 Service Unavailable):

• Returns an RFC 9457 Problem Detail (type … /await-overloaded).
• The process is already at AWAIT_MAX_CONCURRENT_WAITS in-flight waiters; rejected immediately (no IMAP work). Clients should retry shortly.

Authentication & Scopes

• Token Scope: Reuses the messages:index scope (no new scope required).
• Authorization: Tokens without messages:index receive a 403 Forbidden.

Example Usage

# Wait up to 30 seconds for a verification email from noreply@example.com
curl -H "Authorization: Bearer <token>" \
  'https://email.squibble.ch/api/v1/email/account/mybox@example.com/messages:await?filter[from]=noreply@example.com&timeout=30'

# Response on success (200 OK):
{
  "data": {
    "collection": [
      {
        "email_id": "123",
        "message_id": "<abc@example.com>",
        "datetime": "2026-05-19T12:34:56+00:00",
        "subject": "Verify your email",
        "addresses": { ... },
        "message": "Click the link to verify: ...",
        ...
      }
    ]
  },
  "meta": {
    "emails_count": 1,
    "imap_search": "FROM \"noreply@example.com\""
  }
}

# Response on timeout (408 Request Timeout):
{
  "type": "https://email.squibble.ch/docs/errors/email/messages/await-timeout",
  "title": "Request Timeout",
  "status": 408,
  "detail": "No matching messages found within 30 seconds.",
  "instance": "https://email.squibble.ch/api/v1/..."
}

Design Constraints & Rationale

Async Bounded-Poll Architecture

The endpoint is an async handler running a bounded polling loop:

1. No threadpool worker held during waits. The route is async def. Between polls it await asyncio.sleep(interval), which yields the event loop and holds no threadpool worker — many thousands of waiters can sleep concurrently on one process.

2. Threadpool used only for the IMAP round-trip. Each poll runs the blocking imaplib connect/search/fetch/logout in the threadpool via run_in_threadpool, so a worker is occupied only for the ~1–2s of an actual IMAP round-trip — never for the full timeout. This is the explicit fix for the “naive long-poll starves the threadpool” foot-gun.

   • Reuses the existing two-pass selective fetch (headers + BODYSTRUCTURE, then text bodies) — memory-safety property preserved.
   • The IMAP connection is opened and closed per iteration (existing try/finally lifecycle); never held open across the wait.

3. Hard concurrency cap. Total in-flight waiters are capped at settings.AWAIT_MAX_CONCURRENT_WAITS (default 50). Over the cap the endpoint returns an RFC 9457 503 immediately — no IMAP work, no unbounded connection/threadpool growth. The counter is mutated only on the event-loop thread (no await between check and increment), so it is race-free without a lock.

4. Server-side timeout cap: hard 120s (le=120 Pydantic type + route annotation); larger client values are clamped, not errored.

5. Poll interval: default 2s between IMAP searches; injectable via the processor’s poll_interval_seconds (0 in tests). Balances responsiveness (~2s detection latency) against IMAP load.

Concurrency Safety

• No worker held while waiting: inter-poll waiting is asyncio.sleep on the event loop; only the brief per-iteration IMAP call uses a worker.
• Connection bound: ~1 IMAP connection per active iteration, closed in a finally; total simultaneous waiters bounded by the hard cap.
• Explicit backpressure: at the cap, new awaits get an immediate 503 problem doc rather than cascading IMAP failures. No deadlock.

Filter Composition

Filters compose with AND logic:

• subject=Invoice AND from=alice@example.com AND since=2026-05-19T00:00:00Z

Maps directly to IMAP SEARCH syntax:

SUBJECT "Invoice" FROM "alice@example.com" SINCE 19-May-2026 ALL

Input Validation & Injection Prevention

• Mailbox names validated with regex ^[^\r\n]+$ (no CRLF).
• Subject filters validated to reject CRLF (prevents IMAP command injection).
• Special characters in filter values are escaped for IMAP (quotes and backslashes).

Testing

The implementation includes 17 comprehensive tests covering:

• ✅ Matching messages returned immediately on first poll
• ✅ 408 timeout returned when no match occurs
• ✅ Client timeout clamped to 120s cap
• ✅ Filters compose into IMAP SEARCH
• ✅ Multiple poll iterations until message arrives
• ✅ CRLF/injection protection in mailbox and filters
• ✅ IMAP connection cleanup on exception
• ✅ since filter using IMAP SINCE (date-only format)

Tests use mocked IMAP to avoid live server dependency and run in <3s.

Production Considerations

1. Monitoring: Track the in-flight-waiter count and 503 rate; raise AWAIT_MAX_CONCURRENT_WAITS if legitimate load is hitting the cap (it bounds simultaneous IMAP connections, so weigh against the IMAP server’s per-user limit).
2. IMAP Load: Poll interval (default 2s) can be adjusted if IMAP server is under strain.
3. Timeout Tuning: Most use cases should use 30–60s; values near 120s should be rare.
4. Rate Limits: Standard rate limit (30 requests/minute) applies; clients should batch waits or use shorter timeouts if needed.