Assembly Line Documentation

@assemblyline-agents/photon connects an agent to iMessage through Photon/Spectrum. It keeps Photon at the channel boundary: inbound webhooks are normalized into Assembly Line turns, and outbound side effects go through a Photon transport, either Photon's Spectrum cloud directly, or a bridge you host.

Photon webhooks are at-least-once. The adapter returns a fast 2xx accepted response and uses X-Spectrum-Webhook-Id + message.id as the default idempotency key, falling back to message.id when the webhook id header is absent.

Photon can emit an iMessage photo and its typed caption as separate webhook events. The adapter assigns adjacent events from the same authenticated sender to a one-second composition window. Each webhook keeps its own durable idempotency record, while the conversation mailbox atomically combines the parts before model execution. The resulting model turn contains all image attachments and the ordered text together, uses the newest message as the reply target, and produces one agent response. An image followed by “I ate 10 of these,” for example, reaches the model as one multimodal request.

Delivery Modes

The adapter picks the outbound transport from the environment at send time:

Mode	Selected when	Required env
Direct (Spectrum cloud)	No transport URL is set and both project credentials are set	`PHOTON_PROJECT_ID`, `PHOTON_PROJECT_SECRET`
Bridge (self-hosted)	`PHOTON_TRANSPORT_URL` is set	`PHOTON_TRANSPORT_URL`, `PHOTON_BRIDGE_TOKEN`

Direct talks to Photon's Spectrum cloud with the project's credentials and requires no bridge. It loads spectrum-ts dynamically from the agent's dependencies, so bridge-only agents do not need that package. Direct mode supports text and Markdown, media by URL, typing, app cards, backgrounds, and DM destinations. Reactions and polls depend on the installed spectrum-ts version. The adapter skips unsupported reactions and tells you to configure a bridge for unsupported polls. Inbound attachment downloads always require a bridge; see Inbound Files.

Bridge posts every side effect to a transport bridge you host (see Outbound Bridge Contract), authenticated with Authorization: Bearer <PHOTON_BRIDGE_TOKEN>. When a transport URL is set without a token, sends fail with PHOTON_BRIDGE_TOKEN is required for Photon delivery.; when neither mode is configured, sends fail with PHOTON_TRANSPORT_URL is required for Photon delivery (or set PHOTON_PROJECT_ID/PHOTON_PROJECT_SECRET for direct delivery).

One-Line Channel Setup

Select the built-in Photon channel profile in agent.md:

---
model: openrouter/openai/gpt-5.4
channels: [photon]
---
Reply helpfully over iMessage.

This compiles the reviewed definePhotonChannel() provider to an HTTP channel on /photon/events, including inbound normalization, typing lifecycle, and outbound delivery. It also activates the plugin-owned conversation tools; do not repeat photon under capabilities:. Inbound audio uses the channel profile's default OpenRouter transcription without a separate declaration. Options remain declarative:

channels:
  photon:
    audio: openai

The Photon plugin owns /photon/events, its description, authentication, and delivery transport. Agent Markdown only selects its typed audio exception.

Set audio: disabled only when voice notes should remain files without a transcript.

The profile stamps ingress-auth requirements but no flat delivery environment requirement because bridge and direct mode use different credential pairs. It resolves and validates the selected delivery mode at send time.

Ingress Auth

Set PHOTON_WEBHOOK_SIGNING_SECRET to verify X-Spectrum-Signature (HMAC-SHA256 over v0:<timestamp>:<body>, with a configurable timestamp tolerance). Alternatively set PHOTON_INGRESS_TOKEN and have the sender pass Authorization: Bearer <token>. Unsigned Photon ingress is local/dev-only; production runtime boot rejects a Photon channel when neither a signing secret nor bearer token is configured.

The production boot check and per-request verification both accept exactly PHOTON_WEBHOOK_SIGNING_SECRET or PHOTON_INGRESS_TOKEN.

Typing Lifecycle

For fast-ack HTTP ingress, the runtime starts the typing lifecycle after any bounded composition window and capacity admission. It does this while runtime initialization and durable run creation continue. Rejected turns and idempotent replays do not start a duplicate indicator. Direct runtime calls start the lifecycle during run setup. Both paths start typing before attachment intake and model work, then stop it immediately before delivery. definePhotonChannel() maps this lifecycle to Photon typing signals.

The indicator is refreshed every PHOTON_TYPING_REFRESH_MS (default 4 seconds). In bridge mode, typing signals are posted to /v1/messages/interact with action: "typing" and state: "start" | "stop"; in direct mode they map onto the Spectrum typing API. Typing failures are logged as warnings and never fail the run, and when no transport or destination is available the lifecycle is a no-op.

Markdown Replies

Photon's Spectrum bridge renders full CommonMark in iMessage, so the adapter sends replies as textFormat: "markdown" whenever the response contains renderable markdown, headings, lists, tables, fenced or inline code, blockquotes, links, or bold/italic. Plain casual messages (no markdown syntax) are sent as plain so they read like a normal text, with casual sentence-ending punctuation softened.

You can override the detection per delivery with a textFormat field on the delivery payload, or call photonTextForReply(body, "markdown") directly.

Rich Feature Tools

Selecting the Photon channel activates these official tools automatically:

photon_react: tap back the user's latest message (like, love, laugh, emphasize, dislike, question)
photon_poll: send a native iMessage poll (title + 2–10 options)
photon_app_card: send a styled link card (caption, subcaption, image)
photon_background: set or clear the chat background image
photon_current_location: read the bridge's cached location context for the current user

Each tool resolves the transport and reply destination from the run's channel context, so the model supplies only content. The location tool calls POST /v1/location/current; it reports cached, stale, refreshing, or unavailable state and never blocks a turn on a live location lookup. In direct mode, reactions and polls depend on the installed spectrum-ts version (see Delivery Modes).

The corresponding definePhoton*Tool factories remain exported for a custom plugin that needs a different description or approval policy.

Outbound Bridge Contract

In bridge mode, the adapter expects PHOTON_TRANSPORT_URL to point at a bridge exposing the Photon transport endpoints:

POST /v1/messages/send: final replies: text or markdown body, native reply targets, link previews, and optional media fields
POST /v1/messages/interact: reactions (action: "react") and typing signals (action: "typing", state: "start" | "stop")
POST /v1/messages/poll
POST /v1/messages/app
POST /v1/messages/background
POST /v1/location/current: cached current-location context used by photon_current_location

The direct transport additionally routes a dedicated /v1/messages/typing path ({ "action": "start" | "stop" }) for hosts that address typing explicitly; bridges only receive typing through /v1/messages/interact.

Optional media fields on a send:

{
  "mediaUrl": "https://example.com/image.png",
  "mediaFilename": "image.png",
  "mediaMimeType": "image/png"
}

Every request carries an idempotency key. Bridge responses are parsed as JSON and read up to a fixed 512 KiB cap; non-2xx responses raise Photon transport returned HTTP <status>: <payload>.

Outbound Files

Use the framework's standard deliver_artifact tool for generated images and other /workspace files. The runtime stores the file through the configured blob adapter; Photon recognizes the resulting files, artifacts, or attachments delivery entries and sends each one as media. A response with multiple files produces one idempotent Photon send per file, with response text attached only to the first send. A custom Photon delivery wrapper is not needed.

Photon gives the bridge a short-lived HTTPS URL rather than exposing blob credentials. Signed downloads use GET /photon/events?photon_media=1&..., are recognized directly from that URL before webhook authentication, are served with no-store and nosniff headers, and expire after 15 minutes by default. The bridge does not need to attach webhook credentials when it fetches a valid signed URL. Set ASSEMBLY_LINE_PUBLIC_URL or APP_PUBLIC_URL to the deployed agent origin. The signature uses PHOTON_MEDIA_URL_SIGNING_SECRET when set, then falls back to the existing Photon bridge, ingress, or project secret. Production public URLs must use HTTPS.

Inbound Files

Photon attachment content is preserved as runtime-visible files, not just message metadata. When Spectrum sends attachment content with fields such as name, mimeType, size, and downloadUrl/contentUrl/url. The adapter keeps those references in ChannelTurn.attachments. The runtime then downloads the bytes after the webhook ACK, stores them through the configured blob adapter, and exposes them under /files/original/... with entries in /files/manifest.json. Attachment downloads are restricted to the PHOTON_TRANSPORT_URL origin, which is why inbound files require a bridge even when outbound delivery runs in direct mode.

Photon bridge multipart/form-data forwarding is also supported. When the bridge sends asset_manifest_json plus matching file parts. The Node host parses the file bytes and the Photon adapter converts them into inline attachments before runtime storage. Prefer form forwarding for uploaded files; JSON forwarding can describe attachments, but it cannot carry the actual file bytes unless it includes an explicit downloadable URL.

Markdown/text uploads read back as UTF-8 through read; binary uploads remain byte-accurate when hydrated under /files. ZIP uploads are kept as their original archive under /files/original/... and, when extraction succeeds, safe entries are also exposed under /files/extracted/<archive-name>/... so the agent can open and project files from a zipped folder directly. If the download URL points at the Photon bridge origin. The runtime uses PHOTON_BRIDGE_TOKEN for the fetch without exposing that token to the model context.

Environment Reference

PHOTON_* variables are canonical on this page; runtime-wide ASSEMBLY_LINE_* variables live in Runtime Configuration.

Variable	Values	Default	Effect
`PHOTON_TRANSPORT_URL`	URL	unset	Bridge base URL; presence selects bridge mode.
`PHOTON_BRIDGE_TOKEN`	string	unset	Bearer token for bridge requests and bridge-origin attachment downloads; required in bridge mode.
`PHOTON_PROJECT_ID`	string	unset	Spectrum project id for direct delivery.
`PHOTON_PROJECT_SECRET`	string	unset	Spectrum project secret for direct delivery.
`PHOTON_WEBHOOK_SIGNING_SECRET`	string	unset	HMAC secret for `X-Spectrum-Signature` verification.
`PHOTON_INGRESS_TOKEN`	string	unset	Expected webhook `Authorization: Bearer` token.
`PHOTON_WEBHOOK_TOLERANCE_SECONDS`	integer, 30–86400	300	Maximum accepted signature timestamp age.
`PHOTON_SEND_REQUEST_TIMEOUT_MS`	integer, 1000–120000	30000	Timeout for sends, polls, app cards, and backgrounds.
`PHOTON_TYPING_REQUEST_TIMEOUT_MS`	integer, 500–15000	3000	Timeout for typing and reaction requests.
`PHOTON_TYPING_REFRESH_MS`	integer, 1000–30000	4000	Typing indicator refresh cadence.
`PHOTON_LOCATION_TOOL_TIMEOUT_MS`	positive integer milliseconds	1500	Timeout for cached bridge location reads.
`PHOTON_MEDIA_URL_SIGNING_SECRET`	string	Photon delivery or ingress secret	Dedicated HMAC secret for outbound artifact URLs.
`PHOTON_MEDIA_URL_TTL_MS`	positive integer milliseconds, up to 24 hours	900000	Lifetime of signed outbound artifact URLs.
`ASSEMBLY_LINE_PUBLIC_URL`	HTTPS URL	`APP_PUBLIC_URL`	Public agent origin used for signed artifact URLs.

Exports

@assemblyline-agents/photon exports, grouped by concern:

Channel: definePhotonChannel, normalizeHttp, startTurn, send, resolveAttachment
Transport helpers: sendPhotonReply, sendPhotonReaction, sendPhotonPoll, sendPhotonAppCard, sendPhotonBackground, startPhotonTyping
Text formatting: photonTextForReply, photonTextFormatForReply, shouldSendPhotonMarkdown, containsRenderableMarkdown, softenIMessageBubbleEndings
Tools: definePhotonReactionTool, definePhotonPollTool, definePhotonAppCardTool, definePhotonBackgroundTool, definePhotonCurrentLocationTool, photonTransportFromToolContext, photonDestinationFromToolContext
Mode and scoped input constants: directPhotonEnabled, PHOTON_BRIDGE_CONFIG, PHOTON_BRIDGE_CREDENTIALS, PHOTON_DIRECT_CONFIG, PHOTON_DIRECT_CREDENTIALS, PHOTON_WEBHOOK_CREDENTIALS, PHOTON_INGRESS_SECRET_ENV

Channels: MD-first channel selection and provider behavior.
Adapters: the channel role matrix and the other channel providers.
Runtime Configuration: runtime ASSEMBLY_LINE_* environment variables.
Troubleshooting: production ingress boot failures and webhook 401s.

Photon iMessage Channel