Collection behavior and limits

Mosaic’s YouTube collector observes configured public Channels without authenticated sessions, identity spoofing, proxy rotation, challenge bypass, or arbitrary JavaScript. Observed description matches do not prove payment; inferred sponsorship is a separate derived signal described below. The private Meta actor follows a separate reviewed public-data transport policy (ADR 0009), including its residential-proxy and existing challenge-handling approach. That policy is not a claim of legal or platform compliance. A missing observation is Coverage, never a negative match.

Source boundary

1. When configured, Mosaic reads the Channel’s uploads playlist (its UC… ID with the UC prefix replaced by UU) with playlistItems.list and batched videos.list descriptions. It calls channels.list only when that derived playlist is not found, to confirm whether the Channel is unavailable or has a different uploads playlist. RSS is the bounded fallback, not a completeness signal. Both paths validate the requested Channel identity. A Channel’s first successful collection considers its newest 50 uploads (the Channel-addition baseline; RSS exposes at most 15); later collections consider the newest 15.
2. Data API snippet.description is recorded as youtube.data_api.video_snippet_description; RSS media:description retains its own provenance. One official batch supplies descriptions and format metadata, reused for up to 60 seconds within the job. Workers do not fetch watch pages for evidence. Titles cannot create Mentions; official video metadata excludes upcoming, live, and archived broadcasts (ADRs 0012–0014).
3. The Worker records new RSS and Data API observations under v2, which scans the complete accepted description stored after first successful acquisition. v1, the opening section (the first 1,000 Unicode code points of the decoded official RSS media:description, with the whole hostname inside that window), remains only for frozen historical observations. Data API provenance is not relabeled as RSS, and retained credit or legacy Event history still prevents replay. Activity and catalog Mention resources report the original acquisition source even after deferred evaluation or raw-text expiry. Historical youtube_rss_observation time-basis and youtube_rss_description relationship-reference tags remain for compatibility; they do not override the accepted description’s explicit provenance source.
4. Collection stops rather than bypassing authentication, challenge, or throttling responses. It records every unobserved item as bounded Coverage. Collection itself does not follow redirects; shortened links are resolved only during Mention evaluation, as described below.
5. Suppressed and erased subjects are never collected (ADR 0018; see the API reference’s subject suppression and erasure section). A governed Channel, or a Channel whose Creator is governed, is not claimed, scheduled, renewed, or counted; a governed video is dropped before observation and is never refetched. In Meta receipts, a governed ad, Creator Page, or Brand Page is classified held with reason governed_subject and creates no evidence, Creator, avatar work, or Activity, and a row that reveals an erased subject keeps only identifiers and classification. Governance is not Coverage and is never reported as a negative match.

Global Mention evaluation

Each accepted description is admitted once, with a frozen snapshot of every active catalog Brand’s effective confirmed Matchers at the observation time, regardless of Workspace tracking. The resulting Mentions are global catalog evidence. A Workspace receives an Event, a one-credit Work Credit charge, and deliveries only for its Tracked Brands that were active when Mosaic first observed the video and that are outside a silent Brand-activation or Channel-addition baseline. A video whose Workspace evaluation closed as capped or inactive is not replayed when eligibility returns.

When staff confirm a new Matcher (a new Brand’s domain, an added company domain, or a campaign hostname) or reactivate an archived Brand, Mosaic queues a versioned re-evaluation of that Brand against each retained accepted description that has not yet evaluated all of the Brand’s current Matchers and has no Mention for it. The queue row freezes the Brand’s Matcher snapshot and the v2 policy. Re-evaluation reuses the initial evaluation’s official format and paid-promotion label, reads no provider metadata, and writes a new Mention only; the initial evaluation, its candidate snapshot, and existing Mentions are never rewritten. Re-evaluated Mentions of already-observed videos are silent: they never create Events, Work Credit charges, or deliveries. Re-evaluation is bounded by raw-description retention: a catalog change does not reach a video whose retained description has already expired. The migration that introduced global matching queued the same backfill once for Brands that were never candidates.

Expired-description refetch

An evaluation held with accepted_description_expired (its accepted text expired before the evaluation ran) gets one budgeted official refetch (migration 0092). So does a completed video whose catalog re-evaluation was held with accepted_description_expired because the frozen text expired before the re-evaluation ran (migration 0094). The held re-evaluation stays as the record that the frozen text had expired; the refetch’s description_refetch re-evaluation matches its Brand, and every other active Brand without a Mention, against the refetched text:

• Each scheduled pass opens a pending refetch for every such evaluation, reserves one unit against the durable YOUTUBE_DESCRIPTION_REFETCH_DAILY_QUOTA_UNITS budget (50 units per Pacific quota day), and sends at most one videos.list request (part=snippet,liveStreamingDetails,paidProductPlacementDetails) for up to 50 videos. One call costs one unit regardless of batch size, so 50 videos cost 1 unit. The request goes through the same metered Data API adapter as collection: it is recorded in the quota ledger, skipped during a Pacific-midnight quota pause (the unit is returned and the batch waits for the reset), and sent with redirect: "manual". Watch pages are never fetched.
• The returned snippet.description is recorded as a new accepted Data API description (youtube.data_api.video_snippet_description) with its own fingerprint and observation time, and its text follows the same six-day retention. The frozen observation, evaluation, candidate snapshot, and existing Mentions are never rewritten.
• The same official resource decides broadcast format and the paid-promotion label. For an ordinary video or Short, Mosaic queues one description_refetch re-evaluation that freezes every active catalog Brand’s currently effective Matchers (skipping Brands that already have a Mention for the video). Its Mentions report observedAtBasis: "youtube_data_api_refetch" with observedAt at the refetch time, acceptedDescription.provenance: "youtube.data_api.video_snippet_description", evaluation: "reevaluation", and descriptionRefetchId naming the refetch record. Relationship evidence provenance carries the same ID. They are silent: no Events, Work Credit charges, or deliveries.
• A video videos.list no longer returns for its Channel (deleted, private, or moved) becomes terminal unavailable, visible in coverage, and is never retried. A failed request backs off 1 h, 2 h, 4 h, … up to daily. An expired initial evaluation is refetched at most once. A completed video is refetched once per held re-evaluation newer than its last refetch; several holds at once share one refetch, and a video is never refetched while a refetch is pending.
• Collection does not report expired text as a coverage failure (“Required RSS description evidence expired before recovery”) for a video whose canonical evaluation already completed or was excluded: Workspace projection reads the persisted Mentions, not the text. The failure is still recorded while the evaluation is pending or held.
• Evaluations that migration 0065 held with precanonical_observation_not_admitted (RSS observations recorded before canonical Mentions, when their Channel’s Creator identity had not been admitted) are not recovered. Migration 0095 made them terminal excluded with reason precanonical_retired. They have no Creator identity or Matcher snapshot, so they are never refetched or re-evaluated and never produce Mentions, Events, Work Credit charges, or deliveries; re-observing the video keeps the retired evaluation. Collection treats them like other excluded evaluations and records no expired-evidence failure for them. Videos first observed after the Channel’s identity exists are evaluated normally.

A pre-canonical per-Workspace evidence row for the same Brand and video no longer holds the whole video. It gains the canonical detail when its Creator and observation time agree; otherwise only that Brand is skipped. The per-Workspace Competitor and primary-Brand matcher is retired.

Shortened and affiliate links

A link on a listed shortener or affiliate-redirect host (for example bit.ly, geni.us, amzn.to, tinyurl.com, howl.me, or Impact *.pxf.io) qualifies only when Mosaic resolves it to a confirmed company domain (including genuine subdomains) or an exact campaign hostname. The occurrence records the initial URL, redirect hops, and final URL as resolution provenance. Resolution uses the public-link resolver bounds below and at most 10 shortened links per description; other links are never requested, and watch pages are never fetched. A transient failure defers the evaluation for up to three attempts, then completes without that link. A shared linking-platform apex destination identifies a Brand only when staff declared the Brand to be that platform. Resolution runs only where YouTube execution policy permits collection.

Paid promotion label

The official videos.list read that decides format also requests paidProductPlacementDetails (no extra quota). YouTube’s creator-declared “Includes paid promotion” flag is stored with the initial evaluation and returned as video.includesPaidPromotion on Mention resources (null when not observed). It names no Brand and never creates a Mention or an Event.

Brand-link sponsorship inference

A confirmed Brand link beginning in the first half of an accepted description signals inferred sponsorship (youtube.brand_link_first_half.v1), not a verified contract or disclosure. Each occurrence keeps linkStart, the Unicode code-point offset of its URL or bare domain, and the Mention keeps the accepted description’s code-point length. A link qualifies when 2 * linkStart < descriptionCodePoints; the midpoint is excluded. Any qualifying occurrence counts, including a shortened link resolved to a confirmed Brand hostname. Second-half links remain Mentions marked not_inferred. Pre-0103 Mentions are not_evaluated, not retroactively classified. Legacy v1 matching only sees the first 1,000 code points and can miss later qualifying links in a long description. Activity Events and webhooks retain their existing observed-match semantics; the classification is available on the Mention resource.

Returned pages are transient collection input. Full raw descriptions are retained at most seven days solely for operational recovery, then removed. They are not durable evidence or a customer-facing archive; deterministic matching and safe link extraction retain only the bounded durable fields.

Data API upload age uses contentDetails.videoPublishedAt, not playlist insertion time; accepted video metadata uses videos.list snippet.publishedAt. Missing collection envelopes and malformed upload entries are source failures, not observed empty history. A mismatched RSS Channel is rejected without tombstoning the requested Channel.

The current Worker composition enables description monitoring. Its five-minute scheduler uses a six-minute collection floor and a roster-scaled interval sized to the YouTube Data API quota. Google’s quota table (checked 2026-09-27) gives a project 10,000 units per day by default and charges one unit per channels.list, playlistItems.list, or videos.list call. A steady-state run costs two units. Scheduled polling may spend 7,000 units per day even when the maximum 60-second jitter shortens every interval; the remaining 3,000 units cover WebSub-hinted runs, admissions (two units each through the Data API), missing-playlist confirmations, and Channel avatar attempts (one channels.list each, described below). The interval reads the active-Channel count, not the full roster. The last column also budgets the worst-case avatar retries: five first-day attempts per admission plus one daily retry for every Channel still without an avatar.

Active Channels	Interval	Polling units/day	With 2 hints/Channel/day, 25 admissions, and avatar retries
25	12 min	≤ 6,546	≤ 6,846
50	22 min	≤ 6,858	≤ 7,283
100	43 min	≤ 6,858	≤ 7,533
250	104 min	≤ 6,991	≤ 8,416

Rosters of 12 or fewer Channels keep the six-minute floor. Above 3,500 active Channels one daily pass exceeds the polling share, so a larger quota is required. WebSub hints request an immediate collection, so new uploads normally arrive well before the next scheduled pass; a fifteen-minute observation target is still only plausible for small rosters and is not a guarantee about YouTube publication delay, collection success, or delivery. Repository configuration is not verification of the deployed Worker, its WebSub delivery rate, or the provider quota actually granted.

YouTube execution is separately scoped by the Worker bindings MOSAIC_YOUTUBE_EXECUTION_MODE and MOSAIC_YOUTUBE_CANARY_CHANNEL_ID. An absent mode defaults to disabled: no YouTube discovery, feeds, hub subscription requests, format classification, avatar fetches, or link resolution run. canary requires exactly one canonical UC… Channel ID; only that Channel can execute. unrestricted must be explicitly configured and has no Channel ID. Malformed or partial settings reject Worker startup. These plain-text settings do not authorize provider credentials. Channel admission outside the policy returns collection_disabled, not an input error; a handle cannot be admitted in canary mode because resolving its Channel ID would require a provider call before scope is known. Existing unrelated webhook, Brand enrichment, and Meta avatar work continues while YouTube is disabled.

Requests and WebSub hints for other admitted Channels retain their durable due marker without dispatching collection or modifying collection health. Due work is reconsidered after a policy change; canonical Mention evaluations for other Channels remain unclaimed. Blocked avatar queue deliveries are retained in youtube_avatar_hold before acknowledgment, then replayed by the five-minute scheduler after scope permits them. Release rechecks the current policy at queue consumption; a failed hold write retries the message without acknowledgment. If hold storage stays unavailable through the configured five retries, the dead-letter queue needs operator replay. The existing raw-description expiry still runs while execution is disabled or canary-scoped. A pending evaluation whose accepted description expires is held, and pending video evaluations can fail; those old snapshots cannot be promised replay after the retention deadline. Fresh collection can observe available current or later source items when the Channel becomes eligible, but cannot restore expired frozen evidence. Canary observation and re-enable must fit the existing evidence-retention window. These are local implementation properties, not a claim that the hold migration, settings, or Worker is deployed.

Unchanged frozen observations require one read; inserts and upgrades are read back to resolve concurrent writers. A URL blocked by local policy without a network request is not recorded as a link-acquisition failure. Later collection never relabels retained RSS evidence as Data API evidence. Description edits and redirect-based sponsor attribution remain outside this frozen-evidence model.

Successful reconciliation may enqueue independent YouTube avatar acquisition before clearing the durable collection request. It enqueues only when the active Channel has no approved avatar (automatic or manual) and it wins a durable per-Channel attempt claim, so a pending or backing-off attempt is never duplicated and the number of collections never multiplies channels.list calls. The job calls channels.list once, uses the largest official Channel thumbnail URL unchanged, validates actual raster dimensions (at least 128px), and stores immutable bytes in R2 with youtube_channel provenance. Existing selections are preserved. Every outcome acknowledges the queue message; a missing thumbnail, provider/storage failure, or unsuitable image instead waits 1, 2, 4, 8, then 16 hours, and then daily, before a later collection may claim the next attempt (at most five attempts in a Channel’s first day). Success is recorded and ends attempts for that Channel. Queue logs report each outcome, and unavailable attempts log channel_metadata_unavailable, avatar_url_missing, source_unavailable, or storage_unavailable. Availability is not guaranteed, and no ranking-site avatar becomes source authority.

YouTube responses are bounded by a 15-second request timeout; RSS response bodies are capped at 2,000,000 bytes.

Initial Brand enrichment

Creating a canonical Brand through governed profile approval atomically enrolls its authoritative domain for one Context.dev Brand Retrieve operation. The existing five-minute scheduler drains a bounded batch. A successful snapshot captured before Brand creation already satisfies that work. New post-migration Brand/domain bindings are reconciled once per Brand, not once per alias; a migration watermark prevents an automatic paid historical backfill. Domainless Brands remain blocked until an authoritative domain is bound—Mosaic does not derive domains from names or Meta identifiers.

Claims and finite leases are stored in Postgres, and provider dispatch is recorded before HTTP. Context.dev documents Brand Retrieve as costing 10 credits and does not document a provider idempotency key. Therefore a crash after dispatch but before the response is stored is held as outcome_unknown for operator review rather than automatically retried. Undispatched claims can be reclaimed. Stored responses are replayed into immutable attempts, snapshots, and candidates without another provider call. Missing credentials leave work pending. Confirmed timeout/rate-limit failures use bounded backoff with at most three attempts; ambiguous transport/gateway outcomes are held, while not-found and malformed responses are terminal. Staff requests reserve idempotency and validate targets before HTTP; the console retains the operation key across errors and tab reloads. Context’s provider deadline is 25 seconds with behavior: fail; the client deadline is 30 seconds. Provider values never automatically approve profiles, Matchers, source targets, tracking, collection, assets, or activity.

These are implementation properties, not a claim that a production secret is configured or that the migration/Worker has been deployed.

Quarantined creator contacts

Mosaic records creator contact findings (emails, business-inquiry links, management lines, and websites) from text it already collects: Channel profiles read during Channel Discovery qualification and retained YouTube video descriptions, each observation scanned once by the scheduled maintenance drain. Each finding keeps its Channel, source (youtube.channel_description, youtube.channel_links, or youtube.video_description), Channel or video reference, source line excerpt, reader, and extractor version. Collection adds no provider call or quota.

Findings are quarantined (ADR 0025): no REST, MCP, web, or delivery path exposes them, collector roles can only insert them, and only governance erasure reads their identifiers. Governed Channels and videos record nothing; erasure deletes the findings of the erased subject.

Initial Meta catalog seed

New Brands created on or after the immutable Meta rollout watermark are reconciled for their first confirmed Page collection. Context suggestions, missing/future/revoked targets and tombstoned accounts cannot authorize collection. The durable Meta control defaults disabled with zero authorized starts; activation and its cumulative limit require a separately approved operation.

The private Meta Worker reconciles one initial-seed item every five minutes, independently of recurring collection and customer tracking. Admission and paid dispatch recheck eligibility in transactions. Any earlier Brand+Page acquisition—including manual, failed, partial or ambiguous work and prior target decisions—prevents automatic rebilling, not proof of successful Coverage. Old Brands do not enter the cohort merely because a new Page target is confirmed.

At most one automatic provider run may be unresolved: starting, running and start_unknown occupy that slot. Authorized attempts consume the cumulative limit even if rejected or unknown. Disabling or exhausting new starts never blocks bound-run inspection or retained-receipt projection. Missing credentials leave admitted work pending. Reconciliation repairs missed enrollment and resumes unfinished work without launching a replacement run. Expired unbound dispatch is held; capped/interrupted observations remain incomplete. Catalog-only projection creates no Activity or Work Credit charges; avatars retain their independent lifecycle.

Provider-run binding and advancement to running are atomic. Stale-start recovery rechecks that no run is bound under the same row lock. A response received after a committed hold retains its identifiers for operator review without reopening collection.

An ambiguous start_unknown blocks further automatic starts. Existing staff attach/replacement commands do not clear that original slot; proof-backed operator reconciliation remains a separate follow-up before broader activation. The system does not reset it or refund its allowance automatically.

This requires the control migration, Worker deployment and explicit enable/budget approval. None of those production actions is implied by local implementation or test results. This initial-seed flow is separate from recurring Meta monitoring.

Monitoring demand

Recurring collection follows monitoring demand: the canonical Brands that at least one paid Workspace actively tracks (ADR 0022). A paid Workspace has spendable Work Credits and no unresolved dispute freeze. Paused and archived Tracked Brands never count, and a subscription status alone does not create demand. Demand only decides whether collection runs; each live Event still rechecks funds when it is charged.

• YouTube. While no Brand is demanded, the scheduled pass renews no WebSub subscription and enqueues no Channel. A WebSub notification records its durable request but enqueues nothing. A queued collection stands down without claiming its Channel, and an admission records its request without subscribing. The first scheduled pass after demand returns renews lapsed subscriptions and collects due Channels from their newest uploads. While any Brand is demanded, the whole Channel universe is collected, because a Mention can appear on any Channel. Every description is matched against every catalog Brand.
• Discovery. Each active Tracked Brand incarnation (activation or resume, either role) plans that Brand’s search queries once, ahead of the periodic catalog searches. Meta partnership creators of actively tracked Brands are planned first. Admission budgets and qualification are unchanged.

Recurring Meta monitoring

The private Meta Worker’s five-minute drain admits recurring snapshots for the effective confirmed Meta targets of demanded Brands, then advances them through the shared acquisition lifecycle and dispatch fence. meta_recurring_monitoring_control defaults to enabled = false. Its cadence_days (1–28, default 7) sets how often each target gets a cadence snapshot, measured from its previous cadence snapshot. A target also gets a baseline snapshot on the next drain when a pending Tracked Brand baseline for it was admitted after its latest recurring snapshot, so a newly tracked Brand is baselined at once. A baseline snapshot does not move the cadence slot. The kind is fixed on the acquisition when it is created. No slot is admitted beside unfinished recurring work for the target.

A cadence snapshot is the billable check (5 Work Credits per Tracked Brand whose baseline for the target is complete). A baseline snapshot charges no check; its live Events still cost 1 credit each. Mosaic absorbs the provider cost of baseline snapshots.

Each start reserves the Meta per-run ceiling against meta_research_budget_control.daily_usd, the same budget Catalog seeds and Brand research use. The start rechecks the control, active tracking and unresolved Page work. If tracking ended, the snapshot fails unstarted. A start_unknown start blocks further recurring starts for that Page until operator review.

A recurring receipt is admitted as recurring_monitoring. The first complete snapshot after activation, resume or target addition seals that Tracked Brand’s baseline silently. After that, each first-seen Brand–Creator relationship becomes one live meta.creator_relationship.discovered Event, with its usage debit and delivery outbox row in the same transaction. Capped or interrupted snapshots never seal a baseline. Enabling the control, raising the budget, and the migration are separate production approvals.

Funded research and daily Watches: local release boundary

These are local implementation contracts, not verified deployments. Shared REST/MCP operation definitions live in apps/server/src/operations/ and are composed in apps/server/src/mosaic-operations.ts; runtime wiring is implemented locally. No live migration, role provisioning, schedule, funding or collection is authorized by this page.

Verified Stripe invoice.paid funds an explicitly configured monthly Work Credit allowance for the qualifying paid subscription service period, deduplicated by subscription and period start. Included funds expire without rollover; failed/incomplete work releases its hold without reviving expired funds. Successful bounded work, including an empty result, charges. Monitoring spends the same Work Credits (ADR 0023; see the API reference’s monitoring charges).

Cancellation leaves funded credits usable until their original period expiry. A positive successful cash refund, even partial, revokes all unused credits of that invoice; pending/failed refunds are no-ops. Existing holds may settle normally, but release after refund cannot revive credits and settled usage is unchanged. Immutable restrictions survive delivery before the funding invoice. Any dispute freezes new paid Meta and YouTube research, including the same admission used by Watches, until audited review of that specific dispute. Closed/won/reinstated Stripe state never auto-clears it. Paid quotes/admission return billing_frozen (409 in REST/MCP); governed cached reads, idempotent replay and eligible free receipt reuse remain available.

Before paid acquisition, both sources try free exact-scope fresh receipt reuse. maxAgeSeconds defaults to 86,400 (0–86,400, inclusive source-age bound); refresh: true bypasses reuse. A reusable Meta receipt must have classified/projected complete or capped coverage for the confirmed target and pinned scope. YouTube requires a completed matching-scope receipt and an eligible Channel. Generic retained documents do not suffice. A reused request holds no credits and starts no provider; current governance still controls evidence reads. Receipt-scoped Meta evidence classifies explicit partnership ads, excluding generic brand-only ads. Exact known-Brand resolution is read-only and cannot create a Brand or resolve an unknown identity into collection authority.

Daily Research Watches are separate Workspace consent for fresh bounded checks, not Tracked Brand monitoring, Events, notifications or legacy source demand. They require recurringConsent: true, per-run and UTC-calendar-month ceilings, available Work Credits and rechecked actor authority. Each dispatch forces refresh: true, maxAgeSeconds: 0; missed days are not backfilled. Calendar budgets do not follow subscription renewal dates. Pause/cancel prevents future checks but does not refund already-authorized work; failure/blocked coverage is not a no-partnership finding.

Release must explicitly review tariffs (MOSAIC_RESEARCH_TARIFFS, version 1), monthly quantity (MOSAIC_BILLING_WORK_CREDIT_ALLOWANCE), migrations through 0112, customer/billing/source/operator roles and scheduler bindings. Verified webhook persistence requires the separately privileged MOSAIC_BILLING_HYPERDRIVE, with no customer/admin fallback, even without an allowance. Configured allowance without that writer fails startup. Distinct connection strings do not prove least privilege. The private Meta Worker’s five-minute cron is opt-in through the deployment setting MOSAIC_META_RESEARCH_SCHEDULE=enabled; server Watch dispatch and YouTube core execution need separately reviewed runtime/schedule composition. Tariffs and consent do not override provider caps, disabled controls or uncertainty fences. Existing Stripe/Postgres/Worker infrastructure is used; neither Redis nor Autumn is required. No production price or rollout is implied.

Migration 0112 and updated role grants are local and pending; no deployment or live Stripe setup is verified. Subscribe to refund-object events refund.created and refund.updated (charge.refund.updated is supported for legacy delivery), plus charge.dispute.created, charge.dispute.updated, charge.dispute.closed, charge.dispute.funds_reinstated, and charge.dispute.funds_withdrawn, alongside existing invoice.paid and subscription events. The adapter does not handle charge.refunded; it is not a substitute for refund-object events. Dispute review is an internal, short-lived mosaic_research_operator capability, not a public unfreeze tool. The operational procedure is in docs/readiness/creator-partnership-research.md.

Customer Meta Brand research

A paid Workspace Brand research request (see the API reference) admits one acquisition with purpose brand_research on the same acquisition lifecycle as the initial catalog seed. It reuses the pinned Actor build and scope, the atomic dispatch boundary, provider-run binding, the stale-start hold, receipt retention, and catalog projection. It never occupies a Catalog-bootstrap slot or the seed’s start budget, and its acquisition records a Workspace-free requester.

The private Meta Worker’s scheduled seed drain also advances admitted research and then settles the Workspace’s Work Credit hold from the acquisition’s terminal state, including after a crash between provider completion and settlement. A separate durable research control defaults disabled with zero authorized starts. While it is disabled, admitted research stays queued; bound runs still finish. Authorized attempts consume its cumulative limit even if rejected or unknown.

Research whose provider start remains start_unknown is eligible for automatic hold release 24 hours after its persisted dispatch time (starting_at). Scheduled reconciliation releases the hold with start_unconfirmed even without provider configuration or enabled new starts. Queue time does not count, and known running work is not expired by this rule. A released hold cannot later settle: Mosaic bears unresolved provider cost without retroactively charging the customer. The acquisition stays start_unknown, retaining the internal uncertainty fence and any later provider identifiers for operator reconciliation. Release runs on the next available bounded scheduler pass; scheduler outages or backlog can delay it.

Only successful research (complete, or capped at the provider result cap) updates Brand or target coverage. Failed, interrupted, unresolved, or in-flight research is invisible to public coverage. Research that reached a provider attempt (completed, failed after start, or with an unknown start) counts as earlier Brand+Page work for the initial seed, so the seed does not collect or rebill that Page automatically. Research that never dispatched (queued behind its gate, or failed before dispatch) spent nothing and neither blocks seed enrollment nor fails an admitted seed at dispatch.

A provider-uncertainty fence, checked under the Brand lock at admission and again at dispatch, keeps research from adding a paid start beside other work for the same Brand and Page: an unfinished Catalog bootstrap, or any acquisition with the same pinned scope that is starting, running, or start_unknown. It ignores Work Credit state and target decision IDs, so releasing an unknown-start hold or reconfirming the Page does not clear it. Admission answers collection_in_progress without holding credits; a queued request waits at dispatch. An unknown start therefore blocks research for that Page until an operator reconciles the acquisition. Local positive-attribution recovery now verifies a pinned successful existing run and records an immutable audit under migration 0108; it never purchases a replacement or charges a released hold. The operator CLI defaults to dry-run, with separate verification/apply modes and reviewed operator authority required for apply. No live operator role or recovery has been verified.

The original admission slice required migration 0101; the funded/reuse/Watch release now also requires the later migrations above, a Worker deployment, explicit tariffs, funding, reviewed least-privilege runtime permissions, and enable/budget approval. Table ownership remains with the non-runtime migration owner; local grant plans are not live runtime access. The non-owner research fixture uses explicit read/admission grants and denies Credit minting, control enable/budget changes, canonical edits, source-state updates, and settlement. Source execution uses a separate owner connection in that disposable fixture. This verifies the admission boundary, not deployment of the future production role matrix. None of these production actions is implied by local implementation or test results.

Customer YouTube Channel research

A paid Workspace YouTube Channel research request (see the API reference) commits its Work Credit hold, its research_channel job, and that job’s outbox row in one transaction. The ADR 0019 core runtime (createYouTubeCoreShadow) executes it; nothing else does, so execution needs the core in shadow and running. Recurring demand is not involved: research never enrolls a schedule, feed poll, hint, refresh, continuation, or acquisition checkpoint.

One paid start. The job lists the newest Videos-tab page (tab: videos, maxPages: 1, listing-only) through the job-owned Apify protocol (JobOwnedApifyActorRuns), with a one-shot start authority in place of demand. Inside the dispatch transaction, that authority requires the request’s own job, a held hold, an enabled research control, an active ungoverned Channel, no unsettled research start for the same Channel and scope, and fewer cumulative research starts than authorized_start_limit. The core’s Apify daily cap and per-run reservation also apply. A refused dispatch writes nothing and the job retries in an hour. A known run that cannot be read yet resumes on retry; a lost start completes the job as provider_outcome_unknown. Canary execution also constrains research to its selected Channel, at handler execution and paid dispatch. If scope narrows after a known start, the job only recovers provider accounting and releases the customer hold; it does not fetch descriptions outside the new scope.

Reads and documents. Videos that already have a current document are used as they are and never re-observed. The rest are read with at most one budgeted Data API videos.list call per attempt and recorded through the shared ingestion path with acquisition origin research. Projection records suppressed/research for that evidence for every Tracked Brand, now and later, so research creates no Event, Credit, or delivery. A Channel’s monitoring baseline ignores research-only documents via durable monitoring_first_observed_at (migration 0105), set only by the first accepted non-research observation and preserved through purge. Failed/released research without a receipt therefore cannot become monitoring history after retention. Later demand still treats the gap as silent Channel-addition history. Research documents follow the normal 30-day retention and purge; refreshes stay demand-gated.

Settlement. One claim-fenced transaction writes the immutable youtube_channel_research_receipt and settles the hold against it (receipt_kind = youtube_channel_research_receipt, its own youtube_receipt_id RESTRICT reference and unique index; Meta receipts keep receipt_id). The Actor must certify a complete bounded listing page: partial, degraded or unverified listing coverage releases the hold, even when it returned zero rows. Known Shorts and broadcasts are excluded from the sample, including broadcasts identified by official metadata; no extra page is acquired to replace them. A complete empty page can settle as valid bounded work. Gaps, provider verdicts, and Channel loss release instead. The core tick’s reconciliation releases held research whose job stopped: a lost start 24 hours after its dispatch (start_unconfirmed), otherwise acquisition_failed.

Uncertainty fence and operator recovery. The fence is derived from the research jobs’ durable provider_runs: any research start for the Channel and scope with no run ID, or a known run not yet settled, in any Workspace and whatever its hold or key. Hold release does not clear it. An internal operator reconciliation can attach exactly one previously unattributed run, verified against the pinned Actor/build, stored input hash and bounded dispatch window. It records an immutable audit decision and reopens the existing job to resume that run, never to start another. A held request within the 24-hour deadline can complete normally. An already released or expired hold stays uncharged; recovery only settles internal provider spend and does not produce a new customer result. Reconciliation enforces the expired deadline even if scheduled release has not run.

An empty/incomplete provider search is not authoritative proof of no run, so there is no withdrawal, provider-refund or force-clear path. Missing, deleted, delayed, unpinned or ambiguous runs stay fenced. The positive-attribution window is conservative (five minutes before to ten minutes after dispatch), not a guarantee of provider acceptance timing. No customer REST/MCP reconciliation command exists. The local operator CLI and grant plans need a separately reviewed release. Meta now has its own local positive-attribution lifecycle/audit migration (0108); neither source supports absence-based force-clear or direct row edits as recovery.

Least privilege. The disposable customer-role fixture admits and reads with SELECT and INSERT on the request, reservation, youtube_job, and youtube_job_outbox only. It is denied Credit grants, any control change, receipts, settlement or release, job state, budgets, and every canonical or source table. A job it forges outside admission starts nothing. The planned mosaic_youtube_core_shadow role gains read access to the control and request, INSERT on receipts, and SELECT, UPDATE on reservations (settle or release); it cannot grant Credits, change the control, or create requests or holds. Admission and dispatch serialize on a transaction advisory lock and read the control without row locks, so neither role needs a write grant on it. The later funding/reuse/Watch slice adds explicit application grant plans; these fixture boundaries do not prove any production role is provisioned.

The original YouTube slice required migrations through 0105; the combined release now includes the later funding/reuse/Watch chain above, core composition, an explicit Work Credit tariff, funding, reviewed runtime permissions, and enable and start-limit approval. None of these is implied by local implementation or tests.

External customer sign-in is separately controlled by MOSAIC_EXTERNAL_CUSTOMER_ADMISSION, which defaults to disabled. Enabling it admits verified, well-formed non-staff WorkOS identities; Workspace authority still comes from active signed membership, never an email-domain override. Global curation and governance require a verified staff identity with an admin membership. Before deploying this authorization change, inspect staff memberships: a staff member with role member no longer receives implicit admin capabilities. Changing the live admission binding requires explicit approval and does not enable research or authorize collection.

Meta identity images

New staff-confirmed Meta targets and newly projected eligible creator evidence enroll independent avatar work in the same transaction. Candidate targets, held-only evidence, and replay of an already-projected receipt do not enroll work. The migration does not scan or backfill historical accounts. A newly projected eligible observation may enroll a previously known account.

The existing scheduler claims at most ten images per invocation, one at a time. Each subject has one durable lifecycle with a two-minute lease, five attempts including crashed attempts, and exponential retry delays starting at one minute. Expired claims can be reclaimed; stale claims cannot settle a newer attempt. Terminal outcomes are retained rather than reset on rediscovery. Graph Page picture acquisition validates actual dimensions of at least 720×720 and stores immutable bytes in R2. Ads Library 60×60 thumbnails are not accepted. Target ownership and Creator account assignments are rechecked before fetching; existing image selection history, including revocation, prevents automatic acquisition. A racing staff selection wins through the revision guard.

Image failures never rerun paid Ads Library collection or change its completeness, Activity, or Credits. This is bounded retry, not exactly-once HTTP or guaranteed image availability. It requires the avatar-work migration and Worker deployment; it is not a production deployment claim.

Historical Meta evidence admission and release scope

The earlier Skylight and AG1 milestone was a one-time release, not a recurring fleet rollout. Existing daily reconciliation and weekly claims are implementation infrastructure, not proof that collection or publication is activated. Missing APIFY_TOKEN is not a scheduler off switch: reconciliation can still write roster claims and enqueue work. Production operations require a separate approved plan.

Local implementations #144 and #145 replace the former nonempty-dataset/public-ingest behavior. The loader binds a successful provider run, approved immutable build, Page, US/ACTIVE input, dataset, key-value store, OUTPUT and counts. Build approval defaults empty. A historical pointer is not a fallback proof of fresh complete collection; SUCCEEDED alone is not completeness.

Validated complete, capped and interrupted collections may be retained as immutable private receipts and allowlisted rows. Admission is atomic per receipt. Private retention precedes the complete-only pointer freshness fence; incomplete collections never advance that pointer or write public ad rows. Verified complete zero-row collections are valid coverage, not generic failure. Malformed or incompletely fetched datasets are not valid interrupted evidence.

An identical retry preserves the receipt and its timestamps. Recovery after retention targets that exact receipt/run, not a newly discovered run. Transient persistence failures may retry; permanent pointer conflict preserves private evidence, reports complete_failed and does not Queue-retry indefinitely. A later daily reconciliation may revisit the still-pending weekly claim. retained_incomplete is acknowledged as incomplete coverage, never collected.

Retained rows are minimized at admission (ADR 0016). A brand-only ad — one Mosaic classifies as having no tagged party, or the same display and tagged Page — keeps its archive ID, Brand Page, display/tagged Page identity and classification, but its creative copy, title, link and media are stored as empty. Explicit creator-partnership ads, held rows and ads between two other Pages keep their full allowlisted fields. The receipt replay hash covers the minimized rows, so an identical retry still matches. Migration 0088_retire_meta_publication applies the same rule to stored rows that already have a pinned brand-only classification and support no Relationship evidence; their receipts keep the original acquisition-time replay hash.

These are locally tested implementation claims, not production acceptance. Partnership Ads publication is retired: Mosaic produces no Public projections, signed Public snapshots, or permitted durable previews.

Exactly-once Channel observation

A canonical Channel is collected once for all interested Workspaces. Duplicate Workspace interest, scheduled work, WebSub notifications, and queue retries coalesce through a durable request marker and collection lease. One source observation is persisted before Workspace-specific Competitor and Event projection. Retries reuse durable observation and Event identities rather than duplicating source I/O or customer Evidence.

A completely unobserved transient pass releases its lease for retry. A terminal source result is persisted as Coverage and may tombstone the canonical Channel with a bounded reason. Tombstones do not disclose which Workspaces previously referenced a Channel.

WebSub hints and renewal

WebSub is an untrusted best-effort hint. The callback URL is an opaque bearer capability; subscription verification also requires the exact topic. Notification bodies and hub signatures are not currently authenticated or parsed, so notifications remain Channel-only wake-up hints. Data API/RSS reconciliation remains authoritative recovery for notification loss, duplicate notifications, and enqueue loss. Description edits are not new observations in the frozen-evidence model, and notification video IDs outside the recent-upload window are not targeted.

Scheduled subscription scans include active Channels missing subscription rows, recovering admission/setup crashes. Initial and renewal requests both send the configured lease bound. A notification arriving during collection preserves one newer durable request; duplicates coalesce, and scheduled reconciliation recovers a lost follow-up enqueue.

The scheduler renews pending or due subscriptions with a bounded lease. Production defaults use a two-day renewal lead and a seven-day maximum lease. Repeated verification of an active subscription echoes the new challenge without creating another subscription. Hub failures retain durable intent and become sanitized subscription Coverage.

Scheduled reconciliation

Each Cron invocation is bounded and fair:

• persisted cursors rotate across due Channels and subscriptions;
• defaults allow 500 IDs per page, 24 pages, and no more than 12,000 unique IDs per pass;
• default enqueue concurrency is 8, independently from Queue consumer concurrency;
• collection completion schedules the next observation using the roster-scaled interval and stable per-Channel jitter; the historical OFF configuration retains its 24-hour minimum;
• a busy Channel retries after the full 15-minute collection lease;
• queue payloads contain only canonical collection or creator-partnership Destination jobs.

Legacy operator YouTube universe seed

The staff milestone uses staff-added Channels only (ADR 0008); there is no required 100-Channel launch roster. The existing seed CLI below remains a legacy tool, not a milestone prerequisite or permission to seed production. It accepts JSON with exactly 100 unique canonical YouTube Channel IDs:

{ "sourceChannelIds": ["UC...", "..."] }

Validate the roster without writing:

pnpm --filter @mosaic/api seed:youtube-universe -- --roster /absolute/path/roster.json

Apply to an owned local Postgres database. Put one loopback URI in a file that is not committed. Mosaic never reads DATABASE_URL. The URI must be:

postgres://USER[:PASSWORD]@127.0.0.1:PORT/DATABASE

or the same form with postgresql://. localhost may replace 127.0.0.1. Port and database are required. Query strings, fragments, extra @, comma-separated hosts, IPv6, and unix sockets are rejected. Mosaic parses that URI once and connects with the resulting host, port, database, and user — not the raw string.

pnpm --filter @mosaic/api seed:youtube-universe -- --roster /absolute/path/roster.json --apply --local-postgres-url-file /absolute/path/local-postgres-url

Repeating apply is safe: existing Channels keep their original admission time. Production apply remains separately authorized. The old 100-Channel roster is no longer a required launch input. Eligible staff additions use the human admission route; the newly agreed silent Channel-addition baseline is a requirement to implement and verify, not a claim of live behavior. Staff Workspaces fund monitoring with operator Work Credit grants (ADR 0023).

Coverage

GET /v1/sources/youtube/coverage reports shared indexed/healthy counts, Channel tombstones, and sanitized recent source failures. It never exposes Workspace interest, raw descriptions, credentials, callback capabilities, or provider internals. An observed description with no Matcher hit is not a Coverage failure.

YouTube Data API errors and quota

A failed Data API request keeps Google’s HTTP status and error.errors[].reason (plus error.details[].reason) tokens, never Google’s message, the API key, or a request URL with the key. The Worker logs one youtube_data_api_error line with callType, kind, httpStatus, reasons, and, on quota exhaustion, pausedUntil. The persisted Coverage code stays collector_unavailable; the Coverage message names the reason:

Google reason	kind	Coverage message detail	Behavior
quotaExceeded, dailyLimitExceeded	quota_exhausted	quota exhausted (HTTP 403 …); Data API paused until the Pacific reset	No Data API call until the next Pacific midnight
(while paused)	—	paused after quota exhaustion until …; no request sent	RSS fallback only
rateLimitExceeded, userRateLimitExceeded	rate_limited	rate limited (HTTP 403 …)	Next run retries
keyInvalid, keyExpired, API_KEY_INVALID	key_invalid	API key rejected as invalid or expired (HTTP 400 …)	Next run retries; fix the Worker secret
accessNotConfigured, SERVICE_DISABLED	access_not_configured	YouTube Data API v3 is not enabled for the key’s Google Cloud project	Next run retries; enable the API on the key’s project
forbidden or another 403	forbidden	request forbidden for this key (HTTP 403 …)	Next run retries
other non-2xx	http_error	HTTP status and reasons	Next run retries

Each failure falls back to official RSS without fetching watch pages. When RSS succeeds, the Data API failure is still recorded as Coverage for that Channel, because an RSS fallback is not complete official coverage. While the Data API is paused, format classification also waits, so new Mention evaluations stay pending until the reset.

Coverage youtubeDataApi reports Mosaic’s estimate of Data API spend: for each of the newest seven UTC days, calls, failed calls, and estimated units per channels.list, playlistItems.list, and videos.list (one unit per call, including rejected requests), beside the modeled 10,000-unit default. pausedUntil is present during a quota pause, and lastError is the most recent classified failure. Google’s quota day resets at midnight America/Los_Angeles, so a UTC day can span two quota days. These are estimates from this Worker, not Google Cloud console figures. The field is omitted when no quota ledger is configured.

youtubeDataApi.descriptionRefetch reports the expired-description refetch: the Pacific quotaDay, dailyBudgetUnits and spentUnits reserved that day, counts of pending, admitted, and terminal unavailable refetches, lastAttemptAt, and up to 20 recentUnavailable videos.

ADR 0019 core in shadow

The ADR 0019 core (documents, the link index, Mentions, and the subscription projection) can run on the server Worker in shadow beside the current pipeline, which stays authoritative. In shadow the core writes only its own tables (youtube_video … youtube_projection_decision and the migration 0100 tables youtube_core_budget, youtube_core_video_hint, and youtube_core_channel_schedule), plus the intentionally shared link-resolution and Data API quota cache. It never delivers, routes webhooks or Slack messages, bills, or writes a current-pipeline table: its youtube_event rows are projections to compare, it never charges Work Credits (ADR 0023), and its Event outbox is not relayed.

Each current core link keeps its first starting position in the latest description. Every observation re-derives youtube_mention.sponsorship_inference using the first-half rule. Moving the link to the second half or removing it produces not_inferred; removal also sets link_removed. Discovery evidence keeps its original position and description length, and retirement keeps the last inference beside purged or unavailable. This core value remains shadow-only; the public Mention resource uses canonical accepted-description evidence.

Setting (server Worker)	Default	Meaning
MOSAIC_YOUTUBE_CORE_MODE	off	off or shadow; any other value is off
MOSAIC_YOUTUBE_CORE_DAILY_DATA_API_UNITS	500	Core Data API units per Pacific quota day (0–2,000)
MOSAIC_YOUTUBE_CORE_DAILY_APIFY_USD	1	Core Apify USD per UTC day; an overrun closes the day (0–20; see below)
MOSAIC_YOUTUBE_CORE_FEED_POLL_MINUTES	30	Minutes between RSS polls of one Channel (10–120)
MOSAIC_YOUTUBE_CORE_RECONCILIATION_HOURS	24	Hours between Actor reconciliations of one Channel
MOSAIC_YOUTUBE_CORE_DEMAND_CHANNELS	absent	1–25 distinct canonical UC… Channel IDs, comma-separated
MOSAIC_YOUTUBE_CORE_DEMAND_UNTIL	absent	Required finite ISO-8601 expiry for that operator demand

An invalid value fails closed to off. Shadow additionally needs the APIFY_TOKEN secret and the HYPERDRIVE_YOUTUBE_CORE_SHADOW binding; without either, every tick logs not_configured with the missing names and does nothing, and a WebSub callback records nothing for the core. With MOSAIC_YOUTUBE_EXECUTION_MODE disabled it reports execution_disabled; in canary it schedules only the canary Channel, and only when that Channel is explicitly demanded.

Recurring acquisition is default-denied. Both demand settings are required. Catalog inclusion, one-shot research, an existing subscription, a retained document, unrestricted execution, or canary selection does not authorize recurring acquisition. Demand gates enrollment, feed/reconciliation claims, hint batching, WebSub, refresh hints, queued-job execution and new paid dispatch. Expiry is exclusive. Missing demand or expiry stops new acquisition but keeps catalog data, retention/purge, local matching and source accounting. An already-issued network request is not canceled by expiry. Mixed queued batches stand down and return all their hints to pending; the next claim selects only demanded Channels. This is a bounded operator safety control, not a customer Watch API or recurring Credit budget. It does not authorize live collection or establish source rights.

Change detection. The core adds no second subscription system. Its triggers are:

• WebSub. The existing subscriptions and renewals are unchanged. After the legacy callback accepts a verified notification, the Worker reads the body (at most 256 KiB), parses at most 50 Atom entries (videoId, channelId, updated, and deletions), drops entries for any Channel other than the callback’s, and persists one hint per video revision before answering 202. The hint is a trigger only; the video is always read from the Data API. A repeated revision coalesces, and a description edit (a new updated) reads the video again even when its document is current. A core failure is logged and never changes the legacy answer.
• RSS. Every demanded Channel’s public feed is polled every 30 minutes on a staggered slot, as recovery for missed pushes; entries become hints the same way, so a revision already pushed coalesces.
• Refreshes. Demanded Channels’ retained documents due for their weekly refresh become hints too.
• Reconciliation. Once a day per demanded Channel (staggered; a new demanded Channel’s first one, the admission backfill, within an hour), the pinned Actor lists the Videos and Shorts tabs from the reconciliation checkpoint with three days of overlap. When the two-page limit ends a listing first, the core persists the Actor’s continuation token with the listing so far and enqueues the next run instead of declaring completion; the checkpoint advances only when the chain completes. The feed checkpoint and the reconciliation checkpoint are separate.

Pending hints are coalesced into one videos.list read of up to 50 videos (one unit). A partial batch waits at most 15 minutes for more hints. Channels are escalated one at a time instead of polling everyone more: three consecutive feed failures or a feed whose 15 entries are all newer than its last checkpoint (a possible gap) bring the reconciliation forward; those, more than 20 hinted videos in a day, and a continuation chain longer than five runs poll that Channel’s feed every 10 minutes for 24 hours.

Scheduling. Each cron tick (every 5 minutes) gives new demanded Channels a schedule row, claims due feed polls, reconciliations, and ripe hint batches, enqueues one match_brand per active catalog Brand whose rules changed (keyed by the rules digest) and the retirement and Tracked Brand projection work, then runs the recovery sweep and relays due job IDs onto the collection queue as youtube_core_job messages. Every claim is one transaction: it locks the Channel’s schedule row or the pending hints (FOR UPDATE SKIP LOCKED), rechecks the due instant and open jobs with a compare-and-set, and inserts the job and its outbox row, keyed by the claimed slot or batch. Two overlapping ticks or isolates therefore never both create or pay for the same work. A consumer claims a job with a compare-and-set on a 20-minute lease; every message is acknowledged, retries and deferrals go back through the job outbox, and an expired lease or lost message is re-sent by the next sweep. A tick failure is logged and never fails the current pipeline’s scheduled run.

Sources. Listings come from brianatpaceline/youtube-channel-videos build 8cDx80kbsQWtfnHEK (started as version 0.2.3 and rejected unless the run reports that build ID), listing-only (includeExtras: false), each run started with 512 MB, a 240-second timeout, and maxTotalChargeUsd 0.02. Descriptions come from Data API videos.list; the Actor’s watch-page extras (enrichOnlyVideoIds, at most five watch fetches) are asked only for videos the Data API returned without a description. A paused or failed Data API read defers the work instead of moving the load to Apify.

Paid runs. Each Actor run belongs to its claimed core job, recorded by input SHA-256 in youtube_job.provider_runs (migration 0102). Dispatch records the marker and budget reservation in one transaction fenced on the claim token. A known 4xx rejection removes the marker and releases the reservation. A lost start response or crash before saving the run ID keeps the reservation and prevents another start by that job. Reconciliation reports provider_outcome_unknown without advancing its checkpoint; a later daily slot can collect again. Retries of known runs resume the saved ID after transient poll, INPUT, or dataset-read failures. A run’s outcome and its charge are recorded separately. A reported charge settles once on the reservation’s day. A finished run with no reported charge keeps its full reservation, and the charge is recorded as unknown. Each core tick re-reads such charges for up to 7 days after the run finished, whatever the job or Work Credit hold state. It settles a later-reported charge once and never changes Work Credits. Health reports those runs as unsettledRuns. Exhausting the five job attempts while a run remains in flight conservatively retains its reservation; a known failed or unpinned run is not re-paid on retry. If Videos succeeds but the Shorts start is lost, the combined listing is deferred until a later daily slot.

After demand ends, recovery may poll and settle known unsettled runs, but never starts another run. Recovered listing data is discarded and the checkpoint does not advance. Unknown starts keep their internal reservation/fence. Expired demand does not grant permission to finish a new Videos/Shorts acquisition chain.

Budgets. Budgets live in youtube_core_budget, one typed, nonnegative amount per kind and day. Every core Data API call first reserves one unit under the core’s own cap for the Pacific quota day, in the same transaction as the UTC-day report row, with one conditional atomic update; a refused reservation defers the job to the next Pacific midnight without calling Google, and a refund returns the unit to the day it was reserved on. Calls are also metered on the shared ledger, and a Google quota error pauses every consumer, the core included, until the reset. Every Actor run reserves $0.02 under the core’s UTC-day Apify cap before it starts and settles, on the reservation’s day, to Apify’s reported usageTotalUsd (or keeps the reservation when none is reported); a refused reservation defers the job to the next UTC midnight. Apify cannot enforce the reservation: the Actor is compute-priced, not pay-per-event, so maxTotalChargeUsd does not bound it. Memory and timeout bound its compute to about 0.034 CU (about $0.013), but residential-proxy traffic is billed on top, so one run can exceed its reservation. Mosaic therefore treats any run charged above its reservation as proof the reservation is not a bound: the excess is recorded for that UTC day, the day closes to every further run, and research health raises youtube_spend_overrun. The day’s spend is then at most the cap plus runs already in flight. Other Mosaic Data API use at a 100-Channel roster is estimated at about 7,600 units per day (scheduled polling capped at 7,000, plus WebSub hints, admissions, avatars, and the refetch), and Discovery’s own cap is 300, so the core’s 500 fits inside the 10,000-unit project quota.

Comparison. GET /control-plane/youtube/shadow-comparison?since=…&until=… (Mosaic staff browser sessions only; at most 31 days; default the last 7) compares, per catalog Brand × video first discovered in the window by either pipeline, the current pipeline’s Mention (relationship evidence and its Events) with the core’s youtube_mention (and its projected Events). It returns the core’s status, summary counts, up to 500 disagreeing pairs, and each pipeline’s Data API units and Apify USD per UTC day. The core’s spend comes from its own budget rows. The current pipeline’s spend is the shared ledger minus the core’s, for the seven days the ledger keeps; it runs no Apify Actor. The route is not an OpenAPI operation or MCP tool and is excluded from request logs.

Disagreement	Meaning
only_old	Only the current pipeline has the Mention
only_new	Only the core has the Mention
timing	Both have it; first discovery differs by more than twice the core’s detection latency
kind	Both have it; live, silent, or no Event differs

The core’s detection latency is one feed-poll interval plus the 15-minute batch delay (45 minutes by default), so timing tolerates 90 minutes. Reasons, where derivable: channel_not_acquired, description_not_fetched, video_unavailable, document_purged, no_matching_link (only old); link_via_redirect, reverse_match, edit_discovery, description_not_fetched_by_old (only new); acquisition_lag, old_lag (timing); outside_tracking_interval, unsubscribed, capped, channel_addition_baseline, brand_activation_baseline, matcher_addition_baseline, refresh_baseline, not_projected (kind). Pairs whose video or Channel is suppressed or erased are counted as governed and never listed. The first tick of each UTC day logs one youtube_core_shadow line with kind: "daily_summary", the previous day’s counts, and both pipelines’ spend; other ticks log kind: "tick" only when they did work or the core is not running.

Core shadow release checklist

The repository names what a release provisions; it performs none of it. In order:

1. Migrations. 0096–0100, 0102, and 0103 applied to mosaic/main (0100 adds budgets, hints, schedules and core job kinds; 0102 adds paid-run records; 0103 adds link positions and sponsorship inference). Migration 0101 separately provisions Meta research; 0104 adds YouTube Channel research and is needed once the 38-grant plan is applied.
2. PlanetScale managed login. pscale role create mosaic main mosaic-youtube-core-shadow-<yyyymmdd> --org brianjyip --format json, with no inherited roles. Capture the one-time credential directly into the Hyperdrive create step; never print it.
3. Grants. An administrator runs youtubeCoreShadowGrantStatements(<login>) from packages/infra/youtube-core-shadow-release.ts: CONNECT, USAGE on public, and exactly the 38 planned youtube-core-shadow table grants: writes to the 16 core tables and the shared link_resolution_cache; one-shot research execution (read the research control and requests, insert receipts, settle or release reservations); and reads of the roster and WebSub subscriptions, catalog, tracking, billing periods and allowance, governance, and, for the comparison, the current pipeline’s relationship evidence, Events, and observations. It is a new capability; no role another consumer uses is changed. Verify with has_table_privilege that it cannot write any legacy table (observations, coverage failures, evaluations, subscriptions, scheduling state). The application role gains nothing.
4. Hyperdrive. Create mosaic-youtube-core-shadow-<yyyymmdd> on us-west-2.pg.psdb.cloud:5432, database postgres, caching off, with that login. Record its ID.
5. Release. Create one server version that binds HYPERDRIVE_YOUTUBE_CORE_SHADOW to that ID (never another login’s Hyperdrive) and sets MOSAIC_YOUTUBE_CORE_MODE=shadow plus any core setting that should differ from its code default. Every other binding is inherited from the reviewed version. Then pin the new version, its core Hyperdrive and its settings in creatorProfileReleaseBaseline (packages/infra/creator-profile-release-guard.ts), so the next release inherits exactly them.
6. Verify not_configured. The released core logs youtube_core_shadow ticks with state: "not_configured" and missing: ["apify_token"], and does nothing else.
7. Secret. The user adds APIFY_TOKEN to mosaic-server-production-77wkqyg2g4lfh3gf with cf workers secrets update, which creates a successor version.
8. Pin the successor. Review that version and deployment, set them in creatorProfileReleaseBaseline.server, and add APIFY_TOKEN to its optionalSecrets, so the next server release inherits the token from exactly that version. No guard accepts an unpinned successor. Every server release, including the full-stack deploy, derives its inheritance from the accepted version with serverReleaseInheritance: whatever that version carries and the release does not set itself, APIFY_TOKEN, the core Hyperdrive and the core settings included, is inherited from that version, and a release that would drop any of it is refused.
9. Verify. Watch youtube_core_shadow logs with cf observability, read the comparison route as staff, and confirm that the current pipeline’s deliveries and usage are unchanged.

The historical release settings above define no demand. Even with the token and database, the updated shadow runtime performs no new acquisition until a separately reviewed, explicitly approved deployment sets both demand bindings for the permitted Channels. Do not amend historical release pins to bypass their guards. Source-policy clearance and live collection approval remain prerequisites for such a deployment.

Expected spend, from measured runs. Apify: a reconciliation is two listing-only runs per Channel per day. A live listing run of the pinned build with the core’s bounds (512 MB, 240 s, 2026-09-30) cost $0.00024; the recorded 2026-09-29 listing run cost $0.0013 and the recorded two-video gap fill $0.0024. At $0.00024 a run, reconciliation costs about $0.0015 per day at the three production roster Channels and about $0.12 per day at 250 Channels (at most about $0.65 at the 2026-09-29 run’s cost), plus one gap fill per Data API description gap and one more reconciliation per admitted Channel’s backfill. Data API: about five units per day at the three roster Channels in the real-data replay. At 250 Channels the estimate is under 200 units per day: at most 96 partial batches, one unit per 50 hinted videos, the weekly refreshes in batches of 50, and a read only when a reconciliation lists a video the core has not read.

Research identity, classification and operations

Asynchronous Brand identity requests are distinct from source collection and from tracking-role Brand requests. They check the confirmed catalog first. Optional Context.dev retrieval is gated by brand_resolution_control.enabled and a cumulative max_external_starts; both default off/zero. Runtime can increment the external-start counter but cannot enable or raise its limit. A committed dispatch marker and shared enrichment fence prevent automatic retry after uncertain provider cost. An unknown numeric Meta Page requires curation; provider name/domain candidates never establish company ownership or admit collection. Requests consume no customer Work Credits.

Staff-only review approves a request against an already curated Brand/profile and confirmed Meta Page without invoking tracking-role Brand-request resolution. Immutable approval records bind the original reference to a governance snapshot. Public reads revalidate that snapshot and report stale on reassignment/revocation; they never trust mutable request JSON as approval. The customer role can read approvals but cannot insert them; a separately provisioned reviewer connection owns writes.

The offline classification challenge set exercises actual production Meta and YouTube rules. Run bun --no-env-file packages/api/src/classification-evaluation/run.ts from the repository. Its synthetic labels, source-specific confusion counts and rule/source hashes are diagnostics, not human-reviewed real-world accuracy. The current YouTube positional rule is inference and can misclassify affiliate links, self-promotion and ordinary citations. Explicit Meta attribution and inferred YouTube sponsorship remain different claims; neither verifies payment.

The separate offline youtube.attributed_signals.offline.v2 candidate distinguishes attributed disclosure, affiliate, self-promotion, ordinary links and uncertainty; it does not replace production v1. The --corpus evaluation path requires versioned lawful capture provenance, hashes, independent annotations and adjudication, and separate development/holdout groups. No real human-reviewed corpus is bundled. Research evidence exposes only bounded redacted hostname support from matching, unexpired discovery/verification observations; missing raw support is unavailable, not reconstructed.

The server scheduler drains identity requests and signed research callbacks and emits structured research_health snapshots and research_health_alert warnings. Signals cover hour-old research holds/identity queues/callbacks, ambiguous starts, 24-hour unknown-start release deadlines, and dead callbacks. Unknown-start deadlines use dispatch time, not admission time. Health checks never release funds, retry providers, or change source authority. Snapshot reads have a ten-second transaction-local timeout. research_health_unavailable means no health result, not zero failures. billing_callback_processing_failed reports rejected processing without payloads or credentials.

Persisted YouTube run costs distinguish reserved from settled USD. Meta catalog/bootstrap and research dispatch reserve $1 per physical start under the independent UTC-day meta_research_budget_control gate (disabled, zero USD by default), in addition to existing cumulative start gates. Unknown starts retain reservations even after customer holds expire; only positive pre-start rejection releases them. A unique run ID prevents duplicate USD charges on shared reattachment. This is admission accounting, not an invoice guarantee: actual costs above the reservation are recorded unclamped.

The source drain reads owner-authenticated Apify usageTotalUsd independently of OUTPUT, including failed runs, after a terminal stabilization delay of at least ten seconds. Its token must read both actor-runs and users/me. Non-owner, missing or unavailable totals remain unknown, not zero; there is no provider final-cost flag. Bounded later reconciliation reads up to twenty runs even after acquisition completion. readMetaResearchCostReport accepts at most 31 UTC days and groups costs by reservation day, purpose and outcome with nullable actual USD and explicit unknown counts.

Complete health snapshots drive a private persisted operator alert outbox with firing/recovered transitions. Configure the paired RESEARCH_OPERATOR_ALERT_URL / RESEARCH_OPERATOR_ALERT_SECRET (at least 32 characters); partial configuration fails startup, and absent configuration retains signals without delivery attempts. Delivery uses public-connect transport, not customer endpoints. An independent monitor calls POST /internal/research/heartbeat-check every five minutes with a separate RESEARCH_HEARTBEAT_CHECK_TOKEN Bearer token (at least 32 characters). It must also alert on timeout, auth failures, and non-2xx responses: the Worker cannot detect its own total outage. The endpoint returns 200 when healthy, 503 when a successful scheduler completion is at least 15 minutes overdue, 401 for bad auth, 405 for non-POST, and 503 when unconfigured. It never advances the heartbeat; only a fully successful scheduled workload does.

Operator payloads carry {version,eventId,code,lifecycle,revision,timestamp,count,units} and the same HMAC headers as research callbacks. Receivers deduplicate eventId and ignore older per-code revisions: retries can arrive out of order. Eight attempts use 60-second leases, 15-second HTTP timeouts and 30-second doubling backoff capped at an hour for network/408/425/429/5xx; other 3xx/4xx are terminal. Retained dead/unknown counts are inventories, not incident rates. Billing failures remain private to its inbox, not the application health reader. No live alert routing, provider prices, delivery latency SLA or real agent onboarding has been verified by local tests.

Public-link resolution

Mosaic preserves every bounded URL occurrence and a structured outcome:

• 10-second request timeout and at most 5 redirects per occurrence;
• at most 25 occurrences per description and a 30-second total sequential deadline;
• at most 64 KiB drained from a response;
• only safe public HTTP(S) hosts and default ports;
• GET-only requests with an explicit Mosaic user agent and no copied credentials;
• successful cache entries revalidated after 1 hour; blocked or failed entries after 5 minutes;
• fragment-free cache keys and strict expiry.

A Mention matches a confirmed hostname as it appears in the accepted frozen official description, or the resolved destination of a listed shortened or affiliate link as described above. Failed or blocked targets never become domain matches. Shortener labels identify transport infrastructure only, not verified brand or partnership identity.