# Questions and answers about Ruslan Vasylev's work

Grouped by system. Items marked "(proposed)" or "Proposed application" are designs, not delivered work.

## API

Request validation, access checks and application responses. The API service connects browser requests to application records and background execution. Fastify routes in the independent reconstruction work authenticate callers and expose durable job and result state; separate employer API teams retain their own ownership.

### How does the browser request work?

Follow-up: What would your API return before analysis is finished?

**Approach.** Return a durable job identity and explicit pending state, with separate authorized queries for status and verified results.

1. Authorize and validate the command: Check project access, request shape, supported inputs, and admission limits before accepting analysis.
2. Return durable acceptance: Respond with an accepted status, job and revision identities, and a way to query progress once the request is safely recorded.
3. Define a clear status contract: Distinguish queued, running, verifying, failed, incomplete, and available states with useful error and retry information.
4. Expose results with scope: Return findings tied to their revision and run, with pagination and explicit coverage when only some checks are complete.
5. Enforce access on every read: Authorize status, findings, and artifact access independently and test repeated submissions and cross-project identifier guesses.

I would return an accepted response after the request and its queue handoff are durably recorded. The response would identify the job and source revision, describe its current state, and provide a way to query status. Acceptance would mean that processing was requested successfully, while result availability would require completed validation. Status responses would distinguish queueing, execution, verification, failure, and incomplete analysis, with stable error categories and retry guidance. Findings would be queried separately and tied to the exact run, with pagination and explicit check coverage. An empty findings list during processing would never mean a pass. Authorization would apply to submission, status, findings, and artifact access, so possession of an identifier grants nothing by itself. I would test duplicate requests, delayed updates, expired access, and attempts to read another project's results.

Tradeoffs and limits:

- Polling is simple but adds request load; event updates need reconnect and reconciliation behavior.
- Partial results can be useful if the contract makes their coverage and remaining work unmistakable.

**Work example.** 
- double.digital, API admission and result states: Routes authenticate the caller and resolve access to the job. Admission returns durable job identity, queued state and execution readiness: 201 for new work or 200 for an idempotent replay. State and artifacts are queried separately. Completion can return 202 while verification is pending, and blocked, retryable, terminal and partial states stay explicit.

**Proposed application, not delivered work.** For CAD, I would retain explicit accepted, processing, incomplete, failed and available states with actionable error codes. 

Hypothetical examples:

- Commands and queries: request analysis, fetch status, list findings. Example: POST starts work; GET returns status and paginated results.
- Access control: enforce project/tenant permissions on every operation. Example: a guessed job ID must not expose another customer’s model.
- Consistency: distinguish accepted work from completed results. Example: the UI displays ‘processing’ rather than interpreting no findings as ‘passed’.

## Database

Application records, relationships and transactional changes. Database records connect requests, jobs, revisions and saved findings. Stable identities and atomic admission prevent concurrent or repeated requests from creating conflicting work. Local IndexedDB persistence and server-side PostgreSQL have different scopes.

### Which revision produced this result?

Follow-up: How would you prevent stale results from replacing newer findings?

**Approach.** Bind every result to immutable inputs, and move the current-result pointer only when revision and attempt checks still agree.

1. Define the complete run identity: Record the source revision, configuration, pipeline, rules, reference data, preprocessing, and model versions that affect the result.
2. Keep attempts and results distinct: Write artifacts under separate attempt and result identities so concurrent executions cannot overwrite each other's output.
3. Validate before publication: Check required artifacts, integrity, and stage coverage while the result remains unavailable to ordinary readers.
4. Fence obsolete completion: Conditionally accept publication only from the active attempt, rejecting late writes from superseded workers.
5. Update the current pointer conditionally: Publish the validated result and update the selected result only if the intended revision and configuration still match.
6. Guard the viewer selection: Render responses only when their revision and run match the current selection, and test completion in reverse order.

I would give each analysis an immutable identity that includes its source revision and the versions of settings, rules, reference data, preprocessing, and models that affect the output. Attempts would write to separate artifact locations, and a result would remain unavailable until its manifest and required stage coverage were validated. Publication would check that the completing attempt still owns the work. A transaction would then expose the result metadata and conditionally update the current-result pointer only if the user's intended revision and configuration still match. Older valid runs could remain available in history without replacing the selected result. The viewer would also reject delayed responses for a previous selection. I would test two revisions finishing in reverse order, a superseded worker completing late, and a crash between artifact upload and metadata publication.

Tradeoffs and limits:

- Version identity enables traceability but does not by itself guarantee identical output from nondeterministic computation.
- Atomic visibility comes from publishing verified metadata together; artifact uploads need separate validation and orphan cleanup.

**Work example.** 
- double.digital, Asset + job lineage: The database connects a result to the job and source that produced it. A completed processing job and a useful, reviewed reconstruction are separate facts.
- double.digital, Completion and recovery: The worker uploads outputs and saves the exact completion report before sending it to the API. A lost response can be checked by report hash; after host loss, reconciliation can recover the stored report. A durable finalization queue verifies artifacts and commits them with job state. Attempt, worker, lease and input checks fence stale writers.
- double.digital, Stale viewer responses: The viewer rejects responses that no longer match the selected capture and reconstruction run. Job source, processing options and pipeline identity remain attached to that run.

**Proposed application, not delivered work.** For CAD, I would retain separate result sets per revision/configuration and transactionally compare the intended revision before updating a current-result pointer. Revision/configuration result sets and a transactional current-result pointer.

Hypothetical examples:

- Reproducibility: identify input revision, pipeline, rules, reference data and ML versions. Example: explain why yesterday’s analysis differs from today’s.
- Atomic publication: avoid exposing half-written result sets. Example: write a result set, validate completeness, then mark it available.
- Concurrency: an older analysis may finish after a newer one. Example: preserve both runs without overwriting the current revision’s status.

## Object storage

Large source files, generated outputs and completion artifacts. Object storage holds file bytes while database records retain ownership and relationships. The reconstruction workflow uses S3 for source/result bytes and stored completion reports; verification connects those objects to the correct processing attempt.

### Where does each kind of data live?

Follow-up: Which data belongs where, and why?

**Approach.** Choose storage around consistency, size, and query needs, adding specialized systems only when a feature justifies them.

1. Identify transactional relationships: Put ownership, revisions, jobs, and result identities in relational storage where constraints protect valid state changes.
2. Separate large artifact bytes: Use object storage for source files and derived assets, with checksums and access-controlled references in the database.
3. Version variable payloads: Keep flexible diagnostics or settings in validated JSON with explicit schema versions and indexes for demonstrated queries.
4. Justify similarity retrieval: Add a vector index only for a defined retrieval feature, keeping candidates distinct from verified geometric findings.
5. Verify lifecycle and access: Test publication, retention, authorization, and recovery across database records and artifact storage.

I would start from the operations the product must support. Ownership, model revisions, jobs, and result relationships fit relational storage because transactions and uniqueness constraints help keep them consistent. Large source files and derived geometry fit object storage, with checksums, sizes, and controlled references recorded alongside their metadata. Variable settings and diagnostic fields can often live in versioned JSON within the relational system, provided they are validated and important queries remain practical. I would introduce a separate document store only for a demonstrated access or scaling need. A vector index would serve a specific similarity-retrieval feature, with embedding versions and permission filtering; retrieved candidates would still require relevance checks. Since object storage and database commits are separate, I would stage artifacts, verify them, publish references transactionally, and reconcile abandoned uploads through a retention policy.

Tradeoffs and limits:

- Each additional store adds consistency, backup, access-control, and operational work.
- Flexible payloads reduce schema friction but still need versioning, validation, and migration rules.

**Work example.** 
- double.digital, Relational state and object bytes: I use PostgreSQL for ownership, jobs, revisions, artifact relationships and transactional state changes; S3 holds large source and result bytes. Versioned JSONB holds variable settings, manifests and diagnostics while retaining relational identities. This reconstruction path does not use a separate document database or vector index.
- double.digital, Asset + job lineage: The database connects a result to the job and source that produced it. A completed processing job and a useful, reviewed reconstruction are separate facts.

**Proposed application, not delivered work.** For CAD, I would add a vector index only for a concrete retrieval need and keep candidates separate from verified geometric findings. Vector similarity retrieval only when required.

Hypothetical examples:

- SQL: relationships, transactions and uniqueness constraints. Example: models, revisions, jobs, findings and ownership.
- Flexible documents: variable analysis payloads with deliberate schema/version handling. Example: different check types produce different diagnostic fields.
- Vector retrieval: similarity searches over embeddings. Example: retrieve potentially relevant prior parts or review lessons; validate relevance separately.
- Object storage: large source files and derived artifacts. Example: keep CAD files outside database rows; store references/checksums in the database.

## Cache

Reusable model bytes with freshness checks and invalidation. The Augrade browser-loading path caches model bytes with version/checksum metadata and invalidation. This supports reuse instead of repeating the same loading work; it does not establish a measured speedup or deployment of a particular inspected revision.

## IFC model loader

Load model bytes and rebuild interactive IFC fragments. Augrade’s browser loading path adds IFC support to the earlier GLB viewer and reconstructs usable model fragments. Its cache/version behavior is part of the inspected loading implementation.

## Geometry preprocessing pipeline

Convert imaging masks into validated browser-ready meshes. Q Bio’s Python pipeline consumes masks supplied by the imaging team, builds meshes, cleans and validates them and prepares GLB levels of detail. Reconstruction registration checks and art-preparation tools are separate work examples.

### What does the model represent?

Follow-up: What information would you extract before running analysis?

**Approach.** Extract structure, placement, representation, and validity first so each geometric rule knows what its input can support.

1. Recover units and assembly structure: Read unit metadata and separate reusable part definitions from placed instances and their parent relationships.
2. Resolve placement explicitly: Compose instance transforms into a documented coordinate frame while preserving the original placement information.
3. Identify the geometric representation: Distinguish B-Rep faces, edges, and surfaces from mesh vertices and triangles, including any approximation settings.
4. Extract what checks require: Collect bounds, topology, adjacency, and available material or semantic attributes without silently inventing missing values.
5. Validate and retain identity: Record invalid or unsupported geometry and bind extracted entities to their source revision for later findings.

I would first build a structural view of the model: units, assembly hierarchy, part definitions, placed instances, and the transforms connecting their coordinate frames. A repeated part should share its definition while retaining distinct instance identities and positions. Next I would identify the representation available for each check. B-Rep describes surfaces and their topological relationships; a mesh represents an approximation through triangles and may lack the original surface semantics. I would extract required properties such as bounds, adjacency, surface types, and supplied material attributes, and validate topology before using them. Missing units or invalid solids would become explicit limitations. Every extracted entity would remain tied to the source revision. I would verify the import using repeated instances, nested transforms, mixed representations, and known measurements before allowing downstream rules to rely on it.

Tradeoffs and limits:

- B-Rep can support measurements tied to analytic surfaces, while mesh measurements depend on tessellation and the method used.
- Extracting every possible property up front can be expensive; start with the capabilities the selected checks require.

**Related model-data work.** Native CAD topology extraction is proposed.


- double.digital, Source manifest: I preserve source identity and placement evidence before reconstruction: hashes, file type and size, capture lineage, available EXIF/GPS and coordinate-frame evidence. The manifest also identifies the frame set, settings and toolchain versions.
- Augrade, Selection + measurements: I integrated selection, transforms and library-backed measurement tools into the BIM viewer so the general contractor could inspect geometry while discussing corrections.

**Proposed application, not delivered work.** For native CAD, I would extract instances, part identity, units, transforms, topology and analytic surfaces while separating B-Rep from display tessellation and preserving original data. Native CAD geometry SDK: B-Rep topology and analytic surfaces.

Hypothetical examples:

- Assembly: distinguish a reusable part definition from its placed instances. Example: one bolt definition appears 200 times with different transforms.
- Representations: B-Rep topology/surfaces differ from tessellated triangles. Example: a cylindrical B-Rep face supports an exact radius check; a mesh approximates it.

### Which units and tolerances apply?

Follow-up: How do you distinguish a real defect from numerical noise?

**Approach.** Separate the engineering limit from numerical uncertainty, and make borderline or invalid geometry explicit.

1. Normalize dimensions and placement: Resolve units and transforms before comparing measurements with requirements expressed in the same frame and units.
2. Separate two kinds of tolerance: Treat allowed design variation as an engineering requirement and computational tolerance as a limit of the representation and algorithm.
3. Inspect geometry quality: Detect gaps, slivers, invalid topology, and tessellation error that could undermine the proposed measurement.
4. Handle the decision boundary: Use defensible uncertainty bounds where available and mark results near the threshold inconclusive when precision is insufficient.
5. Validate across difficult cases: Test known geometry near thresholds at different scales and placements, and record any repairs as explicit transformations.

I would first normalize units and assembly placement, because a scale or transform error can look like a defect. Then I would separate the design requirement from the numerical tolerance used by the geometry algorithm. The former describes acceptable engineering variation; the latter reflects representation quality and computational precision. Neither should silently replace the other. I would inspect invalid topology, tiny gaps, sliver faces, and mesh approximation error before trusting a measurement. Where uncertainty can be bounded, I would compare that range with the requirement; otherwise, near-threshold cases would be marked inconclusive and sent for a more suitable calculation or review. Any repair would be recorded and checked for its effect on the result. Known reference shapes, boundary cases, and scale changes would test whether apparent defects survive a justified change in numerical precision.

Tradeoffs and limits:

- One global epsilon rarely suits every model scale and operation, and a larger epsilon can hide a real defect.
- Finer tessellation or higher precision can reduce uncertainty but cannot recover missing source information.

**Work example.** 
- double.digital, Registration and quality gates: Local reconstruction results retain arbitrary units. Registered packages validate transforms, scale and registration evidence. COLMAP component selection uses registration and reprojection-quality metrics, with corrupt-input and selection tests.

**Proposed application, not delivered work.** For CAD, I would separate numerical validity from engineering acceptance, normalize units, respect kernel tolerance and use a scale-aware allowance. I would retain uncertainty, test boundary cases and send ambiguous cases to review. CAD unit conversion, geometry-SDK tolerances and boundary fixtures.

Hypothetical examples:

- Units and coordinates: establish consistent dimensions and assembly placement. Example: convert inches to millimetres before applying a thickness threshold.
- Tolerances and defects: choose numerical tolerances and detect invalid geometry. Example: tiny gaps or sliver faces can create misleading results or failed operations.

## 3D renderer

Render scenes and spatial assets within browser resource constraints. IMERZA work owns the viewer architecture and renderer integration built on an early prototype. Loading, allocations and resource lifetimes support the sales-agent presentation workflow; this is not a claim of authoring Three.js or the entire original engine.

## Volumetric renderer

Show source MRI volume context alongside derived organ surfaces. Q Bio’s browser work combines GLSL volume rendering with organ models and source slices for doctor-led conversations. It consumes imaging data; segmentation AI remained with the imaging team.

## Splat rendering integration

Integrate captured surroundings with modeled property scenes. Gaussian-splat content was fitted to loading, detail and GPU resource budgets in the property viewer. Dataset size, resident data and drawn splats remain distinct.

## CMS and scene-state integration

Connect property content and selections to scene configuration. Property records and sales-agent selections drive camera presets, highlighted units and filters in the viewer. This is application/content integration rather than ownership of every CMS service.

## Device-state transport

Carry interaction state between iPad controls and the rendering machine. MQTT is recorded in the multi-device presentation work. The agent operates the iPad and the buyer watches TV output; the evidence does not establish authorship or operation of a separate messaging broker.

## Browser database

Persist BCF topics, comments, viewpoints and model relations locally. The inspected Augrade implementation uses IndexedDB CRUD and clash/topic mappings. Local persistence and file exchange do not establish a verified remote synchronization roundtrip.

## BCF importer/exporter

Read and write coordination topics through BCF ZIP/XML files. BCF file import/export is a separate exchange mechanism from the local database. Its implementation is inspected; arbitrary revision preservation and deployed feature scope remain qualified in the source work.

## Model-record integration

Map IFC/custom model schemas into browser interactions and updates. Ruslan worked on frontend model-record mapping and change interactions with backend/integration teams. Their separate APIs and geometry kernel were not his owned backend.

## Blender preparation tools

Prepare authored content for repeatable viewer ingestion. Art-team scripts establish export, naming and ingestion conventions for engine-ready content. This is concrete preparation tooling, separate from the renderer.

## Capture-processing tools

Automate processing and delivery around physical capture work. Scanlab’s processing and delivery tools grew out of its capture-service workflow. Historical capture work remains separate from the later independent reconstruction product.

## Processing workers

Background execution for reconstruction and geometry tasks. The independent reconstruction path uses Batch workers and supervised processes. Scheduling, bounded retries, leases and recovery are recorded parts of that path, with staging verification and production-adoption limits retained.

### How does an upload become a result?

Follow-up: How would you design the upload-to-result pipeline?

**Approach.** Make upload acceptance durable, then move through explicit stages to a verified result tied to one revision.

1. Define the input contract: Specify supported formats, size limits, ownership, and the checks required before an upload can enter analysis.
2. Record the source and request: Store the original bytes and checksum, assign revision and job identities, and durably record the intent to enqueue work.
3. Run stages with explicit outputs: Parse, normalize, analyze, and assemble findings using versioned inputs and declared dependencies between stages.
4. Retain reusable intermediate work: Keep an intermediate artifact only when its integrity and compatibility can be verified for the resumed stage.
5. Publish after validation: Check the result manifest and required stage outcomes before making the result available in one controlled state transition.

I would start with a small intake contract: who may upload, which formats are supported, and what limits apply. The service would store the original file, verify its integrity, and create immutable model revision and analysis identities with the selected settings. It would return the job identity after acceptance is durable, with a recoverable handoff to the queue. Workers would then parse, normalize units and placement, run the requested checks, and persist findings as separate stages with explicit dependencies. Each stage would record its version, inputs, outputs, and failure state. If analysis fails after parsing, a validated, compatible parsing artifact could support a retry. Finally, a publication step would check required outputs and expose a coherent result set. I would test interrupted uploads, queue handoff failures, and crashes between storage and publication.

Tradeoffs and limits:

- More stage boundaries improve recovery and diagnosis but add storage, coordination, and retention work.
- A saved artifact supports reuse only when the next stage can verify its identity and compatibility.

**Work example.** 
- double.digital, Source manifest: I preserve source identity and placement evidence before reconstruction: hashes, file type and size, capture lineage, available EXIF/GPS and coordinate-frame evidence. The manifest also identifies the frame set, settings and toolchain versions.
- double.digital, Atomic job admission: Job admission needs to be atomic so concurrent requests cannot each create work for the same capture. The job and its dispatch state belong in one consistent transaction.
- double.digital, Request → artifact: I separate upload acceptance from GPU execution. A Batch worker receives the durable job ID, fetches authorized inputs and runs frame extraction, COLMAP and LichtFeld reconstruction. Generated artifacts are verified before browser review.

**Proposed application, not delivered work.** For CAD, I would retain this orchestration and replace reconstruction stages with parsing, normalization and checks. 

Hypothetical examples:

- Intake: store the original file, assign model/revision IDs, validate supported input. Example: a STEP upload creates a queued analysis record; the API returns a job ID immediately.
- Stages: parse → normalize → analyze → persist findings. Example: parsing completes, but analysis fails; retain the parsed artifact where safe.

### How is work scheduled and tracked?

Follow-up: How would users see progress, and how would you detect stuck work?

**Approach.** Show durable stage progress, and distinguish a missing worker from a live worker that has stopped making useful progress.

1. Queue within capacity: Accept work within explicit backlog and tenant limits while workers claim only the concurrency they can support.
2. Expose meaningful stage state: Report queued, running, verifying, available, and failed states with timestamps and completed work where measurable.
3. Track worker ownership: Use renewable leases and heartbeats to identify the attempt currently allowed to update the job.
4. Detect stalls separately: Compare progress age and stage runtime with appropriate limits because a heartbeat alone does not prove useful work.
5. Recover and verify: Reconcile expired attempts, fence stale updates, and exercise recovery with worker crashes and deliberately stalled stages.

I would separate the web request from queued execution and give the user a durable job record to follow. That record would show the current stage, time queued, last meaningful progress, and any action needed. Percentages would use actual work counts where possible; otherwise I would show stage progress without inventing a completion estimate. A worker would claim a renewable lease and send heartbeats, while stage progress would be recorded independently. Missing heartbeats suggest a lost worker; fresh heartbeats with no progress suggest a stuck operation or an unsuitable progress signal. Recovery would check durable results, expire ownership when appropriate, and prevent the old attempt from publishing. Queue age and saturation would drive admission limits and capacity decisions. I would validate the behavior under slow jobs, disconnected workers, and crashes during final verification.

Tradeoffs and limits:

- Short leases recover failures quickly but can cause unnecessary retries during pauses or network disruption.
- Stage deadlines need workload context because healthy geometry operations can have long intervals without measurable progress.

**Work example.** 
- double.digital, Job state and progress: The browser polls durable job state every ten seconds. Ownership and useful progress are separate: the native supervisor tracks advancing training iterations and detects stalls, while the control plane reconciles expired leases against provider execution state.
- double.digital, Claims + leases: The job record tracks who owns an attempt and which provider execution is running. Leases and reconciliation let the control plane identify abandoned work and recover under bounded retry rules.
- double.digital, Managed compute scaling: My scaling example is managed compute around batch jobs. It is distinct from running a continuously autoscaled API or a real-time fleet.

**Proposed application, not delivered work.** For CAD, I would expose the current stage and last meaningful progress time, distinguishing capacity waits from a stalled parser. 

Hypothetical examples:

- Queue: separate short API requests from long-running workers; handle backpressure. Example: 100 uploads wait in a queue instead of exhausting web-server memory.
- State: queued/running/succeeded/failed; worker heartbeats reveal abandoned jobs. Example: a crashed worker leaves a job running; an expired lease permits recovery.

### How does interrupted work recover?

Follow-up: What happens if a worker finishes but crashes before acknowledging the job?

**Approach.** Treat a lost acknowledgement as uncertain delivery: inspect durable completion before deciding whether computation must run again.

1. Separate work from acknowledgement: Assume the queue may deliver the job again even when computation or result publication has already succeeded.
2. Look up a durable completion receipt: Use the analysis and result identities to find a validated completion record before starting another attempt.
3. Commit completion before acknowledging: Verify artifacts and conditionally persist their manifest and completion receipt while checking the active attempt token.
4. Recover only verifiable work: Reuse compatible durable artifacts or supported checkpoints when present, and recompute any work whose completion cannot be established.
5. Retry by failure class: Apply bounded backoff with jitter to transient failures, and route invalid inputs or exhausted retries to explicit failure states.
6. Reject obsolete attempts: Fence late writes from superseded attempts and test crashes around artifact storage, completion commit, and queue acknowledgement.

I would assume that the job can be delivered again, then ask what was durably recorded before the crash. If a validated result and completion receipt already exist for the analysis identity, the next delivery can acknowledge that result without creating duplicate findings. If only artifacts exist, a recovery step must verify their manifest and compatibility before completing publication. If no trustworthy durable output exists, computation may need to run again. Completion would conditionally commit the result identity, manifest, and receipt against the active attempt before acknowledging the queue. A newer attempt would invalidate older attempts' permission to publish. Transient storage or network failures would receive bounded retries with backoff and jitter; malformed input would receive a useful terminal error. I would test each crash boundary, including a lost completion response, to verify recovery rather than infer success from silence.

Tradeoffs and limits:

- Idempotent publication prevents duplicate visible results but does not guarantee that computation happens exactly once.
- Recovery can resume only at validated durable boundaries or checkpoints the underlying computation actually supports.

**Work example.** 
- double.digital, Completion and recovery: The worker uploads outputs and saves the exact completion report before sending it to the API. A lost response can be checked by report hash; after host loss, reconciliation can recover the stored report. A durable finalization queue verifies artifacts and commits them with job state. Attempt, worker, lease and input checks fence stale writers.
- double.digital, Claims + leases: The job record tracks who owns an attempt and which provider execution is running. Leases and reconciliation let the control plane identify abandoned work and recover under bounded retry rules.

Hypothetical examples:

- Classify failures: malformed geometry is different from a temporary network outage. Example: reject an unreadable file with a useful error; retry a transient storage failure.
- Idempotency: repeated delivery must not create duplicate jobs or findings. Example: key an analysis by revision + configuration version; enforce uniqueness in storage.
- Checkpoints: resume a safe stage without redoing expensive earlier work. Example: reuse a validated parsing result after an inference timeout.

### How is a large job contained?

Follow-up: How would you stop one huge model from disrupting other jobs?

**Approach.** Give each job a resource budget and enforce admission, isolation, and cleanup so one model cannot consume shared capacity unchecked.

1. Estimate and bound the workload: Check file and expanded sizes, assembly complexity, and requested operations before selecting a worker class.
2. Enforce per-job budgets: Set CPU, memory, scratch disk, runtime, and any GPU limits rather than relying on cooperative behavior alone.
3. Control concurrent admission: Reserve capacity and cap parser concurrency with fair queueing so large jobs cannot crowd out every smaller job.
4. Constrain the execution boundary: Run native processing in restricted processes or containers with minimal privileges, limited mounts, and controlled network access.
5. Clean up by attempt ownership: Remove attempt-owned scratch data after exit or expiry while retaining source artifacts and bounded diagnostic records.
6. Exercise failure containment: Test oversized inputs, memory exhaustion, hung parsers, and worker crashes while checking that neighboring jobs remain serviceable.

I would combine admission control with enforced resource limits. Intake would estimate workload from file size, expanded content, and model complexity, then route larger jobs to suitable workers or reject work beyond supported limits. Each job would have explicit CPU, memory, disk, runtime, and relevant GPU budgets. The scheduler would cap concurrency against actual capacity and preserve fair access for other users. Native parsing would run outside the request server in a restricted process or container, with minimal privileges and narrowly scoped storage and network access. Cancellation or limit exhaustion would stop the attempt and record a clear outcome. Attempt-owned scratch space would be cleaned after termination, with an expiry process covering crashes. I would test resource exhaustion and malformed inputs alongside normal jobs to confirm that containment and cleanup work under pressure.

Tradeoffs and limits:

- Resource estimates are imperfect, so oversized jobs need clear retry, rerouting, or rejection policies.
- A child process or container alone is not complete protection against hostile input; parser maintenance and isolation strength must match the threat model.

**Work example.** 
- double.digital, Admission and worker limits: I bound source count, source bytes and runtime before dispatch and validate a fixed one-GPU Batch profile. Upload scanning runs in a separate supervised child process with a deadline, bounded replies and owned temporary-directory cleanup.

**Proposed application, not delivered work.** For large CAD assemblies, I would add explicit CPU, memory and disk quotas, worker size classes and concurrency caps. Worker size classes, CPU/memory/disk quotas and cgroup limits.

Hypothetical examples:

- Sizing: CPU, RAM, disk and runtime vary with model complexity. Example: route a large assembly to a higher-memory worker; limit concurrent parsers.
- Isolation: contain a native parser crash or hostile/malformed input. Example: process each file in an isolated worker with resource limits.
- Cleanup: remove temporary files and failed partial outputs safely. Example: expire scratch artifacts while preserving source files and diagnostic metadata.

## Browser geometry worker

Execute mesh intersection work away from the main browser interaction path. The Augrade Web Worker receives mesh data and instance transforms, builds BVHs and returns intersections for semantic/rule filtering. Clash detection is the capability; the worker is an implementing component.

## Supervised upload scanner

Bound upload scanning in a separate owned process. Upload scanning has a deadline, bounded replies and owned temporary-directory cleanup. The fixed execution profile is not a complete security sandbox or automatic model-size routing.

## Processing supervisor

Track advancing work and detect stalled native processing. The native supervisor observes iteration progress and stalls. Useful progress is separate from ownership/lease state and does not establish periodic API heartbeat renewal.

## Job reconciler

Check abandoned attempts against provider execution state. Lease and provider-state reconciliation supports bounded recovery of abandoned work. It is a concrete control-plane mechanism within the processing workflow, not a new orchestration-product claim.

## Completion finalizer

Verify artifacts before committing durable final job state. A stored completion report, report hash and finalization queue allow recovery after a lost response or host loss. Attempt and input checks fence stale writers; this is not exactly-once compute or general checkpoint resume.

## Processing telemetry

Collect job-correlated durations, progress, errors and optional GPU samples. CloudWatch EMF/Logs Insights and native-stage telemetry retain job/attempt, settings and image identity for diagnosing processing behavior. Optional samples are not new performance or production-scale claims.

### Where do time and cost go?

Follow-up: How would you diagnose a pipeline that became slower and more expensive?

**Approach.** Separate workload changes from implementation changes, then trace time and cost to the stage that actually regressed.

1. Normalize the comparison: Compare similar inputs and settings across versions while accounting for traffic mix, concurrency, hardware, and pricing changes.
2. Break down latency and spend: Connect request, job, attempt, and stage identities to queue delay, computation, storage, finalization, and billed resources.
3. Investigate plausible causes: Check retries, duplicate work, input growth, cache behavior, resource saturation, and idle accelerator time against the observed regression.
4. Test one causal change: Replay representative inputs under controlled conditions and change the suspected factor while measuring quality, latency, and cost together.
5. Apply targeted capacity or reuse: Adjust bottleneck concurrency or reuse verified results only when dependency capacity, version identity, and access permissions support it.
6. Verify and retain rollback: Confirm the improvement under realistic load, preserve provenance without exposing source data, and revert if the regression persists.

I would first determine whether the same work became more expensive or whether the workload changed. I would compare similar inputs and settings, accounting for concurrency, hardware, and pricing, then split end-to-end time into queueing, computation, storage, and finalization. Correlated job, attempt, and stage records would expose retries, repeated computation, long transfers, and idle or saturated resources. I would form a specific hypothesis and replay representative inputs while changing one factor where practical, rather than treating correlation as proof. The fix might be algorithmic, a concurrency adjustment, or reuse of a verified result with matching versions and permissions. More workers would help only if the bottleneck and downstream limits justify them. I would confirm quality, latency distributions, throughput, and billed cost under realistic load, with protected diagnostic data and a rollback path.

Tradeoffs and limits:

- A faster isolated run can still raise total cost or worsen queueing under load, so measure the full operating context.
- Caching requires compatible identities, permission checks, and retention rules; a high hit rate alone does not establish correctness.

**Work example.** 
- double.digital, Processing telemetry: Telemetry links jobs and attempts to native-stage durations, iteration progress, errors and optional GPU samples. Settings and image identity remain available for comparing runs.
- double.digital, Managed compute scaling: My scaling example is managed compute around batch jobs. It is distinct from running a continuously autoscaled API or a real-time fleet.
- double.digital, Idempotent requests: A repeated request should not accidentally create another processing job. The identity checks also need to reject a reused key when the inputs have changed.
- double.digital, Completion and recovery: The worker uploads outputs and saves the exact completion report before sending it to the API. A lost response can be checked by report hash; after host loss, reconciliation can recover the stored report. A durable finalization queue verifies artifacts and commits them with job state. Attempt, worker, lease and input checks fence stale writers.

**Proposed application, not delivered work.** I would compare identical inputs/settings across versions, split queue, compute, storage and finalization time, investigate input size, settings, utilization and retries, then verify spend and bottlenecks before changing capacity or adding a cache. 

Hypothetical examples:

- Observability: connect API requests, jobs, stages and model versions. Example: a job ID links structured logs, timings and error traces.
- Capacity/backpressure: scale bottleneck workers without overwhelming dependencies. Example: cap concurrent inference while keeping accepted work queued.
- Cost controls: cache reusable results and avoid redundant analysis. Example: identical revision/configuration reuses a valid result when permissions allow.
- Reproducibility/security: retain provenance and protect customer CAD data. Example: restrict artifact access and record which input/model versions produced a finding.

## Frame extraction stage

Prepare image frames before reconstruction. Frame extraction is a recorded stage in the independent Batch reconstruction path; FFmpeg is among the used processing tools.

## Photogrammetry stage

Run COLMAP reconstruction and select usable components. The processing path runs COLMAP and uses registration/reprojection-quality metrics for component selection. Local reconstruction units and registered-package evidence remain explicit.

## Splat reconstruction stage

Run the recorded LichtFeld reconstruction step and retain its artifacts. LichtFeld is a processing stage executed by the independent reconstruction worker. This does not establish a production training fleet or authorship of the underlying reconstruction tool.

## Registration validator

Validate transforms, scale and registration evidence. Registered packages are checked for valid placement evidence while local outputs retain arbitrary units. Rotation validity is not a manufacturing-tolerance guarantee.

## Source manifest builder

Retain source identity, capture lineage and available placement evidence. Hashes, file types/sizes, capture lineage, available EXIF/GPS, coordinate-frame evidence and tool/settings identity are preserved before reconstruction.

## Shared settings contract

Validate and retain the exact processing configuration. A shared versioned schema rejects unknown fields and bounds settings. Admission saves the full snapshot/hash and completion checks what was applied.

## Request authentication and access checks

Authenticate callers and resolve access to processing records. The API uses caller authentication and job access checks; Auth0 is recorded in that implementation. Evidence sharing and revocation retain their own saved-assessment scope.

## CAD model adapter (proposed)

Proposed: preserve native instances, topology, units and analytic surfaces. The CAD proposal separates original B-Rep data from display tessellation and preserves identity/placement evidence. Browser IFC work is related experience, not native CAD-kernel authorship.

## Precise geometry-check adapter (proposed)

Proposed: narrow broad-phase candidates with trusted precise checks. The design uses conservative candidate rejection followed by precise geometry checks and boundary fixtures. Existing mesh intersections and reconstruction validation do not establish native-kernel clearance accuracy.

## CAD rule contract (proposed)

Proposed: validate customer-specific rules and isolate executable extensions. Versioned reconstruction settings are implemented. Typed CAD rule contracts, reference-data extensions and constrained executable rules are proposed extensions.

### How can checks evolve safely?

Follow-up: How would you add customer-specific checks without destabilizing the system?

**Approach.** Make customer variation validated, versioned configuration first, then isolate any new executable logic behind a tested contract.

1. Separate settings from algorithms: Represent approved thresholds, materials, enabled checks, and reference selections as typed configuration wherever possible.
2. Define the rule contract: Specify supported model representations, required inputs, units, parameters, outputs, and failure behavior for every rule.
3. Pin the effective configuration: Validate bounds and unknown fields, then snapshot the resolved settings and reference-data versions for each run.
4. Test the change before enabling it: Compare reference and boundary cases, execution cost, and existing rule outcomes under the proposed configuration or implementation.
5. Constrain execution and rollout: Apply resource limits and stronger isolation for untrusted code, then enable versions gradually with a clear rollback path.

I would first ask whether the customer difference can be expressed as data: approved thickness limits, material properties, selected checks, or reference tables. Those settings would use a typed schema with explicit units, permitted ranges, and rejection of unknown fields. Each analysis would retain the resolved configuration and reference versions. When a new algorithm is necessary, it would implement a rule contract describing supported geometry, required inputs, output structure, and failure behavior. I would test it against known models, threshold boundaries, invalid inputs, and the existing regression suite, including runtime and memory budgets. New versions would be enabled for a limited scope before broader use, with rollback to a pinned version. Customer-supplied executable code would receive a separate trust decision and constrained execution environment rather than being treated like ordinary configuration.

Tradeoffs and limits:

- Configuration is safer to govern than arbitrary code, but even valid settings can produce misleading outcomes if their engineering meaning is wrong.
- An extensible rule contract needs deliberate versioning so new capabilities do not silently change old analyses.

**Related configuration work.** The implemented example is versioned reconstruction settings. CAD rule contracts, reference-data extensions and executable-rule isolation remain proposed.


- double.digital, Versioned processing settings: A shared versioned schema rejects unknown fields. Admission saves the complete settings snapshot and its hash; completion checks what was actually applied. Contract and fixture tests cover the boundary.

**Proposed application, not delivered work.** For CAD, I would make customer differences validated configuration: thresholds, materials, enabled checks and reference versions. A typed rule contract and reference-model tests would gate changes; executable rules need a constrained boundary. Typed CAD plugin/rule contracts and constrained executable-rule boundaries.

Hypothetical examples:

- Rule interface: define inputs, parameters, outputs and supported model types. Example: a thickness check accepts material-specific limits.
- Configuration/versioning: distinguish customer settings from executable logic. Example: two customers use the same algorithm with different approved thresholds.
- Safe execution and testing: bound runtime and validate new rule behaviour. Example: test a new rule against a reference model suite before enabling it.

## Vector retrieval index (proposed)

Proposed only when a concrete retrieval need justifies an index. The current reconstruction path uses relational records and object storage, without a separate document database or vector index. The reference keeps retrieval candidates distinct from verified geometric findings.

## ML preprocessing package (proposed)

Proposed: share a versioned preprocessing component between training and serving. The design preserves units, transforms, feature order and preprocessing/model identity. Actual examples are downstream imaging geometry and versioned reconstruction contracts; the imaging team owned segmentation AI.

### How is geometry prepared for a model?

Follow-up: How would you keep preprocessing consistent between training and production?

**Approach.** Version preprocessing with the model and run the same raw fixtures through training and serving to detect divergence.

1. Define the feature contract: Specify units, coordinate transforms, feature order, missing values, sampling, and entity mappings expected by the model.
2. Share the production transformation: Use one versioned preprocessing implementation for common training and inference transforms, with training-only augmentation kept explicit.
3. Pin learned preprocessing state: Fit normalization or encoding state on training data only and package that state with compatible model and preprocessing versions.
4. Compare both execution paths: Pass identical raw fixtures through training preparation and serving, checking features within justified numerical tolerances.
5. Preserve identity and monitor drift: Record preprocessing and embedding versions with outputs, and investigate production feature changes or incompatible inputs.

I would define a feature contract before training: units, coordinate frames, feature order, missing-value behavior, sampling rules, and how features map back to model entities. Shared transformations would live in one versioned component used by training preparation and serving. Training-only augmentation would remain explicit. Any fitted normalization or encoding state would be learned from the training split and packaged with the model, avoiding information leakage from evaluation data. Deployment would check model and preprocessing compatibility. The same raw geometry fixtures would pass through both paths, with comparisons for expected features and justified floating-point differences. Predictions and embeddings would retain their preprocessing identity; a changed embedding model could require rebuilding its retrieval index. In production I would monitor unexpected input and feature changes, and reject or mark unsupported inputs rather than silently applying a different transformation.

Tradeoffs and limits:

- Sharing code reduces divergence, but dependency, hardware, and fitted-state differences still need compatibility checks.
- Random sampling needs controlled evaluation behavior and recorded settings; it cannot always promise bit-for-bit reproducibility.

**Related preprocessing work.** A shared ML training/serving pipeline is proposed.


- Q Bio, MRI to browser geometry: I built the processing path from the imaging team's segmentation masks to browser-ready organ geometry, including cleanup, validation and multiple levels of detail.
- double.digital, Source manifest: I preserve source identity and placement evidence before reconstruction: hashes, file type and size, capture lineage, available EXIF/GPS and coordinate-frame evidence. The manifest also identifies the frame set, settings and toolchain versions.
- double.digital, Versioned processing settings: A shared versioned schema rejects unknown fields. Admission saves the complete settings snapshot and its hash; completion checks what was actually applied. Contract and fixture tests cover the boundary.

**Proposed application, not delivered work.** For ML review, I would use one versioned preprocessing component for training and serving, preserving units, transforms and feature order. The same raw fixtures would exercise both paths and predictions would retain model/preprocessing identity. Shared preprocessing package, golden fixtures and model/preprocessing compatibility checks.

Hypothetical examples:

- Features: turn geometry/context into an agreed representation. Example: encode face attributes and adjacency for a graph model.
- Embeddings: produce vectors for retrieval or downstream tasks. Example: similar geometry retrieves candidate reference examples.
- Training/inference consistency: transformations must match. Example: different unit normalization at inference changes predictions.

## Inference result adapter (proposed)

Proposed: connect predictions to explicit review and availability states. The design distinguishes unavailable/incomplete inference from a valid no-defect result, retains deterministic findings and allows bounded retry or human review.

### How does inference join the workflow?

Follow-up: What should the product do when inference is unavailable?

**Approach.** Keep completed checks usable, label inference coverage as incomplete, and offer recovery without implying that missing output is a pass.

1. Separate analysis-stage status: Track deterministic checks and inference independently so an unavailable model does not erase already valid work.
2. Choose and bound the inference boundary: Run inference in a worker or service according to resource needs, with explicit deadlines, concurrency limits, and health signals.
3. Show the missing coverage: Tell the user which checks completed, which remain unavailable, and whether the review can satisfy its completion policy.
4. Provide controlled recovery: Retry transient failures within a budget or offer manual review while retaining the same input and model identities.
5. Validate recovered predictions: Check output shape and entity mapping before displaying predictions as review candidates with appropriately interpreted confidence.

I would make inference a separately observable stage with an explicit unavailable or incomplete state. Completed deterministic checks could remain visible if they do not depend on the missing predictions. The interface would identify the missing coverage and any review decision that must wait; an empty prediction set would never imply that no defects exist. Depending on resource and scaling needs, inference could run within a batch worker or behind a dedicated service, but either boundary would have deadlines and concurrency limits. Temporary failures would receive bounded retries; prolonged outages would offer manual review or a later rerun. Recovery would use the recorded input and model versions, validate the returned outputs, and map predictions to the correct entities. I would test timeouts, unavailable versions, malformed responses, and partial completion through the actual user workflow.

Tradeoffs and limits:

- Graceful degradation is appropriate only when the product clearly states which decisions remain unsupported.
- A model confidence score needs validation for its intended use and should not be presented as certainty that a finding is correct.

**Related API-state work.** Model inference integration is proposed.


- double.digital, API admission and result states: Routes authenticate the caller and resolve access to the job. Admission returns durable job identity, queued state and execution readiness: 201 for new work or 200 for an idempotent replay. State and artifacts are queried separately. Completion can return 202 while verification is pending, and blocked, retryable, terminal and partial states stay explicit.

**Proposed application, not delivered work.** For CAD, I would mark inference unavailable or incomplete, retain valid deterministic findings, and offer bounded retry or manual review. Missing predictions must never mean no defects. Independent deterministic-check and inference-stage states.

Hypothetical examples:

- Deployment boundary: batch worker versus separate inference service. Example: expensive GPU inference runs separately from CPU geometry parsing.
- Output interpretation: map predictions/confidence back to model entities. Example: face-level labels become highlighted candidate findings, not unquestioned facts.
- Failure behaviour: timeouts and unavailable models must produce explicit states. Example: mark AI analysis incomplete while retaining completed deterministic checks.

## Further questions

### How are checks correct and fast?

Follow-up: How would you make a geometric check both correct and fast?

**Approach.** Define correctness first, then use conservative candidate filtering to reserve precise geometry work for plausible violations.

1. Specify the measurement: Define the rule's geometry, units, threshold, supported representation, and expected treatment of invalid inputs.
2. Build a conservative broad phase: Use bounds or spatial indexes that retain every pair that could violate the rule, including its clearance and numerical margins.
3. Apply a suitable narrow phase: Evaluate retained candidates with the required geometric operation and record any approximation or uncertainty affecting the decision.
4. Reuse only equivalent work: Cache reusable geometry or results against the relevant revision, placement, algorithm, and settings identities.
5. Validate correctness and cost together: Compare with trusted reference cases, exercise near-threshold and degenerate geometry, and profile representative model sizes.

I would begin by specifying exactly what the check measures and what counts as a violation. A wall-thickness rule, for example, needs a defined measurement method and supported geometry, not just a distance threshold. To reduce work, a conservative broad phase would discard candidates that cannot violate the rule, using bounds expanded for any required clearance and numerical margin. A narrow phase would then perform the appropriate detailed measurement on the remaining candidates. B-Rep and mesh inputs may require different methods and uncertainty treatment. I would reuse indexes or measurements only when their input and configuration identities still match. Correctness would be tested against reference cases, threshold boundaries, and difficult topology before optimizing the measured bottleneck. Performance tests would track candidate counts and expensive operations as well as total runtime, while incomplete calculations would remain explicit.

Tradeoffs and limits:

- A conservative filter may retain extra candidates, but excluding a real violation would invalidate the check.
- Sampling can accelerate some measurements only when its coverage and approximation limits are acceptable for the rule.

**Related geometry tooling.** Browser mesh-clash detection is implemented. Native CAD-kernel checks and verified manufacturing-tolerance accuracy are not established.


- Augrade, Selection + measurements: I integrated selection, transforms and library-backed measurement tools into the BIM viewer so the general contractor could inspect geometry while discussing corrections.
- double.digital, Registration and quality gates: Local reconstruction results retain arbitrary units. Registered packages validate transforms, scale and registration evidence. COLMAP component selection uses registration and reprojection-quality metrics, with corrupt-input and selection tests.
- Augrade, Worker-based mesh clash detection: I developed a browser clash-detection path that sends geometry and instance transforms to a worker, builds BVHs and returns intersections for semantic and rule filtering. General contractors can review the resulting conflicts and select findings for local BCF issue creation. This is mesh-based analysis using Three.js and three-mesh-bvh.

**Proposed application, not delivered work.** For CAD clearance checks, I would establish a trusted result, conservatively reject impossible pairs with bounding boxes or a spatial index, then apply precise checks. Fixtures, boundary cases, a slow reference and representative large assemblies would validate optimization. Bounding-box/BVH broad phase and geometry-SDK narrow phase.

Hypothetical examples:

- Measurements: compute properties needed by a rule. Example: estimate wall thickness or measure a hole diameter against an allowed range.
- Candidate filtering: avoid expensive comparisons where cheap tests suffice. Example: bounding boxes reject distant part pairs before detailed intersection tests.
- Validation: compare findings against known cases and difficult boundary cases. Example: test values just above/below a threshold and across different model scales.

### How do findings survive revisions?

Follow-up: How would a finding remain meaningful after a model update?

**Approach.** Preserve the original finding, then establish explicit correspondence and recheck it against the new revision.

1. Anchor the original finding: Store the source revision, part and instance identities, relevant entities, and spatial evidence in a defined coordinate frame.
2. Preserve its explanation: Record the measured value, units, threshold, rule version, and limitations that support the original conclusion.
3. Establish revision correspondence: Match entities using verified persistent identifiers or explicit geometric and topological evidence rather than assuming face IDs survive.
4. Represent ambiguous mapping: Record splits, merges, unmatched entities, and confidence so uncertain correspondence can remain unresolved.
5. Recheck and link the history: Run the relevant rule on the new revision and link outcomes without overwriting the original evidence.

I would make a finding an immutable statement about a particular revision. It would include the part and placed instance, relevant geometry, a spatial anchor, the measured value, units, threshold, and rule version. That preserves what the finding originally meant even if the model changes. For a new revision, I would establish correspondence explicitly, using persistent identifiers only when their continuity is trustworthy and otherwise comparing geometry, topology, and placement. A face split or merge may produce several candidates or no reliable match, so the system must allow an unresolved mapping. After correspondence, the rule would run again to determine the current condition. The interface could then show linked histories and distinguish continued, changed, resolved, and unassessed findings. I would validate this using face splits, reordered entities, moved instances, and removed parts.

Tradeoffs and limits:

- Cross-revision matching is evidence about correspondence, not proof that the old measurement is still valid.
- Uncertain mapping may need human review, especially when topology or assembly meaning changes substantially.

**Work example.** 
- double.digital, Assessment + corrections: I pin an assessment to its exact source or result, run, content digest, spatial region and placement snapshot. Retried identical commands are safe; changed commands reject key reuse. Corrections create an explicit revision linked to the previous assessment rather than rewriting its original basis.
- double.digital, Review exact versions: The review opens a specific reconstruction version and its source and placement basis. Linked comparisons help inspect differences while the saved assessment remains attached to the exact evidence it used.

**Proposed application, not delivered work.** For CAD, I would record instance, affected geometry, measured value and rule version. After revisions, I would re-evaluate or establish correspondence explicitly; unresolved matches stay unresolved. CAD entity correspondence and re-evaluation.

Hypothetical examples:

- Location: attach findings to parts, instances, faces or a spatial region. Example: the viewer highlights the face that triggered a check.
- Explanation: preserve measured value, threshold and rule version. Example: “Measured 0.8 mm; required minimum 1.2 mm.”
- Revisions: topology can change; old face IDs may not survive. Example: splitting one face creates multiple faces, requiring remapping or an unresolved state.

### How is a model change evaluated?

Follow-up: How would you demonstrate that a model update improves review quality?

**Approach.** Compare versions on trustworthy, leakage-resistant cases and require improvement in review outcomes before a reversible rollout.

1. Define the review decision and limits: Agree on relevant defect classes, critical misses, reviewer effort, and acceptable latency and cost before evaluating a candidate.
2. Build reliable evaluation labels: Use representative expert-reviewed cases, adjudicate ambiguity, and retain label provenance and uncertainty.
3. Prevent evaluation leakage: Keep related parts and revisions together across splits, with temporal separation where deployment conditions require it.
4. Replay a paired comparison: Run old and new versions on the same held-out inputs and compare class-specific precision, recall, critical misses, effort, and cost.
5. Inspect shadow disagreements: Observe the candidate on production-like inputs without changing decisions, then review disagreements and distribution differences.
6. Roll out with stop criteria: Promote a pinned version gradually only after agreed gates pass, while retaining monitoring and rollback capability.

I would define improvement in terms of the review task before inspecting candidate results. Experts would label representative cases, adjudicate ambiguous defects, and record uncertainty rather than treating every historical comment as ground truth. Related parts and revisions would stay on the same side of the training and evaluation split, with time-based separation where appropriate. I would replay both versions on identical held-out inputs and compare precision, recall, critical missed defects, reviewer effort, latency, and cost by relevant case groups. Sample counts and uncertainty would accompany the metrics, especially for rare critical defects. A shadow run would reveal production disagreements without changing decisions, and targeted reviewer studies could test whether the new output actually helps. Promotion would use predefined gates, a pinned version, gradual exposure, and rollback if quality or operational limits regress.

Tradeoffs and limits:

- Better aggregate accuracy can conceal worse performance on rare or consequential defects.
- Repeated tuning against a holdout weakens its evidential value, so final decisions may need fresh evaluation cases.

**Related regression and review work.** ML evaluation and shadow rollout are proposed.


- double.digital, Testing blind spot: The route tests mocked the database, so they missed the older SQL rule. I reproduced the refusal with the real API input builder against PostgreSQL and kept that boundary regression with the fix.
- double.digital, Review exact versions: The review opens a specific reconstruction version and its source and placement basis. Linked comparisons help inspect differences while the saved assessment remains attached to the exact evidence it used.
- double.digital, Assessment + corrections: I pin an assessment to its exact source or result, run, content digest, spatial region and placement snapshot. Retried identical commands are safe; changed commands reject key reuse. Corrections create an explicit revision linked to the previous assessment rather than rewriting its original basis.

**Proposed application, not delivered work.** For ML updates, I would define improvement around review decisions, use expert-labelled holdouts with related revisions kept on the same side of a split, compare critical misses, precision, reviewer effort, latency and cost, shadow on identical inputs, then pin rollout and retain rollback. Expert-labelled evaluation sets, family/time-aware splits, shadow comparison and pinned model rollout.

Hypothetical examples:

- Ground truth: define reliable labels and representative evaluation cases. Example: experts adjudicate ambiguous findings instead of treating every comment as correct.
- Metrics: false positives, missed defects and performance matter differently. Example: a high overall accuracy can hide poor detection of rare critical issues.
- Versioned rollout: compare versions and retain rollback paths. Example: evaluate a candidate on saved cases before making it the default.

