04 / Test infrastructure

Firmware TestAutomation Platform

A traceable, recoverable firmware-validation loop that connects WSL-hosted services with physical Windows test stations and makes every job visible from intake to report.

01 What it solves

What it solves

Firmware validation can begin with Jenkins or a webhook, then cross firmware retrieval, board-specific execution, report collection, and hardware-facing troubleshooting. This project turns that distributed manual flow into one visible job pipeline with explicit recovery boundaries.

02 Execution flow

Execution flow

01Webhook or manual submission
02FastAPI records the task
03Redis notification
04Celery routes by board queue
05Windows Worker runs pytest
06Allure report + result index
07Dashboard

03 Architecture & responsibilities

Architecture & responsibilities

API & dashboard

FastAPI exposes webhook, task, report, and health endpoints while serving the operational dashboard.

Task routing

A publisher consumes Redis notifications and follows pipeline → board ID → Celery queue routing so physical test resources remain addressable, isolated, and extensible.

Test execution

Windows Workers obtain firmware, invoke pytest, produce Allure output, and persist task_result.json for each execution.

Visibility & recovery

The dashboard combines task state, Worker and Publisher heartbeats, report indexes, and delayed-result recovery instead of treating a published job as a finished job.

04 Operational evidence

Operational evidence

**Cross-boundary execution architecture**Jenkins provides approved build context through a secure ingress. The service layer relays the task to Windows test stations, which retrieve firmware and test code, execute against a virtual meter, and return the result to the dashboard and notification flow.

**Operational dashboard**The dashboard consolidates active, queued, and failed work with pipeline trends, test-station status, task risk, and report links so operators can understand the current state without joining several machines manually.

**Allure test evidence**Each completed pytest run is published as an Allure result. Engineers can move from a test case to its execution log and failure context, keeping diagnosis grounded in recorded evidence.

**Report index & task traceability**Generated reports are collected into one searchable index and linked back to their task context, avoiding manual report-file discovery.

05 Reliability & operations

Reliability & operations

Duplicate-claim guards protect a task from being claimed twice while still allowing its separately routed boards to execute.
Recovery boundaries distinguish transient broker disconnects, lost Worker tasks, stale Celery task IDs, and late-arriving reports rather than hiding the first failure.
Report uploads are validated; temporary upload directories are cleaned up, and unsafe archive entries are rejected before they become report content.
A Worker launcher, preserved local serial-port configuration, versioned EXE builds, and release documentation make the test-station side deployable and maintainable.

06 Verification approach

Verification approach

Repository regression tests cover task claiming, result reconciliation, queue availability, report synchronization, Worker packaging and launch conditions, and serial-port configuration.
An end-to-end PowerShell entry point exercises the service check, webhook submission, Worker processing wait, and dashboard result lookup as one minimum workflow.
Acceptance is based on pytest outcomes, service logs, generated reports, and real device behaviour—not on unverified efficiency claims.

07 Engineering with Codex

Engineering with Codex

Engineering with CodexCodex accelerated implementation; I retained responsibility for system boundaries, integration choices, hardware debugging, failure attribution, and acceptance.

Used Codex to accelerate module boundaries, task decomposition, initial configuration and script work, regression-test examples, dashboard iteration, refactoring, and documentation.
Captured repeatable project rules in AGENTS.md, including WSL deployment, cache diagnosis, Vue template constraints, Chinese-text safety, and source → deployed file → browser runtime validation.
Kept execution deterministic: pytest, service logs, reports, and physical-device results remain the source of truth rather than generated code alone.

08 AI evolution

AI evolution

AI is an implementation aid in this version, not the execution authority. A future extension could use structured task results and logs to support failure summaries, similar-issue retrieval, and human-reviewed triage.

09 Stack

Stack

FastAPI · SQLite · Redis · Celery · pytest · Allure · WSL · Windows Worker

View source on GitHub