Case A3 / human-in-the-loop product evaluation

Working prototype & tested

Neural Commander.

Hands-on playtesting became explicit gameplay, architecture, content-security, and acceptance criteria for a deterministic host-authoritative isometric sci-fi RTS vertical slice.

  • Prototype
  • Tested
  • Packaged
  • Live-verified
  • One open regression
Platform: C# · .NET 8 · Godot 4.6.2 MonoEvidence: Tests, release build, native captures, audio analysisCurrent maturity: Tested local vertical slice

Operational problem

A playable build can still be architecturally wrong.

Game behavior that looks plausible in isolation can violate economy semantics, network authority, content integrity, navigation, feedback, or long-term determinism. The work required converting hands-on observations into rules that both the simulation and the evidence system could enforce.

Scope & constraints

A bounded vertical slice with host authority and deterministic state.

  • Deployment, construction, extraction and gathering, production, movement, combat, docking, AI opponents, terrain, HUD, minimap, fog, and audio share one simulation contract.
  • The host remains authoritative; multiplayer-facing contracts establish the current local multiplayer contract; broader internet-security validation is a later release stage.
  • Immutable snapshots and deterministic behavior make state changes inspectable and testable.
  • Strict content validation and deterministic ZIP/SHA-256 admission prevent arbitrary content drift.
  • A runnable native package is required; editor-only or screenshot-only progress is insufficient.

Architecture & tradeoffs

Separate authoritative simulation, admitted content, and presentation evidence.

Conceptual Neural Commander evidence flow.

Commands and content

Versioned input and strictly admitted data enter a deterministic host-authoritative simulation.

Authoritative state

Economy, units, terrain, combat, navigation, production, and AI resolve through immutable state transitions.

Godot presentation

Data-driven isometric and PBR presentation, HUD, fog, minimap, and audio reflect the accepted state.

Automated evidence

xUnit, native capture, build validation, and PCM/WAV diagnostics expose regressions and remaining failures.

Conceptual public view; source, package paths, internal task identity, and raw captures are excluded.

What Shayne owned

Playtest interpretation, semantic correction, and human acceptance.

Field observationPlayed native builds and translated usability, pacing, behavior, visual, and audio observations into testable requirements.
Semantic correctionSeparated Alloy as an economic resource from Mass as a physical load after identifying a false underlying assumption.
Architecture tradeoffsDirected deterministic host authority, content admission, runnable-build discipline, and specialist workstream boundaries.
AcceptanceReviewed tests, native captures, audio diagnostics, known failures, and release evidence before accepting checkpoints.

AI role

Specialist workstreams, one human product and evidence authority.

AI agents contributed implementation, test design, visual/audio work, review, and packaging in bounded areas. Shayne retained product intent, playtest interpretation, architecture decisions, correction of false assumptions, workstream direction, and acceptance. The work demonstrates AI-directed software engineering with human product and systems authority.

Verification

Strong evidence with one open deterministic regression.

Broad regression suiteDeterministic gameplay coverage with a known later-wave correction still tracked.
Focused acceptance laneCombined validation of the current bounded release.
Native visual reviewAccepted full-resolution runtime evidence with successful exit.
Audio diagnosticsBounded stereo analysis checked continuity and transition behavior across the main audio buses.

The Release build completed cleanly. One deterministic later-wave regression remains the next focused correction and acceptance target.

Result & next stage

A tested, packaged, native-running vertical slice.

The current result is a working deterministic host-authoritative prototype with an admitted local content package and live-verified native Windows runs. The next stage is to close the remaining regression and extend multiplayer, security, and release validation beyond the current local environment.

Open validation item: a deterministic later-wave gameplay condition remains the next correction target.
Next AI Systems Lab case studyPegasus Agent Mail