The Problem:
Modern reinforcement-learning agents are increasingly being deployed in adversarial environments, yet evaluating them remains stubbornly difficult. Standard RL metrics like cumulative reward or win rate tell you whether an agent succeeded, but not how or why. Two agents can have identical win rates while exhibiting wildly different strategies, decision quality, and failure modes; conversely, an agent can score high on aggregate reward while making consistently poor tactical choices that get masked by lucky outcomes. For organizations like Lockheed Martin's Advanced Technology Labs, where AI agents may eventually inform real-world strategic systems, this opacity is a serious limitation. There is no general-purpose, dimensional framework for inspecting what an adversarial AI is actually doing turn by turn, and without one, model iteration becomes guesswork. Compounding the problem, AI systems are typically evaluated in isolation from one another (one model per study), but real adversarial deployments increasingly involve coordinated multi-model pipelines where a classifier may route decisions to a specialist policy. Existing evaluation tooling rarely keeps pace with that architectural complexity.
Our Approach:
This project builds an end-to-end evaluation harness for adversarial AI in a turn-based tactical strategy game. A human player controls three "mech" units, while a multi-headed reinforcement-learning model controls the enemy ("vek") units; the human's job is to survive five turns or eliminate all enemies, while the AI's job is to destroy them. Around this game, we have built a complete evaluation pipeline with five tightly integrated components. (1) An API server sits between the game and the AI models, translating the game's internal state into the exact 2,372-dimensional observation tensor that the trained policy networks expect, and decoding the model's action logits back into legal moves and attacks. (2) A multi-model AI stack combines a VQ-VAE transformer classifier (model 2) that detects which of five strategic patterns the human player is using each turn, with a multi-headed PPO adversary (model 3) whose head selection is dynamically routed by that classification, so the enemy AI literally adapts to the player's playstyle in real time. (3) A deterministic per-turn grading system scores every enemy-AI decision across nine dimensions (target hit rate, threat coverage, kill potential, damage threatened, position quality, mech HP lost, mechs killed, buildings destroyed, and vek lost), combining them into a 0–10 composite per turn and a 0–100 aggregate per game. (4) An LLM-powered analyst layer (currently using Groq's Llama 3.1) feeds the deterministic metrics into a language model after each game, producing a 2–3 sentence qualitative narrative that contextualizes the numbers ("the AI maintained 100% target hit rate but failed to convert any to kills, suggesting its targeting prioritization is sound but its positioning is too cautious"). (5) A live web dashboard persists every game's metrics and summary to JSON, displays performance trends, and lets evaluators expand any historical mission to inspect its full per-turn breakdown. Together, these components turn what would normally be an opaque "black-box adversary" into a fully instrumented system whose strategic behavior can be measured, compared across model versions, and explained in plain English