← Projects · Flagship · Agentic AI

FLAME — Agentic Recommendation Engine for Gamified Crowd-Labelling

Google ADK · Gemini 2.5 Flash · Vertex AI Search · Firestore · Cloud Run

The AI backend of a gamified platform that turns plant workers at a global enterprise into a distributed labelling workforce for workplace-safety imagery — deciding what each player sees, in which of four mini-games, at what difficulty, and processing every submission through consensus, ranking, trust, and anti-gaming pipelines.

Problem

Building a high-quality labelled dataset of workplace-safety (HSE) images normally means paying annotators. This platform gamifies it: workers play four mobile mini-games — a quiz, a binary swipe, bounding-box annotation, and pairwise comparison — and each play produces structured labels. FLAME is the recommendation and scoring engine behind it: it must keep 4 game queues personalized per player, keep label quality trustworthy, and do it with sub-second question serving — designed for 100K+ concurrent users and 50K+ new images per day.

My role

I architected the AI backend end-to-end: the ADK multi-agent engine, the async FastAPI hot path, the ranking/consensus/trust algorithms, and the concurrency model. I refactored inherited prototype modules into the canonical ADK Skills pattern, hardened five documented race conditions with Firestore transactions and idempotent claims, and led technical direction for a junior engineer — including presenting the vector-retrieval trade-off analysis (Firestore findNearest vs. Vertex AI Search vs. Qdrant) to senior stakeholders for sign-off.

Architecture

Mobile app4 mini-games API GatewayFirebase JWT Cloud Run (private)FastAPI · async hot pathprefetch queues 12 agentsTopicSelector · ADK LlmAgentQualityGuard · LLM verdictsGameSelector · borderline LLM9 deterministic (Elo, consensus,trust, scoring, ranking…) Gemini 2.5 Flashvia ADK Runner · 5 Skills Vertex AI Searchsemantic retrieval + local re-rank Firestore · 15 collectionsplayers · queues · labels · rankings · configs (TTL-cached) · dead letters Cloud Functions + Schedulerrefill triggers · 7 maintenance crons Hybrid decision logic: deterministic thresholds handle clear cases; the LLM reasons only on the ±0.05 borderline band — with every verdict persisted for audit. Prefetch queue + pre-signed GCS URLs + background refill keep question-serving on the sub-second design target at scale.

LLM-driven reasoning, no hardcoded routing. The flagship TopicSelectorAgent is a Google ADK LlmAgent with a SkillToolset exposing 13 structured Firestore tools. The LLM itself reasons over nine personalization signals — weighted topic distributions, player history, per-player deduplication — and authors the Vertex AI Search query. The orchestrator owns which bucket; the model owns what within it, returning Pydantic-validated structured output.

Deterministic where it should be. Consensus routing, Bradley-Terry Elo pairwise ranking (K=32 cold / 16 settled), trust scoring, anti-gaming (multi-signal: speed, monotony, stagnation, intransitivity — shadow-ban as data, never code paths), and scoring are pure Python. LLMs are reserved for judgment calls; math stays math.

Tech stack

Google ADKGemini 2.5 FlashVertex AI SearchFirestore (async)Cloud RunCloud FunctionsCloud SchedulerFastAPIPydanticscikit-learnPython

Key design decisions & trade-offs

  • Vertex AI Search over Firestore findNearest / Qdrant. Managed retrieval avoids owning an embedding pipeline and prevents Firestore composite-index explosion; the trade-off — less ranking control — is mitigated by a local re-ranking agent. I presented this analysis to stakeholders for architecture sign-off.
  • ADK Skills over direct LLM calls. Five SKILL.md packages with per-skill tool gating load capabilities incrementally, keeping the agent’s context window small instead of dumping every signal into one prompt blob.
  • LLM only on the borderline. Game/difficulty progression uses cheap deterministic thresholds; only the ±0.05 ambiguous band escalates to Gemini — and fails open to a safe default. Cost control plus a full audit trail.
  • Config as data. Eleven Firestore config documents behind a 5-minute TTL in-memory cache: admins tune every threshold — consensus, scoring, trust weights, queue targets — with zero redeployment.
  • Concurrency hardening. Idempotent session claims via create() (AlreadyExists → early return), deterministic label IDs, atomic increments for XP — closing five documented race conditions.

Scope & results

~7,800 lines · 12 agents · 18 tool modules · 5 ADK skills · 15 Firestore collections · 19 composite indexes · 17 unit + 7 integration test files

The engine is designed for 100K+ concurrent users and 50K+ images/day, with a prefetch-queue architecture (pre-signed GCS URLs, background refill service) engineered for sub-second question serving at that target scale. Client and platform details are confidential; the numbers above are verified from the codebase.