← Projects · Flagship · Agentic AI

Multi-Agent Recommendation Engine for Gamified Data Labelling

Python 3.13 · FastAPI · Cloud Run · Google ADK · Gemini 2.5 Flash · Vertex AI Search · Firestore · Cloud Functions · API Gateway

The backend of a gamified platform with two objectives: annotating an industrial safety image dataset, and raising frontline workers’ health, safety and environment awareness. It decides what each worker sees, in which mode, at which difficulty, then processes every submission through consensus, ranking, trust and anti-gaming.

Problem

The platform serves two goals at once: collecting consensus-validated labels on an industrial safety image dataset, and raising frontline workers’ hazard-recognition skill. One mode is a quiz that teaches and gates progression by lesson and topic completion; the others produce structured labels. The engine has to keep every worker’s queue relevant and varied, keep label quality trustworthy, and serve the next session fast.

My role

I was the sole owner of the backend: 12 agents, 3 LLM-backed, and the boundary between LLM-driven and deterministic components. The React Native frontend and the quiz question bank were other people’s work. I also mentored a junior engineer.

Key design decisions & trade-offs

  • The LLM decides what; Python decides which. A deterministic orchestrator schedules which content bucket comes next. Within that bucket, an LLM agent selects the topic and writes the search query. Everything numeric and consequential, including per-player deduplication, IoU bounding-box consensus routing, Bradley-Terry Elo pairwise ranking, trust scoring and anti-gaming, is plain Python: reproducible, cheap and auditable.
  • Skills load only what the current mode needs. Topic selection is a real ADK LlmAgent wired to five filesystem skills. It loads only the skill it needs, and that skill’s allow-list decides which of 13 tools become callable, so a session only sees the tools it needs.
  • Google ADK agent, services on Cloud Run. The LLM agent is built with Google ADK. The deterministic services and the serving API run on Cloud Run, where stateless services scale cheaply.
  • Managed versus self-hosted retrieval. I compared the options and presented the trade-offs: a managed retrieval service avoids building and operating a separate index, at the cost of less control over ranking, which a local deterministic re-rank on the retrieved candidates can recover.
  • LLM only where judgment is needed. Thresholds settle the clear cases. Only an accuracy landing inside a 0.05 band around a threshold calls Gemini, to weigh trend and consistency, and its reasoning is persisted to Firestore for audit. Both progression and difficulty fail open.
  • The hot path never waits on a model. Submitting a session returns the whole next session in one response, so the client never makes a second call. The agents that recompute progression, trust and anti-gaming run in background tasks after the response has gone out. The serving path still writes – labels, sessions, XP – it just never blocks on the model.
  • Configuration as data. Zero hardcoded thresholds: 11 config documents behind a five-minute cache, tuned without redeploying.
  • Safe under concurrency. Transactions, atomic increments, ArrayUnion and dot-notation updates, with five documented race-condition fixes.

What I would do next

Measure serving latency against an explicit SLO, add offline evaluation of recommendation quality, and track label agreement over time as the quality signal.

Employer and client work is described at a general level. Client identities, internal figures and confidential details are omitted.