KPI Lab · Concept · HOVI

The AI sitting next to the manager

I wanted the scene where Tony Stark works while talking to JARVIS. An AI that fully understands our tactical model and game plan, watching the same match alongside me, weighing substitution timing and responses together. So I went and researched it. The result was not what I expected — the rules were already open, and most of the judgements I wanted to hand the AI were not the AI's to make. Stripping those away left something sharper: not an AI that decides, but one that trades evidence.

Hyuckjin Han · Concept 2026.08 Status: concept · research done Built: nothing

What this piece is

This is not about something I built. It is about what I found before building anything. There is no implementation. Instead I checked, against the literature and the regulations, whether this idea is possible, where it is blocked, and how it would have to be built. A good part of what I wanted, I folded away during the research. That process is written down as it happened.

01It has already been built. And then forbidden by its own maker.

In January 2026, Lenovo and FIFA announced Football AI Pro — a generative AI assistant orchestrating multiple agents over 2,000-plus football metrics and petabytes of tracking data, answering in text, video, graphs and 3D. At the 2026 World Cup it was issued to all 48 participating nations, used by players and coaching staff on their phones.

Almost exactly what I had in mind. But FIFA attached one condition.

FIFA's own wording

"The tool may be used for analysis before and after matches, but not during live play."

The best-resourced version of this idea, sitting on the best data available, handed to 48 teams — and then barred from the match itself. No reason was published.

Which raised the first question. Is it the rules?

02The rules are open — far more than I expected

I went and read Law 4.4 of the IFAB Laws of the Game. It says this.

Law 4.4 — Electronic communication (excerpt)

"The use of any form of electronic communication by team officials is permitted where it directly relates to player welfare or safety or for tactical/coaching reasons but only small, mobile, handheld equipment (e.g. microphone, headphone, earpiece, mobile phone/smartphone, smartwatch, tablet, laptop) may be used."

And the EPTS clause contains this — the competition organiser is responsible for the accuracy of "the information and data transmitted from EPTS to the technical area during matches."

Two things become clear.

First, there is no restriction on content. The permitted list governs the form factor of the device, not the kind of information on it. "Tactical/coaching reasons" is undefined and broad. An AI-generated tactical suggestion falls inside it on any natural reading.

Second, the Laws already presuppose live data reaching the bench. The phrase "transmitted to the technical area during matches" is the evidence. Since July 2022 FIFA has formally included real-time transmission of tracking data in its validation framework — in FIFA's own words, "in response to the growing desire of the football world for live match analysis."

And the decisive part — the word "artificial intelligence" appears nowhere: not in the IFAB Laws, not in FIFA's regulations, not in UEFA's, not in the Premier League handbook. No provision governing a team's use of AI exists.

So FIFA's restriction was a judgement, not a legal requirement. Not because grounds to forbid it existed, but because they thought it too early.

But there are four lines you cannot cross

ConstraintBasisIf breached
Must be small, handheld equipmentLaw 4.4No large monitors or workstations
Nothing may reach a playerLaw 4.4 — players may not wear or use communication equipment at allAny earpiece-to-player idea is flatly illegal
Nothing about refereeing decisionsUEFA Art. 57.05 — "under no circumstances may any such systems be used in connection with any kind of refereeing decision"Team official sent off (Law 12)
Only one person may convey instructions at a timeLaw 1 — the technical areaThe AI cannot become a second voice

The second row deleted one of my early ideas outright. Putting anything in a player's ear is not worth thinking about. And using broadcast video is not a Laws question but a rights question — which is usually where a real deployment hits the wall.

03Judgement is not the AI's job

Having established that the rules are open, the next question. So what is the AI going to say?

At first I imagined things like these — "substitute now", "that player's fitness is dropping", "there's an injury risk". So I went and checked the evidence for each. All of it was thin.

The line between an observation and a judgement

Inside = what HOVI may say · outside = what a person judges

Legal under the rules — wide Law 4.4 · tactical/coaching purpose · no content limit · AI unmentioned HOVI says these — observations · what happened, described · accumulated high-intensity distance · space occupation, pressing shape · phase and formation detection · set-piece arrangement · retrieval of past cases HOVI does not say these — judgements ✕ "substitute now" → the manager Myers rule failed replication · three-sub era ✕ "his fitness is dropping" → AT · physio EPTS cannot separate fatigue from pacing ✕ "he's playing well today" → the analyst no validated live individual evaluation ✕ "there's an injury risk" → the physio
I first read this diagram as a defeat — plenty of land to build on, very little firm ground to stand on. But the items on the right were never the AI's job. A club already has an athletic trainer, a physio and a performance analyst, and those judgements are their professional territory. Thin evidence means the AI must not judge — not that the tool is useless.
What I wanted it to sayWhat the literature says
Substitution timingThe famous 58/73/79-minute rule (Myers, 2012) failed direct replication. Silva & Swartz re-analysed 4,226 observations and concluded — "there is no discernible time during the second half when there is a benefit due to substitution." And the original study predates five substitutions entirely
Live fatigue assessmentA Champions League study of 244 players: the decline in running output is very large (d = 1.41–2.15), but there is no difference between the final two 15-minute periods of each half. The authors attribute the pattern to pacing strategy, not fatigue. A drop in output is not an identified fatigue signal
Live player evaluationA player has 30–80 on-ball events in a match. VAEP and xT were designed and validated for season-level aggregation, and a within-match estimate is mostly variance. No live evaluation method validated against an external criterion exists
Injury risk predictionA 2026 systematic review: of 22 studies, 17 (77%) at high risk of bias. Reported AUC scatters from 0.42 to 0.91. The conclusion — "ML models should currently be considered decision-support tools rather than standalone injury-prevention systems"

Injury prediction needs particular care. ACWR (acute:chronic workload ratio), long the industry standard, has already collapsed — an acute-to-random workload ratio turns out to be as associated with injury as the real one. The relationship was a mathematical artefact of the ratio, not a physiological signal. And the metric is still embedded in several commercial products.

I also read a vendor's validation directly. Their own study reports flagging 72.4% of injuries (306/423) one to seven days in advance. But it reports neither the false-positive rate nor the number of alerts raised. Flag everyone every day and detection reaches 100%. Without that number, 72.4% cannot be interpreted.

04So what HOVI does — reporting, not judging

The people are already there. An athletic trainer, a physio, a performance analyst. They are the experts, and they already work with data. They know where overload begins in their own domain. The sense that 12 km in a match is a lot lives with them, not with an AI.

So the AI's job is not to judge but to watch, on their behalf, the thresholds they already know.

How it works

You set it beforehand — "tell me when anyone passes 8 km." The moment it happens, it tells you, with the values actually used in conditioning assessment attached: high-intensity distance, sprint count, accelerations and decelerations. The judgement after that belongs to the trainer and the physio.

Periodic reporting works the same way — a status read-out every fifteen minutes. The interval, the thresholds and which values ride along are all changed by the person using it.

Where HOVI sits

Thresholds and judgement both belong to people · HOVI takes only the middle

Person Set the threshold "tell me past 8 km" HOVI — this span only Live monitoring · spoken report on threshold, and every 15 min high-intensity, sprints, accel/decel attached No inference · no judgement · numbers only Person Judgement Trainer · physio · analyst They adjust thresholds as they use it — this loop fits the tool to the club
Cut this way, the problem in §03 stops being HOVI's problem. It does not adjudicate fatigue versus pacing. It states that a player has covered 8 km. That is an observation, not an inference. And a system that does not infer cannot produce a wrong inference.

But why voice

This is the part of the idea that is actually new. Live GPS dashboards already exist. The problem is that seeing one requires looking at a screen.

During a match the trainer's eyes and hands are already occupied — watching the pitch, treating someone, talking to the bench. A tablet demands eyes. The ears are free.

Sometimes hearing it and speaking back is easier than looking. If an automatic reporting system watches the screen on your behalf, you get to do the thing you are better at.

There is a technical benefit too: the voice commands become a closed vocabulary. "Tell me who passes 8 km" is a fixed sentence pattern, not open dialogue. Speech recognition in stadium noise is, as we will see, the weakest link in this whole idea — and a closed vocabulary shrinks that risk considerably.

05Where this goes next — from reporting to conversation

§04 is the floor — watching thresholds and reading them out. That can be built with today's technology and leaves almost no room to be wrong.

But that is not what I originally wanted. I wanted a conversation in which evidence moves both ways. Not being notified of a number, but asking and getting the evidence for it, then asking again from what you see. And eventually an AI that knows what kind of thing I tend to want to know, and puts it up before I ask.

Three stages, and the line not to cross

What the AI's job becomes at each stage

Stage 1 · build now Reporting threshold → alert "past 8 km" no inference Stage 2 · next Conversation question → evidence "why? compare the last three" retrieval and narration only Stage 3 · later Personalisation patterns → surfaced first offered before you ask Safe personalisation — what to show he always asks for high-intensity after a distance alert → include it Personalisation that crosses the line — what to decide he always substitutes in this situation → recommend the substitution Wrong on the left is showing something useless. Wrong on the right is having made the decision for you. And the right has a worse trap — an AI that has learned to agree with me is not a second opinion.
Stage three is not the problem; what stage three is applied to is. Learning which evidence a person wants is personalised retrieval, and at worst it shows you something you will not use. Learning which decisions a person makes and then recommending them inherits every evidence problem from §03 intact.

Stage two is the real destination. An alert is the opening of a conversation, not the end of one. You hear "past 8 km", ask "why? compare it with the same point in the last three matches", get that evidence, and from it ask "and what about the last twenty minutes". Evidence moving both ways rather than being announced in one direction — that was the picture I wanted.

And this fits the architecture in §07. Conversation does not mean the AI does the arithmetic. Its job runs from translating the question into predefined metrics to putting the resulting value into words. Everything between is done by deterministic code.

What can be said about stage three now

Learning a user's patterns is an easier problem than learning football. One person makes the same kinds of request repeatedly, the signal is consistent, and errors are corrected immediately. The negative result in §03 — that player embeddings do not improve prediction — is about modelling players, not about modelling users.

Two cautions, written down in advance.

So personalisation points at "what to show" and never at "what to decide". Tailor the choice of evidence to me; do not tailor the conclusion. Hold that line and stage three does not kill the second opinion.

06So I changed the question — coverage, not accuracy

Stuck here, I went back to why I wanted this tool in the first place.

I never wanted the AI to be right. Running a substitution or a tactical change through beforehand makes what you expect explicit, and makes it easier to explain to the players. And if it does not appear, you have the next response ready at the same time. That was the point.

That difference turns out to be large.

Read as prediction, read as preparation

What you evaluate the same simulation output against

As prediction — verification is blocked now "this is what will happen" → judged on being right → counterfactuals have no ground truth As preparation — verification becomes possible now if they drop off if nothing changes if we press higher → judged on whether reality landed inside → this can be counted
Every tactical counterfactual study is stuck at "was the prediction right", because there is no ground truth. So results in this field are validated by expert preference rather than accuracy — and that preference moves a great deal depending on whom you ask and what you ask them. The widely quoted "90%" means that five people at Liverpool FC (three data scientists, one video analyst, one coaching assistant) assessed 50 scenarios, and in 45 of them the majority favoured TacticAI's version. Another study asked 36 people (3 club experts, 9 coaches, 24 players): overall 44% judged the AI's version better, but among the three club experts it was 34% — and on multi-step scenarios those same experts rose to 41%. Asking for coverage sidesteps the problem. Whether reality landed inside the branches you prepared can be counted independently of anyone's taste.

The question, changed

❌ What happens if I make this substitution — unanswerable. No ground truth exists.
✅ What should I prepare for if I make this substitution — answerable, and countable afterwards.

There is a practical benefit too. Instead of telling the players "this is what will happen", a manager can say "one of these three will come, and here is what we do in each". It is easier to explain, and when the prediction misses, the team does not fall apart.

07If you build it — never let the LLM do the arithmetic

I researched the architecture too. Here the answer is comparatively clear.

The most natural implementation is "hand the AI the match data and ask it". That fails.

LLM doing the arithmetic versus tools doing it

A football question-answering benchmark (top) and a structured-data query benchmark (bottom)

Football QA · SoccerNet 2026 Asking a general LLM directly 39% Tool-routing specialist system 91% Structured-data queries · dbt 2026 benchmark LLM writes the query itself 51~90% Through a semantic layer 98~100% Aggregation accuracy over tabular data is 7–35%. Past 8K tokens of context, 70–90% of performance is lost.
Four independent lines of evidence point the same way — football agent research, data-query benchmarks, and a paper that has an LLM narrate the output of an xG model. Use the LLM as a router and a narrator. Never as a calculator.

"How's our left side doing?" is a filtered aggregation. It is the weakest item in these benchmarks, and long context with missing values — real conditions — is exactly where performance collapses.

And the way it fails is the worst possible. It will willingly, quietly, plausibly hand you the wrong number. For someone deciding a substitution inside thirty seconds, that is the one fatal failure mode.

Through a semantic layer, by contrast, it says it does not know when it does not know. It fails safe. The price is coverage — zero on questions outside its scope. You narrow the list of answerable questions in exchange for not being wrong inside it, and for this use that is the right trade.

For reference, LLMs do not know football that well

2026 World Cup, 104 matches
68.4%
best-performing LLM
same conditions
68.3%
bookmakers
same conditions
69.7%
ordinary fans — above the LLM

And on tasks that combine the Laws of the Game with visual evidence, the best models manage 44% on foul classification and 6% on visual grounding. The researchers' conclusion was "superficial pattern matching rather than genuine reasoning." "What happens if we switch to a back three" is not a question retrieval can answer. A simulator or a fitted model computes it; the LLM puts it into words.

08Two components

HOVI is not one object. It needs two organs, and I am working on each separately.

The structure of HOVI

One hub, two components — each at a different stage of verification

HOVI Conversational football AI partner · concept stage ■ prototype · stage 1 verified ① Ears and eyes Dual-View glasses turning speech and gaze into data □ researched only · nothing built ② The what-if engine Tactical simulator a device for preparing branches
① is the input organ that captures what the manager says and sees and turns it into data; ② is the computational organ that runs the "what if". HOVI is the conversational layer on top of both. Only ① actually exists, and within it only one stage is verified.

① has a piece of its own — the tool that hears the manager and cuts the passage that just went by, from two angles, with only its time-synchronisation stage verified.

Component ①The manager speaks during the match — Dual-View →

② is researched only. The early idea of showing it as a hologram is physically blocked — volumetric displays are additive and cannot occlude what is behind, so a crowded penalty box renders as an overlapping smear — and counterfactual simulation has no validation criterion to begin with. The coverage argument in §06 is what came out of that research. When there is something to verify, it gets its own piece.

09What would have to be true

Narrowing the scope to reporting changed the risk list. Inference risk fell; plumbing risk moved to the front. None of the following is confirmed.

  1. Live EPTS data can actually be received. This is now the hardest premise. The Laws permit it and FIFA has validated real-time transmission since 2022, but owning GPS vests for training and having live telemetry arrive at the bench mid-match are different product tiers. Whether this works at a club I can reach has to be checked first. If it does not, nothing else matters
  2. Voice works in a stadium. Not a single published error rate exists for speech recognition in live sport. Industrial settings put 16–24% as the floor, and a stadium is louder. A closed command vocabulary shrinks the risk substantially — a handful of fixed patterns rather than open dialogue, with keyword boosting on player names and numbers. Until it is measured, I do not know
  3. Alerts do not fire too often. The classic failure of threshold alerting, well documented in clinical settings — fire too often and people stop listening. The false-positive question I put to the injury-prediction vendor earlier comes straight back at my own system. How many alerts in ninety minutes is right, I do not yet know
  4. No default thresholds are shipped. If the product arrives with "8 km" pre-filled, that is a claim — an unevidenced one. Leave it empty and let the expert fill it. This is a design principle rather than a technical constraint, and it will not be easy to hold
  5. Ears really are better than a screen. My assumption, not a measured fact. One trainer, one match, and you would know. It is the cheapest assumption to test, so it goes first
  6. The list of answerable questions can be pinned down narrowly. If it reaches the conversational layer, a semantic layer is only safe inside its scope. What share of the questions actually asked on a touchline can be answered from predefined metrics — nobody has counted
  7. Reality lands inside the branches prepared. The simulator-side assumption. If coverage is the validation criterion, the number has to be measured
  8. Personalisation does not cross the line. Stage three in §05 must attach only to what to show. This is a line I have to hold myself, not one the technology enforces, and the more useful it gets the more tempting crossing it becomes. The test for whether it has been crossed needs setting in advance — "is this output evidence or a conclusion?"

The first thing to do

Number five. One trainer, one match, a spoken report every fifteen minutes. No AI and no learning required — a person with a timer reading it out will do. Whether receiving it by ear beats a screen, which values they want, how often becomes annoying. That single match settles half the list above.

10What I am holding onto in this idea

Researching this folded away much of the Iron Man image. The hologram is physically blocked, the player earpiece is illegal, and most of the judgements I wanted had no evidence behind them.

But one thing got firmer the more I looked.

A manager is not someone who decides but someone who chooses. The coach's view, the analyst's view and the AI's view each arrive carrying their own reasoning, and the manager picks among them. To win, I think you need several perspectives and several approaches.

That view lines up with the research. A legal commentary on the regulatory vacuum around AI landed on a single principle — "ultimate responsibility clearly remains with the coach or the referee" — and every shipped AI assistant attaches the same caveat: "it does not replace practitioner expertise."

Put the AI forward as the answer and it collapses for want of evidence; put it forward as one vote and it is useful even on thin evidence. The coach and the analyst can use the same tool to develop and argue their own reasoning. The quality of the options placed in front of the manager goes up. That is what I actually want from this.

And this is also why the line in §05 matters. Once the AI starts tailoring conclusions to me, it is not one vote but a copy of my own. Saying you need several perspectives while manufacturing one more voice that sounds like you does not hold together. Tailor the evidence to me, never the conclusion — that is not a technical constraint but the thing I want from the tool.

The next step is item six in §09 — collect the questions a manager actually asks during a match and count what share can be answered from predefined metrics. That can be done right now, with no AI at all.