KPI Lab · Concept · Component ② Tactical Simulator

A tool that evaluates the plan

The final goal is to watch the same match together, talk about it, and have that conversation feed the manager's decisions. Working towards it I ran four preliminary measurements, and on their results fixed the currently buildable scope as plan evaluation rather than prediction. Below: the measurement data, the verdicts, the design decisions that follow, and what remains out of reach.

Hyuckjin Han Sept 2026 Status Concept · 4 preliminary measurements Implementation None

01The problem — substitutions

Permitted substitutions went from three to five. Actual usage is 4.4 per team per match at FC Seoul and 4.9 at Hwaseong FC, or 8.84 per match across both sides. It is the largest single variable a manager controls directly during a match.

Ten cards per match

FC Seoul, 25 K League 1 matches, 2026 · timing of all 221 substitutions, both teams

02State of the field

As of September 2026, generating match continuations from event data is already published and in some cases open-sourced. The table lists what the review turned up.

Both generative lines — LEM and ScoutGPT — swap players in the starting line-up. Neither covers in-game substitutions or the opponent's adjustment to them. No study in the reviewed literature treats substitutions as a sequence exchanged between the two sides.

The absence of prior work does not vouch for the direction. The possibility that no signal exists has to be ruled out first. §03 and §04 are that check.

03Measurement ① — interaction in substitution timing

Hypothesis — opponent substitutions respond to ours.
Data — Hwaseong FC, 2026 season, 23 matches, 224 substitutions across both sides. For each of our 114 substitutions, the share of opponent responses within N minutes.
Null model — the opponent's set of substitution times replaced with another match's, 3,000 times (each side's timing distribution preserved; only the fact of being the same match removed).

Opponent response rate after our substitution

● observed · ▬ 95% interval of the shuffled null (Hwaseong FC, 23 matches · 224 substitutions)

Verdict — unsupported. In all four windows the observed value sits inside the null 95% interval. The two sides' median timings are also close, at 71′ and 68.5′. Prediction of opponent substitution timing is removed from the design.

Limitation — one team, one season, one league. This is "no signal detected", not proof that no interaction exists.

04Measurement ② — substitution type

Hypothesis — the response is in type rather than timing; an attacking change is answered with a defensive one.
Data — FC Seoul, 25 K League 1 matches, 221 substitutions across both sides. Classified attacking / like-for-like / defensive by the difference in positional rank (GK 0 · DF 1 · MF 2 · FW 3).

Hwaseong FC data was excluded here: K League 2 has no league-wide player file, leaving 46% of incoming players without a confirmed position.

Secondary result — score state governs substitution type

Substitution type by score state

FC Seoul, 25 K League 1 matches, 2026 · 221 substitutions, both teams

Primary result — no type-level response detected

Rank correlation between the actor's substitution type and the responder's, over pairs where a response occurred:

One subgroup showed direction. Where an attacking card was played while trailing, defensive responses ran at 21.1 / 26.3 / 36.8% against 4.7 / 6.2 / 10.9% otherwise — three to four times higher. However, the minimum p across 3 states × 3 windows = 9 tests is 0.014, which corrects to 0.126 and is not significant. The five responses concerned came from four matches. Retained as a hypothesis only.

Limits of the instrument

By positional rank, two-thirds of substitutions register as "no change"

FC Seoul, 25 matches · classification of 221 substitutions

Verdict — type-level response prediction is held. Absence of signal cannot be separated from insufficient resolution in the instrument. What is required is not a positional label but a role label.

05Measurement ③ — the variance structure of usage

Question — does "using the same player differently" exist in the data?
Data — 519 player-matches drawn from 34,518 raw events across FC Seoul's 25 matches (20+ minutes played, 24+ events, goalkeepers excluded). Variance decomposed on the 261 player-matches from the 17 players with four or more appearances.
Variables — nine describing how a player was used rather than how well (2 positional · 5 action mix · 2 pass character).

Coordinate handling — raw event coordinates are not normalised to attacking direction; they flip with home/away × half. The rule was confirmed against goalkeeper mean X and corrected (after correction: GK 8–9 · DF 40 · MF 50–56 · FW 58–61). Existing FC Seoul analysis scripts already apply this correction, so published reports are unaffected.

Result ① — role is fixed, task varies

ICC 0.69 for advancement and 0.69 for width. About 70% of variance is explained by player identity. Regulars play in the same place.

Reading a low ICC as "instructions change from match to match" is an error. Variation has three sources — changed instruction, opponent and situation, and measurement error. To separate them, events within a single match were split into odd and even halves, measured separately and compared (split-half reliability, Spearman-Brown corrected). Within one match the task is constant, so any discrepancy between the two halves is measurement error.

What each variable's variance is made of

FC Seoul, 25 matches · 17 players with 4+ appearances · 261 player-matches · variance decomposition + within-match split-half reliability

Split-half reliability is 0.00 for dribble share, 0.04 for duels, 0.02 for shots and 0.24 for defensive actions. A match contains only a handful of these events, so the share is unstable. These four variables are unusable under the current measurement scheme.

Result ② — measured amplitude on the positional axis

81% of the match-to-match variation in advancement reproduces within the same match (75% for width). About 25% of total variance is real match-level variation.

Role is fixed across a season; within that role the position taken up moves by about 8 m per match and as much as 27 m across matches. That variation is what "task" refers to.

Result ③ — the link between task and output was not established

In a within-player cross-validation with player identity removed, usage explained individual attacking output at R² 0.117. Restricted to variables with confirmed reliability, this falls to R² 0.035.

The 0.117 was driven mainly by shot share and dribble share, while the output metric itself contains penalty-area shots and completed dribbles. The same events appear on both sides of the equation, and both variables are unusable by the reliability criterion above. The valid figure is 0.035. For context, a 2026 study reports that adding player and team embeddings does not significantly improve next-event prediction, while noting that those embeddings do carry meaningful patterns about role and team style.

Verdict — no evidence was obtained that a change of task changes outcomes. The choice of output metric may be responsible: the effect of an inverted winger appears not in that player's numbers but in the space the full-back gains. Re-measurement against team-phase output is required and is constructible from event data.

Result ④ — coming on as a substitute does not itself change usage

For the 11 players with at least three starts and two substitute appearances, compared on the reliable variables: advancement increased in 5 of 11 and width in 3, with mean changes of −0.28 pp and −0.77 pp. The act of being introduced does not determine how a player is used.

06Design decision ① — the task is declared by a person

§04 showed the absence of role labels; §05 showed that only the positional axis is measurable. Both converge on one design. Do not infer the role from data; have the user declare it.

What is declared is not the role but the task. Within a season a regular's role is fixed (ICC 0.69), and it is rare to declare a change of position at the moment of a substitution. What is actually instructed is a task inside the role — "cutting inside from the right" — and that corresponds to the ±8.3 m positional variation measured in §05.

The query changes shape, from P(outcome | player) to P(outcome | player, declared task, context). The same design is supported by a limitation stated in the prior literature.

A generative model reproduces observed usage; newly instructed usage lies outside the training distribution. This design supplies that input from a person.

The opponent side follows the same principle. An opponent substitution is not an object of prediction but an observed event. The system looks up the incoming player after the fact and opens response branches, so it does not require the timing prediction removed in §03.

68 minutes, touchline

⚠️ Interface mockup — the figures on screen illustrate behaviour and are not measurements

07Design decision ② — evaluation, not prediction

Matching the four measurement results against the two candidate feature sets:

Prediction depends on four unverified premises. Evaluation depends on zero or one. "What this substitution will produce" has no ground truth and cannot be validated, whereas "was the declared plan carried out" is an observation. Returning references is retrieval. Neither involves inference.

The loop

Declared midweek · measured in the match · scored afterwards · accumulated

08Which instructions are measurable

The split-half reliabilities from §05 define the feature range directly.

A game plan written in the language of position and direction can be evaluated. A game plan written in the language of intensity and frequency cannot, at this stage.

One response therefore has to exist on the input side — "That instruction cannot be quantified under the current measurement scheme. Please state it in positional terms." It is the Tier discipline of §09 applied to input.

Match phase (build-up, transition, defending) is not detected automatically. Accuracy from events alone is insufficient, and the user already holds the information. It becomes a declared field.

09Tier discipline — stating the thickness of the evidence

Data support for a declared task is not uniform. Across the same 17 players as §05, those with three or more matches in each of two distinct usage modes number 41–59%, depending on how finely usage is partitioned. The remainder is extrapolation.

Answering without distinguishing support means returning unsupported values in the same format as supported ones. For a user deciding within thirty seconds, that is the critical failure mode.

Tier discipline — the thickness of the evidence travels with the answer

The same question must be answered differently depending on data support

10Data constraint — the opponent

Conditioning on an opponent's usage requires repeated observations of that player.

Observation depth: our players vs opponents

FC Seoul, 25 matches · 519 player-matches (same filter as §05)

11Operating structure — compute midweek, retrieve in-match

There is no time in-match to run a counterfactual generative model through hundreds of Monte Carlo samples, and by the principle set out in the HOVI write-up the computation cannot be delegated to the LLM either. Moving computation into the training week removes the constraint.

Compute midweek, retrieve during the match

When the tool is used determines what it costs to run

A declared task with its expected effect and falsifier is a labelled prediction, and can therefore be scored after the match. At roughly 100 per season, the role and task labels whose absence §04 identified accumulate without a separate collection effort.

Risk — declaring only the expected effect invites confirmation bias. The falsifier has to be declared alongside it for the record to carry verification value.

12The final goal and the current boundary

The final goal is to watch the same match together, exchange football conversation, and have that conversation used in the manager's decisions. This write-up does not reduce that goal. It fixes the point reachable now.

The staircase to the final goal

Scope at each stage, and what blocks the next

What blocks stage 3

Two of the six are solvable directly. The rest require access inside a club or progress elsewhere. The scope of this stage is therefore limited to stage 1.

Stage 1 is also a precondition for stage 3. Repeating declaration and scoring across a season accumulates the task labels whose absence §05 identified, and those labels become the training data for task-conditioned simulation. Stage 2 opens on top of stage 1's output.

13What has to be verified

Nothing is confirmed yet. In order of verification:

1. The share of a manager's instructions expressible in positional language. §08 pinned the feature range to this. A full match of instructions has to be transcribed and the share expressible as a measurement formula counted. A low share halves the feature range — the first item to check, and it needs one match and one sheet of paper
2. Whether declaration is sustained in practice. Writing a task, expected effect and falsifier for every substitution is a question of operating habit, not technology
3. Whether team-phase output detects the task effect. Individual output gave R² 0.035. This is the route to the open question in §05
4. Whether league-wide events can be collected across multiple seasons. The constraint in §10 is resolved here
5. Whether Tier 3 discipline holds. "Do not answer" is a rule whose violation becomes more tempting as the tool becomes more useful. The criterion for detecting a violation has to be defined in advance
6. Testing the trailing-and-attacking hypothesis. It currently rests on eight matches and needs multi-season expansion
7. How §04's result changes under role labels. The figure was computed with an instrument blind to 66.5% of cases. A different instrument may change it — or may not

Three write-ups, one system

HOVI is the conversational layer; two organs sit beneath it

The answer range of this tool ends at "was the plan carried out" and does not include "was the plan right". The latter is judgement, and sits on the other side of the line drawn in the HOVI write-up.