Parts 4 and 5 kept one network responsible for everything: reading the image, reading the words, and flying. The words kept losing to the image. This part tests a different division of labour, stated as a design rule:

The sentence should only choose the target. The image and the state should do the flying.

So the words get exactly one job and no path to the motors. A world model learns where every target is without ever seeing a sentence. A tiny selector turns the sentence into a colour. A planner, which is not learned, flies to the chosen target.

1. The system: perceive, select, plan

1. Perceive (learned, no words). An encoder of the same kind as the BC policy’s turns the image and state into 64 numbers, z. A small head reads out, for each of the four colours, the offset from the drone to that colour’s hover point. The head was trained against the simulator’s true positions; those positions are training labels only, never inputs.
2. Select (learned, 340 parameters). Word embeddings, averaged, then one linear layer to four colours. It is 100% correct on the train and validation sentences. In this task one colour word decides the target, so that is expected, not a result.
3. Filter (rule). Single-frame estimates are noisy (about 0.17 m near the goal), and acting on them raw makes the commands jitter. The filter predicts the offset forward by the last command and blends in each new measurement at 30%.
4. Plan (rule). The cross-entropy method (CEM): sample 64 sequences of 8 commands (1.6 s), score each by where it would leave the drone relative to the goal, keep the best 8, resample around them, four rounds. Fly the first command and plan again 0.2 s later.
5. Stop (rule). The expert’s own rule: Stop when the estimated goal is within 0.12 m and the drone is slower than 0.08 m/s.

The planner is short (full code):

def plan(self, g0, displacement):        # g0: filtered goal offset (2,), body frame
    mean, std = self.mean, np.full((8, 2), 0.3)
    for _ in range(4):
        A = cap(mean + std * rng.standard_normal((64, 8, 2)))   # 64 sequences of 8 commands
        off = g0 + displacement(A)                              # where the goal would be
        cost = (off ** 2).sum(-1).mean(1)                       # stay close to it
        cost += 0.5 * jerk(A) + 1.0 * (A[:, -1] ** 2).sum(-1)   # smooth, arrive slowly
        elite = A[np.argsort(cost)[:8]]
        mean, std = elite.mean(0), elite.std(0) + 1e-3
    return elite[0][0]                                          # fly step 1, re-plan next tick

integrator = lambda A: -np.cumsum(A, axis=1) * 0.2               # variant (A)

The only difference between the two variants is displacement. (A) assumes the drone moves exactly as commanded. (B) starts from z, applies the world model’s learned dynamics z’ = LN(z + f(z, a)) step by step, and reads the head after each step.

All planner settings were tuned with the true target positions on training scenes, before any learned variant was run on validation.

2. What the world model learned

The world model has 574,515 parameters and was trained on CPU for 12 epochs (36 minutes). Its training data was the 100 demonstration layouts plus exploration flights over the same layouts: noisy expert, random walks, and overshoots past the goal. That is 30,448 rows, three times the states the BC policies saw. The loss unrolls the dynamics 4 steps from each start state with the recorded commands (full code):

z = model.encode(rgb[:, 0], state[:, 0])
loss = heads_loss(z, k=0)                                  # state + every colour's offset
for k in range(1, 5):
    z = model.step(z, command[:, k - 1])                   # z' = LN(z + f(z, a))
    target = model.encode(rgb[:, k], state[:, k]).detach() # what the encoder sees at t+k
    loss += 0.9 ** k * (((z - target) ** 2).mean() + heads_loss(z, k))

• A, perception is good enough to plan with. On unseen layouts the estimated goal position is off by a median 0.15–0.26 m, under the 0.4 m goal radius in 68–92% of frames. Training on 100 scenes instead of 20 roughly halved the error. The bottleneck was the number of distinct scenes, not the model.
• B, the learned dynamics are worse than doing nothing clever. Starting from the same perceived position, rolling the learned dynamics forward 2 s gives 0.39 m error on expert flights and 0.57 m on random walks. Simply integrating the commands stays near 0.30 m. In this simulator the low-level controller tracks commands closely, so an integrator is almost perfect dynamics.
• C, and the learned dynamics are biased by direction. Hold one command for 1 s: the model predicts forward and left moves about right, but backward moves twice as far as commanded and right moves at 0.58 of the commanded distance. Most training commands point forward, toward the targets.

3. Results

(A) World model + integrator planner.

• All 10 pairs split the right way (no end-to-end model splits more than 5 correctly; Fisher’s exact test for 10/10 vs 5/10: p ≈ 0.03).
• Success is 15/20 and pair success 6/10, 95% interval 0.31–0.83. Against the best end-to-end model’s 3/10 (0.11–0.60) that difference is not yet beyond chance.
• It reaches the 0.4 m goal region in all 20 flights.
• The five misses all come after it has reached the region: two stops just outside it and three collisions. The same planner fed the true target position succeeds 20/20, so what is left is perception error close to the target.

(B) The same system planning through the learned dynamics scores 0/20: 13 timeouts and 5 collisions. The GIF above shows why. CEM searches for commands that look good according to the model, and panel C showed the model exaggerates some directions. The planner keeps choosing the commands whose effect the model gets most wrong, and the drone jitters. Two things make this worse: the model was trained to unroll 4 steps and is asked for 8, and most of its training commands point forward, while a planner tries every direction.

4. Is this a fair comparison? No, so here is what it does and doesn’t explain

The modular system differs from the BC policies in more than its structure:

Neither the labels nor the states, alone or together, gave an end-to-end policy the modular system’s 10 of 10 correctly split pairs. What’s left is the structure: the words can only choose, and nothing downstream can ignore that choice. I can’t yet separate how much comes from the selector and how much from the rule-based Stop.

Two smaller differences:

• Randomness. The planner’s random seed differs between the two flights of a pair; BC evaluation is deterministic.
• Training budget. The world model trained for 36 minutes, about 4–5 times a Part 4 BC run.

What this part does not show

• One training seed and ten pairs. 6/10 has a 95% interval of 0.31–0.83.
• The selector’s job is trivial here. Two pillars always differ in colour, so one word decides. Relational instructions (“the pillar left of the red one”) would need a real grounding model.
• The integrator works because the simulator is kind. Its controller tracks commands almost perfectly. With wind, drag or a real airframe, the dynamics would have to be learned or identified, and section 3 shows that naively learned dynamics are a liability for a sampling planner.
• Validation only. It was used for screening; final numbers need a fresh test split.

Next

Season 2 ends with a working answer to Part 4’s question, for this two-pillar, one-colour-word task: yes, the words can be made to decide, if they are only allowed to decide. Season 3 turns to engineering. One evaluation contract will run the same policies in Python and C++, so every later change (ONNX, INT8, a C++ runner) is measured against the same numbers.

Appendix

python -m dronevla.world_model train --data data/v0.2 --explore data/explore_v0.2 --out runs/wm_v1
python -m dronevla.world_model eval --run runs/wm_v1 --out reports/wm_v1_prediction.json
python scripts/wm_diagnostics.py --run runs/wm_v1 --out reports/wm_v1_diagnostics.json
python -m dronevla.planner train-selector --data data/v0.2 --out runs/goal_selector
python -m dronevla.evaluate --data data/v0.2 --split val --policy plan-oracle \
    --policy plan:runs/wm_v1:integrator --policy plan:runs/wm_v1:wm \
    --out reports/eval_planner_val.json
python scripts/figures/blog06.py
bash scripts/figures/render_blog06_videos.sh

References: what I took from each

• World Models (Ha and Schmidhuber, 2018) and PlaNet (Hafner et al., 2019) learn a compact latent state and its dynamics from images. → The encoder, latent z and residual dynamics.
• PETS (Chua et al., 2018) plans with CEM through learned dynamics. → The planner. Its use of model ensembles to avoid exploiting model error is exactly what variant (B) lacks.
• When to Trust Your Model (Janner et al., 2019) shows how model error compounds over long rollouts. → Why the planner was given an integrator first, and why (B) fails at an 8-step horizon after 4-step training.
• TD-MPC (Hansen et al., 2022) plans in a latent space learned for control. → A direction for making (B) work, not tried here.
• CLIPort (Shridhar et al., 2021) separates what (language-conditioned semantics) from where (spatial precision). → The same split, in its simplest possible form.

Summary (STAR)

Part 4 ended with a diagnosis. The words matter on one frame per episode, the first, which is 2% of the training rows. On that frame the policy answered with the average of the two sentences’ answers. On every other frame the image already showed where to go.

Large vision-language-action models (VLAs) don’t output actions the way that policy did. RT-2 and OpenVLA turn each action dimension into one of 256 tokens and train with cross-entropy. OpenVLA-OFT and ACT predict a chunk of several future actions at once. This post borrows those two output formats and nothing else. The encoders keep their design, the data and the parameter budget stay as they were, and the image layer shrinks from 128 to 104 numbers to keep the parameter counts comparable. There is still no pretrained model; that arrives in Season 4.

How these experiments were run

This post is a screen, not a final result, so the rules are lighter than Part 4’s and are stated up front:

• One training seed per model (seed 0). Part 4 used three.
• Validation only: 10 pairs, 20 flights. The test split is untouched.
• Same training recipe as Part 4: 20 epochs, keep the last, dataset v0.2 (100 training pairs).
• Same size: every model has 80–100% of Part 4’s 480,901 parameters.

Single-pair differences are noise at this sample size, so the post gives intervals where it matters.

1. What changed: only the output

Tokens. For each axis, the training speeds are split into 256 equal bins between their 1st and 99th percentiles, as OpenVLA does. The head outputs 256 scores per axis instead of one number. Training asks it to score the expert’s bin highest (cross-entropy); flying uses the centre of its top bin (model code):

def set_action_bins(self, actions):            # actions: (N, 4) training speeds / limits
    for i, d in enumerate((0, 1)):             # vx, vy; vz and yaw rate are always 0 here
        lo, hi = torch.quantile(actions[:, d], torch.tensor([0.01, 0.99]))
        self.bin_edges[i] = torch.linspace(lo, hi, 257)

def tokenize(self, actions):                   # training target: which bin
    return torch.stack([torch.bucketize(actions[:, d], self.bin_edges[i, 1:-1])
                        for i, d in enumerate((0, 1))], dim=1)

def detokenize(self, logits):                  # flying: the centre of the most likely bin
    centres = (self.bin_edges[:, :-1] + self.bin_edges[:, 1:]) / 2
    return centres.gather(1, logits.argmax(-1).T).T

Chunks. The head predicts the next 8 steps at once. A shared trunk produces one feature, a learned embedding for each step k is added, and one decoder turns each into bins plus a Stop logit. The loss covers every future step that exists in the episode (training code):

logits_h, stops_h = model.heads(feats)[2]          # (B, 8, 2, 256), (B, 8)
target = model.tokenize(chunk_action[idx])         # expert's next 8 actions, as bins
ce = cross_entropy(logits_h.flatten(0, 2), target.flatten(), reduction="none")
motion = (ce.view(B, 8, 2).mean(-1) * mask).sum() / mask.sum()   # mask: steps past the end
stop = (bce(stops_h, chunk_stop[idx]) * mask).sum() / mask.sum()
loss = motion + stop

The model is still asked every 0.2 s, and only the first step is flown. So the chunk changes what the model is trained to predict, not how it flies. Section 3 also tests flying all 8 steps.

2. Why tokens fly better: averaging versus choosing

On the first frame, the image and the state are identical for both sentences of a pair, and their labels differ. Regression with a Huber loss is pulled toward the middle. Over all 200 training first frames, the Part 4 model is closer to its own sentence’s answer in exactly 100 and to the other sentence’s answer in the other 100. It sits on average 0.03 m/s from the midpoint.

A token head can’t place its answer between two bins it was never shown. It has to put its probability somewhere, and it mostly puts it on one expert answer. It picks the right one in 135 of 200 training first frames. Its median error is 0.0005 m/s, but its mean is 0.066 m/s, because the wrong picks are a whole answer away. Training pair 0, sentence B in the figure is such a wrong pick.

In flight, a regression policy that starts in the middle tends to hover between the pillars. A token policy commits to a side. That alone moves the numbers:

This is the multimodality problem that Diffusion Policy and Behavior Transformers were built for: when two answers are both right, a model that averages them is wrong for both. Here the two answers come from two sentences rather than from two demonstrators, but the loss can’t tell the difference on that frame.

3. Results: the chunk is the best end-to-end model so far

• Tokens alone don’t make the drone choose. With 256 bins, 7 of 10 pairs still go to the same pillar for both sentences. The token head commits, but on most first frames it commits to whatever the image suggests.
• Tokens + an 8-step chunk is the best end-to-end model: 4 pairs split the right way, and pair success is 3/10. One plausible reason: predicting 1.6 s ahead forces the representation to encode where the flight is going, not just the next step. Its 95% interval is 0.11–0.60, which overlaps Part 4’s 0/10 (0–0.28). With one seed and ten pairs this is a lead, not a proven gain.
• Flying all 8 predicted steps before asking again flies about as well (9/20) but listens less (1 correct pair). My guess: re-asking every step lets each new image correct the course, and flying 8 steps blind gives that up.

The first three validation pairs, unselected, show how far from solved this is:

4. More states: it reacts, then it can’t stop

Part 4’s diagnosis was that the words decide only on frames the policy rarely sees. Part 6’s world model was trained on three times as many states: expert flights plus exploration flights (noisy expert, random walks, overshoots past the goal). So I gave the chunk policy the same 400 exploration flights (19,896 states), each state labelled with what the expert would do from there toward a target picked at random for that episode, with the matching sentence. For exploration rows only the current step is a label; their future chunk steps are masked, because a random walk isn’t the expert’s plan.

On training frames, the swap test now looks like the expert. Changing only the sentence moves the first command by a median 0.196 m/s (the expert: 0.200), and it moves in the right direction in 180 of 200 frames. In closed loop it splits 5 of 10 pairs the right way, the most of any end-to-end model (cross-attention + counterfactual relabelling also reaches 5). But 8 of 20 flights overshoot the target and leave the room, success falls to 6, and pair success is 0.

I didn’t separate the causes. Three candidates:

• Stop is diluted. The exploration rows contain only 6 Stop examples, so Stop becomes a rarer event in the training mix.
• The bins got coarser. The bin range comes from the training speeds, and exploration widens the vy range from about ±0.17 to ±0.42 m/s. Each bin is 2.4 times wider, which is coarse near the goal.
• Recovery labels near the goal. Overshoot flights teach large corrections exactly where the expert usually slows down.

With exploration states the first-frame reaction matched the expert’s on training frames (median 0.196 vs 0.200 m/s), but pair success fell from 3/10 to 0/10. The experiment changes what the policy reacts to without changing how it stops. That split, choosing versus flying, is what Part 6 builds in on purpose.

What this part does not show

• One training seed and ten pairs. Wilson 95% for 3/10 is 0.11–0.60. Single-pair differences, and the bin-count comparison in the appendix, are within noise.
• Validation was used both to screen models and to set the Stop threshold, so these are not generalisation estimates. The original test seeds were seen in an earlier round; final numbers need a fresh test split.
• The swap test runs on training frames. Near-zero token errors there may partly be memorisation.
• The language is easy. The two pillars always differ in colour, so one word decides.
• It is not a VLA. Only the output format is borrowed; there is no pretrained vision or language model.

Next

No end-to-end model splits more than 5 of 10 pairs the right way, and the best one completes 3. Part 6 tries the opposite design: a model that perceives where every target is without reading the words, a tiny selector that lets the words pick one, and a planner that flies there.

Appendix

A. Everything else I tried (validation, one seed)

model	params	correct / swapped / same-target pairs	success /20	pair success /10
Part 4 regression	480,901	0 / 0 / 10	3	0
+ counterfactual relabelling	480,901	2 / 3 / 5	0	0
+ counterfactual + FiLM	492,517	4 / 3 / 3	0	0
+ target-position aux head	493,773	0 / 0 / 10	8	0
+ cross-attention	420,997	0 / 0 / 10	0	0
+ cross-attention + counterfactual	420,997	5 / 3 / 2	0	0
256-bin tokens	469,609	3 / 0 / 7	8	1
64-bin tokens	471,289	1 / 1 / 8	7	1
1024-bin tokens	475,609	2 / 0 / 8	4	0
256-bin + paired goal loss	480,555	2 / 0 / 8	7	0
256-bin + cross-attention	477,289	1 / 0 / 9	9	0
256-bin + chunk 8, fly step 1	470,633	4 / 1 / 5	8	3
256-bin + chunk 8, fly all 8	470,633	1 / 0 / 9	9	1
chunk 8 + exploration states	470,633	5 / 2 / 3	6	0
chunk 8 + exploration + aux head	468,909	5 / 2 / 3	2	0

Notes:

• Counterfactual relabelling adds, for every training state, the expert's action toward the other target with the other sentence. The words start to matter, but correct and swapped pairs are about equal, and flying collapses. Part of that is a confound: the added rows contain no Stop positives, while the Stop loss weight stayed at its original value.
• Aux head: an extra output trained to predict where every colour's target is, from the simulator's true positions (the same labels Part 6's world model gets). It flies better (8/20) and listens no more.
• FiLM and cross-attention let the sentence modulate the image features (FiLM scales and shifts the convolution channels; cross-attention uses the sentence as a query over 6 × 8 image patches). The FiLM row and the aux-head row predate the 80–100% parameter rule (FiLM 102%, aux 103%).
• Bin count: 64, 256 and 1024 bins are within one seed's noise of each other.

python -m dronevla.train --data data/v0.2 --epochs 20 --select last --seed 0 \
    --action-bins 256 --img-dim 104 --out runs/bc_tok256_v0.2_s0
python -m dronevla.train --data data/v0.2 --epochs 20 --select last --seed 0 \
    --action-bins 256 --img-dim 104 --chunk 8 --out runs/bc_tokchunk8_v0.2_s0
python -m dronevla.train --data data/v0.2 --epochs 20 --select last --seed 0 \
    --action-bins 256 --img-dim 104 --chunk 8 --explore data/explore_v0.2/train \
    --out runs/bc_tokchunk8_explore_v0.2_s0
bash scripts/run_tokens_s0.sh && bash scripts/run_tokens2_s0.sh   # evaluate (and train if missing)
bash scripts/run_explore_s0.sh
python scripts/season2_summary.py                 # every number in this post (needs all run_*_s0.sh)
python scripts/figures/blog05.py                  # figures
bash scripts/figures/render_blog05_videos.sh      # GIFs

References: what I took from each

• OpenVLA (Kim et al., 2024) and RT-2 (Brohan et al., 2023) discretise each action dimension into 256 bins and predict them as tokens. → The token head and its 1st–99th percentile bins.
• OpenVLA-OFT (Kim, Finn and Liang, 2025) and ACT (Zhao et al., 2023) predict chunks of future actions. → The 8-step chunk. Here only the first step is flown.
• Diffusion Policy (Chi et al., 2023) and Behavior Transformers (Shafiullah et al., 2022) show that averaging a multimodal action distribution fails. → The explanation for section 2.
• Stop Regressing (Farebrother et al., 2024) finds classification losses train value networks better than regression. → Why switching the loss alone was worth testing.
• Causal Confusion in Imitation Learning (de Haan et al., 2019) describes policies that latch onto the wrong cue. → The image shortcut from Part 4, which tokens don’t remove.

Summary (STAR)

Three parts of setup lead to this one: a simulator, a scored task, and 100 pairs of flights in which the same image comes with two different sentences. Now the first policy. I expected it to be bad at flying. I didn’t expect it to fly toward a pillar and stop there while having no idea which one.

1. The model: small on purpose

Three inputs, one output, no pretrained parts:

• Image: four strided convolutions shrink 96 x 128 x 3 to a 6 x 8 x 64 map, then to 128 numbers.
• Sentence: each word gets a learned 32-number vector, and the vectors are averaged. With only 15 distinct words in the dataset, this is the simplest text encoder that can still tell “red” from “blue”.
• State: the 11 numbers from Part 2’s contract, normalised with training-set statistics.

They are joined into 192 numbers, and a small MLP outputs four velocities (squashed by tanh and scaled to the limits) plus one Stop logit. Total: 480,901 parameters. It runs in 2.7 ms per decision on the CPU (median), far inside the 200 ms budget (simplified below; model code):

def forward(self, rgb_u8, proprio, text_vec):
    """rgb_u8 (B, 96, 128, 3) uint8; proprio (B, 11) normalised; text_vec (B, 32)."""
    x = rgb_u8.permute(0, 3, 1, 2).float().div(255.0).sub(0.5)       # to (B, 3, 96, 128)
    feats = torch.cat([self.img_fc(self.cnn(x)),                      # (B, 128)
                       text_vec,                                      # (B, 32)
                       self.prop(proprio)], dim=1)                    # (B, 32)
    out = self.head(feats)                                            # (B, 5)
    return torch.tanh(out[:, :4]), out[:, 4]                          # velocities, Stop logit

2. Training: behaviour cloning on CPU

Behaviour cloning means copying the expert: for every recorded frame, predict the action the expert took. Two losses are added together: a Huber loss on the four velocities, and a binary cross-entropy on Stop, weighted up because only 6.7% of rows are Stop (full code):

huber = nn.SmoothL1Loss(beta=0.1)
bce = nn.BCEWithLogitsLoss(pos_weight=torch.tensor(13.9))   # from training rows only

def train_step(idx):
    rgb, prop, ids, act, stp = prep.batch(train, idx)       # act is scaled by the limits
    m, logit = model(rgb, prop, model.encode_text(ids))
    loss = huber(m, act) + bce(logit, stp)
    opt.zero_grad()
    loss.backward()
    opt.step()

Settings: Adam, learning rate 10⁻³, batch 16, 9,952 training rows. Each seed takes 7–8.5 minutes on the laptop CPU.

The evaluation rules, fixed before running

An earlier round of experiments taught me these rules by breaking them, so this time they were written down first:

1. Decisions use validation only. The 10 test pairs aren’t touched, and no number in this post comes from them. (Those test seeds were looked at during the earlier, discarded round, so final numbers will come from a fresh test split.)
2. Three training seeds. With 10 validation pairs, one seed’s luck can look like an effect.
3. One stopping rule for every run: train 20 epochs and keep the last. Picking the “best” epoch by validation loss favoured whichever epoch the Stop term happened to like.
4. “Which target did it go for” = the one it came closest to. Scoring by the final position misreads flights that end outside the room.

The Stop threshold is the only thing tuned, on validation rows.

3. Results: it flies, it doesn’t listen

The expert splits all 10 pairs. The policy split 0, 2 and 0 of them: for almost every layout it flew to the same pillar whichever sentence it was given.

The middle panel follows from that. Within a pair, each sentence names a different pillar. A policy that flies to the same pillar for both is right for exactly one of the two sentences, so any word-blind policy scores exactly 10 of 20, as Part 2 predicted. Seeds 0 and 2 are that case. Seed 1’s two split pairs land on 10 too: it split one pair the right way round and one the wrong way round. And no seed completed both sentences of any pair: pair success was 0/10 for all three seeds.

The top view shows it directly:

Its most common failures are stopping away from both hover points and stopping at the wrong pillar (appendix A). It does fly toward the pillars and it does stop. What it lacks is the one decision the sentence exists for.

4. Why: the words matter for one frame

My hypothesis was that the image makes the words unnecessary, and I tested it directly. From every recorded training state, I asked each trained policy for its action twice: once with the episode’s own sentence, and once with its pair partner’s. Then I compared how much the sideways command changed with how much the expert’s changes (measurement script).

A. On the identical first frame, swapping the sentence moves the expert’s sideways command by 0.200 m/s. It moves the policy’s by 0.004–0.007 m/s, 30 to 50 times less, for all three seeds. At later steps the gap is just as small. The policy has nearly disconnected the words. (The expert’s later-step bars are computed actions: from each logged state, what the expert would do for the other sentence. Those states never occur with the other sentence in the data, which is why its gap grows later in the episode.)

B. On the training set, the policy matches the expert to about 0.01 m/s on every step except the first, where its error is 0.10 m/s. That’s half of the 0.2 m/s gap between the two sentences’ labels, consistent with predicting the average of the two.

The mechanism is simple:

• On the first frame, both sentences share one image, and only the words say left or right. That frame is 2% of the training rows.
• On every later frame, the drone has already started moving to one side (its heading never changes; only its position does). The image itself shows which way it is going, so the image predicts the label. That’s 98% of the rows.
• Behaviour cloning minimises the average loss over all rows. The cheapest solution is to read the image and average over the 2% it can’t explain. In closed loop, the first move is then made without the words, and every frame after it confirms whichever way the drone happened to lean.

That is why the counterfactual pairs exist. Without them, this policy’s success rate could have looked like a language effect; with them, the word-blindness is visible.

What this part does not show

• 10 validation pairs is a small sample; a policy that listened only a little could hide in it.
• One small architecture and one way of encoding words. A bigger text pathway could behave differently; that is a question for the next season, not a result here.
• Simulation only, two pillars, two sentence templates.

Next

Season 2 asks whether the shortcut can be removed. One idea is to give the words a reason to matter on every frame. Another is to split the problem in two: first perceive where every target is, then choose the one the sentence names.

Appendix

for s in 0 1 2; do
  python -m dronevla.train --data data/v0.2 --epochs 20 --select last --seed $s --out runs/bc_v0.2_s$s
done
python -m dronevla.evaluate --data data/v0.2 --split val \
    --policy runs/bc_v0.2_s0 --policy runs/bc_v0.2_s1 --policy runs/bc_v0.2_s2 \
    --out reports/eval_v0.2_val.json
python scripts/language_use.py reports/eval_v0.2_val.json
python scripts/instruction_sensitivity.py --data data/v0.2 --split train \
    --runs runs/bc_v0.2_s0 runs/bc_v0.2_s1 runs/bc_v0.2_s2 \
    --out reports/instruction_sensitivity_v0.2_train.json
bash scripts/figures/render_blog04_videos.sh
python scripts/figures/blog02_04.py

References: what I took from each

• RT-2 (Google DeepMind) represents actions as text tokens inside a vision-language model. → That’s one end of the action-output spectrum; this policy is the opposite end.
• SmolVLA (Hugging Face) pairs a small VLM with a flow-matching action expert that outputs action chunks. → The target for Season 4, and why the (5,) contract will need a mapping step.
• A Recipe for Training Neural Networks (Karpathy) says to become one with the data and verify the evaluation before trusting a model. → Pair success and the swap test are how this project checks the policy instead of trusting the loss.

Summary (STAR)

Part 2 fixed the task and the scorer. Now the policy needs examples to learn from. The question of this series sets one hard requirement on the data: listening has to be necessary. If one image could only ever lead to one target, a policy could ignore the words and still match every label. So every example in this dataset comes as a pair.

1. How one pair is made

1. A seed picks a layout: two pillars of different colours, box or cylinder, somewhere in front of the drone. Layouts that would make a bad test are thrown away and resampled. Examples: hover points closer than 1.5 m, or a straight path to one pillar that brushes the other.
2. One start snapshot is shared by both flights. Everything random (the layout, the position-noise stream) comes from the seed, never from which target is named. That’s why the two first frames are byte-identical.
3. The scripted expert flies both targets through the same action adapter, controller and physics a policy would use. It is an oracle: it reads true positions, which a policy never gets. That’s fine for a teacher, as long as the dataset says so (it does).
4. The recorder writes every 0.2 s: the PNG image, the 11-number state, the action actually applied, the Stop label, and, in separate priv_* columns that policies never read, the true position for later analysis.

The sentences come from two human-written templates, with no language model involved: “Go to the {colour} {shape} and stop.” and “Approach the {colour} {shape}, then hold position.” Both flights of a pair use the same template, so the only difference between them is the colour and shape words.

2. What is in v0.2

Splits are by layout, not by frame. If frames from one flight landed in both training and validation, a model could memorise the scene and look good for the wrong reason. Here, both flights of a pair and every frame of a flight stay in one split, and no layout appears twice. The validation and test seeds are the same as in the first, smaller version of the dataset, so results stay comparable across versions. One caveat: those test seeds were looked at during an earlier, discarded round of experiments, so final test numbers will come from a fresh test split.

Two label properties shape everything in Part 4:

• Stop is rare. Only 6.7% of action rows say “stop”. The training loss weights those rows up, with the weight computed from training data only.
• Most of the time the expert flies at full speed. 75% of commands are at the 0.5 m/s cap. The interesting part of each label is its direction, and at the very start that direction depends only on the sentence.

The colour-by-side table is reported, not enforced: green goals were on the left 42 times and on the right 26. The splits are small, so a balanced design would have to come from the generator, not from filtering afterwards.

3. Proving the files are what they claim

A dataset that changes silently invalidates every result built on it. So the recorder writes a manifest: the code commit, the simulator commit, the full task configuration and its hash, the camera parameters, a SHA-256 for every non-image file, and one hash over all 12,198 images. Re-running the command on the same machine reproduces the images byte for byte.

Then a validator reads everything back from disk and runs 14 checks: files present and hashes matching, images decoding to 96 x 128 x 3, every pair complete and identical at the start, splits disjoint, every sentence naming its goal, actions aligned with observations. Here is the pair check, slightly simplified (full code):

for pid, eps in by_pair.items():
    a, b = sorted(eps, key=lambda e: e["goal_index"])           # goal 0 and goal 1
    for key in ("split", "layout_id", "snapshot_seed", "first_rgb_sha256", "instruction_family"):
        if a[key] != b[key]:
            pair_problems.append((pid, f"{key} differs"))       # same start, same first image
    if a["instruction"] == b["instruction"]:
        pair_problems.append((pid, "same instruction for both goals"))
    if load_steps(root, a["episode_id"])[0]["proprio"] != load_steps(root, b["episode_id"])[0]["proprio"]:
        pair_problems.append((pid, "first proprio differs"))    # same state, too
check("pairs_complete_and_identical_at_start", not pair_problems, ...)

4. Two bugs that only reading the output caught

Both happened while I was building the first version of the generator. Each is now covered by a test.

A good layout was rejected: the visibility check rendered from inside the drone. One candidate layout was thrown away because “a target covers 0 pixels at the start”. The real first frame showed it clearly, at 346 pixels. Hypothesis: the check and the recording render from different places. Confirmation: the check rendered from the nominal start position. Position noise leaves the hovering drone a few centimetres away from there, so the check’s camera sat inside the real drone’s arm and propeller, which filled the whole view. Fix: render the check from the pose of the actual first observation. The layout is now test pair 5, with both targets clearly visible.

The teacher was “speeding” 37% of the time. The expert commanded exactly 0.5 m/s. Stored as float32, that became 0.5000001 m/s, so the adapter flagged a cap on 37% of the expert’s actions. Numerically harmless, but it would have reported a teacher that never exceeds the limit as clipping more than a third of the time. The expert now commands 0.5 x (1 − 10⁻⁵): zero flags.

What this dataset can’t support

• Two pillars in an empty room, yaw fixed, flat motion. It says nothing about open-vocabulary language or real scenes.
• 10 validation and 10 test layouts are few, so success rates will carry wide intervals.
• The two pillars always differ in colour, so colour alone always decides the target.

Next

Part 4 trains a policy on these 100 pairs, and it does not listen.

Appendix

python -m dronevla.record --out data/v0.2 --pairs 100 10 10 --version 0.2.0    # ~9 min on CPU
python scripts/inspect_episode.py --dataset data/v0.2 --episode val-004-g1      # every frame + action arrow
python scripts/figures/blog02_04.py

References: what I took from each

• Datasheets for Datasets (Gebru et al.) proposes a standard record of a dataset’s purpose, composition and collection. → The recorder writes a datasheet with every v0.2 number in this post computed from the files.
• LeRobotDataset stores episodes as Parquet plus media with metadata. → I kept a simple Parquet + PNG layout now and will convert when SmolVLA needs it.

Summary (STAR)

Part 1 ended with a fast simulator. Fast isn’t the same as believable, and believable isn’t the same as testable. Before training anything, I wanted three things fixed: which parts of a real drone the simulation copies, what exactly a policy sees and outputs, and how a flight is scored. If those move later, every result moves with them.

1. A spec sheet, filtered by scale

I took the spec sheet of a real consumer drone, the DJI Mavic 4 Pro, and went through it item by item. The simulated airframe is a Bitcraze Crazyflie 2.x: the physics parameters in gym-pybullet-drones (mass, inertia, thrust, drag) come from that real product.

A 40x difference in mass means absolute numbers such as “25 m/s” can’t transfer. So every spec item went into one of three bins:

2. The camera: a gimbal, tilted down

In Part 1 the camera was bolted to the body and looked straight ahead. That has two problems. Half of every frame was sky, and when the drone pitches forward to accelerate, the whole image tilts with it. The Mavic carries its camera on a 3-axis gimbal, which keeps the image level. So the simulated camera now does the same, tilted 20° down, with the Mavic’s 72° (diagonal) field of view.

Tilting 20° down also moves the pixels from sky to floor and targets, which fixes Part 1’s “half the frame is sky” finding. The targets became 1.2 m pillars so they stay in view all the way to the stopping point. At the start, each target fills 1.5–4.5% of the frame; at the stopping point, the named one fills 36–43%.

3. Position noise, chosen against the expert

A real drone doesn’t know exactly where it is. Its position estimate drifts slowly. I added a noise process to the position the controller sees. The camera still renders from the true position, because it is physically on the drone. The size, 3 cm horizontally and 1 cm vertically, is a choice. It keeps the Mavic’s 3:1 ratio, scaled down to fit a small room.

The timing of the noise took more work. My first version was jumpy, and the scripted expert (a controller that knows the true positions) began to fail: it couldn’t hold still after stopping, because the controller chased the jumping estimate as if it were real motion.

This choice was tuned on one set of seeds, so those 20/20 don’t count as evidence. On 40 held-out episodes that I never looked at while choosing, the expert succeeded in all 40.

Wind and a tilt limit exist as well. The dataset in Part 3 uses noise and the tilt limit but keeps wind off; the details are in the appendix.

4. The contract every policy must satisfy

This is the most important picture in the series. Every policy, from the tiny one in Part 4 to SmolVLA later, has to fit into it.

Two details matter:

• The action is a velocity request, not motor speeds. The policy says “go 0.4 m/s forward, 0.1 m/s left”. A PID controller underneath turns that into motor commands 12 times per decision. So the policy only has to learn where to fly.
• One adapter enforces the limits, for everyone. The expert, the learned policies and later the C++ runtime all go through the same function (full code):

def adapt(raw, limits=ActionLimits(), planar=True):
    """Validate and cap one raw policy action. Never raises on bad numbers -- it reports."""
    raw = np.asarray(raw, dtype=np.float64).reshape(-1)
    if raw.shape != (5,):
        return AdaptedAction(..., rejected=f"wrong shape {raw.shape}, expected (5,)")
    if not np.all(np.isfinite(raw)):
        return AdaptedAction(..., rejected="non-finite value in action")

    vx, vy, vz, yaw_rate, stop_logit = raw
    speed = math.hypot(vx, vy)
    if speed > limits.horizontal_mps:            # 0.5 m/s, capped as a vector, not per axis
        vx, vy = vx * limits.horizontal_mps / speed, vy * limits.horizontal_mps / speed
    ...                                          # same for vz (0.3 m/s) and yaw rate (0.5 rad/s)
    return AdaptedAction(applied=np.array([vx, vy, vz, yaw_rate]),
                         stop_positive=bool(stop_logit > 0.0), flags=flags)

The adapter records which limit engaged in flags, so “the policy asked for too much” is always visible in the logs instead of silently corrected.

5. The task, and how a flight is scored

The scene. An 8 x 8 x 3 m room. The drone starts on one side at 1 m altitude. Two pillars with different colours stand on the other side. Each has a hover point 0.75 m in front of it, where the drone should stop.

One episode. The drone gets one sentence, such as “Go to the red box and stop.”, and flies. To stop, the policy must signal Stop three decisions in a row (0.6 s). The drone then holds position for one second while the evaluator watches.

Success means all of these, judged on the true simulator state:

• within 0.4 m of the named hover point,
• slower than 0.1 m/s for the whole second after Stop,
• no collision, and within 30 s.

Everything else is a named failure, because “failed” alone teaches nothing:

Pair success is the metric the series is built around. Each layout is flown twice from the identical start, once per sentence. A pair counts only if both flights succeed. A policy that ignores the words and always picks one pillar can still score 50% on single episodes, but its pair success is zero.

Testing the scorer itself. An evaluator with a bug produces believable wrong numbers. So each outcome has a test that causes it on purpose: fly to the wrong pillar, ram a target, signal Stop while still moving, and so on. The evaluator must name every one correctly. The same test suite (79 tests at the commit below) also covers the action contract, the environment API and pair determinism: both flights of a pair get byte-identical first images.

What this part does not show

• This is not a Mavic. It is a 27 g quadrotor with a Mavic-like camera and noise profile.
• The 3 cm noise size is a choice, not a measurement. The 72° field of view assumes DJI quotes it diagonally, their usual convention.
• Everything is simulation.

Next

The task and scorer are fixed. Part 3 builds the dataset: 120 pairs of flights where the same image comes with two different sentences.

Appendix

python -m pytest tests -q                    # includes one fixture per outcome
python scripts/try_env.py --headless         # same start, two instructions, two outcomes
python scripts/playground.py --headless      # camera mount, noise, wind: change and re-run
python scripts/figures/blog02_04.py          # every figure in parts 2-4

References: what I took from each

• DJI Mavic 4 Pro specs give the camera, gimbal, tilt and hover-accuracy numbers. → I copied only the items whose meaning doesn’t depend on size.
• Bitcraze Crazyflie 2.1 is the real airframe behind the simulated one. → Its mass and motor model are why the absolute speeds stay small.
• Gymnasium: create a custom environment defines reset/step, terminated vs truncated. → A timeout is a truncation, not a failure state, which is why timeout is scored separately.

Summary (STAR)

If I change only the words, will the drone fly somewhere else? Before I could even ask that, my simulator produced 1,000 frames that passed every automated check I had written, and a contact sheet showed the targets cut out of view. This post is about turning a CPU-only simulator into something that produces usable observations, and measuring what each recorded frame costs.

The first recording looked valid, and wasn’t

The setup is a laptop with no NVIDIA GPU: WSL2 + Ubuntu, PyBullet through gym-pybullet-drones, and a camera on the drone that records a 128 x 96 image every 0.2 simulated seconds.

My first 1,000-frame recording passed every check I had written: right count, right shape, no NaNs, no duplicate images, nothing blank. Then I laid 20 frames out as a contact sheet and looked at them. In most of them the targets were cut off at the bottom edge.

Hypothesis: the drone was flying too high to see objects on the floor. Check: it flew at about 1 m, and the targets sat about 1.2 m ahead at 0.15 m. Looking down at them takes atan(0.85 / 1.2) ≈ 35°, but the level camera’s 60° vertical field of view only reaches 30° below its centre line.

Fix and verification: fly at 0.4 to 0.6 m and record again. The new contact sheet shows both targets in view:

A frame of sky and floor is non-blank and unique, so none of the checks I had written could see this. Looking at the data did. I’ll come back to this lesson throughout the series. The guard that would catch it automatically is a target-visibility count from the segmentation mask.

The minimal simulator

Two choices matter for everything after this:

• Linux, outside OneDrive. The code runs in WSL2’s own filesystem. Later stages (ROS 2, PX4, a C++ core) are Linux-first, and I didn’t want a sync client or a folder name with spaces and Korean letters in the build path.
• A CPU renderer. PyBullet’s TinyRenderer draws the camera image in software, so the missing GPU never enters the picture.

Three clocks

• Physics, 240 Hz. The engine advances the world 1/240 s per step.
• Controller, 60 Hz. A PID controller turns “fly this way” into four motor speeds. It reacts to the current error, how that error has added up, and how fast it is changing.
• Camera and policy, 5 Hz. Every 0.2 s the drone takes a picture and the policy decides what to do next.

So one recorded frame is 48 physics steps and 12 controller updates. I started with the examples’ 48 Hz, but that puts 9.6 updates in a frame, so I switched to 60 Hz: my fixed-tick recording loop then runs exactly 12.

What the drone actually sees

To see what is in those pixels, I rendered one frame with PyBullet’s segmentation mask, which labels every pixel with the object it belongs to, and counted:

When so little of the image carries the answer, a small policy can latch onto an easier cue instead. Part 4 shows a policy doing exactly that.

The camera call is short (simplified; full code in dronevla/camera.py):

def render(self, pos, quat, client=0):
    """Return one frame as (height, width, 3) uint8, alpha dropped."""
    w, h, rgba, _, _ = p.getCameraImage(
        width=self.width, height=self.height,          # 128 x 96
        viewMatrix=self.view_matrix(pos, quat),        # where the drone is and where it looks
        projectionMatrix=self._projection,             # 60 deg vertical FOV, square pixels
        shadow=0,                                      # off: the default doubles render time
        flags=p.ER_NO_SEGMENTATION_MASK,               # off: the policy never sees it
        renderer=p.ER_TINY_RENDERER,                   # CPU software rasteriser, no GPU
        physicsClientId=client,
    )
    return np.asarray(rgba, dtype=np.uint8).reshape(h, w, 4)[:, :, :3]

Where one recorded frame spends its time

I timed every stage of producing a recorded frame over 1,000 frames, after 50 warm-up frames that were thrown away. I report medians (p50). I repeated the whole run in a second process, and throughput agreed within 1.7%.

Two findings changed decisions:

1. The controller costs more than the physics. Twelve calls to the pure-Python PID controller take 9.5 ms. The 12 environment steps they steer, 48 physics steps inside, take 3.5 ms. Every policy I evaluate later pays this cost, so it is a fixed part of every experiment’s time budget.
2. Defaults matter. The library’s built-in camera renders shadows and a segmentation mask. Neither is useful here, and together they double the render stage, from 12.8 to 26.5 ms per image. That is why the project has its own camera.

The timing loop puts a stopwatch around each stage (simplified; full code):

for i in range(n_frames):
    t_physics, t_control = 0.0, 0.0
    for _ in range(ctrl_per_frame):                  # 12 controller periods per frame
        t0 = time.perf_counter()
        obs, *_ = env.step(action)                   # 4 physics steps each
        t_physics += time.perf_counter() - t0
        t0 = time.perf_counter()
        action[0], *_ = ctrl.computeControlFromState(...)
        t_control += time.perf_counter() - t0

    t0 = time.perf_counter(); rgb = cam.render(pos, quat); t_render = time.perf_counter() - t0
    t0 = time.perf_counter(); png = encode_png(rgb);       t_encode = time.perf_counter() - t0
    t0 = time.perf_counter(); path.write_bytes(png);       t_write  = time.perf_counter() - t0

Can I afford the dataset?

The plan assumed 6 to 15 KB per frame, stored as JPEG. Measured:

For these 128 x 96 frames of flat colour, lossless PNG is the smallest option: 1.4 KB, against 2.7 to 3.9 KB for JPEG. At the measured rate, the planned 200,000 frames would take about 1.6 hours and 0.3 GB, or 2.4 hours if I keep the plan’s 1.5x safety margin. These numbers extrapolate the measured rate; that dataset doesn’t exist yet.

What this part does not show

• The scene is a floor plus two plain shapes. Richer scenes render slower and compress worse.
• The laptop was busy during the runs, with an editor and a browser open. A quiet machine could land on either side of these numbers.
• Everything here is simulation. Nothing has flown a real drone.

Next

A fast simulator is not yet a believable one. Part 2 takes a real drone’s spec sheet and asks what a 27 g simulated quadrotor can honestly borrow from it, then fixes the task and how it is scored, before any model is trained.

Appendix

git clone https://github.com/Ckck12/Drone_VLA_Simulation.git dronevla && cd dronevla
git checkout --detach 98490502e80c0cf50a65765619e6d72f03ee88b0
SHA=$(sed -n 's/.*gym-pybullet-drones\.git@\([0-9a-f]\{40\}\).*/\1/p' env-lock-candidate.txt)
git clone https://github.com/learnsyslab/gym-pybullet-drones.git third_party/gym-pybullet-drones
git -C third_party/gym-pybullet-drones checkout --detach "$SHA"

python3.12 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install "$(grep '^numpy==' env-lock-candidate.txt)"          # before pybullet
pip install --no-build-isolation "$(grep '^pybullet==' env-lock-candidate.txt)"
pip install "$(grep '^torch==' env-lock-candidate.txt)" --index-url https://download.pytorch.org/whl/cpu
grep -v -e '^-e ' -e '^torch==' env-lock-candidate.txt > /tmp/req.txt
pip install -r /tmp/req.txt
pip install --no-deps -e third_party/gym-pybullet-drones
pip check

python -m dronevla.profile_env --frames 1000 --renderer tiny --out reports/env.json
python scripts/figures/blog01_environment.py      # every figure in this post

References: what I took from each

• gym-pybullet-drones provides a quadrotor model and a PID controller on top of PyBullet. → I pinned it by commit and kept its controller, but replaced its camera, whose defaults cost twice the render time.
• PyBullet Quickstart Guide documents getCameraImage and its CPU TinyRenderer. → On a laptop without a GPU, that path takes the GPU out of the question entirely.
• pip: build system interface explains that builds run in an isolated environment by default. → Good for reproducibility, but it hid numpy from pybullet’s compiler, so I turn isolation off for that one package and assert the result.

Summary (STAR)