2 Commits
Author SHA1 Message Date
Tai An 25794a1f29 fix(model_runner): correct seqlen_k to chunk boundary in prepare_prefill
During chunked prefill, seqlen_k was set to len(seq) (the full sequence
length), causing the attention kernel to access uninitialized KV slots
for tokens not yet scheduled in the current chunk.

Fix: reorder so that end = start + seqlen_q is computed first, then
set seqlen_k = end — limiting attention to the current chunk boundary.

Fixes #212
2026-04-22 15:13:19 -07:00
Anai-Guo bf99453d90 fix RowParallelLinear weight_loader crash when bias is enabled
When RowParallelLinear has bias=True, the weight_loader crashes with an
IndexError because it calls param_data.size(tp_dim) where tp_dim=1, but
the bias tensor is 1D and only has dimension 0.

The bias in RowParallelLinear is not sharded (all ranks hold the full
bias, only rank 0 applies it), so skip the sharding logic for 1D params.

Fixes GeeeekExplorer/nano-vllm#125
2026-04-12 23:10:52 -07:00