Rain-Bus
ffd2defdfc
add Chinese annotations to all source files for learning purposes
...
Annotated 16 source files covering the full architecture:
engine (scheduler, block manager, model runner), layers (attention,
linear, sampler, etc.), model (qwen3), and utils.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-25 21:33:15 +08:00
f64d821c20
fix chunked prefill bugs and refactor
2026-04-26 02:53:06 +08:00
25794a1f29
fix(model_runner): correct seqlen_k to chunk boundary in prepare_prefill
...
During chunked prefill, seqlen_k was set to len(seq) (the full sequence
length), causing the attention kernel to access uninitialized KV slots
for tokens not yet scheduled in the current chunk.
Fix: reorder so that end = start + seqlen_q is computed first, then
set seqlen_k = end — limiting attention to the current chunk boundary.
Fixes #212
2026-04-22 15:13:19 -07:00
8d63a98c03
support chunked prefill and fix minor bug
2026-04-14 03:05:35 +08:00
9e8507ef41
minor simplify
2026-04-13 22:09:46 +08:00
2f21442653
support qwen2
2025-11-04 01:44:42 +08:00
df99418f7d
simplify
2025-08-31 20:02:51 +08:00
f5b4840276
fix(model_runner): correct position indexing to be 0-based
...
- Change position calculation from len(seq) to len(seq) - 1
2025-07-04 14:29:12 +08:00
cb0b3dec3f
remove rng state
2025-06-27 22:50:33 +08:00
1caeec8dfa
same as vllm
2025-06-27 18:50:56 +08:00
658520b788
warmup and allocate
2025-06-27 01:51:57 +08:00
03cfc13bb3
faster pickle
2025-06-23 00:51:52 +08:00
cde3fc22c2
simplify
2025-06-21 17:19:15 +08:00
ffafaeb133
Release CUDA Graphs resource before exit.
2025-06-18 16:17:31 +08:00
bc0ad5a116
better
2025-06-17 23:33:38 +08:00
7e42fa6f63
fix
2025-06-15 13:28:29 +08:00
fc778a4da9
better
2025-06-15 10:36:45 +08:00
53b3ef2e32
support tensor parallel
2025-06-15 01:31:24 +08:00
b6136383c9
support fast pickle
2025-06-14 13:36:57 +08:00
4a8aa090a7
fix
2025-06-14 00:56:07 +08:00
98a1551a7d
support CUDA_VISIBLE_DEVICES
2025-06-12 23:14:01 +08:00
fee58d44e4
fix
2025-06-12 01:00:31 +08:00
08c84ec08d
multi file loader
2025-06-12 01:00:09 +08:00
386290d69e
refactor
2025-06-11 21:12:57 +08:00
b98e1ca305
fix
2025-06-10 21:25:54 +08:00
a5a4909e6a
init commit
2025-06-10 00:27:01 +08:00
Rain-Bus
GeekExplorer