Rain-Bus
ffd2defdfc
Annotated 16 source files covering the full architecture: engine (scheduler, block manager, model runner), layers (attention, linear, sampler, etc.), model (qwen3), and utils. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
13 lines
537 B
Python
13 lines
537 B
Python
# nano-vllm: 一个轻量级的 vLLM 实现
|
|
#
|
|
# 核心架构:
|
|
# LLM (入口) → LLMEngine (引擎) → Scheduler (调度器) + ModelRunner (模型运行器)
|
|
# ↓ ↓
|
|
# BlockManager (KV缓存管理) Qwen3ForCausalLM (模型)
|
|
#
|
|
# 数据流:
|
|
# 用户请求 → Sequence → Scheduler 调度 → ModelRunner 准备输入 → 模型前向 → 采样 → 返回结果
|
|
|
|
from nanovllm.llm import LLM
|
|
from nanovllm.sampling_params import SamplingParams
|