Install
Please confirm you are human
This browser or connection looks automated. Press and continuously hold the control for 3 seconds to enable Google-hosted web results and, when separately allowed, AI-assisted answers.
A successful check enables 100 search requests. Interactive access does not authorize scraping, systematic collection, or reuse of search output.
News
dspark
2+ hour, 40+ min ago (47+ words) vLLM CPU DeepSeek-V4 DSpark (speculative decoding) — deferred, not implemented. DSparkDeepseekV4ForCausalLM is only instantiated when the speculative- decoding registry resolves it by name for a DSpark speculative config; a plain (non-spec) DeepSeek-V4 CPU model never constructs this class. This stub exists solely…...
kv_transfer
9+ mon, 3+ week ago (280+ words) Configuration for distributed KV cache transfer. WARNING: Whenever a new field is added to this config, Policy for handling KV cache load failures. The rank of this vLLM instance in the KV cache transfer. Typical value: Experiment feature flag to…...
scheduler
10+ mon, 3+ day ago (822+ words) WARNING: Whenever a new field is added to this config, If set to False, disable async scheduling. Async scheduling helps to If set to true and chunked prefill is enabled, we do not want to If set to True, KV…...
On-Policy Distillation??? Vime
2+ day, 11+ hour ago (370+ words) Enable on-policy distillation. Required flag to use OPD. Type of OPD: vllm or megatron. Required when --use-opd is set. OPD KL penalty coefficient (default: 1.0). Controls the weight of the distillation signal relative to the RL advantage. Path to teacher Megatron…...
DeepSeek R1 with 128xH100??? Vime
2+ day, 12+ hour ago (580+ words) This is an example of doing DeepSeek R1 RL training using 128xH100 GPUs. We will use bf16 for training, and an fp8 format with 128x128 blockwise quantization for inference. The maximum response length is 32k, and dynamic sampling will be used to filter data during training....
External Rollout Engines Roadmap??? Vime
2+ day, 11+ hour ago (685+ words) An external rollout engine is a vLLM engine that is not launched by the vime training job. Another system deploys and owns the engine lifecycle; vime connects to those engines during training, registers a router, and syncs updated actor weights…...
Trace Viewer??? Vime
2+ day, 11+ hour ago (309+ words) vime can attach lightweight execution traces to each rollout sample. These traces capture span-style events such as generation and reward-model calls, and they can be inspected later from a saved rollout debug dump. To inspect traces later, save rollout debug…...
PD Disaggregation??? Vime
2+ day, 11+ hour ago (288+ words) PD Disaggregation separates Prefill and Decode workers in vLLM rollout. This is especially useful for multi-turn, long-context, and agentic RL workloads where prompt processing and token generation have very different compute and memory profiles. Use PD Disaggregation when: rollout contexts…...
GLM-4.7-Flash with 8??H100??? Vime
2+ day, 13+ hour ago (364+ words) The environment setup, data, and checkpoint conversion are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with GLM-4.7-Flash. To convert the Hugging Face checkpoint to torch_dist format: Execute the training script: Here, we…...
DeepSeek-V4-Flash-Vision-Exp (Experimental)
3+ day, 2+ hour ago (920+ words) DeepSeek-V4-Flash-Vision-Exp is a multimodal mixture-of-experts model in the DeepSeek-V4 family. It combines the DeepSeek-V4 language model with a vision encoder and aligner, and accepts text, single-image, and multi-image requests through the OpenAI-compatible chat API. Support on vLLM Ascend is…...