vllm.v1.worker.gpu.spec_decode

Modules:

• acceptance_estimator –
• adaptive_verification –

  Adaptive verification for DSpark speculative decoding.

• dflash –
• dflash2 –
• draft_logits –
• dspark –
• eagle –
• extract_hidden_states –
• gemma4 –
• ngram –
• rejection_sampler –
• speculator –
• speculator_config –

  Per-speculator configuration.

• standalone_ar –
• suffix –
• target_dependent_ar –
• utils –
• v1_speculator_configs –

  Speculator configs that only the V1 model runner needs.

• xpress –

Functions:

• init_speculator –

  Build the speculator named by speculative_config.speculator_config.

init_speculator(vllm_config, device, req_states)

Build the speculator named by speculative_config.speculator_config.

The speculator is constructed with (vllm_config, device), plus req_states if its constructor accepts it.

Source code in vllm/v1/worker/gpu/spec_decode/__init__.py

def init_speculator(
    vllm_config: "VllmConfig",
    device: torch.device,
    req_states: "RequestState",
):
    """Build the speculator named by `speculative_config.speculator_config`.

    The speculator is constructed with `(vllm_config, device)`, plus
    `req_states` if its constructor accepts it.
    """
    from vllm.v1.worker.gpu.spec_decode.speculator import BaseSpeculator

    speculative_config = vllm_config.speculative_config
    assert speculative_config is not None
    speculator_config = speculative_config.speculator_config
    qualname = speculator_config.speculator_class
    if qualname is None:
        raise NotImplementedError(f"{speculative_config.method} is not supported yet.")
    speculator_cls = resolve_obj_by_qualname(qualname)
    if not (
        isinstance(speculator_cls, type) and issubclass(speculator_cls, BaseSpeculator)
    ):
        raise TypeError(
            f"speculator_class '{qualname}' of {type(speculator_config).__name__} "
            f"must be a subclass of {BaseSpeculator.__module__}."
            f"{BaseSpeculator.__qualname__}."
        )
    # BaseSpeculator doesn't define the constructor signature.
    factory = cast(Callable[..., "BaseSpeculator"], speculator_cls)
    if "req_states" in inspect.signature(factory).parameters:
        return factory(vllm_config, device, req_states=req_states)
    return factory(vllm_config, device)