Skip to content

vllm.models.kimi_k3.amd.specialized_decode.mono.common.debug

A timeline build's per-CTA phase stamps (debug only; ATOM_MONO_TIMELINE, saved by atom.mono.runtime.timeline): the s_memrealtime (100 MHz) at which the CTA passes point k, kept in LDS until the kernel ends -- a global store per stamp would sit on every later s_waitcnt vmcnt(0).

on is the build's compile-time switch: off, every call traces to nothing.

Functions:

  • given_up_waits –

    Each CTA's first wait that gave up, the earliest first, from a rank's

  • region_id –

    A mailbox region's id in a wait record: its name's CRC-32 (31 bits), the

  • region_ids –

    Name -> region_id: a Mailbox's table.

  • stamp_begin –

    Zero the LDS record, then point 0.

  • stamp_flush –

    The CTA's record -> tl [CTA][points] int64.

given_up_waits(health, names)

Each CTA's first wait that gave up, the earliest first, from a rank's health words; names the regions a record may name.

Source code in vllm/models/kimi_k3/amd/specialized_decode/mono/common/debug.py 
71
72
73
74
75
76
77
78
79
80
81
82
def given_up_waits(health: torch.Tensor, names) -> list[str]:
    """Each CTA's first wait that gave up, the earliest first, from a rank's
    ``health`` words; ``names`` the regions a record may name."""
    by_id = {region_id(n): n for n in names}
    records = health.view(BLOCKS + 1, RECORD_WORDS)[1:]
    records = records[records[:, 0] != 0].tolist()
    t0 = min((r[6] for r in records), default=0)
    return [
        f"+{(at - t0) / 100:.0f} us CTA {cta}: {by_id.get(rid, rid)} pair {idx}"
        f" waited for tag {tag}, saw {seen}"
        for _, rid, idx, tag, seen, cta, at, _ in sorted(records, key=lambda r: r[6])
    ]

region_id(name)

A mailbox region's id in a wait record: its name's CRC-32 (31 bits), the same in every kernel and table that names it.

Source code in vllm/models/kimi_k3/amd/specialized_decode/mono/common/debug.py 
60
61
62
63
def region_id(name: str) -> int:
    """A mailbox region's id in a wait record: its name's CRC-32 (31 bits), the
    same in every kernel and table that names it."""
    return zlib.crc32(name.encode()) & 0x7FFFFFFF

region_ids(names)

Name -> region_id: a Mailbox's table.

Source code in vllm/models/kimi_k3/amd/specialized_decode/mono/common/debug.py 
66
67
68
def region_ids(names) -> dict[str, int]:
    """Name -> ``region_id``: a ``Mailbox``'s table."""
    return {n: region_id(n) for n in names}

stamp_begin(on, tls, tid, points)

Zero the LDS record, then point 0.

Source code in vllm/models/kimi_k3/amd/specialized_decode/mono/common/debug.py 
29
30
31
32
33
34
35
@traced
def stamp_begin(on, tls, tid, points):
    """Zero the LDS record, then point 0."""
    if const_expr(on):  # noqa: SIM102
        if tid < points:  # the stamping thread's wave: ordered
            fx.ptr_store(fx.Int64(0), tls + tid)
    stamp(on, tls, tid, 0)

stamp_flush(on, tls, tl, tid, bid, points)

The CTA's record -> tl [CTA][points] int64.

Source code in vllm/models/kimi_k3/amd/specialized_decode/mono/common/debug.py 
46
47
48
49
50
51
52
53
54
55
56
57
@traced
def stamp_flush(on, tls, tl, tid, bid, points):
    """The CTA's record -> ``tl`` ``[CTA][points]`` int64."""
    if const_expr(on):  # noqa: SIM102
        if tid < points:
            fx.generic_store(
                fx.inttoptr(
                    fx.PointerType.get(fx.Int64.ir_type, fx.AddressSpace.Global, 8),
                    tl + fx.Int64((bid * points + tid) * 8),
                ),
                fx.ptr_load(tls + tid),
            )