Hardware Experiment Protocol and Receipt

A training run, a conversion loss report or a target calibration is not evidence of what a device does. That evidence is a run on the device, declared before it happens and sealed afterwards. sc_neurocore.hardware.experiment defines both halves. It never executes anything: execution is the operator's act on the operator's device.

Declare the protocol first

resolve_protocol(document) admits a protocol only when it could serve as evidence, and refuses it with the offending field otherwise.

Field	What it fixes
opt_in	Must be true: the operator explicitly authorises execution on the device
declared_at	ISO 8601 timestamp with a zone offset (for example +00:00); no run may start earlier
operator	name (required) and contact of whoever authorises and runs it
device	vendor, model, serial (required) and firmware
image	kind (bitstream, firmware, container, binary) and the sha256 of what executes
network_sha256, data_sha256	The converted network (converted_network.json, conversion or target report) and the evaluation data
samples	Evaluation samples per measured run
latency	start_event, end_event, clock, warmup_runs, measured_runs and includes_transport: true
power	Optional: instrument (vendor, model, serial), calibration (certificate, calibrated_on, valid_until), measurement_point, sample_rate_hz
criteria	Preregistered metric/threshold pairs: accuracy (at least, in [0, 1]), latency_p50_ms, latency_p95_ms, energy_per_inference_j (at most)

Latency is always measured from the host's request to the host receiving the result, so transport is included; a protocol that does not say so is refused. Warmup runs are declared, recorded and excluded from the percentiles. An energy criterion requires a declared power instrument. to_public_dict() adds the schema version and a sha256 over every field; a stored protocol resubmitted with a digest that no longer matches is refused.

Seal what the run observed

seal_receipt(protocol, observations) takes raw observations: started_at, finished_at, the device_serial and image_sha256 seen at run time, one latency per warmup and per measured run, the number of correct predictions, optional notes and, when power was declared, energy with source: "instrument", the instrument's serial and joules per measured run.

It refuses a different device or image, a run that started before the protocol was declared, run counts other than those declared, a missing energy measurement when power was declared, energy when none was declared, energy from another instrument or on a day outside its calibration's validity, and any energy whose source is not the instrument. Energy is never inferred from operation counts or estimates.

Accuracy, the nearest-rank p50 and p95 latencies, mean and maximum energy per inference and the verdict of every preregistered criterion are computed from the observations, never taken from the caller. The receipt embeds the full protocol, its digest and a sha256 over everything.

Verify a receipt

verify_receipt(document) re-resolves the embedded protocol, checks its digest, recomputes every derived figure and verdict from the raw observations and compares the whole document, digest included. A receipt whose figures do not follow from its observations is refused, so a stored receipt can be checked by anyone holding it.

sc_neurocore.hardware.experiment

Declare a hardware experiment before it runs, then seal what it measured.

A protocol is written before any execution. It records the operator's explicit opt-in, who runs it, on which device, with which image, which converted network and data, how latency is timed (transport included, warmup runs declared and excluded), how power is measured (instrument and a valid calibration certificate) and the preregistered acceptance criteria. Its digest binds all of that.

A receipt binds observations to that digest. It refuses observations from another device or image, run counts that differ from the protocol, a run that started before the protocol was declared, energy from an undeclared or uncalibrated instrument, and any energy figure not measured by an instrument: energy is never inferred from operation counts or estimates. Accuracy, latency percentiles, energy statistics and every verdict are computed here from the raw observations, never taken from the caller, so :func:verify_receipt can recompute a sealed receipt end to end.

Nothing here executes on hardware. Execution is the operator's act on the operator's device; this module states what counts as evidence of it.

Protocol dataclass

A hardware experiment as declared before it runs.

Attributes

declared_at : str ISO 8601 timestamp with zone at which the protocol was fixed. operator : dict name and optional contact of whoever authorised and runs it. device : dict vendor, model, serial and optional firmware. image : dict kind (one of :data:IMAGE_KINDS) and sha256 of what executes. network_sha256, data_sha256 : str Digests of the converted network and of the evaluation data. samples : int Evaluation samples per measured run. latency : dict start_event, end_event, clock, warmup_runs and measured_runs; includes_transport is always true. power : dict or None instrument (vendor, model, serial), calibration (certificate, calibrated_on, valid_until), measurement_point and sample_rate_hz; None when energy is not measured. criteria : list of dict Preregistered metric and threshold pairs.

Source code in src/sc_neurocore/hardware/experiment.py

@dataclass(frozen=True)
class Protocol:
    """A hardware experiment as declared before it runs.

    Attributes
    ----------
    declared_at : str
        ISO 8601 timestamp with zone at which the protocol was fixed.
    operator : dict
        ``name`` and optional ``contact`` of whoever authorised and runs it.
    device : dict
        ``vendor``, ``model``, ``serial`` and optional ``firmware``.
    image : dict
        ``kind`` (one of :data:`IMAGE_KINDS`) and ``sha256`` of what executes.
    network_sha256, data_sha256 : str
        Digests of the converted network and of the evaluation data.
    samples : int
        Evaluation samples per measured run.
    latency : dict
        ``start_event``, ``end_event``, ``clock``, ``warmup_runs`` and
        ``measured_runs``; ``includes_transport`` is always true.
    power : dict or None
        ``instrument`` (vendor, model, serial), ``calibration`` (certificate,
        calibrated_on, valid_until), ``measurement_point`` and
        ``sample_rate_hz``; ``None`` when energy is not measured.
    criteria : list of dict
        Preregistered ``metric`` and ``threshold`` pairs.
    """

    declared_at: str
    operator: dict[str, str]
    device: dict[str, str]
    image: dict[str, str]
    network_sha256: str
    data_sha256: str
    samples: int
    latency: dict[str, Any]
    power: dict[str, Any] | None
    criteria: list[dict[str, Any]]

    def to_public_dict(self) -> dict[str, Any]:
        """Return the protocol as stored, including its opt-in and digest.

        Returns
        -------
        dict
            Every field, ``opt_in: true``, the schema version and ``sha256``.
        """
        body = {"schema_version": PROTOCOL_SCHEMA_VERSION, "opt_in": True, **asdict(self)}
        return {**body, "sha256": _canonical_sha256(body)}

    @property
    def sha256(self) -> str:
        """Return the digest that a receipt binds to."""
        return str(self.to_public_dict()["sha256"])

sha256 property

Return the digest that a receipt binds to.

to_public_dict()

Return the protocol as stored, including its opt-in and digest.

Returns

dict Every field, opt_in: true, the schema version and sha256.

Source code in src/sc_neurocore/hardware/experiment.py

def to_public_dict(self) -> dict[str, Any]:
    """Return the protocol as stored, including its opt-in and digest.

    Returns
    -------
    dict
        Every field, ``opt_in: true``, the schema version and ``sha256``.
    """
    body = {"schema_version": PROTOCOL_SCHEMA_VERSION, "opt_in": True, **asdict(self)}
    return {**body, "sha256": _canonical_sha256(body)}

HardwareExperimentError

Bases: ValueError

Raised when a protocol or its observations cannot be admitted as evidence.

Attributes

field : str The protocol or observation field that was refused. reason : str What is wrong, in words an operator can act on.

Source code in src/sc_neurocore/hardware/experiment.py

class HardwareExperimentError(ValueError):
    """Raised when a protocol or its observations cannot be admitted as evidence.

    Attributes
    ----------
    field : str
        The protocol or observation field that was refused.
    reason : str
        What is wrong, in words an operator can act on.
    """

    def __init__(self, field: str, reason: str) -> None:
        super().__init__(f"{field}: {reason}")
        self.field = field
        self.reason = reason

resolve_protocol(value)

Resolve a declared protocol, refusing anything that could not be evidence.

Parameters

value : mapping The protocol document. opt_in must be true; a stored protocol resubmitted with its sha256 must match it.

Returns

Protocol The protocol exactly as a receipt will bind it.

Raises

HardwareExperimentError A missing opt-in or identity, an unknown field, a latency definition without transport, a power declaration without a valid calibration, or a criterion that cannot be judged.

Source code in src/sc_neurocore/hardware/experiment.py

def resolve_protocol(value: object) -> Protocol:
    """Resolve a declared protocol, refusing anything that could not be evidence.

    Parameters
    ----------
    value : mapping
        The protocol document. ``opt_in`` must be ``true``; a stored protocol
        resubmitted with its ``sha256`` must match it.

    Returns
    -------
    Protocol
        The protocol exactly as a receipt will bind it.

    Raises
    ------
    HardwareExperimentError
        A missing opt-in or identity, an unknown field, a latency definition
        without transport, a power declaration without a valid calibration, or
        a criterion that cannot be judged.
    """
    data = _mapping(value, "protocol")
    _unknown(
        data,
        {
            "schema_version",
            "opt_in",
            "declared_at",
            "operator",
            "device",
            "image",
            "network_sha256",
            "data_sha256",
            "samples",
            "latency",
            "power",
            "criteria",
            "sha256",
        },
        "protocol",
    )
    if data.get("schema_version", PROTOCOL_SCHEMA_VERSION) != PROTOCOL_SCHEMA_VERSION:
        raise HardwareExperimentError("schema_version", "is not the protocol this build reads.")
    if data.get("opt_in") is not True:
        raise HardwareExperimentError(
            "opt_in", "hardware execution runs only on the operator's explicit opt_in: true."
        )
    operator = _mapping(data.get("operator"), "operator")
    _unknown(operator, {"name", "contact"}, "operator")
    device = _mapping(data.get("device"), "device")
    _unknown(device, {"vendor", "model", "serial", "firmware"}, "device")
    image = _mapping(data.get("image"), "image")
    _unknown(image, {"kind", "sha256"}, "image")
    if image.get("kind") not in IMAGE_KINDS:
        raise HardwareExperimentError("image.kind", f"must be one of {', '.join(IMAGE_KINDS)}.")
    power = _power(data.get("power"))
    protocol = Protocol(
        declared_at=_instant(data.get("declared_at"), "declared_at").isoformat(),
        operator={
            "name": _text(operator.get("name"), "operator.name"),
            "contact": _text(operator.get("contact"), "operator.contact", required=False),
        },
        device={
            "vendor": _text(device.get("vendor"), "device.vendor"),
            "model": _text(device.get("model"), "device.model"),
            "serial": _text(device.get("serial"), "device.serial"),
            "firmware": _text(device.get("firmware"), "device.firmware", required=False),
        },
        image={
            "kind": str(image["kind"]),
            "sha256": _sha256_hex(image.get("sha256"), "image.sha256"),
        },
        network_sha256=_sha256_hex(data.get("network_sha256"), "network_sha256"),
        data_sha256=_sha256_hex(data.get("data_sha256"), "data_sha256"),
        samples=_count(data.get("samples"), "samples", minimum=1),
        latency=_latency(data.get("latency")),
        power=power,
        criteria=_criteria(data.get("criteria", []), power),
    )
    declared = data.get("sha256")
    if declared is not None and declared != protocol.sha256:
        raise HardwareExperimentError("sha256", "the stored protocol digest does not match it.")
    return protocol

seal_receipt(protocol, observations)

Seal a hardware run's raw observations against its declared protocol.

Parameters

protocol : Protocol The protocol declared before the run. observations : mapping started_at, finished_at, device_serial, image_sha256, warmup_latency_ms and latency_ms lists, correct and, when power was declared, energy (source: instrument, instrument_serial, joules_per_inference per measured run); optional notes.

Returns

dict The receipt: the full protocol, the admitted observations, derived accuracy, nearest-rank latency percentiles and energy, the verdict of every preregistered criterion, and sha256 over all of it.

Raises

HardwareExperimentError An observation that does not belong to this protocol, or energy that was not measured by its declared, calibrated instrument.

Source code in src/sc_neurocore/hardware/experiment.py

def seal_receipt(protocol: Protocol, observations: Mapping[str, Any]) -> dict[str, Any]:
    """Seal a hardware run's raw observations against its declared protocol.

    Parameters
    ----------
    protocol : Protocol
        The protocol declared before the run.
    observations : mapping
        ``started_at``, ``finished_at``, ``device_serial``, ``image_sha256``,
        ``warmup_latency_ms`` and ``latency_ms`` lists, ``correct`` and, when
        power was declared, ``energy`` (``source: instrument``,
        ``instrument_serial``, ``joules_per_inference`` per measured run);
        optional ``notes``.

    Returns
    -------
    dict
        The receipt: the full protocol, the admitted observations, derived
        accuracy, nearest-rank latency percentiles and energy, the verdict of
        every preregistered criterion, and ``sha256`` over all of it.

    Raises
    ------
    HardwareExperimentError
        An observation that does not belong to this protocol, or energy that
        was not measured by its declared, calibrated instrument.
    """
    body = {
        "schema_version": RECEIPT_SCHEMA_VERSION,
        "protocol": protocol.to_public_dict(),
        "protocol_sha256": protocol.sha256,
        **_derive(protocol, _mapping(observations, "observations")),
    }
    return {**body, "sha256": _canonical_sha256(body)}

verify_receipt(receipt)

Recompute a sealed receipt from its protocol and raw observations.

Parameters

receipt : mapping A document :func:seal_receipt produced.

Returns

dict The receipt, when every digest, derived figure and verdict recomputes to exactly what it states.

Raises

HardwareExperimentError Another schema, a changed protocol or observation, or a derived figure or verdict that does not follow from the observations.

Source code in src/sc_neurocore/hardware/experiment.py

def verify_receipt(receipt: object) -> dict[str, Any]:
    """Recompute a sealed receipt from its protocol and raw observations.

    Parameters
    ----------
    receipt : mapping
        A document :func:`seal_receipt` produced.

    Returns
    -------
    dict
        The receipt, when every digest, derived figure and verdict recomputes
        to exactly what it states.

    Raises
    ------
    HardwareExperimentError
        Another schema, a changed protocol or observation, or a derived figure
        or verdict that does not follow from the observations.
    """
    data = _mapping(receipt, "receipt")
    if data.get("schema_version") != RECEIPT_SCHEMA_VERSION:
        raise HardwareExperimentError("schema_version", "is not the receipt this build reads.")
    protocol = resolve_protocol(data.get("protocol"))
    if data.get("protocol_sha256") != protocol.sha256:
        raise HardwareExperimentError("protocol_sha256", "does not match the embedded protocol.")
    observations = dict(_mapping(data.get("observations"), "observations"))
    energy = observations.get("energy")
    if isinstance(energy, Mapping):
        observations["energy"] = {
            key: energy.get(key) for key in ("source", "instrument_serial", "joules_per_inference")
        }
    expected = seal_receipt(protocol, observations)
    if dict(data) != expected:
        raise HardwareExperimentError(
            "receipt", "its figures, verdicts or digest do not follow from its observations."
        )
    return expected