Evaluation Client

Run a policy evaluation and query station status.

from se3labs.sdk import eval_client
from se3labs.sdk.examples.hold_still import HOLD_STILL_REQUEST

report = eval_client.serve_policy(
    "example-station-id", "example-station-addr", HOLD_STILL_REQUEST,
)
print(report.state, report.results)

Replace example-station-id and example-station-addr with the station ID and relay host:port supplied by your station operator. The saved login from se3labs auth login supplies authentication. serve_policy() runs the station-driven policy lifecycle until the session ends, then returns a SessionReport with its outcome and episode scores. get_status() queries the latest status reported by the station.

serve_policy

serve_policy(
    station_id: str,
    address: str,
    request: JobRequest,
    *,
    connect_timeout_s: float = 30.0,
) -> SessionReport

Connect to address (the relay, or a station directly), run the whole session on station_id, and return the report.

address is host:port and uses a plaintext gRPC connection. The saved SE3 login supplies authorization metadata. connect_timeout_s limits channel setup, not the duration of the evaluation.

Raises SessionRejected if the station refuses the session, grpc.FutureTimeoutError if channel setup times out, grpc.RpcError on RPC failure, or auth.AuthError if the saved login cannot be used.

get_status

get_status(
    station_id: str,
    address: str,
    *,
    timeout_s: float = 10.0,
) -> status_pb.Status

Fetch the latest job status that station_id reported to the relay.

address is the relay's host:port. The saved login supplies authentication; timeout_s limits this RPC, not the evaluation. Each call returns a snapshot. Poll again for updates; a cached status does not establish that the station is still connected.

Returns: status_pb.Status, with the job state, requested episode count, completed pb.EpisodeResult objects, and the update timestamp. Read fields as attributes, such as status.state. The linked message reference describes each field and its possible values.

from se3labs.sdk import eval_client

status = eval_client.get_status("example-station-id", "example-station-addr")
print(status.state, len(status.episodes), status.episodes_total)
for episode in status.episodes:
    if episode.scored:
        print(episode.episode_id, episode.success, episode.score)

Raises: auth.AuthError if the saved login cannot be used, or grpc.RpcError on a failed request. Inspect exc.code() for grpc.StatusCode.NOT_FOUND (no cached status), DEADLINE_EXCEEDED (the RPC exceeded timeout_s), UNAUTHENTICATED (login refused), or UNAVAILABLE (the relay or its authentication service is unreachable).

SessionReport

SessionReport(
    station: Station_Spec | None = None,
    episodes: int = 0,
    steps: int = 0,
    close_reason: str = '',
    state: str = '',
    results: list[pb.EpisodeResult] = <factory>,
    errors: list[str] = <factory>,
    refusals: list[str] = <factory>,
)

Outcome and counters collected during one policy evaluation session.

state describes whether the evaluation job completed; it does not say whether the robot passed the task. Each item in results is a pb.EpisodeResult. Check its scored flag before reading success or score. Nested questions and answers are documented with that type.

Attributes

station: Station_Spec | None = None

Hardware specification received when the station accepts the session; None if no welcome arrived.

episodes: int = 0

Number of episodes initialized, including any that did not finish or get scored.

steps: int = 0

Number of observation messages received.

close_reason: str = ''

Station's terminal reason and optional detail, such as COMPLETED or WALK_FAILED ...; empty without a closing message.

state: str = ''

SUCCEEDED, FAILED, or CANCELLED; see status_pb.Status for job states. Empty if the stream ended without a closing message.

results: list[pb.EpisodeResult] = field(default_factory=list)

pb.EpisodeResult objects supplied when the session closes, in execution order. May include unscored episodes; empty without a closing message.

errors: list[str] = field(default_factory=list)

Exceptions raised by resume(), refused() or infer(), one line per failed step. That step answers "keep executing". Exceptions from initialize(), ended() or reset() propagate to the caller.

refusals: list[str] = field(default_factory=list)

Every chunk the station refused, one line each, such as "ep_0000 step 41: unsafe: left: the grasp site would go 1.2 cm outside its box". A refused chunk did not run; the policy heard why through refused().

SessionRejected

SessionRejected(reject: pb.Reject)

The relay or station refused the evaluation session.

str(exc) includes the rejection name and message. For programmatic handling, inspect exc.reject.code and exc.reject.message; their fields and possible codes are documented under pb.Reject.

Import constants with from se3labs.interface.wire import pb, then compare the code with a value such as pb.REJECT_CODE_STATION_BUSY.

Attributes

reject: pb.Reject

The pb.Reject details, with a numeric code and a human-readable message.