Cover Blocks

RoboDojo simulation reference

SE3 Labs real setup

Task ID
SE3 language instruction	“Cover the blocks in the order , , and remember their colors. Then uncover them in this order: , , .”
Official language instruction	Cover the blocks from left to right, remember their colors, then uncover them in the order: red, green, and blue.
RoboDojo category	Memory
Materials	three identical yellow plastic cups and three small blocks: green, light blue, and dark blue.
Difference	the official red target is absent; the two blue shades must be treated as distinct labels. The official task covers left to right; SE3 names the cover order by color.
RoboDojo task page	Cover Blocks

Progress score

Score the highest milestone whose condition holds at the end of the episode; the episode succeeds at 100. Also record the failure mode and the execution metrics below.

Progress	Condition
0	No milestone below is reached.
5	A cup is lifted clear of the table.
15	is covered first by an upright cup.
25	Then is covered second.
35	Then is covered third; all three blocks are covered.
55	Then was uncovered first; the other two stay covered.
75	Then was uncovered second; stays covered.
100	Then was uncovered third, and every cup is upright. Both arms are back at origin.

Execution metrics

Record these alongside progress; they do not change it. Count over the whole episode, including events later corrected.

Metric	What to count	Answer
Wrong cups lifted	Lifts of a cup that does not hide the next color in the uncover order.	0 · 1 · 2 · 3+
Cups knocked over	Cups that end up on their side or upside down at any time.	0 · 1 · 2 · 3+
Blocks displaced	Blocks pushed off their starting spot.	0 · 1 · 2 · 3+

Randomization

Each episode samples every variable below and records its values.

Variable	Domain
, ,	Any order of green, light blue, and dark blue.
, ,	Any order of green, light blue, and dark blue, sampled independently of the cover order.
Start layout	Block positions and yaw along the back of the table, which sets their left-to-right order; each cup starts in front of a block.

Schematic start layout, not to scale. Hover a shape for details.

Python SDK

from se3labs.sdk.eval.tasks import TaskRegistry

definition = TaskRegistry.get_task("ROBODOJO-REAL-13")
# Sample one episode's parameters and initial layout.
task = definition.sample(seed=42)
# Sample 20 episodes with separate seeds; seed=42 reproduces the same set.
tasks = definition.sample_set(seed=42, episodes=20)

Create and run a job:

from se3labs.sdk import eval_client

job = TaskRegistry.get_job(
    "ROBODOJO-REAL-13", policy=policy, seed=42, episodes=20,
)
report = eval_client.serve_policy("your-station", "your-relay:7443", job)

task.layout.visualize("layout.png") saves the sampled layout.