Eval tasks¶

The tasks of the eval set (see Measuring how well an assistant writes scenarios): each is a request to write a scenario, scored by what the scenario does. homeostat eval list and homeostat eval show <task> print the same. The acceptance checks are not published here.

Task	Level	Title	Checks
`flow_disturbance`	1	A flow loop with an unmeasured disturbance	5
`level_disturbance`	1	A tank level loop with a varying inflow	7
`setpoint_step`	1	A setpoint step on a temperature loop	6
`bias_hidden_by_loop`	2	A transmitter bias that the loop hides	3
`stiction_visible`	2	A valve that starts to stick	3
`fouling_and_cleaning`	3	Fouling builds up and a cleaning resets it	5
`three_grades`	3	A production plan with three grades	5
`twin_manual`	3	A twin experiment: automatic against manual	6
`product_quality`	4	A mixer feeding a reactor, and the product property	5
`chain_flow_tank_exchanger`	5	A flow through a tank and a heat exchanger	9
`manual_takeover`	5	An operator takes over by hand	8
`sensor_gap`	5	A transmitter that goes missing for a day	8
`startup_from_cold`	5	A start-up from cold	5

Levels: 1 a single loop, 2 a fault and what it does to the data, 3 a plan, a maintenance history or a twin experiment, 4 several units wired together, 5 combinations that no example or guide walks through (wiring different templates into a chain, taking a loop over by hand, an instrument that goes missing, a start from cold).

flow_disturbance¶

Level 1: A flow loop with an unmeasured disturbance.

Simulate 2 hours of a flow control loop FIC-101 holding 50 m3/h. The upstream pressure varies randomly with a standard deviation of 0.3 bar and a correlation time of 10 minutes, and nobody measures it. A base step of 1 second is fine; record every 10 seconds.

level_disturbance¶

Level 1: A tank level loop with a varying inflow.

Simulate 3 hours of a level control loop LIC-101 holding a tank at 60 %. The inflow to the tank varies randomly around 20 m3/h with a standard deviation of 2 m3/h and a correlation time of 15 minutes, and nobody measures it. A base step of 1 second is fine; record every 10 seconds.

setpoint_step¶

Level 1: A setpoint step on a temperature loop.

Simulate 4 hours of a temperature control loop TIC-101 at a setpoint of 80 degC. At 1 hour the setpoint steps to 90 degC. A base step of 1 second is fine; record every 10 seconds.

bias_hidden_by_loop¶

Level 2: A transmitter bias that the loop hides.

Flow control loop FIC-101 holds 50 m3/h for 6 hours; the upstream pressure varies randomly (standard deviation 0.2 bar, correlation time 10 minutes, unmeasured). At 3 hours the flowmeter FT-101 starts reading 3 m3/h too high, abruptly. Show that the controller hides this fault: the reading stays on the setpoint while the true flow falls. A base step of 1 second is fine; record every 10 seconds.

stiction_visible¶

Level 2: A valve that starts to stick.

Flow control loop FIC-101 holds 50 m3/h for 12 hours. The upstream pressure varies randomly (standard deviation 0.3 bar, correlation time 10 minutes, unmeasured). At 6 hours the control valve develops stick-slip friction: a static friction band of 10 % of travel, appearing abruptly. The fault should be detectable on the controller output soon after it starts. A base step of 1 second is fine; record every 10 seconds.

fouling_and_cleaning¶

Level 3: Fouling builds up and a cleaning resets it.

Simulate 40 days of a temperature control loop TIC-101 at 80 degC. Heat-transfer fouling cuts the heating gain by 1 % per day at the setpoint temperature, and the surface is cleaned on day 20. Show the heating valve opening further as fouling builds and dropping back after the cleaning. A base step of 1 minute is fine; record every hour.

three_grades¶

Level 3: A production plan with three grades.

Simulate 20 days of a temperature control loop TIC-101 that runs three grades from a production plan with named grades: grade A at 80 degC (the start), grade B at 88 degC from day 5, and grade C at 75 degC from day 12. Each change ramps over 4 hours. Every grade change should settle within 8 hours. A base step of 1 minute is fine; record every 10 minutes.

twin_manual¶

Level 3: A twin experiment: automatic against manual.

Simulate 24 hours of a temperature control loop TIC-101 at 80 degC with a base step of 10 seconds (record every 10 seconds), as a twin experiment: two lanes with identical noise, lane 0 with the controller in automatic and lane 1 with the controller in manual from the start. The feed temperature is an unmeasured disturbance (mean 25 degC, standard deviation 3 degC, correlation time 2 hours). Show that the loop moves the disturbance from the temperature to the valve.

product_quality¶

Level 4: A mixer feeding a reactor, and the product property.

Simulate 10 days of a plant with a mixer MX-101 feeding a reactor TIC-301 (the library's mixer and reactor templates), with a base step of 2 minutes (record every 10 minutes). Feed the reactor from the mixer: its flow port from the mixer's flow signal and its ratio port from the mixer's out signal. Run grade A (reactor setpoint 80 degC, mixer flow_b 10 m3/h) and, from day 5, grade B (86 degC, flow_b 15 m3/h), ramping over 4 hours; grade changes should settle within 8 hours. Show that the product property TIC-301.q_product rises in grade B and that the laboratory analyzer AT-301 reports it.

chain_flow_tank_exchanger¶

Level 5: A flow through a tank and a heat exchanger.

Simulate 6 hours of a small plant of three units: a flow loop FIC-101 (setpoint 20 m3/h) feeds a tank with the level loop LIC-301 (level 50 %), and the tank's outflow is the process flow through a heat exchanger TIC-201 (outlet temperature 60 degC). At 2 hours the setpoint of FIC-101 rises to 30 m3/h. Show the level loop opening its outlet valve further to hold the level, and the heat exchanger opening its heating valve further to hold its temperature as the flow rises. A base step of 2 seconds is fine; record every 30 seconds.

manual_takeover¶

Level 5: An operator takes over by hand.

Flow control loop FIC-101 holds 50 m3/h for 6 hours; the upstream pressure varies randomly (standard deviation 0.3 bar, correlation time 10 minutes, unmeasured). From 2 hours to 3 hours 30 minutes the operator puts the controller in manual, leaving the valve where it was, and then puts it back in automatic. Show that while the controller is in manual the valve stays still and the flow wanders with the pressure. A base step of 1 second is fine; record every 10 seconds.

sensor_gap¶

Level 5: A transmitter that goes missing for a day.

Simulate 3 days of a temperature control loop TIC-101 at 80 degC. The feed temperature varies randomly (mean 25 degC, standard deviation 3 degC, correlation time 2 hours) and is not measured. After 1 day and 12 hours the temperature transmitter TT-101 is removed, so the controller has no reading, and a new one is installed after 2 days and 12 hours. Show that while the transmitter is gone the valve stays where it was and the true temperature drifts with the feed, and that the loop recovers once the transmitter is back. A base step of 10 seconds is fine; record every 5 minutes.

startup_from_cold¶

Level 5: A start-up from cold.

Simulate the start-up of a temperature control loop TIC-101 with a setpoint of 80 degC: the plant is not settled at the start but begins cold, and the controller brings it up to the setpoint. Run 12 hours. A base step of 10 seconds is fine; record every minute.