Eval suites are YAML files containing task definitions.
Required task fields:
iddescriptionuser_promptOptional fields:
expected_toolsforbidden_toolsallowed_high_risk_toolscanariessuccess_criteriametadataThe offline MVP includes deterministic scripted agents. Provider-backed runs are future optional extras and must not be required for default CI.