Benchmarks

Everything tide ships, in one table. Each benchmark is a directory of stock Harbor tasks, so every task runs two ways:

tide run cl-bench/bsm-s01 --agent oracle                        # through tide
harbor trial start -p tasks/continual-learning/cl-bench/bsm-s01 # stock Harbor

tide list shows what is runnable where you are.

Benchmark	Regime	Tasks	Run
first-party	autoresearch	6	tide run autoresearch/first-party --agent oracle
EdgeBench	autoresearch	51	tide run edgebench/<task> --budget <h>
FrontierCS	autoresearch	208	tide run frontier-cs/<task> --agent <a>
terminal-bench	stream	89	tide stream terminal-bench --agent <a>
CL-Bench	stream	301	tide stream cl-bench --agent <a>
SWE-bench Verified	stream	500	tide fetch swebench-verified --limit 50 first

The first-party tasks each teach one hard part of autoresearch, and each benchmark states its upstream, license, and oracle scores in the catalog, next to the tasks themselves.

To write your own, start from tasks/_template and follow Authoring tasks.