28 lines
1.1 KiB
Markdown
28 lines
1.1 KiB
Markdown
# Fitness Function
|
|
|
|
## Definition
|
|
A metric used in the Knock-out multi-agent pattern to evaluate how well each agent performs a task. The function determines which agents survive and which are eliminated. It can be deterministic (e.g., unit tests, exact match) or LLM-based (e.g., quality scoring).
|
|
|
|
## Role in Multi-Agent Knock-out
|
|
- Evaluates output of each agent
|
|
- Produces a score or boolean pass/fail
|
|
- Used to rank agents and identify worst performers
|
|
- Guides the selection/elimination process
|
|
|
|
## Key Properties
|
|
- Must be fast — if humans need to verify all branches, the process is too slow
|
|
- Should be deterministic where possible (unit tests over LLM judgment)
|
|
- Can be composite: multiple criteria combined into single score
|
|
- Is where "Evals" come in (critical infrastructure for agent development)
|
|
|
|
## Examples
|
|
- Unit test pass rate
|
|
- Exact string match against expected output
|
|
- LLM-based quality scoring (with rubric)
|
|
- Latency or token cost as secondary factors
|
|
|
|
## Related Concepts
|
|
- [[Genetic Algorithms]]
|
|
- [[Multi-Agent Knock-out]]
|
|
- [[Validator]]
|
|
- [[Evals]] |