The score of a planner is the number of solved tasks
If a suboptimal or invalid plan is returned, all tasks in the domain are counted as unsolved.
If that happens in more than one domain, the entry is disqualified.
Satisficing Track
single CPU core
8Gb memory limit
30min time limit
Multiple plans can be returned, the one with the lowest cost is counted.
The score of a planner on a solved task is the ratio C/C where C is the cost of the cheapest discovered plan and C is the cost of a reference plan. The score on an unsolved task is 0. The score of a planner is the sum of its scores for all tasks.
If an invalid plan is returned, all tasks in the domain are counted as unsolved.
If that happens in more than one domain, the entry is disqualified.
Agile Track
single CPU core
8Gb memory limit
5min time limit
The cost of the discovered plan is ignored, only the CPU time to discover a plan is counted.
The score of a planner on a solved task is 1 if the task was solved within 1 second and 0 if the task was not solved within the resource limits. If the task was solved in T seconds (1 ≤ T ≤ 300) then its score is 1 - log(T)/log(300). The score of a planner is the sum of its scores for all tasks.
If an invalid plan is returned, all tasks in the domain are counted as unsolved.
If that happens in more than one domain, the entry is disqualified.