International Planning Competition 2023 - Numeric Tracks

IPC 2023 - Numeric Tracks

International Planning Competition 2023 - Numeric Tracks

This is the website for the numeric tracks of the IPC 2023.

Calls

Please forward the following calls to all interested parties.

Preliminary Schedule

Event Date
Domain submission deadline 9th December 2022
Demo problems provided December 2022
Planner submission opens February 28 2023
Final Planner submission April 19 2023
Planner Abstract submission deadline May 24 2023
Contest run May-June 2023
Results announced July 2023
Result analysis deadline August 2023

Tracks

Optimal Track

Satisficing Track

Agile Track

PDDL Fragment

We will use a subset of PDDL 2.1, considering two fragments:

Preconditions and goals are arbitrary propositional formulas. That is, they include quantifiers, negative preconditions, disjunctive preconditions and any number of numeric predicates. Conditional effects are not considered. Terms can either be literals as in classical planning, or numeric terms of the form f(X){ >=,>, =} 0 where f(X) is a linear expressions. Each domain of the competition will be either a SNP or a LNP (LNP is more expressive than SNP). Planners supporting only SNP will be accepted and their performance will be reported when compared with respect to only SNP problems. A planner can also only support SNP. We are considering to have separate rankings.

In terms of the optimisation track, the optimisation functions will always be a minimisation. Problems will either be a minimisation of action costs, or a metric function that is a weighted sum with positive coefficients where each involved variable can only be increased by the actions.

Sample problems are here: Domain1, Problem1, Domain2, Problem2, Domain3, Problem3

Domains

The competition tasks can now be found in the following Github Repository.

Registration

We require that a planner is registered through a team. For this you need to send an e-mail with a subject containing “[Registration for Numeric Tracks]” to jea20@st-andrews.ac.uk. The email must contain:

names of participants, email contacts, track (agile/satisficing/optimal), at least one github account

Based on that, we will create a private repository under the ipc2023-numeric organization and add all participants as users with with write access and participants can both upload their planner there, and then commit to the repository as they wish until the “final planner submission” deadline.

All participants must subscribe to the Google Group. We will announce further details on the submission process there in due time.

To propose a domain for the competition, please contact the organizers (see below).

Planner submission

The competitors must submit the source code of their planners that will be run by the organizers on the actual competition domains/problems, unknown to the competitors until this time. This way no fine-tuning of the planners will be possible. We will host repositories of planners ourselves. The repositories will be hosted on Github under the ipc2023-numeric organisation, and they will be kept private until the end of the competition when we make them public, i.e., after the competition is concluded, we plan to make all planners, domains, and all related data accessible from one place. We require participants to provide self-contained code, with scripts for compilations and running, “compile” and “run”, respectively. The code should be compilable on a fresh installation of Ubuntu 20.04 LTS, and running has to require only three parameters as an input in the following order: the domain file, the problem file, and the name of the file where the last solution found will be saved. The solution format is the usual IPC solution format, which consists of a list of actions separated by a new line.

After Competition!

Participant Teams

Results

The results were presented at the 33rd International Conference on Automated Planning and Scheduling on July 11 in Prague. IPC attendants

The presentation slides of this talk contain additional details. This is a summary:

Winning team

Per-domain tables of solved instances are linked here for the Satisficing Track and for the Optimal Track. Note that when evaluating the tables, planners that were not able to process LNP problems have zeroes in the respective cells. For comparisons reasons, we used a well known greedy best first search algorithm with state of the arte heuristics from the literature (Scala et al. 2020) and $A^*$ with the numeric extension of the $h_{max}$ heuristic (Scala et al. 2017) for the satisficing and the optimal track respectively. These are implemented in the ENHSP planning system.

Organizers

Contact us: ipc2023-numeric@googlegroups.com