This page informs about the Process Model Matching track, which was a new OAEI track in 2016 and is now offered a second time in 2017. We provide a short description and link to the relevant resources.
In 2013 and again in 2015 the community interested in process modeling conducted an evaluation campaign similar to the OAEI. Instead of matching ontologies, the task was to match process models described in different formalisms like BPMN and Petri Nets. More information can be found at the webpage of the 2015 contest.
At the OAEI 2016 we offered a subset of the tasks from the Process Model Matching Contest as OAEI track, by converting the process models to an ontological representation. By offering this track, we hoped to gain insights in how far ontology matching systems are capable of solving the more specific problem of matching process models. This was track was also motivated by the discussions at the end of the 2015 Ontology Matching workshop, where many participants showed their interest in such a track. Unfortunately, in 2016 only around 3 matching systems participated in this track. Nevertheless, the results were surprisingly good compared to the state of the art process model matching systems (see here). In 2017 we are offering the track again in the hope of attracting a higher number of participants. Moreover, we use an additional dataset that has also been used in the original Process Matching contest in 2015.
The final results are now available at the results page!
The first dataset from the original contest that we have chosen is the University Admission dataset. It consist of process models that describe the process of university admission for different universities. Typical activities within that domain are Sending acceptance, Invite student for interview, or Wait for response. These examples illustrate one of the main differences to the ontology matching task: The labels are usually verb-object phrases that are sometimes extended with more words. Another important difference is obviously related to the existence of an execution order (i.e., the model is a complex sequence of activities) which can be understood as the counterpart to a type hierarchy.
The BPMN representation of the process models was converted to a set of assertions (ABox) using the vocabulary defined in the BPMN 2.0 ontology (TBox). For that reason the resulting matching task is a special type of instance matching task where each ABox is described by the same TBox. That means that an instance matching system should be able to create meaningful mappings with a limited amount of adapting the system to the specifics of this track.
The references alignment contains correspondences between instances of the class task as well as some correspondences between events. There are also some cases where tasks are matched on events (where it makes sense). The collection consists of 9 models ("Cologne", "Frankfurt", "FU_Berlin", "Hohenheim", "IIS_Erlangen", "Muenster", "Potsdam", "TU_Munich", "Wuerzburg"), for each pair exists an alignment in the gold standard. However, there is only an alignment named "Cologne-Frankfurt.rdf" and no alignment "Frankfurt-Cologne.rdf".
The second dataset that we have chosen is, which is new in 2017, is the Birth Registration dataset. It consists of process models that describe the birth process and related administrative tasks, like for example deciding upon the name of the child. These process models were originally available as Petrinets as *.pnml. We have also converted these datasets into ontologies, more precisely into ABoxes using the ontology introduced in [1].
The birth registration dataset we make available here contains the old gold standard that has also been used in 2015. This is the only gold standard that we will make available and it will also be part of our evaluation. However, we will also use a newer probabilistic gold standard in our final evaluation (blind evaluation). This gold standard will not be available during or after the campaign.
Note that the old gold standard contains only mappings between transitions (which can be understood as the counterparts to activities), while the new gold standard contains also a few correspondences between places (however, the have usually a rather low probability in the probabilistic gold standard).
We encourage to generate mappings with varying confidence values for both datasets. Especially for the birth registration dataset they will be taken into account.
This track runs in SEALS mode. i.e., the datasets are available as SEALS testsuites that can be executed via the SEALS client made available at the general OAEI webpage. This version is the ontology based version of the dataset. Participants of the OAEI 2017 should use version of the dataset wrapping their tools as SEALS bundle as referred to on the general OAEI 2017 webpage.
Please feel free to write at any time a mail (contact below), if there are problems related to the testsuite!
In paralell, we will also offer the ontologies themselves for download (see above), which might be helpful for local tests. However, these files might differ slightly from the SEALS testsuite, which should be taken as reference.
If you have any questions or remarks, feel free to contact us.