# A Simple Baseline for Inductive Knowledge Base Completion: Supplementary Material


Within this folder you find the code/commands and (links to) all datasets that you need to generate the baseline results presented in the paper "A Simple Baseline for Inductive Knowledge Base Completion".

To run AnyBURL you need to have Java installed on our device. Any version >=12 should work properly (e.g., openjdk12). Type `java -version` to check your installation.

### Results for FB237

We prepared everything to generate the results for the splits of the FB237 datasets from [1]. At the end of this readme we explain where to get the other datasets and how to generate results for them.

In each of the experiments, we first need to learn the rules sets. Make the *suppl* folder the working directory and run AnyBURL with this command:

	java -Xmx12G -cp AnyBURL-23-1x.jar de.unima.ki.anyburl.Learn exp/fb237/config-learn.properties

This will run AnyBURL for 30 seconds in the restricted setting that corresponds to the baseline. You can take a look at the config file to see the parameters that result into this setting. The command generates the rule file *rules-v1-30* in the *exp/fb237* folder. Now you can use these rules to create a ranking file. Use this command:

	java -Xmx12G -cp AnyBURL-23-1x.jar de.unima.ki.anyburl.Apply exp/fb237/config-apply.properties 

The resulting ranking file can be evaluated with this command:

	java -Xmx12G -cp AnyBURL-23-1x.jar de.unima.ki.anyburl.Eval exp/fb237/config-eval.properties

This generates the following command line output. The first value is the hits@1 score, the second value is the hits@3 score, the third value is the hits@10 score and the last value is the MRR score, which we show in the results tables of our paper.

	...
	* loading result set at exp/fb237/preds-v1-30
	0.4000   0.5341   0.5854   0.4720

There might a slight difference compared to the MRR number shown in the FBe v1 column (which is 46.7 in the paper) , which might be caused by the sampling done within AnyBURL. Differences are usually very small, exception might be the v1 splits of the datasets which are very small in number of test triples. 

If you want to generate the results for the v2, v3, and v4 splits of this dataset, you have to change the config-files and rerun the commands above. 

### All Datasets

The other datasets can be obtained from:

  * WNe, FBe, NELLe used in [1] are available at: https://github.com/kkteru/grail/tree/master/data
  * ILPC small and large used in [2] are available at: https://github.com/pykeen/ilpc2022/tree/main/data
  * The datasets from Table 3 used in [3] are available at: https://github.com/bdi-lab/InGram/tree/main/data
  
The file names of the target graph differ between these datasets. In [1] it is the *train.txt* file in the *_ind* folders. In [2] this file is called *inference.txt*. In [3] it is the *msg.txt* file. Once the datasets are downloaded, the config files can be copied and easily adapted to generate the baseline results for all of the datasets. You have to add subfolders manually in the *exp* folder.

### References


[1] Teru, K.; Denis, E.; and Hamilton, W. 2020. Inductive relation prediction by subgraph reasoning. In International conference on machine learning, PMLR.
  
[2] Galkin, M.; Berrendorf, M.; and Hoyt, C. T. 2022. An open challenge for inductive link prediction on knowledge graphs. arXiv preprint arXiv:2203.01520.
 
[3] Lee, J.; Chung, C.; and Whang, J. J. 2023. Ingram: Inductive knowledge graph embedding via relation graphs. In International conference on machine learning, PMLR.