Skip to content

Toy datasets

ToyWSDDataset and ToyELDataset are small, hand-curated, and bundled directly in the code — no network access required. Each picks classically ambiguous words/mentions with two candidates apiece, so a context-blind baseline (e.g. always picking the first candidate) would score no better than chance:

  • ToyEADataset — 3 same-city entities across two knowledge graphs, where the second graph's labels append the country name (e.g. "Paris, France"), so exact label matching misses every pair.
  • ToyELDataset — 6 mentions of "Paris", "Mercury" and "Amazon" against two knowledge-base entries apiece.
  • ToyWSDDataset — 4 mentions of "bass" and "crane" against their two senses apiece (fish/music, machine/bird). The classic "bank" example is deliberately left out here since it's already the walkthrough in the Lesk example.
  • ToyWSADataset — 4 senses of "mouse" and "bat", each glossed differently across two dictionaries.
"""Running the bundled toy datasets end-to-end.

ToyWSDDataset and ToyELDataset need no network access, so they're a quick
way to sanity-check a linker without fetching anything external.

Run with: `uv run python examples/toy_datasets.py`
"""

from linkingtk.algorithms.string_similarity import StringSimilarityLinker
from linkingtk.algorithms.wsd import LeskLinker
from linkingtk.datasets import ToyELDataset, ToyWSDDataset
from linkingtk.eval import Evaluator


def main() -> None:
    mentions, senses, wsd_truth = ToyWSDDataset().load()
    wsd_results = LeskLinker().link(mentions, senses)
    wsd_predictions = [(r.source_id, r.target_id) for r in wsd_results]
    print("WSD:", wsd_predictions)
    print(
        "Metrics:", Evaluator.evaluate(predictions=wsd_predictions, ground_truth=wsd_truth).metrics
    )

    mentions, kb, el_truth = ToyELDataset().load()
    el_linker = StringSimilarityLinker(
        source_field="context", target_field="description", metric="word_overlap"
    )
    el_results = el_linker.link(mentions, kb)
    el_predictions = [(r.source_id, r.target_id) for r in el_results]
    print("EL:", el_predictions)
    print("Metrics:", Evaluator.evaluate(predictions=el_predictions, ground_truth=el_truth).metrics)


if __name__ == "__main__":
    main()

Run with:

uv run python examples/toy_datasets.py

WSD: [('m1', 'bass.n.01'), ('m2', 'bass.n.02'), ('m3', 'crane.n.01'), ('m4', 'crane.n.02')]
Metrics: {'precision@1': 1.0, 'recall': 1.0, 'f1': 1.0}
EL: [('m1', 'Q90'), ('m2', 'Q830149'), ('m3', 'Q308'), ('m4', 'Q925'), ('m5', 'Q3733'), ('m6', 'Q3884')]
Metrics: {'precision@1': 1.0, 'recall': 1.0, 'f1': 1.0}

Naisc ontology-matching datasets

ConferenceDataset and AnatomyDataset load OWL ontology pairs (plus a reference alignment) from the Naisc project's OAEI benchmarks. Unlike the datasets above, these fetch their left.rdf/right.rdf/align.rdf files over the network (cached under ~/.cache/linkingtk/downloads/ after the first load() call), and require the graph optional dependency group (for rdflib) — install with uv sync --extra graph. Only owl:Class entities are extracted; object/data properties are dropped.

  • ConferenceDataset — academic-conference ontologies, well under 100 classes per side. Small enough for quick, exhaustive Entity Alignment / Word Sense Alignment testing.
  • AnatomyDataset — mouse-anatomy vs. NCI Thesaurus ontologies. Despite being listed alongside ConferenceDataset as a "toy" dataset in Naisc, this is the full anatomy track (thousands of classes per side) — prefer ConferenceDataset unless you specifically need a larger benchmark.
from linkingtk.datasets import ConferenceDataset

left, right, ground_truth = ConferenceDataset().load()

Both accept a base_url override (a URL or file:// path) if you want to point them at a local copy instead of GitHub, which is how tests/datasets/test_naisc.py exercises them against fixtures without a network call.