SemEval-2021 Task 11 Shared Task Dataset

NLPContributionGraph - Structuring Scholarly NLP Contributions in the Open Research Knowledge Graph


NLPContributionGraph was introduced as Task 11 at SemEval 2021 for the first time. The task is defined on a dataset of Natural Language Processing (NLP) scholarly articles with their contributions structured to be integrable within Knowledge Graph infrastructures such as the Open Research Knowledge Graph. The structured contribution annotations are provided as (1) Contribution sentences : a set of sentences about the contribution in the article; (2) Scientific terms and relations: a set of scientific terms and relational cue phrases extracted from the contribution sentences; and (3) Triples: semantic statements that pair scientific terms with a relation, modeled toward subject-predicate-object RDF statements for KG building. The Triples are organized under three (mandatory) or more of twelve total information units (viz., ResearchProblem, Approach, Model, Code, Dataset, ExperimentalSetup, Hyperparameters, Baselines, Results, Tasks, Experiments, and AblationAnalysis).

The Shared Task

As a complete submission for the Shared Task, given NLP scholarly articles in plaintext format, systems had to automatically extract the following information: contribution sentences; scientific term and predicate phrases from the sentences; and * (subject,predicate,object) triple statements toward KG building organized under three or more of twelve total information units.

Data and Resources

Cite this as

Jennifer D'Souza and Sören Auer and Ted Pedersen (2021). Dataset: SemEval-2021 Task 11 Shared Task Dataset.

DOI retrieved: 15:36 04 Aug 2021 (GMT)

Additional Info

Field Value
Author Jennifer D'Souza and Sören Auer and Ted Pedersen
Maintainer Jennifer D'Souza
Version 1.0
Last Updated June 17, 2021, 11:49 (CEST)
Created February 25, 2021, 12:15 (CET)
License Creative Commons Attribution Share-Alike 3.0