Open Research Knowledge Graph: Towards Machine Actionability in Scholarly Communication

Despite improved digital access to scientific publications in the last decades, the fundamental principles of scholarly communication remain unchanged and continue to be largely document-based. The document-oriented workflows in science publication have reached the limits of adequacy as highlighted by recent discussions on the increasing proliferation of scientific literature, the deficiency of peer-review and the reproducibility crisis. In this article, we present first steps towards representing scholarly knowledge semantically with knowledge graphs. We expand the currently popular RDF graph-based knowledge representation formalism to capture annotations, such as provenance information and describe how to manage such knowledge in a graph data base. We report on the results of a first experimental evaluation of the concept and its implementations with the participants of an international conference.


INTRODUCTION
Documents are central to scholarly communication, as the vast majority of research contributions are formulated as papers in printed or electronic forms. While document-centred scholarly communication may be sufficient for addressing certain scientific questions, in practice, most of the contemporary scientific discourse (and its underlying problem domains) demanding an interdisciplinary research effort, in which scientists have to deal with a large amount of divers, and (currently) not interlinked information sources. Furthermore, scientific outputs are usually reported in forms of monolithic (electronic) documents. Most of these publications remains hardly accessible for computer analysis and processing, besides running simple text search or limited annotation functions on them. Therefore, despite that the text contained in publications can be indexed and used for searching, images, diagrams, tables, mathematical formulas, and references are hardly accessible to computers. This clearly indicates that scientific publishing has remained almost unchanged during the past several centuries, which is very unsatisfactory in the light of current technical development of the society in general.
There are further arguments pointing towards a major reform of current scientific paper publishing. For instance following recent scientific discourse is difficult (if not impossible), due to exponential expansion of scientific knowledge volume [7,26]. We are entering an era, when individual researchers cannot possibly cope with the extreme volumes of publications, which can easily result in vague, ambiguous and redundant research reporting. Consequently, this phenomenon is also identified as one of the main factors behind the reproducibility crisis [23].
Insufficient peer review is another problem. More than 1 million articles are published annually [6]. Finding referees, and managing high quality peer review process, are oftentimes difficult. This notion results in extended manuscript revision times and feedback cycles, where publication deadlines are often delayed and research outputs become outdated during the publication process. [32].
The challenge, we are facing in scholarly communication is also illustrated by how the digitalization techniques compare to other domains. Most information rich publishing and communication services (e.g. encyclopedias, mail order catalogs, street maps or phone books) went through a total digital transformation in the recent years. Examples are encyclopedias, mail order catalogs, street maps or phone books. In these domains, the traditional documentoriented publications where not just digitized and published digitally as PDF documents, as we do in scholarly communication now. Who would use a PDF of a phone book, mail-order catalog, encyclopedia or phone book now anymore? Which actually would be the comparable state to what we have in scholarly communication right now. Instead in these domains, completely new means how to organize and access the information were developed. Documentoriented mail order catalogs were replaced by e-commerce services, which allow faceted-browsing, filtering, comparisons, and many other features, not available in print. Similarly, street maps were replaced by navigation services. The difference is that these digital services not just digitized the analogue documents, but developed very specific means for collaboration, information sharing and access, which leverage and exploit the new digital possibilities. Another commonality is that instead of unstructured documents, these new services are now based on comprehensive and structured data or even knowledge bases.
As a result there is an urge for a more flexible, fine-grained, semantic and context sensitive representation of scientific knowledge. One way to achieve this, is presenting scholarly information as structured, interlinked, and semantically rich knowledge graphs [3].
A definition to the term "knowledge graph" was given by [13]: "a knowledge graph acquires and integrates information into an ontology and applies a reasoner to derive new knowledge". A knowledge graph, which is utilized to store, exchange and to infer (for example, using a reasoner) scientific knowledge in an open and transparent way, we name as an Open Research Knowledge Graph (ORKG).
In this article, we describe the first complete round-trip engineering of a knowledge graph based scholarly communication approach. We expand the currently popular RDF graph-based knowledge representation methods to be efficiently able to capture annotations, such as provenance information, in the body of scientific knowledge. We also detail our back-end and front-end ORKG implementations, based on the extended graph data model. Finally we report on the evaluation results, pertaining to our initial ORKG concept and software prototype implementation, with the involvement of international conference delegates at the Data Integration in the Life Sciences (DILS) conference.
The remainder of this article is structured as follows: In section 2 we introduce the problem, together with our general research approach. The details of ORKG are presented in section 3, discussing the details of our proposed technical solution. This is followed by the reporting on the process and the date collected during an evaluation study in section 4, with subsequent discussions in section 5. All related work is presented in section 6 and we close this paper with directions for future work and conclusions in section 7.

PROBLEM STATEMENT
Nowadays, published research contributions are in form of (typically digital PDF) documents, where authors structure articles in sections addressing problem statement, approach, evaluation, and conclusions. This approach to scholarly communication is not without challenges. The communicated information is often ambiguous and difficult to reproduce. Moreover, since information is in documents it is difficult for researchers to efficiently explore state-of-the-art research. Reviewing contributions, too, is becoming more resource intensive due to the vast number of publications and general lack of reviewers.
We illustrate the problem with an example from life-sciences. When searching for publications on the popular Genome editing method CRISPR/Cas 1 in scholarly search engines we obtain a vast amount of search results. Google Scholar, for example, returns more than 56.000 results, when searching for the search string 'CRISPR cas' 2 . For answering a specific research question, it is often required to join various keywords or search strings. Examples in the context of CRISPR/Cas include: • How good is CRISPR/Cas (wrt. precision, safety, cost)?  • What specifics have genome editing with insects?
• Who has applied it to butterflies?
Even when adding the search term 'butterfly' to the search query, we still receive more than 500 search results, many of which might be non-relevant. Furthermore, the relevant results might not be included (e.g., due to the fact that the technical term for butterfly is Lepidoptera, which combined with 'CRISPR/cas' returns 1.700 results). We argue that keyword-based information retrieval cannot fulfill the requirements of scholarly communication in the digital age. While there have been some approaches for developing automated techniques to assist researchers in managing the breadth and depth of information in scholarly documents, we suggest that completely new means for representing scholarly communication have to be developed, and that automated techniques can and will not reach the accuracy required. Working with digitized scholarly documents might only cure some symptoms of the problems currently observed in scholarly communication but not result in an effective cure. Imagine, how the analysis of PDF versions of maps or mail order catalogs using natural language processing and information retrieval techniques would compare to the currently prevalent techniques using database backed e-commerce applications or digital map systems such as OpenStreetMaps [35].
The document-based mechanism to represent scholarly communication do not allow to clearly identify concepts and their relationships. This results in ambiguities since different authors refer differently to the same concepts or use the same term for referring to different concepts. Also, the characteristics of concepts or relationships between concepts are not clearly defined in a way that algorithms and machines could support researchers in mastering the information. Automated techniques to identify concepts in a text (e.g., named entity recognition) as well as their characteristics or relationships (relation extraction), despite decades of research, do (and will) not reach a sufficiently high accuracy for meaningful applications. Some people might argue that recent advances in machine learning might be applicable to making sense of scholarly communication. We suggest that this is not possible, due to the lack of training data. Machine learning can only be applied when  algorithms can learn from tagged training data (produced by humans). To represent scholarly communication, training data does not exist and creating it would be extremely time-consuming even if restricting ourselves only to very narrow domains (e.g., consider the effort required to create a systematic literature review). The only way to resolve this problem is to develop a semantic means to represent scholarly communication.

OPEN RESEARCH KNOWLEDGE GRAPH
We propose to leverage knowledge graphs to represent information communicated in scholarly literature. Crucially, knowledge graphs not only contain bibliographic metadata (e.g., about authors, conferences, references) but semantic descriptions of scholarly contributions (i.e., the problem, approach, solution, implementation, evaluation of concrete research investigations). We call this knowledge graph the Open Research Knowledge Graph (ORKG).
One of the approaches to populate the ORKG is to crowd-source the information. To that end, we developed a prototype web application that we envision later to be integrated into journal and conference or open-access repository submission systems. In this section, we present the two main component of the system: the back-end and the user interface (UI).

Scholarly Knowledge Graph Back-end
In order to meet ORKG requirements, the system needs to implement many aspects that govern and control the curation process, from defining the data model, and representing resources, to exposing the system via APIs.
Data model and architecture. The back-end uses a graph model that consists of nodes and edges. This model was chosen to simplify the process of adding information to the system without the need to learn more complex data models, such as RDF 3 . One of the greatest differences to RDF is that everything is modeled as an entity, i.e., it can be referenced by an identifier (ID).
The data model is centered around the concept of a statement. A statement is a triple that consists of a subject and an object (nodes) that are connected by a predicate (relationship). Nodes can have one of two types: resources and literals. Resources represent a concept, such as a scientific method or an author, whereas literals represent values, such as the name of the method or the author's name. Within statements, literals can only appear in the object position of the statement.
The main application is written in the Kotlin [25] programming language, within the Spring Boot 2 framework. The data is stored in a Neo4j graph database accessed by the application via the Spring Data Neo4j OGM 4 . From the architectural perspective, the design follows a classical layered architecture with ports and adapters ("hexagonal architecture"). To evaluate the different behaviors of technologies, adapters to different components are possible. For example, the persistence layer is able to handle linked property graphs (LPG) such as Neo4j as well as triple-stores. The domain layer holds the domain objects, such as statements, resources, and literals. The application layer is responsible for the application logic, such as the REST API. Building on the other layers, the user interface is responsible for querying and displaying the data forming the knowledge graph.
ID generation. In order to be able to link and retrieve statement or resource entities, IDs need to be generated when information is stored. Because we envision a distributed architecture, globally unique IDs are required to synchronize information between instances. However, global unique identifiers are complex to generate and manage. IDs need to be generated in a way that satisfies the requirements of being unique across all entities in the system, immutable, short (for easy input) and can be generated with reasonable performance. As summarized in Table 1, we have taken into account various methods for ID generation.
Universally unique identifiers (UUID) exist but are very hard for humans to read, remember, or understand due to their length.Merging or linking data between several instances of the ORKG infrastructure does not need universal uniqueness. However, the same result can be achieved by having locally unique IDs that are name-spaced. To satisfy the requirement of being short, positive numbers will be used with a prefix that determines the type, e.g., "P123" for a predicate. This is similar to the way Wikidata handles IDs. It has the drawback of being harder to read in a query but has the advantage of being short. Using a human-readable label would also be possible but cannot be automatically generated and therefore needs input from the user at the time of creation. It is technically possible to change to human-readable IDs from numeric ones later but due to the immutability requirement, it should be avoided so it does not cause problems for downstream users. Generating these IDs fast is crucial when a large number of statements is included in the graph. ID generation can be done with good performance by using the HiLo pattern, moving the responsibility of ID generation from the database to the back-end. In this pattern, the database creates a prefix (the so-called high number) that the back-end can then use to construct IDs by generating its own numbers (low numbers) and adding the received prefix. These low numbers are usually from a limited block of numbers (such as 1000) and can be generated without talking to the database. Once the client drained its number pool, it can request another prefix from the server. This reduces the amount of interaction between the database and back-end significantly and therefore improves performance.
RDF representation. The graph model can be exported to RDF via the Neo4j Semantics extension 6 . Due to the differences between our graph model and RDF, a "semantification" needs to occur. Most 6 https://github.com/jbarrasa/neosemantics importantly, the ORKG back-end auto-generates URIs for the data. Mapping (or changing) these URIs to an existing ontology needs to happen manually. The Semantics extension also allows importing RDF data and vocabularies. Hence, existing vocabularies can be imported, modified and extended. Unfortunately, only a subset of OWL is supported, e.g., the reasoning capabilities are very limited at this point and mainly comprise sub-class inference. The query language for the imported data is Cypher, i.e., Neo4j's native query language, as there currently is no support for SPARQL. However, we plan to add SPARQL support in near future based on our work for a query algebra for graph data [37] and the existing implementation for the Gremlin graph query language [38].
Data retrieval and querying. The back-end is exposed via a simple RESTful API 7 to be used by the front-end (User Interface). Data is queried by sending HTTP requests and is returned in JSON format. This allows other applications to talk to the database and work with data in ways other than those we anticipate. This decoupling also allows for greater flexibility while the project is still in development. A technical documentation of the current API specification is accessible online 8 .
Provenance and authentication. The back-end takes into consideration provenance information as metadata on the statements created, such as when and by whom an entity was created. Users will be able to authenticate against ORKG using single sign-on (SSO), e.g., using ORCID. This greatly lowers the entry barrier, fostering collaboration. It also allows us to implement more sophisticated role management.
Linking to other services. In order to enrich ORKG data with other metadata, it will be possible to load or link data from other sources, such as Crossref. A key requirement is to have connection points, such as DOIs. Although bibliographic metadata about scholarly communication is important, building yet another scientific information system is out of the scope of this project. Rather we aim to link and integrate other systems (e.g. DataCite, CrossRef) and their information via identifiers such as DOI, ORCID etc.
Community engagement and development. Interested parties will be able to use and test the infrastructure via the ORKG beta service, a deployment of the latest version of the ORKG. It will support requesting feedback on features early and allow for the project to be guided by the requirements of its users.
Due to the experimental and fast-changing nature of the project, it is vital to provide technical documentation to interested developers. We will also provide an easy way to download and run the ORKG infrastructure for local experiments, e.g., as a Docker image. This will allow interested parties to get a first glance at the technology and spark interest to contribute to the project with software. For an in-depth explanation of the ORKG architecture and technology used, written documentation and developers communication boards will be provided on platforms such as GitLab or GitHub.

Graph Curation and Exploration User Interface
The user interface (UI) facilitates access to the knowledge graph, by providing the ability to search, browse, create and modify the existing research contributions. The user interface was built with the following two key requirements in mind: (1) Usability (i.e., easy to use) to focus on the ability to allow all (co-)authors from any discipline to use the system without any previous knowledge or training on the system and (2) Dynamicity of the system to adapt descriptions of the scholarly data in a way that allows the user to control data curation with a maximum degree of freedom. The user interface design was inspired by the Wikidata project-the central community-created data management platform of Wikipedia [14,40]. The goal of the UI is to provide users with a wizard that guides them in creating graph-based representations of research contributions. This design choice reinforces with the usability requirement of the system. The UI is written in compliance with the ES6 standard of JavaScript using the React framework. The Bootstrap framework is used for responsive HTML design and user interface components.
Scholarly contribution curation. The system allows users to add new scholarly contributions, initially via a DOI using the Crossref infrastructure or by adding all information manually. The user can add new properties or select them from a list via the autocomplete feature. Afterwards, the created property will be linked to a resource either a newly created one or a resource already existing in the graph. The ORKG supports two types of resources (Literals, and Linked resources). Furthermore, the user of the application can edit values of the inserted scholarly contributions, at the resource level or the predicate level. The curation aspect of the system relies on the dynamicity requirement of the user interface to be able to provide a dynamic environment to collect all possible data of the contributions. Figure 6 exemplary shows selected attributes of the "Quick Sort" algorithm contribution with the possibility to edit the information and add new values (i.e., resources and literals). 9 Search and Exploration. To support exploring contributions, the main page lists all available resources. Navigation links to check resources and predicates support narrowing down the field of exploration. Furthermore, for quick access, functionality is available to search for any resource or a predicate via their labels. Each resource in the ORKG is expandable and can be explored to see what other resources are related to it. The relationships are displayed in subject-predicate-object form (similar to RDF triples), whereby the subject is the current research contribution, the predicates are presented in separate blocks on the left, and the related objects are grouped on the right side of each block.
State-of-the-art comparison. One major feature of the ORKG is the ability to perform a state-of-the-art comparison quite easily and quickly. The user can select a scholarly contribution and then compare it to all other contributions in the knowledge graph addressing the same research problem, ranked by similarity. Figure 4 depicts an example of comparing the resource Quick Sort to other known contributions. As a result, the system produces a table showing contributions with a similarity value. The resulting table shows the most common characteristics or properties between the evaluated contributions in an interactive manner that supports sorting the values to obtain deeper understanding of the connections.

EVALUATION
In this section, we detail the aims and outcomes of the evaluation process. For the initial user evaluation, we followed a qualitative approach to identify major (positive and negative) issues and what users perceive when working with the system. The evaluation process had two components: 1) Instructed interaction with the system and 2) A short evaluation questionnaire.

Instructed User Interactions & User Survey
In order to evaluate the system, instructed interaction sessions were conducted with 12 participants (authors from different disciplines such as Computer Science, Data Science, Biology, Life Science) during the 13 t h International Conference on Data Integration in the Life Sciences 2018 (DILS2018) 10 . The aim of these interactions was to get first-hand observations and comments on our approach. Thus, this was the first prototypical implementation in which participants had to semantically describe their DILS papers, thereby creating ORKG content. The interaction sessions were conducted with the help of two instructors. At the start of each individual interaction sessions, the instructor explained the underlying principles of the system in two minutes, including how the system works and what is required of the participant to complete the session. Consequently, participants engaged with the system without any further guidance from the instructor. However, they were allowed at any time to ask the instructor for assistance. The completion time was recorded for each participant, together with the instructor's notes and the participant's comments on the interaction session. Timing was critical to establish the typical length of a user interaction, due to the fact that our system is expected to be integrated with manuscript submission systems and therefore should require little time. The interaction notes, participant's comments and the time recordings were analyzed together with outcomes of the user survey.
As an addition to the instructed interaction sessions, participants were invited to fill out a short evaluation survey. The aim of this questionnaire was to provide further insights into how users from 10 https://www.springer.com/de/book/9783030060152 different scientific disciplines experienced the system (e.g., comprehensiveness of terminology used, user-friendliness, UI design). The questionnaire was thus treated as a qualitative instrument, since the data was insufficient to establish any correlational or causal relationship [24]. This short user survey was pen and paper-based, and contained altogether 11 items. The items were designed to complement participant reactions to the system following their instructed interaction session, so that they could more deeply reflect on the aspects they found critical (both positive and negative) about the system. Participants filled out their surveys after the instructed interaction session. All 12 participants answered the questionnaire. The survey is available online 11 .

Results
As a result of the interaction and survey data (descriptive) analysis, we obtained a number of major topics, which received significant attention from the users.
(1) Difficulties with the HCI 12 Overall, participants found the system to be fairly easy to use, especially after receiving some explanation or examples. 75% of the participants found the UI fairly intuitive and easy to use, while 10% did not need any guidance, with 80% of the participants needed guidance only at the beginning. With regards to the user interface, five out of twelve participants suggested to make the UI more keyboard-friendly, for instance by adding a simple guiding wizard through the UI. As participant #3 stated: "More description in advance can be helpful". Two participants commented that the navigation process throughout the system is complicated for first-time users, and suggested other ways that could be done easier. As an example, participant #5 suggested to "Use breadcrumbs to navigate". Moreover, the timing of the participants averaged approximately 17 min with a maximum of 22 min and a minimum of 13 min. (2) Visualisation of the Knowledge Graph Four participants demanded visualization (i.e., graph chart) to be available when creating a sub-graph. For instance, participant #1 commented that "It could be helpful to show a local view of the graph while editing". This type of visualisation would facilitate the comprehension among users from multiple disciplines. Another participant suggested that we integrate a document (PDF) viewer within the application to highlight relevant passages or phrases for the user. Participant #4 noted the that "If I could highlight the passages directly in the paper and add predicates there, it would be more intuitive and save time". (3) Availability of vocabularies for curation Two participants commented on the use of a controlled vocabulary to guide the curation process. A vocabulary would make the process more fluent and intuitive. Focusing on the UI aspect, participant #6 suggested to "show an overview of the existing ontology to avoid a lot of new properties and objects".
Further details on the survey are presented in Table 2, giving an overview of how participants rated the main items of the survey.  much  in mins   1  4  4  5  3  2  6  16  2  2  3  5  4  8  7  19  3  4  5  5  3  9  7  15  4  3  3  5  3  6  7  13  5  4  3  5  3  6  8  14  6  4  3  5  3  8  9  13  7  3  4  5  3  7  6  19  8  3  2  4  3  8  6  13  9  4  5  3  3  7  5  14  10  4  5  5  1  8  8  22  11  4  5  5  1  8  8  20  12 - Table 2: Overview of answers on the most important concepts in the evaluation survey Again, the results do not provide statistical relevance since the cohort of participants is too small for statistical conclusions. However, the results provide a number of suggestions users indicated to be critical when using the ORKG, and what aspects of the UI and the interaction with the system in general should be improved. Table 3 shows a sample (for the sake of readability) of the usecase data collected during the evaluation of the ORKG with the participants of the international DILS 2018 conference. This table shows the data grouped into four main categories. Research Problem, which indicates what is the main problem or issue that the publication is addressing. Participants used a variety of properties when expressing the problem such as "Problem, Addresses, Focus, Subject, Proposes, and Topic". Approach shows what is the solution or approach taken to face the problem. For instance, the information about the approach was delivered through "Approach, Uses, Prospective work, Method, Focus, and Algorithm". Implementation is one of the most comprehensively described aspects in this study, arguably because it was easier for participants to describe technical details as concepts compared to describing the problem or the approach. Implementation is presented with two columns, one for the property (describing the implementation) and the other for the respective value. Evaluation is the last category participants filled, and is constructed in a similar manner as implementation. The complete dataset obtained in the case study is available online 13 .

DISCUSSION
With this first-hand experience of the ORKG, we were able to put multiple aspects of the system under scrutiny, better understand how users interact with the system, and obtain data and feedback on interaction and user experience. During the evaluation, the main focus was the user interface and how users found it throughout their experience. Since the user interface is the only aspect of the system that the participants would interact with, the survey was directed to certain aspects of user interaction. Early results showed that the system achieved some of the required objectives by being easy to use and capable of adapting to the collected data regardless of the discipline or the domain. The results of the survey ( Table  2) show how the participants reacted to each descriptive concept 13 https://doi.org/10.5281/zenodo.2549916 that we are looking for. Most of the results are above half, except the Guidance needed which shows that most participants did not really need the guidance to use the system, reinforcing the usability requirement of the user interface. All case study participants displayed an interest in the ORKG and provided valuable input on what should be changed, added, or removed. Furthermore, the participants suggested to integrate the system within libraries, universities, and other institutes allowing everybody to exploit the potentials and benefits of the system on a larger scale.
Finally, we highlight an important limitation. Since the ORKG relies on people to curate the data (i.e., expert crowd-sourcing), the process to convert a document-based contribution to a machinereadable model is remains challenging. Difficulties arise when trying to conceptualize key aspects of the scholarly contribution, such as problem or approach. Participants in the case study found this task quite challenging and time-consuming.

RELATED WORK
Representing encyclopedic and factual knowledge using RDF and Linked Data is increasingly feasible. This is underscored by knowledge graphs such as DBpedia [2], Wikidata [41], and Yago [21] as well as industrial initiatives like Google, IBM, Bing, BBC, or Thomson Reuters.
In the library and scholarly communication context, much work has so far focused on representing and managing bibliographic metadata while the formal (i.e., machine readable) representation of scientific information communicated in scholarly literature has received very little attention, with the exception of few initiatives such as the Semantic Publishing and Referencing (SPAR) Ontologies [30] which, however, focus primarily on metadata and to some extent on document structure.
There has been some work on enriching various document formats with semantic annotations. Examples include Dokie.li [9], RASH [31] or MicroPublications [11] for HTML and SALT [18] for LaTeX. We started representing key findings of survey articles focusing on semantically describing research problems, approaches, implementations and evaluations in [15] and integrating bibliographic information in a knowledge graph [33].
Other work focused on developing ontologies for representing scholarly knowledge in specific domains, for example, mathematics [27], the RXNO ontology in chemistry or the OBO Foundry ontologies [34] in the life sciences. A knowledge graph for science must go beyond such efforts, by enabling the parallel and synchronized creation, curation, and augmentation of both terminological/ontological as well as assertional and discourse knowledge. For representing provenance and discourse we can build on the PROV ontology [28] and Document Components Ontology [12].
While there has been work on argumentation and reasoning in AI (e.g., [4,17]) and philosophy (often using specialized formalisms), more work needs to be done to represent argumentation, concept drift, and scholarly knowledge evolution in knowledge graphs.
The RDF data model and respective ontologies arguably appear adequate as a scaffold for representing scholarly knowledge. However, aspects such as provenance, evolution, and discourse are more difficult to represent in pure RDF (see the ongoing discussion about reification  such as RDF singleton properties [42], which can be used for representing and exchanging semantic data, we need to investigate more how graph data management techniques (e.g., using the Gremlin graph query algebra [22]) can be employed to store and manage the extremely large amounts of interconnected scholarly communication data and metadata. Hence, we argue that a knowledge graph for science can be build but must extend the triple (or quad) data model of RDF. The scholarly communication community has initiated numerous related projects. The Research Graph [1] is a prominent example of an effort that aims to link research objects, in particular publications, dataset, researcher profiles. The Scholix project [8], driven by a corresponding Research Data Alliance working group and associated organizations, aims at standardizing the information about the links between scholarly literature and data exchanged among publishers, data repositories, and infrastructures such as DataCite, Crossref, and OpenAIRE.
Other related projects include Research Objects [5], which proposes a machine-readable abstract structure that relates the products of a research investigation, including articles but also data and other research artefacts, as well as the RMap Project [19], which aims at preserving "the many-to-many complex relationships among scholarly publications and their underlying data. "

CONCLUSION AND FUTURE WORK
This article describes the first step of a large research and development agenda. We suggest that the transition to knowledge-based information flows is an absolute imperative in order to adapt scholarly communication to the digital world. This article describes a first complete iteration of the knowledge graph based scholarly communication concept: We described a knowledge-graph based data model, which uses RDF as a scaffold, but adds important features for provenance tracking and annotation. The data model is implemented in a graph database back-end. We also showcased a first prototypical implementation of the user interface, which provides three core features: curation of the scholarly knowledge graph, exploration and retrieval as well as comparisons of approaches for reviewing the state-of-the-art in a certain field. We performed an initial evaluation of the concept and implementation with the participants of the recent Data Integration in the Life Sciences conference.
As the next step, we aim to significantly improve the individual components of the ORKG architecture. For the data model, we aim to introduce the concept of knowledge molecules. They provide reusable, compact, relatively simple, structured units of knowledge, e.g., to represent research problems and contributions. With this, we aim to lay the foundation for a novel quality of (cognitive) knowledge graphs that are better at representing conceptual entities (in addition to the factual ones currently being dealt with by knowledge graphs). We also expect that such a novel concept of cognitive knowledge graphs will be better at dealing with semantics emerging from large scale collaboration as well as concept drift. On the user interface side, we aim to integrate more strategies for crowd-sourcing and human-machine cooperation, where researcher are enabled to contribute their research descriptions using flexible and lightweight widgets that are able to be tailored for the specifics of different disciplines. We plan to integrate an automatic analysis of textual research articles, to provide recommendations for the manual curation of the ORKG. The more contributions and descriptions are available, the more automated techniques using the existing representations as training data can be integrated. Finally, we aim to scale the case studies to larger venues and communities to experiment with various research areas and their representations and refine the authoring, curation and exploration techniques. In particular, we have to realize more applications that provide direct value to researchers, while or after contributing their descriptions, to ultimately render a network effect and thus make the ORKG comprehensively reflecting the world's scientific knowledge.