1 of 34

Jonathan Donais & Joshua Chittle

2 of 34

What is DBpedia?

Crowdsourced community project.
Made publicly available in 2007.
Initialed developed by the Free University of Berlin and University of Leipzig in conjunction with Openlink Software.
Wikipedia is for people - DBpedia is for machines.
Part of the wider semantic web and linked open data initiative.

3 of 34

Motivation

Wikipedia

Rich and diverse.
6th most visited site.
30 million articles, 287 languages.
Open and accessible.

Semantic Web

Structured data.
Semantically rich queries.
Needs large amounts of data.

Foster wider adoption of the semantic web by providing a rich and diverse dataset

4 of 34

How does it work?

Community built ontology.
Community defined mappings.

From raw Wikipedia data (infoboxes), into the ontology

An automatic extraction framework, extracts raw Wiki data to RDF using a mix of mappings and automatic tools.

5 of 34

The Extraction Framework

6 of 34

The Extraction Framework

Writing in Scala.
Input: Wikipedia pages.

Each Wikipedia page is converted to an Abstract Syntax Tree (AST).
ASTs are run through a group of extractors.
RDF statements are produced (24 different extractors, 4 groups).

Output: Collected RDF statements are written to a sink.

7 of 34

Extractors

abstract	article categories	disambig- uation	category label	personen- namendatei	external links
geo coordinates	grammatical gender	homepage	image	infobox	inter- language
label	lexicali- zations	page ID	persondata	category label	redirects
revision ID	SKOS categories	thematic concept	topic signatures	wiki page	mapings

8 of 34

Extractors

Infobox Extraction

Raw Extraction

Automatic extraction, without the use of the community built mappings.

Mapping-Based

Manually created mappings relate infobox elements to DBpedia ontology items.

Feature Extraction

Labels or geographic coordinates.

Statistical Extraction

9 of 34

Infobox Extraction

Infoboxes contain the most valuable information.
Typically list relevant facts in a table of attribute-value pairs.
Grassroots decentralized nature of wikipedia means variation in their structure and format
Two methods - Raw and Mapping based extraction

10 of 34

Raw Extraction

Infobox example for an automobile

{{Infobox automobile

| name = Ford GT40

| manufacturer = [[Ford Advanced Vehicles]]

| production = 1964-1969

| engine = 4181cc (...)}}

Raw extraction

dbr:Ford_GT40 [

dbp:name "Ford GT40"@en;

dbp:manufacturer dbr:Ford_Advanced_Vehicles;

dbp:engine 4181;

dbp:production 1964; (...)] .

Problems: Ontology Class? Literal Typing?

11 of 34

Mapping Extraction

Community effort through the DBpedia Mappings Wiki.
Mapping assigns a type from the ontology to entities in the infobox.
Attributes of the infobox are mapped to properties.
Allows information from different language editions to be interlinked.
Allows many to one mappings for homonyms.

12 of 34

Mapping Example

{{TemplateMapping \|mapToClass = Automobile \|mappings = {{PropertyMapping \| templateProperty = name \| ontologyProperty = foaf:name }} {{PropertyMapping \| templateProperty = manufacturer \| ontologyProperty = manufacturer }} {{DateIntervalMapping \| templateProperty = production \| startDateOntologyProperty = productionStartDate \| endDateOntologyProperty = productionEndDate }}	{{IntermediateNodeMapping \| nodeClass = AutomobileEngine \| correspondingProperty = engine \| mappings = {{PropertyMapping \| templateProperty = engine \| ontologyProperty = displacement \| unit = Volume }} {{PropertyMapping \| templateProperty = engine \| ontologyProperty = powerOutput \| unit = Power }} }} (...) }}

13 of 34

Mapping Vs. Raw

Mapped

dbr:Ford_GT40 [

rdf:type dbo:Automobile;

rdfs:label "Ford GT40"@en;

dbo:manufacturer

dbr:Ford_Advanced_Vehicles;

dbo:productionStartYear

"1964"ˆˆxsd:gYear;

dbo:productionEndYear "1969"ˆˆxsd:gYear;

dbo:engine [

rdf:type AutomobileEngine;

dbo:displacement "0.004181";

]

(...)

] .

Raw

dbr:Ford_GT40 [

dbp:name "Ford GT40"@en;

dbp:manufacturer dbr:Ford_Advanced_Vehicles;

dbp:engine 4181;

dbp:production 1964; (...)] .

14 of 34

NLP Extraction

Lexicalizations

Builds relationships between alternative names for entities and concepts based on statistical scores

Topic Signatures

Vector Space Model.
Strongest word score for an entity used to tie resources to topic signatures.

Thematic Concept

Links in Wikipedia often indicate ‘main topic’

Grammatical Gender

15 of 34

Current Dataset

Version 3.9 released in September 2013.

Improved Infobox matching.
Extended to articles that do not contain an infobox.
Wikidata linking.
Various enhancements

Describes 4 million English entities

3.22 million within a consistent ontology.

24.9 million entries in 119 languages.

16 of 34

Current Dataset

Concepts for DBpedia entities published under different namespaces (URIs)

Ontologies: http://dbpedia.org/ontology/
Resources: http://dbpedia.org/resource/
Properties: http://dbpedia.org/property/

Localizations: http://<lang>.dbpedia.org/...

17 of 34

Ontology

Manually created and maintained by DBpedia’s community
Based on commonly used infoboxes.
Covers 529 classes of entities.
Described by 2,333 properties.
Current version of the ontology available online:

http://mappings.dbpedia.org/server/ontology/classes/

18 of 34

Ontology

19 of 34

Ontology Status

As of April 2013:

Mapping communities for 27 languages, 23 are active
Almost all languages have 20% coverage of property occurrences on Wikipedia
Dutch, English, Greek, Polish and Spanish, 50%+
Bulgarian and Portuguese 70%

20 of 34

Community

14 official DBpedia chapters, each for a different language.
Organize a local community, provide mappings, hosting for services and help to facilitate localized Linked Open Data clouds
Foster ‘healthy competition’ between chapters

Mapping sprints prior to release
Lead to increased interest and contributions

21 of 34

Open Data Cloud

DBpedia has a broad scope covering many areas.
Can be easily linked to external datasets.
Part of the Open Data Cloud

45 million existing interlinks.
Ex.: Freebase, CIA World Factbook, US Census Data

22 of 34

Live Sync

Wikipedia changes all the time

Ex: June 2013, English wikipedia had 3.3 million edits per month - 77 edits per minute

Live Synchronisation

Continuous stream of updates from wikipedia
DBpedia is updated within a few minutes

23 of 34

Live Sync

Local Wikipedia Mirror

Pulls changes from wikipedia.

Mapping Wiki

Source of input for extraction manager.

Live Extraction Manager

Is fed pages to be processed.

Synchronization Tool

3rd party mirrors up to date using the change set

24 of 34

SPARQL

“SPARQL Protocol and RDF Querying Language”.
SQL-like query language for RDF.
SPARQL 1.1, W3C Standard
“the semantic web as a database”
DBpedia can be queried online:

http://dbpedia.org/sparql

25 of 34

SPARQL Example

PREFIX foaf: <http://xmlns.com/foaf/0.1/> �SELECT ?name (COUNT(?friend) AS ?count) �WHERE { � ?person foaf:name ?name . � ?person foaf:knows ?friend . �} GROUP BY ?person ?name

?name	?count
“Alice”	3
“Bob”	1
“Charlie”	1

26 of 34

SPARQL

Machine readable results: XML, JSON, CSV, TSV
Looks similar to SQL

SELECT, WHERE, GROUPBY

Key difference - graph pattern matching

SELECT ?name (COUNT(?friend) AS ?count) �WHERE { � ?person foaf:name ?name . � ?person foaf:knows ?friend . �} GROUP BY ?person ?name
This will give us back entities which match this group graph pattern - entities that have both ‘foafa:name’ and ‘foaf:knows’ properties
Thus <http://example.org/snoopy> doesn’t match, and doesn’t have an entry in the returned data.

27 of 34

SPARQL

Filters / Optional:

{ ?x foaf:name ?name .� FILTER regex(?name, "Smith")

OPTIONAL { ?x prop:height ?h .

FILTER ( ?h < 160 ) }}

Other SQL-like operators: EXISTS, NOT, UNION, MINUS, etc.
Unique Query Forms: CONSTRUCT, ASK, DESCRIBE.

28 of 34

SPARQL

A SPARQL query for DBpedia…
http://dbpedia.org/sparql

select ?abstract ?grossing

where

{

<http://dbpedia.org/resource/Star_Wars_Episode_V:_The_Empire_Strikes_Back> <http://dbpedia.org/ontology/abstract> ?abstract .

FILTER langMatches( lang(?abstract),"en") .

<http://dbpedia.org/resource/Star_Wars_Episode_V:_The_Empire_Strikes_Back> <http://dbpedia.org/property/gross> ?grossing

}

?abstract	?grossing
"Star Wars Episode V: The Empire Strikes Back (also known as The Empire Strikes Back) is a 1980 American epic space opera film…”	"5.38375067E8"^^<http://dbpedia.org/datatype/usDollar>

29 of 34

Uses

Advanced Wikipedia Search
Geographic Applications
Document Classification
Natural Language Processing
Digital Libraries and Archives
Knowledge Exploration

30 of 34

Faceted Browsing

Partial exploitation of DBpedia to allow more complex queries
Utilizes Faceted Search Paradigm
Various Demo Implementations:

31 of 34

NLP Services

DBpedia Spotlight
Quepy
APIs for text-to-entity linking

Ex.: AlchemyAPI, Semantic API from Ontos, Open Calais, Zemanta

Annotations and Tagging

Faviki, BBC

Question Answering

32 of 34

Other Applications

DBpedia Relationship Finder

Provides a simpler interface for finding connections within the huge DBpedia data set.

DBpedia Mobile

Renders a map containing annotations about your surrounding area.

AboutThisDay

Gives events and information about a given day.

33 of 34

Questions

34 of 34

Sources

Lhmann, J.; Schuppel, J.; Auer, S.: Discovering Unknown Connections - the DBpedia Relationship Finder. IN proceedings of 1st Conference on Social Semantic Web, CSSW2007, Leipzig, September 24-28, 2007. Volume P-113 of GI-Edition - Lecture Notes in Informatics (LNI). Bonner Kollen Verlag, ISSN 1617-5468, ISBN 978-3-88579-207-9.
http://www.w3.org/TR/2013/REC-sparql11-overview-20130321/
http://www.w3.org/TR/sparql11-query/
Auer, S.; Bizer, C.; Lehmann, J.; Kobilarov, G.; Cyganiak, R.; Ives, Z.: DBpedia: A Nucleus for a Web of Open Data. In Aberer et al. (Eds.): The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11–15, 2007. Lecture Notes in Computer Science 4825 Springer 2007, ISBN 978–3-540–76297–3.
Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, Christian Bizer. DBpedia – A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Submitted to the Semantic Web Journal.
Mohamed Morsey, Jens Lehmann, Sören Auer, Claus Stadler, Sebastian Hellmann, (2012) «DBpedia and the live extraction of structured data from Wikipedia", Program: electronic library and information systems, Vol. 46 Iss: 2, pp.157 – 181