nodegoat Workshop�2 November 2023�NYU Abu Dhabi
�
Pim van Bree & Geert Kessels
@LAB1100 / @nodegoat
Programme
09:00 Welcoming Remarks
Süphan Kirmizialtin, Visiting Assistant Professor of Middle Eastern History, NYUAD
David Joseph Wrisley, Professor of Digital Humanities, NYUAD
09:30 Introduction to Nodegoat
Pim van Bree and Geert Kessels, LAB1100
10:15 Who are you and what is your research question?
10:30 Learn how to enter geographical data into nodegoat
11:00 Coffee Break
11:15 Learn how to enter relational data into nodegoat
12:15 Learn how to conceptualise a data model
13:00 Lunch
14:00 Learn how to implement a data model in nodegoat
15:00 Enter data in your data model
15:30 Coffee Break
15:45 Filter & visualise data
16:15 Import & export
16:45 Conditional formatting, network analysis, linked data connections
17:30 End of the workshop
Who are we?
LAB1100 is a research and development firm established in 2011 by Pim van Bree and Geert Kessels.
We started in 2011 by developing diachronic spatial and social network graphs for historical research for the Study Platform on Interlocking Nationalisms at the University of Amsterdam.
We then built a web-based research platform to host all the data that is used in the visualisations. We first called this the ‘Chrono Spatial Research Platform’ and then renamed the platform to nodegoat.
2011 “SPINTime”
The Study Platform on Interlocking Nationalisms (Universiteit van Amsterdam) uses nodegoat to map and analyse intellectuals in the 19th century to understand the spread of romantic nationalism.
People, Letters, Paintings, Monuments, Music, Texts, Currency.
12 years, over 400 collaborators.
Usage of nodegoat
nodegoat is used by 2800+ individual research projects.�nodegoat runs at 40 institutes in 18 countries, supporting 80+ projects, including:
Universität Bern: ‘Repertorium Academicum Germanicum’, ‘Forced Academic Migration’, ‘Dynamic Data Ingestion’, etc.
Australian National University / University of New Hampshire: ‘Modelling Ancient Narratives, Territories, Objects’.
Universiteit van Amsterdam: ‘Encyclopedia of Romantic Nationalism in Europe’, ‘Managing multi-level conflicts in commercial cities in northern Europe (c. 1350-1570)’, ‘The European Spa as a Transnational Public Space and Social Metaphor’, etc.
Universiteit Gent: ‘The transnational dynamics of social reform, 1840-1940’, ‘Agents of Change: Women Editors and Socio-Cultural Transformation in Europe, 1710-1920’, ‘Pyramids and Progress: Belgian expansionism and the making of Egyptology, 1830-1952’, etc.
CNRS IRHT: ‘Signifying Objects, Texts and Networks: Digitising Transactions in Jewish European Culture, 1500-1700’.
Colorado College: ‘Al-‘Ulūm al-gharība: Mapping the sciences in al-Jabartī’, ‘Homosexuality in Modern France’, etc.
LMU München: ‘Relocating Modernism: Global Metropolises, Modern Art and Exile’.
Università di Padova: ‘l'Ottocentenario dell'Università di Padova del 2022’, ‘Ruling in hard times. Patterns of power and practices of government in the making of Carolingian Italy’.
Université du Luxembourg: ‘Mapping Wartime Jewish Diaries’, ‘Soldiers and their communities in WWII: The impact and legacy of war experiences in Luxembourg’, ‘Ecrire l'histoire de la place financière d/au Luxembourg’, etc.
Universidade NOVA de Lisboa: ‘Mapping movements in the Portuguese empire’.
�See nodegoat.net/usecases or zotero.org/lab1100/tags/nodegoat for examples.
nodegoat allows you to define any number of custom data models. You are free to define your research environment based on the specific needs of your research project.
After you have configured nodegoat, you can use your environment as a collaborative workspace to create, manage, analyse and visualise data.
Once you have created a data set, you can publish interactive visualisations or export data publications.
1
2
3
RESTful API (JSON & JSON-LD)
Version Control
Clearance Levels
Multiple Users
Multiple Projects
Ingestion Processes
Linked Data Connectivity
Public User Interfaces
Import CSV / Export CSV / ODT
Reconciliation Processes
Find and Replace
Text Tagging
Full Text Editing
Media Upload (PDF, Images, etc.)
Source Referencing
Chronology Statements
Chronological Visualisations
Built-in Diachronicity
Custom Map Tiles
Geographical Visualisations
Built-in Spatiality
Graph-based Network Metrics
Social Visualisations
Built-in Relationality
Geometry Support
Reversed Classification
Path-based Data Analysis
Multi-layered Filtering
Relational Data Entry
Implement your own data model in nodegoat based on your research question and source material
The Repertorium Academicum Germanicum (Universität Bern) uses nodegoat to map and analyse medieval students and their careers in the Holy Roman Empire.
Events, People, Institutes, Sources.
Over 70.000 scholars and 400.000 events in live environment.
Susanna de Beer (Universiteit Leiden) uses nodegoat to map literary texts and ancient locations in Rome.
Interactive web app shows current location of device.
The TIC project (Ghent University) uses nodegoat to map and analyse conference attendance between 1880 and 1920 to analyse intellectual cooperation.
Events, People, Organisations, Topics.
Three years, 40 collaborators, over 20.000 conference attendees in live environment.
Lucie Janotová (Scuola Normale Superiore, Florence, Italy) uses nodegoat to analyse non-conformist art activism in Socialist Czechoslovakia.
Use nodegoat
nodegoat runs in the browser, more info: https://nodegoat.net/.
To get a hosted account, go to: https://nodegoat.net/requestaccount.
For larger projects we provide paid services to run local installations, training events, workshop, support, server management, etc. See: https://nodegoat.net/products.
The latest version of the open source package can be found at http://nodegoat.net/release.
Who are you?
Please introduce yourself and your research question.
Learn how to enter data
We use a shared demo environment for this exercise. Please log out with your personal account you received via email.
Go to nodegoat.net and login with:
Username: workshop[1-40]�Password: denhaag
nodegoat allows you to define any number of custom data models. You are free to define your research environment based on the specific needs of your research project.
After you have configured nodegoat, you can use your environment as a collaborative workspace to create, manage, analyse and visualise data.
Once you have created a data set, you can publish interactive visualisations or export data publications.
1
2
3
Creating a database
Conceptual �Data Modelling
Logical �Data Modelling
Data-based�Application
People |
Cities |
Book | ||
Links to | ||
Person | ||
Cities |
<XML>
Book | ||
Value name | Value type | Links to |
Title | string | |
Publication Date | date | |
Author | reference | People |
Publication City | reference | Cities |
Creating a database
Conceptual �Data Modelling
Logical �Data Modelling
Data-based�Application
People |
Cities |
Book | ||
Links to | ||
Person | ||
Cities |
<XML>
Book | ||
Value name | Value type | Links to |
Title | string | |
Publication Date | date | |
Author | reference | People |
Publication City | reference | Cities |
Creating a database
Conceptual �Data Modelling
Logical �Data Modelling
Data-based�Application
People |
Cities |
Book | ||
Links to | ||
Person | ||
Cities |
<XML>
Book | ||
Value name | Value type | Links to |
Title | string | |
Publication Date | date | |
Author | reference | People |
Publication City | reference | Cities |
How to conceptualise a data model
Person | |
Place of Birth | |
Place of Residence | |
Place of Death | |
Capacity |
City |
Capacity |
Book | |
Author | |
Publication Place |
How to conceptualise a data model
How can you describe each of these Types in-depth?
Person | |||
Value name | Kind of value | Links to | |
Family Name | string | - | |
Given Name | string | - | |
Biographical Notes | text | | |
Capacity | classification | Capacity | |
[Birth] - Date | date | - | |
[Birth] - Location Reference | reference | City |
From a spreadsheet to a data model
Whenever historians gather data, this usually happens in a spreadsheet. How to transfer this to a database?
date | sender | sender gender | sender capacity | recipient | recipient gender | recipient capacity | kind of letter |
07-09-1805 | Grimm Jakob | male | philologist | Zimmer Henriette | female | court lady | cooperative |
18-06-1825 | Grimm Jakob | male | philologist | Dobrovský Josef | male | philologist | cooperative |
04-04-1836 | Štúr Ľudovít | male | linguist | Godra Michal | male | poet | cooperative |
29-12-1858 | Grimm Jakob | male | philologist | Costa Ethbin Heinrich | male | historian | phatic |
26-06-1836 | Štúr Ľudovít | male | linguist | Hamuljak Martin | male | editor | cooperative |
12-11-1837 | Štúr Ľudovít | male | linguist | Gaj Ljudovit | male | linguist | personal |
17-04-1841 | Štúr Ľudovít | male | politician | Amerling Karel | male | teacher | cooperative |
03-04-1851 | Grimm Jakob | male | philologist | La Villemarqué Théodore | male | philologist | phatic |
22-07-1842 | Štúr Ľudovít | male | linguist | Karadžić Vuk | male | philologist | cooperative |
People |
Cities |
Letter | ||
Links to | ||
People | ||
Cities |
From a spreadsheet to a data model
A lot of historians put all their data in a single spreadsheet, which grows, and grows, and grows...
date | sender | sender gender | sender capacity | recipient | recipient gender | recipient capacity | kind of letter | source reference | sending city | sending lat/long | receiving city | receiving lat/long |
07-09-1805 | Grimm Jakob | male | philologist | Zimmer Henriette | female | court lady | cooperative | BBG,57 | Paris | 48.853,2.348 | Kassel | 51.3166,9.5 |
18-06-1825 | Grimm Jakob | male | philologist | Dobrovský Josef | male | philologist | cooperative | AJBG, 8 | Kassel | 51.316,9.5 | Prag | 50.088,14.420 |
04-04-1836 | Štúr Ľudovít | male | linguist | Godra Michal | male | poet | cooperative | DPZ, 96-98 | Preßburg | 48.148,17.106 | Vrbas | 45.57,19.640 |
29-12-1858 | Grimm Jakob | male | philologist | Costa Ethbin Heinrich | male | historian | phatic | AJBG, i | Berlin | 52.524,13.410 | Laibach | 46.05,14.505 |
26-06-1836 | Štúr Ľudovít | male | linguist | Hamuljak Martin | male | editor | cooperative | DPZ, 110 | Preßburg | 48.148,17.10 | Jasenová | 49.17,19.295 |
12-11-1837 | Štúr Ľudovít | male | linguist | Gaj Ljudovit | male | linguist | personal | DPZ, 134 | Preßburg | 48.1481,17.10 | Agram | 45.814,15.977 |
17-04-1841 | Štúr Ľudovít | male | politician | Amerling Karel | male | teacher | cooperative | DPZ, 213 | Preßburg | 48.148,17.106 | Budapest | 47.49,19.039 |
03-04-1851 | Grimm Jakob | male | philologist | La Villemarqué Théodore | male | philologist | phatic | JCLV, 6 | Berlin | 52.5243,13.4 | Paris | 48.85,2.34879 |
22-07-1842 | Štúr Ľudovít | male | linguist | Karadžić Vuk | male | philologist | cooperative | DPZ, 237 | Uhrovec | 48.74818.338 | Belgrade | 44.80,20.4651 |
From a spreadsheet to a data model
Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:
Sheet 1 contains all the information about each letter
ID | date | sender | recipient | kind of letter | source reference | sending city | receiving city |
1 | 07-09-1805 | 1 | 3 | cooperative | BBG,57 | 1 | 2 |
2 | 18-06-1825 | 1 | 4 | cooperative | AJBG, 8 | 2 | 5 |
3 | 04-04-1836 | 2 | 5 | cooperative | DPZ, 96-98 | 3 | 7 |
4 | 29-12-1858 | 1 | 6 | phatic | AJBG, i | 4 | 8 |
5 | 26-06-1836 | 2 | 11 | cooperative | DPZ, 110 | 3 | 9 |
6 | 12-11-1837 | 2 | 7 | personal | DPZ, 134 | 3 | 10 |
7 | 17-04-1841 | 2 | 8 | cooperative | DPZ, 213 | 3 | 11 |
8 | 03-04-1851 | 1 | 9 | phatic | JCLV, 6 | 4 | 1 |
9 | 22-07-1842 | 2 | 10 | cooperative | DPZ, 237 | 6 | 12 |
From a spreadsheet to a data model
Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:
Sheet 2 contains all the information about each person
ID | family name | given name | gender | capacity | VIAF ID | VIAF URL |
1 | Grimm | Jakob | male | philologist | 31997136 | |
2 | Štúr | Ľudovít | male | linguist | 39390566 | |
3 | Zimmer | Henriette | female | court lady | 299713803 | |
4 | Dobrovský | Josef | male | philologist | 56628990 | |
5 | Godra | Michal | male | poet | 84519339 | |
6 | Costa | Ethbin Heinrich | male | historian | 310510743 | |
7 | Gaj | Ljudevit | male | linguist | 12321242 | |
8 | Amerling | Karel | male | teacher | 13331054 | |
9 | La Villemarqué | Théodore | male | philologist | 49227556 | |
10 | Karadžić | Vuk | male | philologist | 7403617 | |
11 | Hamuljak | Martin | male | editor | 14809173 |
From a spreadsheet to a data model
Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:
Sheet 3 contains all the information about each city
ID | current English name | alternative names | latitude | longitude | Geonames ID | Geonames URL |
1 | Paris | | 48.85340881 | 2.34879994 | 2988507 | |
2 | Kassel | Cassel | 51.31666946 | 9.5 | 2892518 | |
3 | Bratislava | Preßburg, Pozsony | 48.14815903 | 17.10674095 | 3060972 | |
4 | Berlin | | 52.52436829 | 13.41053009 | 2950159 | |
5 | Prague | Prag | 50.0880394 | 14.42076015 | 3067696 | |
6 | Uhrovec | | 48.74884033 | 18.33843994 | 3057042 | |
7 | Vrbas | | 45.57138824 | 19.64082909 | 3187297 | |
8 | Ljubljana | Laibach | 46.0510788 | 14.50512981 | 3196359 | |
9 | Jasenová | | 49.17636108 | 19.29567909 | 3059695 | |
10 | Zagreb | Agram | 45.81444168 | 15.9779796 | 3186886 | |
11 | Budapest | | 47.49800873 | 19.03990936 | 3054643 | |
12 | Belgrade | Beograd | 44.80400848 | 20.46512985 | 792680 |
From a spreadsheet to a data model
In between these sheets, we have relationships:
Sheet 1: Letters
Sheet 2: People
Sheet 3: Cities
ID | family name | given name | gender | capacity | VIAF ID | VIAF URL |
1 | Grimm | Jakob | male | philologist | 31997136 | |
2 | Štúr | Ľudovít | male | linguist | 39390566 | |
3 | Zimmer | Henriette | female | court lady | 299713803 | |
4 | Dobrovský | Josef | male | philologist | 56628990 | |
5 | Godra | Michal | male | poet | 84519339 | |
6 | Costa | Ethbin Heinrich | male | historian | 310510743 | |
7 | Gaj | Ljudevit | male | linguist | 12321242 | |
8 | Amerling | Karel | male | teacher | 13331054 | |
9 | La Villemarqué | Théodore | male | philologist | 49227556 | |
10 | Karadžić | Vuk | male | philologist | 7403617 | |
11 | Hamuljak | Martin | male | editor | 14809173 |
ID | current English name | alternative names | latitude | longitude | Geonames ID | Geonames URL |
1 | Paris | | 48.85340881 | 2.34879994 | 2988507 | |
2 | Kassel | Cassel | 51.31666946 | 9.5 | 2892518 | |
3 | Bratislava | Preßburg, Pozsony | 48.14815903 | 17.10674095 | 3060972 | |
4 | Berlin | | 52.52436829 | 13.41053009 | 2950159 | |
5 | Prague | Prag | 50.0880394 | 14.42076015 | 3067696 | |
6 | Uhrovec | | 48.74884033 | 18.33843994 | 3057042 | |
7 | Vrbas | | 45.57138824 | 19.64082909 | 3187297 | |
8 | Ljubljana | Laibach | 46.0510788 | 14.50512981 | 3196359 | |
9 | Jasenová | | 49.17636108 | 19.29567909 | 3059695 | |
10 | Zagreb | Agram | 45.81444168 | 15.9779796 | 3186886 | |
11 | Budapest | | 47.49800873 | 19.03990936 | 3054643 | |
12 | Belgrade | Beograd | 44.80400848 | 20.46512985 | 792680 |
ID | date | sender | recipient | sending city | receiving city |
1 | 07-09-1805 | 1 | 3 | 1 | 2 |
2 | 18-06-1825 | 1 | 4 | 2 | 5 |
3 | 04-04-1836 | 2 | 5 | 3 | 7 |
4 | 29-12-1858 | 1 | 6 | 4 | 8 |
5 | 26-06-1836 | 2 | 11 | 3 | 9 |
6 | 12-11-1837 | 2 | 7 | 3 | 10 |
7 | 17-04-1841 | 2 | 8 | 3 | 11 |
8 | 03-04-1851 | 1 | 9 | 4 | 1 |
9 | 22-07-1842 | 2 | 10 | 6 | 12 |
Creating a database
Based on this, you can now draw out your conceptual data model:
People |
Cities |
Letters | ||
Links to | ||
People | ||
Cities |
Creating a database
Conceptual �Data Modelling
Logical �Data Modelling
Data-based�Application
People |
Cities |
Letter | ||
Links to | ||
People | ||
Cities |
Letter | ||
Value name | Value type | Links to |
Date | date | |
Sender | relationship | People |
Recipient | relationship | People |
Sending city | relationship | Cities |
Receiving city | relationship | Cities |
<XML>
Iterative Data Modelling
People |
Cities |
Letter | ||
Links to | ||
People | ||
Cities |
Letter | ||
Value name | Value type | Links to |
Date | date | |
Sender | relationship | People |
Recipient | relationship | People |
Sending city | relationship | Cities |
Receiving city | relationship | Cities |
Draft a Conceptual Data Model
Draw a first version of a conceptual data model of your project. This can be done on a piece of paper or with any digital tool (PowerPoint, Word, etc).
You can find the slides of this presentation via https://lab1100.com/pres.
Setting up your nodegoat project
For those who have an account on https://nodegoat.net: go to nodegoat.net and login with the credentials you received via email.
If you do not have an account yet, go to https://nodegoat.net/requestaccount and request one.
Some definitions
Objects are things that may have existed (e.g. 'Jacob Grimm', or 'The Lord of the Rings'). Objects are grouped by means of their typology: the Objects of 'Jacob Grimm' and 'Wilhelm Grimm' belong to the Type 'Person'. 'The Lord of the Rings' and the 'Bible' belong to the Type 'Book'.
Sub-Objects define aspects of Objects that have a temporal and/or spatial dimension. 'Birth', 'Place of Residence' and 'Death' define temporal and spatial dimensions of Objects of people. 'Production', 'Ownership', and 'Exhibition' define temporal and spatial dimensions of Objects of artworks.
Categories are labels that are assigned to Objects (e.g. 'female', or 'sketch'). Categories are part of a Classification: the Categories 'female' and 'non-binary' are part of the Classification 'Gender'. The Categories 'sketch' and 'oil painting' are part of the Classification 'Kind of Artwork'.
Complex Temporal Statements: ???
Historians have endless needs to express temporal uncertainty, but lack a vocabulary to represent this (e.g. ‘around’, ‘approx.’, ‘1780?’).
The Repertorium Academicum Germanicum project at the University of Bern commissioned us to extend nodegoat with a feature to handle complex date statements. We first explored using EDTF for this. EDTF was not suitable for our needs due to these shortcomings:
Complex Temporal Statements: ChronoJSON
To produce date statements that are both vague and actionable, we developed 'Chronology Statements'. Chronology Statements implement a human and computational interpretable temporal topology. It provides objects with a position in time by means of statements that combine operators, cyclic periodisation, relationality, and additional offsetting.
Chronology Statements are formatted in JSON, for which the ChronoJSON format has been developed.
Defining Chronology Statements
Period
Date Start
Date End
Period
Date Start
Defining Chronology Statements
Statement (before/after/etc.)
Date End
Statement (before/after/etc.)
Period
Date End
Defining Chronology Statements
Date Start
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Date End
Defining Chronology Statements
Date Start
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Period
Date End
Defining Chronology Statements
Date Start
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Period
cycle
Date End
Defining Chronology Statements
Date Start
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Between Statement
(before/after/etc.)
Period
cycle
cycle
cycle
cycle
reference
reference
reference
reference
cycle
Date End
Defining Chronology Statements
Date Start
1 year after
2 years before
2 months after
1 month before
immatriculation
spring
winter
w.sem.
spring
Luther's imatr.
1504
1500
Erasmus' graduation
sommer semester
Defining Chronology Statements
ChronoJSON
Compute
Complex Temporal Statements: ChronoJSON
Explicit and transparent storage and representation of uncertain and vague dates.
Actionable by design:
Full documentation will be released soon. Guides on usage are available: https://nodegoat.net/guides/chronologystatements, https://nodegoat.net/guides/relationalchronologystatements.
@nodegoat / nodegoat.net / support@nodegoat.net
@LAB1100 / LAB1100.com / info@LAB1100.com