1 of 45

nodegoat Workshop�2 November 2023�NYU Abu Dhabi

Pim van Bree & Geert Kessels

@LAB1100 / @nodegoat

2 of 45

Programme

09:00 Welcoming Remarks

Süphan Kirmizialtin, Visiting Assistant Professor of Middle Eastern History, NYUAD

David Joseph Wrisley, Professor of Digital Humanities, NYUAD

09:30 Introduction to Nodegoat

Pim van Bree and Geert Kessels, LAB1100

10:15 Who are you and what is your research question?

10:30 Learn how to enter geographical data into nodegoat

11:00 Coffee Break

11:15 Learn how to enter relational data into nodegoat

12:15 Learn how to conceptualise a data model

13:00 Lunch

14:00 Learn how to implement a data model in nodegoat

15:00 Enter data in your data model

15:30 Coffee Break

15:45 Filter & visualise data

16:15 Import & export

16:45 Conditional formatting, network analysis, linked data connections

17:30 End of the workshop

3 of 45

Who are we?

LAB1100 is a research and development firm established in 2011 by Pim van Bree and Geert Kessels.

We started in 2011 by developing diachronic spatial and social network graphs for historical research for the Study Platform on Interlocking Nationalisms at the University of Amsterdam.

We then built a web-based research platform to host all the data that is used in the visualisations. We first called this the ‘Chrono Spatial Research Platform’ and then renamed the platform to nodegoat.

2011 “SPINTime

4 of 45

The Study Platform on Interlocking Nationalisms (Universiteit van Amsterdam) uses nodegoat to map and analyse intellectuals in the 19th century to understand the spread of romantic nationalism.

People, Letters, Paintings, Monuments, Music, Texts, Currency.

12 years, over 400 collaborators.

5 of 45

Usage of nodegoat

nodegoat is used by 2800+ individual research projects.�nodegoat runs at 40 institutes in 18 countries, supporting 80+ projects, including:

Universität Bern:Repertorium Academicum Germanicum’, ‘Forced Academic Migration’, ‘Dynamic Data Ingestion’, etc.

Australian National University / University of New Hampshire:Modelling Ancient Narratives, Territories, Objects’.

Universiteit van Amsterdam: Encyclopedia of Romantic Nationalism in Europe’, ‘Managing multi-level conflicts in commercial cities in northern Europe (c. 1350-1570)’, ‘The European Spa as a Transnational Public Space and Social Metaphor’, etc.

Universiteit Gent: The transnational dynamics of social reform, 1840-1940’, ‘Agents of Change: Women Editors and Socio-Cultural Transformation in Europe, 1710-1920’, ‘Pyramids and Progress: Belgian expansionism and the making of Egyptology, 1830-1952’, etc.

CNRS IRHT: Signifying Objects, Texts and Networks: Digitising Transactions in Jewish European Culture, 1500-1700’.

Colorado College:Al-‘Ulūm al-gharība: Mapping the sciences in al-Jabartī’, ‘Homosexuality in Modern France’, etc.

LMU München:Relocating Modernism: Global Metropolises, Modern Art and Exile’.

Università di Padova:l'Ottocentenario dell'Università di Padova del 2022’, ‘Ruling in hard times. Patterns of power and practices of government in the making of Carolingian Italy’.

Université du Luxembourg:Mapping Wartime Jewish Diaries’, ‘Soldiers and their communities in WWII: The impact and legacy of war experiences in Luxembourg’, ‘Ecrire l'histoire de la place financière d/au Luxembourg’, etc.

Universidade NOVA de Lisboa:Mapping movements in the Portuguese empire’.

See nodegoat.net/usecases or zotero.org/lab1100/tags/nodegoat for examples.

6 of 45

nodegoat allows you to define any number of custom data models. You are free to define your research environment based on the specific needs of your research project.

  • Person
  • Letter
  • Event
  • Location

After you have configured nodegoat, you can use your environment as a collaborative workspace to create, manage, analyse and visualise data.

  • Analyse correspondence networks
  • Collect conference attendees
  • Create a literary map of Rome

Once you have created a data set, you can publish interactive visualisations or export data publications.

  • Publish interactive visualisations
  • Publish datasets
  • Publish mobile app

1

2

3

7 of 45

RESTful API (JSON & JSON-LD)

Version Control

Clearance Levels

Multiple Users

Multiple Projects

Ingestion Processes

Linked Data Connectivity

Public User Interfaces

Import CSV / Export CSV / ODT

Reconciliation Processes

Find and Replace

Text Tagging

Full Text Editing

Media Upload (PDF, Images, etc.)

Source Referencing

Chronology Statements

Chronological Visualisations

Built-in Diachronicity

Custom Map Tiles

Geographical Visualisations

Built-in Spatiality

Graph-based Network Metrics

Social Visualisations

Built-in Relationality

Geometry Support

Reversed Classification

Path-based Data Analysis

Multi-layered Filtering

Relational Data Entry

Implement your own data model in nodegoat based on your research question and source material

8 of 45

9 of 45

The Repertorium Academicum Germanicum (Universität Bern) uses nodegoat to map and analyse medieval students and their careers in the Holy Roman Empire.

Events, People, Institutes, Sources.

Over 70.000 scholars and 400.000 events in live environment.

10 of 45

Susanna de Beer (Universiteit Leiden) uses nodegoat to map literary texts and ancient locations in Rome.

Interactive web app shows current location of device.

11 of 45

The TIC project (Ghent University) uses nodegoat to map and analyse conference attendance between 1880 and 1920 to analyse intellectual cooperation.

Events, People, Organisations, Topics.

Three years, 40 collaborators, over 20.000 conference attendees in live environment.

12 of 45

Lucie Janotová (Scuola Normale Superiore, Florence, Italy) uses nodegoat to analyse non-conformist art activism in Socialist Czechoslovakia.

13 of 45

Use nodegoat

nodegoat runs in the browser, more info: https://nodegoat.net/.

To get a hosted account, go to: https://nodegoat.net/requestaccount.

For larger projects we provide paid services to run local installations, training events, workshop, support, server management, etc. See: https://nodegoat.net/products.

The latest version of the open source package can be found at http://nodegoat.net/release.

14 of 45

Who are you?

Please introduce yourself and your research question.

15 of 45

Learn how to enter data

We use a shared demo environment for this exercise. Please log out with your personal account you received via email.

Go to nodegoat.net and login with:

Username: workshop[1-40]�Password: denhaag

16 of 45

nodegoat allows you to define any number of custom data models. You are free to define your research environment based on the specific needs of your research project.

  • Person
  • Letter
  • Event
  • Location

After you have configured nodegoat, you can use your environment as a collaborative workspace to create, manage, analyse and visualise data.

  • Analyse correspondence networks
  • Collect conference attendees
  • Create a literary map of Rome

Once you have created a data set, you can publish interactive visualisations or export data publications.

  • Publish interactive visualisations
  • Publish datasets
  • Publish mobile app

1

2

3

17 of 45

Creating a database

Conceptual �Data Modelling

Logical �Data Modelling

Data-based�Application

People

Cities

Book

Links to

Person

Cities

<XML>

Book

Value name

Value type

Links to

Title

string

Publication Date

date

Author

reference

People

Publication City

reference

Cities

18 of 45

Creating a database

Conceptual �Data Modelling

Logical �Data Modelling

Data-based�Application

People

Cities

Book

Links to

Person

Cities

<XML>

Book

Value name

Value type

Links to

Title

string

Publication Date

date

Author

reference

People

Publication City

reference

Cities

19 of 45

Creating a database

Conceptual �Data Modelling

Logical �Data Modelling

Data-based�Application

People

Cities

Book

Links to

Person

Cities

<XML>

Book

Value name

Value type

Links to

Title

string

Publication Date

date

Author

reference

People

Publication City

reference

Cities

20 of 45

How to conceptualise a data model

Person

Place of Birth

Place of Residence

Place of Death

Capacity

City

Capacity

Book

Author

Publication Place

21 of 45

How to conceptualise a data model

How can you describe each of these Types in-depth?

Person

Value name

Kind of value

Links to

Family Name

string

-

Given Name

string

-

Biographical Notes

text

Capacity

classification

Capacity

[Birth] - Date

date

-

[Birth] - Location Reference

reference

City

22 of 45

From a spreadsheet to a data model

Whenever historians gather data, this usually happens in a spreadsheet. How to transfer this to a database?

date

sender

sender gender

sender capacity

recipient

recipient gender

recipient capacity

kind of letter

07-09-1805

Grimm Jakob

male

philologist

Zimmer Henriette

female

court lady

cooperative

18-06-1825

Grimm Jakob

male

philologist

Dobrovský Josef

male

philologist

cooperative

04-04-1836

Štúr Ľudovít

male

linguist

Godra Michal

male

poet

cooperative

29-12-1858

Grimm Jakob

male

philologist

Costa Ethbin Heinrich

male

historian

phatic

26-06-1836

Štúr Ľudovít

male

linguist

Hamuljak Martin

male

editor

cooperative

12-11-1837

Štúr Ľudovít

male

linguist

Gaj Ljudovit

male

linguist

personal

17-04-1841

Štúr Ľudovít

male

politician

Amerling Karel

male

teacher

cooperative

03-04-1851

Grimm Jakob

male

philologist

La Villemarqué Théodore

male

philologist

phatic

22-07-1842

Štúr Ľudovít

male

linguist

Karadžić Vuk

male

philologist

cooperative

People

Cities

Letter

Links to

People

Cities

23 of 45

From a spreadsheet to a data model

A lot of historians put all their data in a single spreadsheet, which grows, and grows, and grows...

date

sender

sender gender

sender capacity

recipient

recipient gender

recipient capacity

kind of letter

source reference

sending city

sending lat/long

receiving city

receiving lat/long

07-09-1805

Grimm Jakob

male

philologist

Zimmer Henriette

female

court lady

cooperative

BBG,57

Paris

48.853,2.348

Kassel

51.3166,9.5

18-06-1825

Grimm Jakob

male

philologist

Dobrovský Josef

male

philologist

cooperative

AJBG, 8

Kassel

51.316,9.5

Prag

50.088,14.420

04-04-1836

Štúr Ľudovít

male

linguist

Godra Michal

male

poet

cooperative

DPZ, 96-98

Preßburg

48.148,17.106

Vrbas

45.57,19.640

29-12-1858

Grimm Jakob

male

philologist

Costa Ethbin Heinrich

male

historian

phatic

AJBG, i

Berlin

52.524,13.410

Laibach

46.05,14.505

26-06-1836

Štúr Ľudovít

male

linguist

Hamuljak Martin

male

editor

cooperative

DPZ, 110

Preßburg

48.148,17.10

Jasenová

49.17,19.295

12-11-1837

Štúr Ľudovít

male

linguist

Gaj Ljudovit

male

linguist

personal

DPZ, 134

Preßburg

48.1481,17.10

Agram

45.814,15.977

17-04-1841

Štúr Ľudovít

male

politician

Amerling Karel

male

teacher

cooperative

DPZ, 213

Preßburg

48.148,17.106

Budapest

47.49,19.039

03-04-1851

Grimm Jakob

male

philologist

La Villemarqué Théodore

male

philologist

phatic

JCLV, 6

Berlin

52.5243,13.4

Paris

48.85,2.34879

22-07-1842

Štúr Ľudovít

male

linguist

Karadžić Vuk

male

philologist

cooperative

DPZ, 237

Uhrovec

48.74818.338

Belgrade

44.80,20.4651

24 of 45

From a spreadsheet to a data model

Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:

Sheet 1 contains all the information about each letter

ID

date

sender

recipient

kind of letter

source reference

sending city

receiving city

1

07-09-1805

1

3

cooperative

BBG,57

1

2

2

18-06-1825

1

4

cooperative

AJBG, 8

2

5

3

04-04-1836

2

5

cooperative

DPZ, 96-98

3

7

4

29-12-1858

1

6

phatic

AJBG, i

4

8

5

26-06-1836

2

11

cooperative

DPZ, 110

3

9

6

12-11-1837

2

7

personal

DPZ, 134

3

10

7

17-04-1841

2

8

cooperative

DPZ, 213

3

11

8

03-04-1851

1

9

phatic

JCLV, 6

4

1

9

22-07-1842

2

10

cooperative

DPZ, 237

6

12

25 of 45

From a spreadsheet to a data model

Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:

Sheet 2 contains all the information about each person

ID

family name

given name

gender

capacity

VIAF ID

VIAF URL

1

Grimm

Jakob

male

philologist

31997136

2

Štúr

Ľudovít

male

linguist

39390566

3

Zimmer

Henriette

female

court lady

299713803

4

Dobrovský

Josef

male

philologist

56628990

5

Godra

Michal

male

poet

84519339

6

Costa

Ethbin Heinrich

male

historian

310510743

7

Gaj

Ljudevit

male

linguist

12321242

8

Amerling

Karel

male

teacher

13331054

9

La Villemarqué

Théodore

male

philologist

49227556

10

Karadžić

Vuk

male

philologist

7403617

11

Hamuljak

Martin

male

editor

14809173

26 of 45

From a spreadsheet to a data model

Instead of working with one massive sheet, it's better to work with separate sheets for different kinds of information:

Sheet 3 contains all the information about each city

ID

current English name

alternative names

latitude

longitude

Geonames ID

Geonames URL

1

Paris

48.85340881

2.34879994

2988507

2

Kassel

Cassel

51.31666946

9.5

2892518

3

Bratislava

Preßburg, Pozsony

48.14815903

17.10674095

3060972

4

Berlin

52.52436829

13.41053009

2950159

5

Prague

Prag

50.0880394

14.42076015

3067696

6

Uhrovec

48.74884033

18.33843994

3057042

7

Vrbas

45.57138824

19.64082909

3187297

8

Ljubljana

Laibach

46.0510788

14.50512981

3196359

9

Jasenová

49.17636108

19.29567909

3059695

10

Zagreb

Agram

45.81444168

15.9779796

3186886

11

Budapest

47.49800873

19.03990936

3054643

12

Belgrade

Beograd

44.80400848

20.46512985

792680

27 of 45

From a spreadsheet to a data model

In between these sheets, we have relationships:

Sheet 1: Letters

Sheet 2: People

Sheet 3: Cities

ID

family name

given name

gender

capacity

VIAF ID

VIAF URL

1

Grimm

Jakob

male

philologist

31997136

2

Štúr

Ľudovít

male

linguist

39390566

3

Zimmer

Henriette

female

court lady

299713803

4

Dobrovský

Josef

male

philologist

56628990

5

Godra

Michal

male

poet

84519339

6

Costa

Ethbin Heinrich

male

historian

310510743

7

Gaj

Ljudevit

male

linguist

12321242

8

Amerling

Karel

male

teacher

13331054

9

La Villemarqué

Théodore

male

philologist

49227556

10

Karadžić

Vuk

male

philologist

7403617

11

Hamuljak

Martin

male

editor

14809173

ID

current English name

alternative names

latitude

longitude

Geonames ID

Geonames URL

1

Paris

48.85340881

2.34879994

2988507

2

Kassel

Cassel

51.31666946

9.5

2892518

3

Bratislava

Preßburg, Pozsony

48.14815903

17.10674095

3060972

4

Berlin

52.52436829

13.41053009

2950159

5

Prague

Prag

50.0880394

14.42076015

3067696

6

Uhrovec

48.74884033

18.33843994

3057042

7

Vrbas

45.57138824

19.64082909

3187297

8

Ljubljana

Laibach

46.0510788

14.50512981

3196359

9

Jasenová

49.17636108

19.29567909

3059695

10

Zagreb

Agram

45.81444168

15.9779796

3186886

11

Budapest

47.49800873

19.03990936

3054643

12

Belgrade

Beograd

44.80400848

20.46512985

792680

ID

date

sender

recipient

sending city

receiving city

1

07-09-1805

1

3

1

2

2

18-06-1825

1

4

2

5

3

04-04-1836

2

5

3

7

4

29-12-1858

1

6

4

8

5

26-06-1836

2

11

3

9

6

12-11-1837

2

7

3

10

7

17-04-1841

2

8

3

11

8

03-04-1851

1

9

4

1

9

22-07-1842

2

10

6

12

28 of 45

Creating a database

Based on this, you can now draw out your conceptual data model:

People

Cities

Letters

Links to

People

Cities

29 of 45

Creating a database

Conceptual �Data Modelling

Logical �Data Modelling

Data-based�Application

People

Cities

Letter

Links to

People

Cities

Letter

Value name

Value type

Links to

Date

date

Sender

relationship

People

Recipient

relationship

People

Sending city

relationship

Cities

Receiving city

relationship

Cities

<XML>

30 of 45

Iterative Data Modelling

People

Cities

Letter

Links to

People

Cities

Letter

Value name

Value type

Links to

Date

date

Sender

relationship

People

Recipient

relationship

People

Sending city

relationship

Cities

Receiving city

relationship

Cities

31 of 45

Draft a Conceptual Data Model

Draw a first version of a conceptual data model of your project. This can be done on a piece of paper or with any digital tool (PowerPoint, Word, etc).

You can find the slides of this presentation via https://lab1100.com/pres.

32 of 45

Setting up your nodegoat project

For those who have an account on https://nodegoat.net: go to nodegoat.net and login with the credentials you received via email.

If you do not have an account yet, go to https://nodegoat.net/requestaccount and request one.

33 of 45

Some definitions

Objects are things that may have existed (e.g. 'Jacob Grimm', or 'The Lord of the Rings'). Objects are grouped by means of their typology: the Objects of 'Jacob Grimm' and 'Wilhelm Grimm' belong to the Type 'Person'. 'The Lord of the Rings' and the 'Bible' belong to the Type 'Book'.

Sub-Objects define aspects of Objects that have a temporal and/or spatial dimension. 'Birth', 'Place of Residence' and 'Death' define temporal and spatial dimensions of Objects of people. 'Production', 'Ownership', and 'Exhibition' define temporal and spatial dimensions of Objects of artworks.

Categories are labels that are assigned to Objects (e.g. 'female', or 'sketch'). Categories are part of a Classification: the Categories 'female' and 'non-binary' are part of the Classification 'Gender'. The Categories 'sketch' and 'oil painting' are part of the Classification 'Kind of Artwork'.

34 of 45

Complex Temporal Statements: ???

Historians have endless needs to express temporal uncertainty, but lack a vocabulary to represent this (e.g. ‘around’, ‘approx.’, ‘1780?’).

The Repertorium Academicum Germanicum project at the University of Bern commissioned us to extend nodegoat with a feature to handle complex date statements. We first explored using EDTF for this. EDTF was not suitable for our needs due to these shortcomings:

  • Missing the flexibility in defining vagueness by using relationships.
  • Missing the flexibility to define custom (i.e. historical) intervals/groupings.
  • Too much summarised in one date 'rule', losing context.
  • Well suited for representation, less suited for computation.

35 of 45

Complex Temporal Statements: ChronoJSON

To produce date statements that are both vague and actionable, we developed 'Chronology Statements'. Chronology Statements implement a human and computational interpretable temporal topology. It provides objects with a position in time by means of statements that combine operators, cyclic periodisation, relationality, and additional offsetting.

  • Chronology Statements apply operators similar to Allen operators and are in line with the computational efficiency of, for instance, temporal relational primitives.
  • Offsetting in time is expressed by specifying an offset amount in combination with an offset unit. Offsets can also be expressed by self-defined cycles.
  • Chronology Statements enable temporal relationality by allowing users to select a reference to another stored Chronology Statement instead of specifying a date.

Chronology Statements are formatted in JSON, for which the ChronoJSON format has been developed.

36 of 45

Defining Chronology Statements

Period

Date Start

Date End

37 of 45

Period

Date Start

Defining Chronology Statements

Statement (before/after/etc.)

Date End

Statement (before/after/etc.)

38 of 45

Period

Date End

Defining Chronology Statements

Date Start

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

39 of 45

Date End

Defining Chronology Statements

Date Start

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Period

40 of 45

Date End

Defining Chronology Statements

Date Start

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Period

cycle

41 of 45

Date End

Defining Chronology Statements

Date Start

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Between Statement

(before/after/etc.)

Period

cycle

cycle

cycle

cycle

reference

reference

reference

reference

cycle

42 of 45

Date End

Defining Chronology Statements

Date Start

1 year after

2 years before

2 months after

1 month before

immatriculation

spring

winter

w.sem.

spring

Luther's imatr.

1504

1500

Erasmus' graduation

sommer semester

43 of 45

Defining Chronology Statements

ChronoJSON

Compute

44 of 45

Complex Temporal Statements: ChronoJSON

Explicit and transparent storage and representation of uncertain and vague dates.

Actionable by design:

  • Chronology Statements are computed into ranges and available in all filter, analysis, and visualisation functionalities.
  • Chronology Statement options can be used as filters, e.g.: ‘show all letters starting two years after Ľudovít Štúr’s second move to Preßburg’.

Full documentation will be released soon. Guides on usage are available: https://nodegoat.net/guides/chronologystatements, https://nodegoat.net/guides/relationalchronologystatements.

45 of 45

@nodegoat / nodegoat.net / support@nodegoat.net

@LAB1100 / LAB1100.com / info@LAB1100.com