Logging samples and events�

Luke Marsden, UNIS

(data.nleg@unis.no)

10/08/2021

​

​

Starting at 10:02

Recording presentation

Picture taken by Andreas Wolden

Why do we log samples and events?

• Provide overview of all data collected (searchable on SIOS website).

​

• Labelled physically, logged electronically

​

• Log ‘virtual’ samples or events without samples

​

• Identify samples years in future

​

• Easier to publish data

​

UUIDs

• Universally unique ID

​

• 32 digit hex number, can be represented as data matrix for simplicity:

f69d4072-49be-11eb-b378-0242ac130002

​

• Chance of collision: 10^-14

​

​

Data matrix

Hierarchy

UUID #1

UUID #2

UUID #3

UUID #4

UUID #9

UUID #10

Station name

UUID #7

UUID #8

UUID #11

UUID #12

UUID #13

UUID #14

Activities

Samples

Subsamples

CTD, Box core, Drone, Dive…

Niskin bottles, ice cores…

UUID #5

UUID #6

Consider hierarchy before cruise, ask me if unsure

Hierarchy: Example 1

CTD

MSS

L-ADCP Master

L-ADCP Slave

P2 (NLEG04)

Activities

Niskin bottle #1

Niskin bottle #2

Samples

Subsamples

100 ml water

100 ml water

200 ml water

100 ml water

Hierarchy: Example 2

Dive missions

Drone flight

P3 (NLEG07)

Activities

Samples

Subsamples

100 ml water

Photo #1

Photo #2

Hierarchy: Example 3

Ice work

NLEG14 Ice

Activities

Samples

Subsamples

MSS through hole in ice

Weather station on ice

Ice core

Ice core 0-10cm

Ice core 10-20cm

Station names

• Check if station name has previously been used https://sios-svalbard.org/aen/stations

​

​

​

​

​

• Around 500 different stations

• Check spelling of station
• P4 (NLEG11)
• NLEG12

​

• Sensible naming of new stations
• Close to P4 (NLEG11) could be P4 (NLEG11) a
• Ice stations new station because of drift, e.g. P4 (NLEG11) ice

​

​

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Creating sample logs

• Before cruise if possible https://www.sios-svalbard.org/cgi-bin/darwinsheet/?setup=aen
• Descriptions tell you how to fill in each field
• New logs for each cruise
• Corrupted files
• Updates

​

​

​

Really important!

​

​

​

Labelling home page on cruise

http://10.3.65.20

How it works

• Hidden 3^rd row – don’t change this! Corresponds to standard name (Darwin Core usually)
• Columns created without 3^rd row will not be uploaded or checked.

​

​

​

​

​

Fill in what you can before cruise

Abstract like in paper. Could be used when publishing dataset.

Separate PIs with semicolon ;

All info for all PIs, even if from same institution

Fill in what you can before cruise

Know in advance

Sample type

Dropdown list frequently updated, ask me if something missing

Ideally should refer to physical thing, not a method

​

Consistency important between cruises

​

I can update log for past cruises

Sampling protocols

One of most important fields

Refer to version and section

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Labelling home page on cruise

http://10.3.65.20

Printing labels

Large: 25 x 51 mm.

• Data matrix + 5 lines text (20-25 characters per line)

​

Medium: 25 x 19 mm

• Fits Eppendorf tubes and large containers.
• Data matrix + 4 lines text (18 characters per line)

​

Small: 10 mm:

• Data matrix only.
• Pre-printed, located by printers.

​

Printers located in Education Lab.

​

​

Printing labels

​

Common causes of error:

​

• Not updating text when new labels printed

​

• Printing new labels and forgetting to scan them

​

​

​

​

​

​

​

​

​

Different on cruise, but automatically populated

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Scanning labels into sample logs

• Select cell in Excel
• Use QR code reader

​

​

​

​

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Logging virtual samples/events

Samples not preserved, or events without samples, need logging too.

​

​

​

​

​

​

Copy and paste into sample log

Get new UUID every time even if you only use one.

Need a lot of UUIDs?

If online: https://www.uuidgenerator.net/version1 (before, start or end of cruise?)

Other sites available

​

Or contact me

​

​

​

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Parent event IDs

UUID #1

UUID #2

UUID #3

UUID #4

UUID #9

UUID #10

Station ID

UUID #7

UUID #8

UUID #11

UUID #12

UUID #13

UUID #14

Activities

Samples

Subsamples

CTD, Box core, Drone…

Niskin bottles, ice cores…

UUID #5

UUID #6

Copy and paste relevant UUID

Consider hierarchy before cruise, ask me if unsure

Parent event IDs

Own copy of activity log using generator

​

​

​

​

​

​

Activities

Samples

Subsamples

CTD, Box core, Drone…

Niskin bottles, ice cores…

Consider hierarchy before cruise, ask me if unsure

Own copy of Niskin log using generator (new this cruise)

​

​

​

​

​

​

Recorded by your or your group

​

​

​

​

​

​

Labelling home page on cruise

http://10.3.65.20

Used to log activities

e.g. CTD, Trawls, sea ice work…

​

Activity log

• Pulls data from ‘Toktlogger’

​

• IMR and AeN using same UUIDs

​

​

Person in charge of data management on cruise only:

• Can update log and reupload – your changes will be preserved in new version
• Generate .txt file of raw data at end of cruise.

​

​

​

​

​

​

​

​

​

Gear/Activity log

Add manually:

• Station name,
• Sample depths
• PI details
• Sampling protocols
• Check gear types

​

​

Don’t run through checker

​

​

Gear/Activity log: How does one know which activity is which?

Gear/Activity log: How does one know which activity is which?

Instrument crew include comment for each activity.

​

The cruise leader can use this to make each activity identifiable

​

Used to populate ‘Event remarks’ in log

Niskin log

• Data from toktlogger and .btl files

​

• Takes few minutes

​

​

​

​

​

​

​

​

​

​

Niskin log

Don’t run through checker

​

​

If you need to create another column…

​

​

​

​

​

1. revisit the template generator page

2. Find what you need to add

3. Create new template

4. Copy and paste entire column from Excel sheet created into old sheet

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

Labelling home page on cruise

http://10.3.65.20

Checker

• Check all sample logs before end of cruise, until they pass.

​

• Identifies:
• Invalid metadata: wrong format, value not from drop-down list…
• Duplicate event IDs in same file
• Missing required columns

​

• Does not identify:
• Copy and paste errors
• Mismatches between physical samples (labels) and sample log
• Duplicate event IDs between different files

​

​

​

​

​

​

​

​

​

​

​

​

Checker

What do researchers need to do

1. Create sample log templates before cruise

​

• Print labels with UUIDs for all physical samples

​

• Scan labels into sample logs

​

• Log virtual samples or events without samples

​

• Add other metadata for each event

​

• Run logs through checker onboard before end of cruise. Fix errors until it passes.

​

• Deposit data in cruise folder

​

​

​

​

​

Deposit data

• Store data and sample logs on onboard cruise folder

​

• Let instrument crew know where data are – they have write access to cruise data folders on NIRD.

​

​

​

​

​

​

​

Sample logs in a folder with your name

Suggested file name

AeN_CruiseNumber_InstrumentName

Do not include:

• Multiple versions of the same file
• Files belonging to other people

Metadata catalogue

https://sios-svalbard.org/aen/tools

Have a good cruise!

data.nleg@unis.no

• Create in advance

• Ask me if any questions

• New files for new cruise