DSpace Developer Meet-Up and Q&A

​

1

Priorities for DSpace 8

IIIF Support and/or Basic Image/Video viewer

DSpace 8.0 had a shortened development cycle and DSpace Steering established four priorities

FEATURES INCLUDE

FEATURES INCLUDE

FEATURES INCLUDE

2

8.0 Statistics

IIIF Support and/or Basic Image/Video viewer

3

DSpace 8.0 Feature Preview (Part 1)

OpenAIRE Data Correction & Publication Claim,

COAR Notify, and more…

4

4

• Content exists elsewhere: assure that your repository showcases all the production of your community
• It is the result of the OpenAIRE ELD project awarded by 4Science back in 2020 and first implemented in DSpace-CRIS in 2021 now finally available in DSpace 8

​

A new researcher joins the institution and logins for the first time in the repository.

The publication claim service found most of their publications and prompts for import.

The researcher reviews the list, confirms the authorship and imports

the publication saving a significant amount of time….

​

​

5

DSpace 8 Features - Publication Claim

5

6

DSpace 8 Features - OpenAIRE Publication Claim

6

7

DSpace 8 Features - OpenAIRE Publication Claim

7

8

DSpace 8 Features - OpenAIRE Publication Claim

8

9

DSpace 8 Features - OpenAIRE Publication Claim

9

Configurable entities

​

​

​

Researcher Profile

Requires:

10

DSpace 8 Features - OpenAIRE Publication Claim

free icons from https://fontawesome.com/

10

• It uses the generic service Suggestion Service to store, retrieve and process suggestions related to content potential of interest for researchers affiliated to the repository in SOLR (new suggestion core)
• It is open to future extensions by mean of additional suggestion providers (i.e. ORCID)
• The openAIRE provider includes a framework to evaluate the quality of the suggestions (i.e. assign a score)

11

DSpace 8 Features - OpenAIRE Publication Claim

11

• Show reliable and accurate information in your repository
• It is the result of two innovative projects, the previous mentioned OpenAIRE ELD project and the ELD ADVANCE project supported by the European Open Science Cloud Digital Innovation Hub

​

A repository manager of a repository indexed in OpenAIRE

login in the repository and see the list of improvements and

corrections identified, among others one publication that has a PMID that

was unknown to the repository and a link to a project.

They click on the “accept the suggestion” button and the new information

is stored in the local record.

​

​

12

DSpace 8 Features - OpenAIRE Data Correction

12

13

DSpace 8 Features - OpenAIRE Data Correction

13

14

DSpace 8 Features - OpenAIRE Data Correction

14

15

DSpace 8 Features - OpenAIRE Data Correction

15

Your repository must be registered as an OpenAIRE provider

​

Configurable entities

​

Researcher Profile

​

Quality Assurance

Requires:

16

DSpace 8 Features - OpenAIRE Data Correction

free icons from https://fontawesome.com/

16

• It uses a generic service QAEventService to store, retrieve and process corrections related to content already in the repository, data are stored in SOLR (new qaevent core)
• It is open to future extensions by mean of additional correction providers (see next slides)
• It includes a framework to automatically process corrections according to configurable rules

17

DSpace 8 Features - OpenAIRE Data Correction

17

18

DSpace 8 Features - DSpaceUsers Data Correction

18

19

DSpace 8 Features - DSpaceUsers Data Correction

19

• It is another provider of the Data Correction functionality just presented, it generates QAEvent
• It collects correction proposals by the DSpace Users (DSpaceUsers source), currently the following CorrectionType(s) are provided:
• request to withdrawal
• request to reinstate
• Thanks to the previous mentioned processing framework it allows automatic approval of request according to flexible rules (based on the ItemFilter framework)

20

DSpace 8 Features - DSpaceUsers Data Correction

20

21

DSpace 8 Features - COAR Notify

I have a publication that uses your dataset

Link to a dataset is in the metadata

check if automatic

notification should

be triggered

LDNMessageConsumer

Link the dataset back to DSpace

21

22

DSpace 8 Features - COAR Notify

22

23

DSpace 8 Features - COAR Notify

23

24

DSpace 8 Features - COAR Notify

24

25

DSpace 8 Features - COAR Notify

25

26

DSpace 8 Features - COAR Notify

26

27

DSpace 8 Features - COAR Notify

27

28

DSpace 8 Features - COAR Notify

28

29

DSpace 8 Features - ROR Integration

29

30

DSpace 8 Features - ROR Integration

30

DSpace 8.0 Feature Preview (Part 2)

Improvements to Processes, embedded Search, Duplicate Detection, Administrative Reports, and more…

31

31

Processes page redesigned

32

32

Processes page redesigned

33

33

Processes page redesigned

34

34

Processes page redesigned

35

35

Basic Duplicate detection

Basic detection mechanism to prevent duplicate items in the repository.

• Compares item titles in SOLR with the submitted item’s title
• Configurable degree of variation (uses levenshtein distance)

​

Duplicates can be found:

• By submitters or editors via a submission step to warn submitters or editors of potential duplicates
• By Workflow reviewers or editors through a warning for claimed and pooled tasks indicating the total number of potential duplicates.
• Through a new /duplicates REST item link which will search and retrieve a paged list of potential duplicates for any item

36

36

Basic Duplicate detection

37

37

Basic Duplicate detection

38

38

Search tab on Community/Collection pages

39

39

Search tab on Community/Collection pages

40

40

Search Facets on Homepage

41

41

Advanced Search filters

42

42

Advanced Search filters

43

43

Add relationships via external sources

44

44

Administrator Reports (beta)

DSpace 8 will introduce basic administrator reports. There are two reports available:

​

​

• Filtered Collection reports: these reports count the numbers of item per community/collection. Administrators will also have access to a number of filters they can apply to narrow down the result.

​

• Metadata Query reports: These will provide item counts based on metadata queries to specify what should be counted and what not (and some other available filters).

​

​

​

45

45

Administrator Reports (beta)

46

46

Bug fixes

https://github.com/orgs/DSpace/projects/27/views/2?filterQuery=done+label%3Abug

​

​

47

47

Overview of Upgrade process from v7 to v8

overview of backend major changes

48

48

Upgrading underlying DSpace 8 dependencies

• Java 17 (OpenJDK or Oracle)
• Apache Maven 3.8.x or above
• Tomcat is now OPTIONAL, if you want to use it you need version 10. You may instead choose to use the Runnable JAR approach to deploying the DSpace 8 backend.

java -jar [dspace]/webapps/server-boot.jar

​

• PostgreSQL 12.x - 15.x (with pgcrypto installed) - unchanged
• SOLR 8.11.1+ - unchanged

in bold the version currently used by the CI and dockerfile

49

49

How to update the server.xml with the embedded Tomcat?

DSpace installation requires you to configure the URIEncoding=”UTF-8” attribute over the Connector in the server.xml luckily this is not required with the embedded tomcat as it is configured by default

Good default are provided also for the X-Forwared-For Header and to trust internal proxies making available out-of-box a production ready configuration.

If you need to customize other aspect of the server.xml (port, extra connector, etc.) you can do that using env properties and/or modifying the spring boot application.properties file

Spring boot reference documentation

50

50

Update your code base

• For upgrade from version prior of DSpace 7 there are some extra steps required (older is your version more extra steps are needed) be sure to look closely to the documentation
• if you are coming from a version prior of DSpace 7 consider to start with a fresh codebase and manually port (reconsider) your changes
• if you are already on DSpace 7 you can merge or rebase your change on top of DSpace 8 (there is a GitHub issue to improve the merge experience)
• include in the new local.cfg your changes or merge the change in the configuration files with your, for example

db.dialect = org.dspace.util.DSpacePostgreSQLDialect

51

51

Major changes in the backend code

• JAVA 11 → 17:
• most javax.* dependencies migrated to jakarta.*
• Hibernate 5 → 6:
• HibernateProxyHelper is now in the org.dspace.core package
• Query created via HQL/JPQL have been updated to use CriteriaBuilder
• Spring boot 2 → 3
• Spring framework 5 → 6

52

https://github.com/DSpace/DSpace/pull/9321

52

Major changes in SOLR

• There are no major changes 🙂 still 8.11.x
• Two new cores have been introduced: suggestion, qaevent. The could contain data not available elsewhere (decision taken over the proposed suggestion or correction) so it is a good idea to include them in your backup strategy

53

53

Overview of Upgrade process from v7 to v8

overview of frontend major changes

54

54

• 1. Merge & fix conflicts
• 2. Upgrade node & install dependencies
• 3. Upgrade to standalone components
• 4. Run ESLint to apply the new rules

​

Upgrading the UI from 7 to 8 - Overview

55

Checkout the DSpace 8 tag from the community github

​

​

Merge in your DSpace 7 branch

​

​

​

1. Merge & fix conflicts

$ git checkout -b dspace-8-upgrade dspace-8.0.0

$ git merge prod

56

Why in that order?

• Ensures that your customizations show up as changes in the merge commit
• You’ll know why those changes are there
• You can re-evaluate if they are still necessary

​

1. Merge & fix conflicts

57

• Fix any merge conflicts you might have with custom code
• For lists (e.g. of injected services in a constructor, or config properties etc), it’s good practice to put the community changes on top, and yours at the bottom
• Leads to fewer merge conflicts down the line, because new community changes will happen above your changes, not in between

​

1. Merge & fix conflicts

58

• 1. Merge & fix conflicts
• 2. Upgrade node & install dependencies
• 3. Upgrade to standalone components
• 4. Run ESLint to apply the new rules

​

Upgrading the UI from 7 to 8 - Overview

59

Switch to node 18 or 20 if you haven’t already

​

​

​

Install the dependencies

​

​

​

2. Upgrade node & install dependencies

$ nvm install 20�$ nvm use 20

$ yarn run clean

$ yarn install

60

• 1. Merge & fix conflicts
• 2. Upgrade node & install dependencies
• 3. Upgrade to standalone components
• 4. Run ESLint to apply the new rules

​

Upgrading the UI from 7 to 8 - Overview

61

PR #2750 converted the UI to use standalone components:

• Every component specifies its own dependencies
• They don’t need to be declared in a module
• Can be more easily reused in a different context
• Can be lazy-loaded separately, so better for performance

​

3. Upgrade to standalone components

62

Differences to the component decorator:

• Property standalone: true
• Optional list of imports

3. Upgrade to standalone components

@Component({

selector: 'ds-base-login-page',

styleUrls: ['./login-page.component.scss'],

templateUrl: './login-page.component.html',

standalone: true,

imports: [ThemedLogInComponent, TranslateModule],

})

export class LoginPageComponent implements OnDestroy, OnInit {

63

More info on the wiki: https://wiki.lyrasis.org/display/DSPACE/Migration+to+Standalone+Components

3. Upgrade to standalone components

64

Convert your custom components to standalone. Path is optional

​

​

Choose the option: “Convert all components, directives and pipes to standalone”

​

​

​

3. Upgrade to standalone components

? Choose the type of migration: (Use arrow keys)

❯ Convert all components, directives and pipes to standalone

Remove unnecessary NgModule classes

Bootstrap the application using standalone APIs

$ ng generate @angular/core:standalone --path src/themes/your-theme-name

65

The script will sometimes not add all required imports to the component decorator

Compare the list of imports in your themed components with the custom theme.

E.g. after the tool

​

3. Standalone components - Fix imports

@Component({

selector: 'ds-footer',

styleUrls: ['footer.component.scss'],

templateUrl: 'footer.component.html',

standalone: true,

imports: [NgIf, RouterLink]

})

66

The custom theme version:

​

​

​

​

​

So copy those imports to your version

3. Standalone components - Fix imports

@Component({

selector: 'ds-footer',

styleUrls: ['footer.component.scss'],

templateUrl: 'footer.component.html',

standalone: true,

imports: [NgIf, RouterLink, AsyncPipe, DatePipe, TranslateModule]

})

67

If you’ve added additional components to your templates and they haven’t been detected by the script you’ll get errors like this, when you build:

​

​

​

​

​

​

The solution is then to add the corresponding component class to the imports:

3. Standalone components - Fix imports

Error: src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html:3:5 - error NG8001: 'ds-thumbnail' is not a known element:

1. If 'ds-thumbnail' is an Angular component, then verify that it is included in the '@Component.imports' of this component.

2. If 'ds-thumbnail' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the '@Component.schemas' of this component to suppress this message.

​

3 <ds-thumbnail [thumbnail]="(object.logo | async)?.payload">

​

imports: [NgIf, RouterLink, ThemedThumbnailComponent]

68

Then the next error is about the async pipe:

​

​

​

​

So add that as well

​

​

​

3. Standalone components - Fix imports

Error: src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html:3:47 - error NG8004: No pipe found with name 'async'.

3 <ds-thumbnail [thumbnail]="(object.logo | async)?.payload">

​

imports: [NgIf, RouterLink, ThemedThumbnailComponent, AsyncPipe]

69

Find the correct component class by ctrl/cmd clicking the HTML in your IDE

If that doesn’t work, search for “selector: 'ds-name-of-selector'“

3. Standalone components - Fix imports

@Component({

selector: 'ds-thumbnail,

styleUrls: [],

templateUrl: '../shared/theme-support/themed.component.html',

standalone: true,

imports: [ThumbnailComponent]

})

export class ThemedThumbnailComponent extends ThemedComponent<ThumbnailComponent>

70

• 1. Merge & fix conflicts
• 2. Upgrade node & install dependencies
• 3. Upgrade to standalone components
• 4. Run ESLint to apply the new rules

​

Upgrading the UI from 7 to 8 - Overview

71

PR #2343 added stricter ESlint rules

Goals:

• fewer merge conflicts
• Imports in alphabetical order
• multiple imports from same file on separate lines
• enforce trailing commas

​

​

4. ESLint - New rules

72

Goals:

• Remove ambiguity
• disallow !(value$ | async) in Angular templates
• disallow != and == in Angular templates
• disallow “harmful” types
• E.g. use string instead of String or () => {} instead of Function

​

​

4. ESLint - New rules

73

Goals:

• Follow RxJS recommendations
• Don’t import from internal modules
• Instead of���Instead of��

​

4. ESLint - New rules

import { of as observableOf } from 'rxjs/internal/observable/of';

import { of as observableOf } from 'rxjs';

74

Goals:

• Follow RxJS recommendations
• Don’t nest subscribes
• Instead of�����Use�

​

4. ESLint - New rules

obs$.pipe(

switchMap((v1) => this.someService.someMethod(v1)),

).subscribe((v2) => console.log(v2));

obs$.subscribe((v1) =>

this.someService.someMethod(v1).subscribe((v2) => console.log(v2))

);

75

Run lint-fix to automatically fix most issues

​

​

You may get a few errors that can’t be fixed automatically:

​

​

4. ESLint - Running ESLint

$ yarn run lint-fix --quiet

src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html�6:24 error Async pipe results should not be negated. @angular-eslint/template/no-negated-async

7:15 error Expected `!==` but received `!=` @angular-eslint/template/eqeqeq

7:54 error Expected `===` but received `==` @angular-eslint/template/eqeqeq

76

“Async pipe results should not be negated”

​

�Reason: isLoading$ could be undefined, null or 0 too

�Fix it by explicitly checking if it’s false

​

​

4. ESLint - Fix issues

<div *ngIf="!(isLoading$ | async)">

<div *ngIf="(isLoading$ | async) === false">

77

“Expected `!==` but received `!=`” or “Expected `===` but received `==`”

​

�Reason: != and == follow complicated rules. ��E.g. 100 == ‘100’ would be true �true == ‘true’ would be false

​

Fix it by using !== and ===

​

​

4. ESLint - Fix issues

<a *ngIf="linkType != linkTypes.None" [target]="(linkType == linkTypes.ExternalLink) ? ...

<a *ngIf="linkType !== linkTypes.None" [target]="(linkType === linkTypes.ExternalLink) ? ...

78

In DSpace 7, to make MyComponent themeable:

• Create ThemedComponent<MyComponent>
• Use selector ds-themed-my-component
• Replace all usages of ds-my-component with ds-themed-my-component
• Easy to forget a place it’s used
• When you forget, the app keeps working
• You’ll only notice when you theme it and realize the themed version isn’t used in a certain context

​

4. ESLint - New themed component rules

79

In DSpace 8:

• Create ThemedComponent<MyComponent>
• But use the original selector ds-my-component
• Replace the selector of MyComponent with ds-base-my-component
• All existing usages now refer to the themed version automatically
• When using the component in a theme use ds-themed-my-component
• This is a convection: not strictly necessary
• The selector for a themed version of a component is never used directly

​

4. ESLint - New themed component rules

80

• PR #2865 adds an ESLint plugin to enforce these rules
• If you run yarn run lint-fix it will automatically convert from the old style to the new one

​

​

becomes:

​

​

4. ESLint - New themed component rules

<ds-themed-thumbnail [thumbnail]="(object.logo | async)?.payload">

<ds-thumbnail [thumbnail]="(object.logo | async)?.payload">

81

becomes:

​

​

4. ESLint - New themed component rules

@Component({

selector: 'ds-themed-thumbnail,

...

})

export class ThemedThumbnailComponent extends ThemedComponent<ThumbnailComponent>

@Component({

selector: 'ds-thumbnail,

...

})

export class ThemedThumbnailComponent extends ThemedComponent<ThumbnailComponent>

82

becomes:

​

​

4. ESLint - New themed component rules

@Component({

selector: 'ds-thumbnail,

...

})

export class ThumbnailComponent implements OnChanges {

@Component({

selector: 'ds-base-thumbnail,

...

})

export class ThumbnailComponent implements OnChanges {

83

Q&A

and

Open Discussion

84