1 of 10

CIP Distributed Test Infrastructure

Chris Paterson

Renesas

Updated: 09/11/18

2 of 10

CIP Testing

  • Originally CIP defined 5 actions to be taken to fulfil the project’s testing requirements.
  • The first step was to create an easy to use test environment that could be downloaded and run by developers who want to run tests on a local platform – “Board at Desk”.
  • The remaining steps are to gradually expanded the functionality of B@D.

​

  • In parallel to B@D, CIP have been planning to create a distributed test environment…

CIP Distributed Test Infrastructure

2

3 of 10

CIP Distributed Testing

  • The trend in open source testing is moving towards a distributed architecture consisting of:
    • Automated software builds (release, push, time based etc.)
    • Common/shared test cases
    • Remote board farms
    • Centralised test reporting
  • The end goal is to improve code quality by detecting issues through continuous integration and testing.
  • The whole Linux community can benefit from shared tools, results and bug tracking.

CIP Distributed Test Infrastructure

3

4 of 10

CIP Distributed Testing Architecture

  • The block diagram on the following slide shows the current architecture I have been working towards.
    • As you can see, it’s very similar to AGL!
  • It would be good to get feedback from other CIP members for this approach, so we can all be on the same track.

CIP Distributed Test Infrastructure

4

5 of 10

CIP Distributed Testing Architecture - Draft

CIP Distributed Test Infrastructure

5

TEST SCHEDULER

LAVA Master

  • Test job scheduling
  • Resource allocation
  • Test results
  • Test logs

DUT CONTROL

LAVA Worker

  • DUT communication
  • Power management
  • Logging

DUT

Physical Hardware

  • CIP reference platforms
  • Member company platforms

TEST RESULTS

KernelCI / Mailing List

  • Test results
  • Test logs

CIP TESTS

Git

  • Test scripts
  • Test data

BUILD SERVER

Jenkins / GitLab (?)

  • CIP/LTS/mainline(?) kernels
  • Device trees
  • CIP Core
  • Other distributions

CIP SOURCE CODE

Git

  • CIP SLTS
  • CIP Core

CIP CONTRIBUTIONS

Git

  • Mailing list(s)
  • Pull requests (GitLab)
  • Patchwork

EXTERNAL SOURCE CODE

Git

  • Mainline kernel
  • LTS kernel
  • Other

EXTERNAL TESTS

Git

  • Real-time tests
  • LTP
  • kselftest
  • etc.

ARTIFACT STORAGE

Apache / Nexus / Artifactorial (?)

  • Binaries
  • Test applications
  • Test data

6 of 10

CIP Distributed Testing Architecture – Current Status

  • Renesas have already been working on:
    • Configuring the CIP LAVA master
    • Setting up a LAVA worker in our UK office
      • Currently with 2 devices connected:
        • QEMU
        • iwg21m (RZ/G1M)
        • iwg22m (RZ/G1E) – in progress
    • Daily healthchecks are running on these devices.

​

  • Links

CIP Distributed Test Infrastructure

6

7 of 10

CIP Distributed Testing Architecture – Current Status

CIP Distributed Test Infrastructure

7

TEST SCHEDULER

LAVA Master

  • Test job scheduling
  • Resource allocation
  • Test results
  • Test logs

DUT CONTROL

LAVA Worker

  • DUT communication
  • Power management
  • Logging

DUT

Physical Hardware

  • CIP reference platforms
  • Member company platforms

TEST RESULTS

KernelCI / Mailing List

  • Test results
  • Test logs

CIP TESTS

Git

  • Test scripts
  • Test data

BUILD SERVER

Jenkins / GitLab (?)

  • CIP/LTS/mainline(?) kernels
  • Device trees
  • CIP Core
  • Other distributions

CIP SOURCE CODE

Git

  • CIP SLTS
  • CIP Core

CIP CONTRIBUTIONS

Git

  • Mailing list(s)
  • Pull requests (GitLab)
  • Patchwork

EXTERNAL SOURCE CODE

Git

  • Mainline kernel
  • LTS kernel
  • Other

EXTERNAL TESTS

Git

  • Real-time tests
  • LTP
  • kselftest
  • etc.

ARTIFACT STORAGE

Apache / Nexus / Artifactorial (?)

  • Binaries
  • Test applications
  • Test data

8 of 10

CIP Distributed Testing Architecture – Next Steps

  • Set up centralised binary storage.
  • Report test results to the CIP KernelCI server.
  • Report test results to cip-test-results mailing list.
  • Start creating/adding test cases to the CIP test scripts repository.

​

  • Links

CIP Distributed Test Infrastructure

8

9 of 10

CIP Distributed Testing Architecture – Next Steps

CIP Distributed Test Infrastructure

9

TEST SCHEDULER

LAVA Master

  • Test job scheduling
  • Resource allocation
  • Test results
  • Test logs

DUT CONTROL

LAVA Worker

  • DUT communication
  • Power management
  • Logging

DUT

Physical Hardware

  • CIP reference platforms
  • Member company platforms

TEST RESULTS

KernelCI / Mailing List

  • Test results
  • Test logs

CIP TESTS

Git

  • Test scripts
  • Test data

BUILD SERVER

Jenkins / GitLab (?)

  • CIP/LTS/mainline(?) kernels
  • Device trees
  • CIP Core
  • Other distributions

CIP SOURCE CODE

Git

  • CIP SLTS
  • CIP Core

CIP CONTRIBUTIONS

Git

  • Mailing list(s)
  • Pull requests (GitLab)
  • Patchwork

EXTERNAL SOURCE CODE

Git

  • Mainline kernel
  • LTS kernel
  • Other

EXTERNAL TESTS

Git

  • Real-time tests
  • LTP
  • kselftest
  • etc.

ARTIFACT STORAGE

Apache / Nexus / Artifactorial (?)

  • Binaries
  • Test applications
  • Test data

10 of 10

Other Tasks

  • CIP Wiki update.
  • Blog post/presentation?

CIP Distributed Test Infrastructure

10