1 of 42

Intro TestKube

Gonzalo Izquierdo - SysEleven

2 of 42

About me:

Gonzalo Izquierdo

Software Engineer/SRE- SysEleven

  • All started with a bunch of Raspis… Great times
  • Working as SRE/DevOps focused on cloud/K8s
  • Maintainer of “clusters of clusters” with huge number of “child” clusters.
  • Using/learning K8s since it was a “cool word” and it was implemented everywhere, for everything.
  • Always trying to use OSS. OSS is the way.
  • Always learning, curious by default.

3 of 42

Scope of this talk:

  • Have a brief intro about TestKube as possible candidate as testing framework.

  • Checkout main components/features. Have a “general overview” of how it works to check if makes sense in our system/infra.

  • Show other ways to integrate testing in a more independent way.

  • Uncouple the testing part from OPS team. Try to enhance automation for testing.

  • Give an overview of some examples of use cases.

4 of 42

What is TestKube

“TestKube is an open-source, cloud-native testing framework which simplifies the testing of Kubernetes applications. It allows users to store, execute and manage tests as part of a Kubernetes cluster. With Testkube, users can utilize multiple testing platforms, orchestrate tests and perform automated testing.”

5 of 42

Some interesting features of Testkube.

  • TestKube allow to create its resources as Kubernetes components.
  • It allows to use multiple testing integrations (executors) like Postman collections, K6, Cypress, etc.
  • Allows to run tests in our infra instead of launching tests from outside (reduced costs and enhanced security).
  • Easy to integrate with some CI/CD systems like Jenkins, ArgoCD, etc.
  • Gives us the opportunity to simplify our Pipelines and delegates the testing workload to TestKube/K8s in a sync or async way.
  • Many of the components of TestKube core are free and OSS.

6 of 42

Testkube Integrations

More info at:

7 of 42

TestKube architecture

(krew plugin)

(dashboard)

8 of 42

TestKube storage

9 of 42

SOME “OLD WAY OF TESTING MODES” EXAMPLES

10 of 42

Example of testing, I:

Local machine/laptop (true story)

11 of 42

Example of testing, II:

12 of 42

Example of testing, III:

13 of 42

TestKube

TESTING/FEATURES

14 of 42

How TestKube components are installed?

The simplest way: Helm (customizable values for some features like Minio or similar)

After having all the components running in our cluster, we can interact using:

  • Dashboard. Requires a Pro plan (free plans available, without advanced support and blablabla) after version 1.17, their subscription plans changed one month ago… Looks cool, but this will not block us.

  • TestKube cli (part of the OSS): It interacts with the TestKube API, allowing us to do more things than using the dashboard, and we like to automate things so…

… We will see many pics of dashboard, but it can be made also with testkube CLI/CRDs :)

15 of 42

How to create/run a simple test (using CLI):

$ testkube create test --name curl-test --type curl/test --test-content-type git-file --git-uri https://github.com/userHere/testkube-demo.git --git-branch main --git-path test/curl/hello-test.json

$ testkube run test curl-test

$ testkube get execution curl-test-1

16 of 42

TestKube CLI

17 of 42

TestKube Dashboard

18 of 42

TestKube functions/components

  • Tests: The base objects based on CURL, Postman, K6, etc.
  • TestSuites: Orchestration for different test steps that allows us to run them sequentially or in parallel.
  • Triggers: Configurable actions that allows us to run tests automatically.
  • Webhooks: Similar to triggers but invoked via HTTP POST call.
  • Global source definition: Base repo for our tests.
  • Status page: Dedicated Status Pages for different components that allows us to display the status of our apps.

19 of 42

Example test with K6:

import http from 'k6/http';

import { check, sleep } from 'k6';

export const options = {

stages: [

{ duration: '30s', target: 20 },

{ duration: '1m30s', target: 10 },

{ duration: '20s', target: 0 },

],

};

export default function () {

const res = http.get('https://httpbin.test.k6.io/');

check(res, { 'status was 200': (r) => r.status == 200 });

sleep(1);

}

sample-test.js

20 of 42

Components as K8s resource. Test (from string):

apiVersion: tests.testkube.io/v3

kind: Test

metadata:

name: testing-curl

namespace: testkube

labels:

executor: curl-executor

test-type: curl-test

spec:

type: curl/test

content:

type: string

data: "{\n \"command\": [\n \"curl\",\n \"https://example.org\",\n \"-H\",\n \"'Accept: application/json'\"\n ],\n \"expected_status\": \"200\"\n}"

21 of 42

Components as K8s resource. Test (from Git):

apiVersion: tests.testkube.io/v3

kind: Test

metadata:

name: k6-demo

namespace: testkube

spec:

type: k6/script

content:

type: git-file

repository:

type: git

uri: https://github.com/user/testing.git

branch: main

path: tests/k6/sample-test.js

22 of 42

Test example on UI:

23 of 42

Triggers on TestKube

Triggers allows us to execute tests when an event happens to certain K8s resources. Examples:

  • An image is updated on a Deployment.
  • A ConfigMap changes it’s data.
  • A service its created.
  • Etc.

24 of 42

Trigger example on UI:

25 of 42

Triggers as K8s resource:

apiVersion: tests.testkube.io/v1

kind: TestTrigger

metadata:

name: testing-demo

namespace: testkube

spec:

action: run

event: modified

execution: test

resource: deployment

resourceSelector:

name: testkube-demo-blue

namespace: default

testSelector:

name: hello-k8s

namespace: testkube

26 of 42

TestSuites

Testsuites are the feature that allows us to orchestrate multiple tests in a simple way. Example:

“ We need to execute the backend testing first with Postman and just after that, check that frontend works as expected with Cypress tests.”

TestSuites allows:

  • QA team organize the flow for their test execution.
  • Allow parallel processing of tests.
  • Acts “similar” to a Pipeline, but for testing.
  • It allows us to add delays between tests, allowing us to do some checks before executing the next test steps.

27 of 42

TestSuites on UI

28 of 42

TestSuite as K8s resource/CLI:

apiVersion: tests.testkube.io/v3

kind: TestSuite

metadata:

name: test-demo

namespace: testkube

spec:

steps:

- execute:

- delay: 0s

test: hello-k8s

- delay: 0s

test: testing-curl

stopOnFailure: false

- execute:

- delay: 10s

stopOnFailure: false

- execute:

- delay: 0s

test: k6-demo

stopOnFailure: false

29 of 42

WebHooks:

TestKube allows to setup webhooks that can be integrated with third party systems if some action is executed, example:

  • Send a message to MS Teams if a test start, ends successfully or it ends with failure.
  • Send alerts to OPSGenie/Pagerduty.
  • Send alerts and notifications to your monitoring and observability tools like Prometheus and Grafana.

Mainly used for communication/info purposes about testing execution/events.

30 of 42

Sources:

  • Manage in a centralized way sources for our tests
  • Uses that sources as “Source” on test creation to simplify management.

Example CR:

apiVersion: tests.testkube.io/v1

kind: TestSource

metadata:

name: test-demo

namespace: testkube

spec:

repository:

type: git

uri: https://github.com/username/testkube-demo

31 of 42

Status Pages (Pro):

  • Gives an overview of how components are passing (or not) the testing. This can be useful for PO/Team leads to check the current status of the development, to improve product, etc.

  • Incidents can be added also to the StatusPages.

  • They can be customized.

  • Can be public or private.

32 of 42

Executors:

“The Testkube Container Executor allows you to run your own container images for executing tests. Testkube orchestrates the Tests using the container image as Kubernetes Jobs.”

kubectl -n testkube describe job 6609[…]

Containers:

6609b5909da8aaa5718b8267:

Image: kubeshop/testkube-postman-executor:1.16.39

Port: <none>

Host Port: <none>

Command:

/bin/runner

{"id":"6609b5909da8aaa5718b8267","testName":"hello-k8s","testNamespace":"testkube","testType":"postman/collection","name":"hello-k8s-17","number":17,"command":["newman"],"args":["run","\u003crunPath\u003e","-e","\u003cenvFile\u003e","--reporters","cli,json","--reporter-json-export","\u003creportFile\u003e"],"content":{"type":"string","data":"{\n \"info\": {\n \"_postman_id\": \"02c90123-318f-4680-8bc2-640adabb45e8\",\n \"name\": \"Testkube Demo\",\n \"schema\": \"https://schema.getpostman.com/json/collection/v2.1.0/collection.json\"\n },\n \"item\": [\n {\n \"name\": \"hello-k8s test\",\n \"event\": [\n {\n \"listen\": \"test\",\n \"script\": {\n \"exec\": [\n \"pm.test(\\\"Body matches string\\\", () =\u003e {\",\n \" pm.expect(pm.response.text()).to.contain(\\\"Hello Kubernetes\\\")\",\n \"})\",\n \"\",\n \"pm.test(\\\"Body matches string\\\", () =\u003e {\",\n \" pm.expect(pm.response.status).to.equal(\\\"OK\\\")\",\n \"})\"\n ],\n \"type\": \"text/javascript\"\n }\n }\n ],\n \"request\": {\n \"method\": \"GET\",\n \"header\": [],\n \"url\": {\n \"raw\": \"http://hello-kubernetes-service.default\",\n \"protocol\": \"http\",\n \"host\": [\"hello-kubernetes-service\", \"default\"]\n }\n },\n \"response\": []\n }\n ]\n}\n"},"startTime":"2024-03-31T19:12:16.962984859Z","endTime":"0001-01-01T00:00:00Z","executionResult":{"status":"running"},"labels":{"app.kubernetes.io/instance":"testkube-tests","executor":"postman-executor","test-type":"postman-collection"},"bucketName":"958b2a4a-a2e0-4eca-98c3-606b50318dfb","runningContext":{"type":"user-cli"}}

33 of 42

Executors CRD (Integrated Postman)

apiVersion: executor.testkube.io/v1

kind: Executor

metadata:

name: postman-executor

spec:

types:

- postman/collection

executor_type: job

image: kubeshop/testkube-postman-executor:1.16.39

args:

- "run"

- "<runPath>"

- "-e"

- "<envFile>"

- "--reporters"

- "cli,json"

- "--reporter-json-export"

- "<reportFile>"

command:

- "newman"

features:

- artifacts

content_types:

- string

- file-uri

- git-file

- git-dir

- git

meta:

iconURI: postman

docsURI: https://kubeshop.github.io/testkube/test-types/executor-postman

$ kubectl get executor --all-namespaces

NAMESPACE NAME AGE

testkube artillery-executor 11d

testkube curl-executor 11d

testkube cypress-executor 11d

testkube ginkgo-executor 11d

testkube gradle-executor 11d

testkube jmeter-executor 11d

testkube jmeterd-executor 11d

testkube k6-executor 11d

testkube kubepug-executor 11d

testkube maven-executor 11d

testkube playwright-executor 11d

testkube postman-executor 11d

testkube soapui-executor 11d

testkube tracetest-executor 11d

testkube zap-executor 11d

34 of 42

Custom Executors

testkube create executor --name curl-container-executor --executor-type container --image curlimages/curl:7.85.0 --types curl-container/test --command curl

apiVersion: tests.testkube.io/v3

kind: Test

metadata:

name: test-demo-custom-exec

namespace: testkube

spec:

type: curl-container/test

executionRequest:

args:

- https://example.org/

$ testkube run test test-demo-custom-exec

35 of 42

DEMO TIME

36 of 42

Source control for testing files (ArgoCD + TestKube Plugin):

apiVersion: argoproj.io/v1alpha1

kind: Application

metadata:

name: testkube-tests

namespace: argocd

spec:

project: default

source:

repoURL: https://github.com/lalotone/testkube.git

targetRevision: HEAD

path: tests/postman

plugin:

name: "testkube-v1.0"

destination:

server: https://kubernetes.default.svc

37 of 42

Trigger example with CI/CD: ArgoCD + TestKube

38 of 42

Trigger example with CI/CD (Recap): ArgoCD + TestKube

  • User pushes changes to Git repo (App)
  • CI Pipeline gets executed and new image is built with new app version (with integration testing).
  • CI/CD Pipeline modifies the Helm chart with the new image version to deploy.
  • ArgoCD checks that some changes were applied on Helm Chart.
  • ArgoCD syncs the new changes and deploy them in cluster (new App version).
  • TestKube (controller) is listening for new changes (trigger).
  • TestKube triggers the executor with the required action.
  • Test is run.
  • We can push a notification to Teams/Slack with the test results.

39 of 42

Source control for testing files:

├── README.md

└── tests

├── k6

│ └── sample-test.js

└── postman

└── hello-k8s.json

Example content for repo:

  • Test “types” can be easily organized on folders and also referred in a separated way on sync process with K8s resources.
  • Testing team can work/maintain their tests independently.
  • Possibility to attach this repo to auto-update the test manifest on K8s for next runs.
  • All the advantages of SCM to maintain/version our tests.

40 of 42

TestKube Monitoring (Prometheus + Grafana)

  • Exposes metrics natively on TestKube API /metrics endpoint
  • Easy dashboards for Grafana integration. Metrics pretty straightforward.
  • Possible Alert integration with Prometheus alert manager.

41 of 42

Conclusions

Pros:

  • TestKube is a nice framework for testing, it’s flexible, easy to use and allows to be customized.
  • Allows multiple integrations with CI/CD systems.
  • TestSuites can be pretty useful to orchestrate the tests and combine to have a complete testing-stack in an organized way.
  • Triggers allows us to improve the automation side to launch triggers in an async-way without human interaction.
  • Allows interesting tools like StatusPages to inform teams about system performance.

Cons:

  • Some features are more “Pro-license” like like organization management (RBAC also).
  • Need to be conscient that all the testing part should be well-organized to avoid chaos.

42 of 42

Thanks for your attention :)

Any question?