1 of 21

Making Cloud Interoperability across OpenStack vendors a possibility using RefStack

Catherine Diep, Solutions Architect & RefStack PTL, IBM

Chris Hoge, Interop Engineer, OpenStack Foundation

Paul Van Eck, Software Developer, IBM

Sergey Slipushenko, Software Developer, Mirantis

​

​

​

​

April 25, 2016 (Updated June 3, 2016)

OpenStack Summit, Austin

2 of 21

Agenda

  • DefCore and Interoperability
  • Introducing RefStack
  • Testing with refstack-client
  • RefStack website
  • Tempest configuration for RefStack test
  • Future plans

3 of 21

OpenStack, Interoperability, and DefCore

https://wiki.openstack.org/wiki/Governance/DefCoreCommittee

  • OpenStack Mission:
    • Ubiquitous
    • Open Source
    • Cloud Computing Platform
  • Ubiquity through Interoperable Interfaces that are:
    • Understood
    • Durable
    • Unrestricted
  • DefCore is the OpenStack Interoperability Project
    • Testing Public Clouds, Private Clouds, and Distributions since 2015.
    • The Future is Now!

4 of 21

Defcore Guidelines

  • The DefCore Committee creates Guidelines which products must adhere if they use the OpenStack Powered logo.

​

  • Guidelines define Components consisting of
    • API Capabilities that a product must expose.
      • Each Capability includes a set of must-pass tests.
    • Designated Sections, OpenStack code that products must contain to provide the capabilities.

5 of 21

How You Can Participate

Get the Powered Logo for your OpenStack distribution, public cloud, or private cloud.

​

Run refstack against your installation and share your results with the OpenStack Foundation and community.

6 of 21

RefStack

There are two parts in the Refstack project:

  • RefStack Client is a command line tool that runs test tools (Tempest or non-Tempest), summarizes results, and communicates with the refstack api. Vendors can use refstack client to run the tests themselves in a controlled and repeatable way.
  • RefStack Server
    • RefStack API is a central repository for the collection of results and is currently hosted at https://refstack.openstack.org/. It can also be hosted locally within any organization's infrastructure.
    • RefStack UI is a web interface for interacting with data collected with the api and client ( https://refstack.openstack.org/ ).

A client and server for OpenStack interoperability testing. Data collected by these tools can be used to validate against the DefCore Guidelines.

7 of 21

RefStack Client Installation

​

  • RefStack provides a one-step setup command (setup_env) which takes care of installation of all dependencies including Tempest. (https://github.com/stackforge/refstack-client/blob/master/README.rst)
  • By default, RefStack will install a specific Tempest SHA which was used by the DefCore committee to define the must-pass tests. RefStack also provides options allowing users to override the default Tempest version.
  • Once refstack-client is installed, Tempest is located at <refstack-client home directory>/.tempest.

​

8 of 21

Testing & Uploading Data

  • Running tests
    • Run a full Tempest API test set with the following command:

./refstack-client test -c <path your tempest.conf file> -v

    • Run only the tests listed in a file with the following command:

./refstack-client test -c <path your tempest.conf file> -v

--test-list <your test-list file>"

​

  • Upload data anonymously
    • Once you’ve completed a test run and you are happy with the results, you can upload it to the API server.

./refstack-client upload <Path of results file>

    • This defaults to uploading the data to refstack.openstack.org. However, you can override that by using the --url flag if you want to keep track of results on your own running API server.

​

​

9 of 21

Uploading data with signature

  • Benefits
    • Data management
      • Decision of sharing or not sharing
      • Deleting the data etc.�
  • Upload data with signature
    • Log in and import your keys to https://refstack.openstack.org

(Instruction https://github.com/openstack/refstack/blob/master/doc/uploading-private-results.md)

    • Upload data

./refstack-client upload -v your-refstack-test-result.json -i your-private-key-file

​

​

​

10 of 21

RefStack Website

https://refstack.openstack.org

​

- Users can view and manage test results.

​

- Allows users to compare their results to DefCore guidelines.

11 of 21

Viewing DefCore Guidelines

Click here to get test list matching selected capabilities.

12 of 21

Getting Test List for Testing

The test list here can be used with refstack-client using the --test-list argument.

​

When generating the test list, it is safer to keep aliases included to account for any test name changes. Tests in the list that do not exist in Tempest will simply be ignored.

13 of 21

Interpreting Test Results

On the results report page for each test run, a user can select multiple guideline versions and target programs.

​

Compliance with the selected guideline and target program is denoted by the percentage and the colored YES/NO.

Can view all passed tests by clicking on the count.

14 of 21

Associating Test Results

Can associate results to your target guideline and target program.

​

This will make the report page for your results default to these selections when calculating and displaying your compliance.

​

Otherwise the latest guideline will be used with the ‘Platform’ target.

15 of 21

Tempest configuration for RefStack Test

  • Tempest Configuration Guide can be found at: http://docs.openstack.org/developer/tempest/configuration.html
  • Tempest default settings
    • Tempest assigns default values to most of the configuration parameters [1].
    • A few of these default values needed to be altered for RefStack test.
    • For example, the default value for “resize” is set to “false”. If you want to run the resize related test cases included in the DefCore must pass test list, the “resize” parameter should be changed to “true”.
  • Enabling services
    • Make sure to enable the service you want to test.
    • For example, the default value for Neutron service is “false”.
  • User accounts
    • Use the accounts.yaml files to configure pre-provisioned users (locking test counts) .
    • Only one user account is needed.
    • User with “admin” role is not required.
  • Images
    • Two different images ID are needed for image_ref and image_ref_alt.
    • Some tests will be skipped if the two image IDs are identical. It’s ok to upload the same image twice.

​

16 of 21

Common tempest.conf settings for RefStack (1)

Following are the known common configuration parameters that need update.

​

  • [auth]
    • use_dynamic_credentials = false
    • test_accounts_file = { your accounts.yaml file}
  • [compute]
    • fixed_network_name = { shared network name if applicable} **
    • flavor_ref = { flavor 1 }
    • flavor_ref_alt = { flavor 2 }
    • image_ref = { image ID 1}
    • image_ref_alt = { image ID 2}
  • [compute-feature-enabled]
    • resize = true
  • [identity]
    • uri = { your v2 Keystone URL }
    • uri_v3 = { your v3 Keystone URL }
    • auth_version = { v2 or v3 , default is v2 }

​

** One shared network as defined in fixed_network_name, or one network per tenant as defined in accounts.yaml

​

​

17 of 21

Common tempest.conf settings for RefStack (2)

​

  • [network]
    • public_network_id = { your public network ID }
    • public_router_id = { your router ID }
  • [object-storage]
    • operator_role = { default is “Member”, update if this, if the role for your clouds is not “Member” }
  • [service_available]
    • neutron = True (default is False)
    • swift = { default is True, skip the Swift tests by setting this to False, if your test is for Compute Platform trademark only }
  • [validation]
    • run_validation = true
    • image_ssh_user = { default is root, set this to the default user of your image }

​

​

​

18 of 21

Troubleshooting

  • I don’t have enough accounts available?
    • You may have a stale locks directory. Clean it up and try again.
  • SSH to VM failures
    • Make sure that the 2 images used have the same username for ssh.
    • Make sure that your image ssh setting matches ssh configuration in the tempest.conf (i.e. use keypair or password for ssh)
  • Certificate validation issues with test or upload
    • May need to use the -k or --insecure optional with refstack-client
  • Check that tempest.config matches the Tempst version used
    • Tempest default settings may have changed
    • Configuration parameters may have moved to different sections.
    • Obsoleted configuration fields.

19 of 21

Future plans

  • Vendor/product registration
  • Associating test results to a product/vendor
  • Centralize testing initiated from RefStack website

20 of 21

Austin summit sessions

  • DefCore sessions
    • https://www.openstack.org/summit/austin-2016/summit-schedule/events/8645
    • Room: Austin Convention Center - Level 4 - MR 17 A/B
    • Monday (April 25), 3:40 PM - 04:20 PM
  • RefStack work sessions
    • https://www.openstack.org/summit/austin-2016/summit-schedule/global-search?t=refstack%3A
    • Room: Hilton Austin - MR 415B
    • Wednesday (April 27), 9:00 AM - 11:40 AM
  • Heat work sessions (Heat scoring and Tempest Plugin discussion)
  • DefCore work sessions
    • https://www.openstack.org/summit/austin-2016/summit-schedule/global-search?t=defcore
    • Room: Hilton Austin - Level 6 - Salon J
    • Thursday (April 28), 9:50 AM - 11:40 AM
  • Tempest/QA Work Session (DefCore

​

​

21 of 21

Thank you!

​

​

Questions?