1 of 6

OpenStack Tacker Testing improvement

2 of 6

Need to create pre-conditions

  • API Conformance Testing Specification describes the preconditions of the test cases. We recognize necessary to create preconditions of each test cases.

​

​

​

​

  • The test case codes only contain API validation, but not creating test preconditions.

Pre-conditions

Test Case

3 of 6

Issues

  • The preconditions specifications of NFV TST are undefined, missing or ambiguous. We must consider the preconditions for each test cases.

4 of 6

Issues

  • NFV TST currently creates test cases to test one API.
  • Preconditions cannot be created by executing the test cases as they are.
  • Because Robot framework cannot specify the order of test cases.
    • For example (Heal VNF test)
      • VNFLifecycleManagement-API.VNFInstances (Create)
      • VNFLifecycleManagement-API.InstantiateVNFTask
      • VNFLifecycleManagement-API.HealVNFTask
      • VNFLifecycleManagement-API.TerminateVNFTask
      • VNFLifecycleManagement-API.IndividualVNFInstance (Delete)

​

    • Actual execution order
      • VNFLifecycleManagement-API.HealVNFTask
      • VNFLifecycleManagement-API.IndividualVNFInstance (Delete)
      • VNFLifecycleManagement-API.InstantiateVNFTask
      • VNFLifecycleManagement-API.TerminateVNFTask
      • VNFLifecycleManagement-API.VNFInstances (Create)
  • We need to implement preconditions codes in addition to current NFV TST codes.

Alphabetical order

5 of 6

W Release

  • One compliance test consists of:
    • Create preconditions
    • Test one API (NFV TST one test case)
    • Post processing

​

  • For now, we make functions to create the preconditions.
    • We create as few preconditions as possible for easy maintenance.
    • We implement preconditions codes for Tacker
      • Plan A. Make preconditions codes with Robot framework.
        • Use the subroutines which are already prepared by NFV TST.
      • Plan B. Make with python.
      • Plan C. Make with Shell scripts.
    • We need to discuss with NFV TST.

​

  • Make post processing to remove all generated by preconditions and tests.

6 of 6

Discussion with NFV TST

  • Our goal is to verify Tacker by NFV TST test code only.
    • If we make functions to create preconditions, our tests include our own understanding.
    • We want NFV TST to maintain preconditions.

​

  • We will propose a discussion on preconditions to NFV TST.
    • Organization of preconditions
    • Precondition codes implemented in Robot framework (Plan. A)