.. _1-introduction: Introduction ============ .. raw:: html
Figure 1-1: RC1 Scope.
This document covers the realisation aspects of conformance of both NFVI and VNFs. The document will cover the following topics: - Identify in details the Requirements for conformance Framework. - Identify in details the Requirement of Test Cases (and mapping them to requirements from The Reference Model and The OpenStack Based Reference Architecture ). - Analysis of existing community projects. - Propose an E2E Framework for conformance of NFVI and VNFs. - Playbook of instructions, user manuals, steps of how to perform verification and conformance for both NFVI and VNFs using the proposed E2E Framework. - Gap analysis to identify where the Gaps are in the industry (tooling, test cases, process, etc). - Identify development efforts needed to address any gaps identified. **Not in Scope** - Functional testing / validation of the application provided by the VNF is outside the scope of this work. - ONAP is not used in the process flow for NFVI verifications, or validations. - Upgrades to VNFs, and the respective processes of verifying upgrade procedures and validating (testing) the success and compatibility of upgrades is not in scope. .. _13-relation-to-other-communities: Relation to other communities ----------------------------- .. raw:: htmlFigure 1-2: RC-1 Relations other communities.
.. _14-principles-and-guidelines: Principles and Guidelines ------------------------- The objectives of the verification program are to deliver a validated implementation of reference architecture which satisfies infrastructure needs for VNF-developer teams, leveraging the OVP ecosystem as the vehicle for delivering validated NFVI. These core principles will guide NFV verification deliverables: .. _141-overarching-objectives-and-goals: Overarching Objectives and Goals ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Deliver verified implementation of reference architecture which satisfies infrastructure needs for VNF-developer teams. 2. All accomplished with augmentation to the current OVP ecosystem. 3. Increase probability VNFs will on-board and function with minimal problems, or issues, during initial instantiation of VNF. 4. Test Harnesses will be portable, or compatible, across all RAs/Distributions which already conform to standard interfaces and services. .. _15-best-practices: Best Practices -------------- The following best practices have been adopted to ensure verification and validation procedures are repeatable with consistent quality in test results, and RI conformances: - Standardized test methodology / flow, Test Plan, and Test Case Suites - Integration with Anuket Upstream Projects and OVP flow (code, docs, cert criteria, etc.) - Leverage Network and Service Models, with identified VNF-specific parameters - Standardized conformance criteria - Define Anuket RA as scenarios, and have all test cases for the RA be involved in OVP - Add test cases from operators, which operators already tested in their environment .. _16-verification-methodologies: Verification methodologies -------------------------- Perform VNF interoperability verifications against an implementation of Anuket reference architecture, leveraging existing Anuket Intake Process. Upstream projects will define features/capabilities, test scenarios, and test cases to augment existing OVP test harnesses to be executed via the OVP Ecosystem. 3rd Party test platforms may also be leveraged, if desired. .. raw:: htmlFigure 1-3: Conformance Methodology
.. _17-assumptions--dependencies: Assumptions & Dependencies -------------------------- **Assumptions** NFVI+VNF testing will be considered **Testable** if the follow qualifiers are present in a test execution, and subsequent result: - Ability to perform Conformance, or Verification of Artifacts to ensure designs (RM/RA/RI) are delivered per specification - Ability to Control (or manipulate), manifestations of RM/RA/RI for the purposes to adjust the test environment, and respective cases, scenarios, and apparatus, to support actual test validations - Ability to monitor, measure, and report, Validations performed against a target, controlled system under test In addition, respective Entrance criteria is a prerequisite which needs to be satisfied for NFVI+VNF to be considered **Testable**. **Dependencies** NFVI+VNF verification will rely upon test harnesses, test tools, and test suites provided by Anuket projects, including dovetaill, yardstick, and Bottleneck. These upstream projects will be reviewed semi-annually to verify they are still healthy and active projects. Over time, the projects representing the conformance process may change, but test parity is required if new test suites are added in place of older, stale projects. - NFVI+VNF verifications will be performed against well defined instance types consisting of a HW and SW Profile, Configured Options, and Applied Extensions (See image.) |Instance Type| .. raw:: htmlFigure: Instance Type
**NFVI+VNF Instance Type:** - Standard compute flavours to be tested are defined in `chapter 4.2.1 <../../../ref_model/chapters/chapter04.md#4.2.1>`__ - Performance profiles come in the form of Basic, Network Intensive, and Compute intensive. Refer to `chapter 2.3 <../../../ref_model/chapters/chapter02.md#2.3>`__ for details on these profiles. .. _18-results-collation--presentation: Results Collation & Presentation -------------------------------- Test suites will be categorized as functional or performance based. Results reporting will be communicated as a boolean (pass/fail). The pass/fail determination for performance-based test cases will be made by comparing results against a baseline. Example performance-based metrics include, but are not limited to: resource utilization, response times, latency, and sustained throughput per second (TPS). **Placeholder to document where results will be posted (e.g. Dovetail dashboards.)** .. _19-governance: Governance ---------- 1. Conformance badges will be presented by the CVC 2. CVC will maintain requirements for conformance .. |Instance Type| image:: ../figures/RC_NFVI_VNF_Instance_Type_25Nov2019.jpg