.. _9-adoption: 9. Adoption =========== .. raw:: html
Figure 1: Transition Plan for NFVI solutions within CNTT
VNF/CNF Transition Plan Framework ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exceptions will be clearly recorded in the appropriate specification Appendix which will service as guidance to VNF/CNF application vendors of what Exceptions will be allowed in each CNTT release. **Figure 2** below demonstrates the concept. - It is expected that over time, as technology matures, there will be a decreasing numbers of Exceptions allowed in CNTT releases. - For each CNTT Release, VNF/CNF can be either: - **Fully Conformant**: No Exception used. - **Conformant with Exception**: One or More of the allowed Exceptions in the Reference Model have been used. .. raw:: htmlFigure 2: Transition Plan for VNFs/CNFs within CNTT
.. _95-cntt-field-trial-approach: 9.5 CNTT Field Trial/ Approach ------------------------------ This portion of Chapter 9 is segmented into two subsections. Section 9.5.1 provides a summary and overview of the trials activities specifically targeted to potential trials participants. Section 9.5.2 addresses the overall CNTT approach to field trials as a method of ensuring consistency between releases. .. _951-summaryfield-trials-participants-overview: 9.5.1 Summary/Field Trials Participants Overview ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Reference Implementation (RI-1) and Reference Conformance (RC-1) requirements are defined by the Reference Architecture (RA-1). To ensure that Telecom industry operators, suppliers, and end user customers will derive benefit for the effort, CNTT is running field tests to validate the feasibility, utility, and effectiveness of its requirements and methods (RI-1/RC-1). .. _9511-field-trials-intentions: 9.5.1.1 Field Trials Intentions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The field trials are viewed as a partnership of CNTT with participants to validate that the community is adopting a consistent approach. This is not a VI badging exercise. The trials will validate the RI-1 and the RC-1 test suite requirements and methods themselves, not VNFs or VI systems under test. .. _9512-expectations-and-assumptions-of-field-trials: 9.5.1.2 Expectations and Assumptions of Field Trials ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ CNTT expects to exit the trials with either validation of RI-1 and RC-1 or a set of actions to review and possibly modify the RI-1 or RC-1 to address any gaps identified. By taking advantage of the community continuous improvement process, the lessons learned in the field trials will be applied to the badging processes to refine/define the criteria with the intention of making the badges meaningful and mutually beneficial to operators and suppliers. Performance testing is not included in the field trials. .. _9513-pre-trials-activities: 9.5.1.3 Pre-trials activities ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Prior to the comencement of any field trials, the CNTT community will define an operational plan, secure resources, and provide all designated contact information required to support trial participants. As the results of the trails may produce data and information that could be considered sensitive by participants, CNTT will establish standard data set requirements and secure collection methods to ensure participant privacy protection. .. _9514-expectations-of-trials-participants: 9.5.1.4 Expectations of Trials Participants ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Trials participants will be expected to commit to establishing an RA-1 compliant NFVI, in whatever manner best suited to the participant. The first step is for the participant to secure appropriate environment space from pre-existing space, newly built space or securing LaaS. The environment can exist in any mix of participant owned, private or community hardware infrastructure. Second, the participant will build/setup/configure the environment space using their preferred method. This can include the use of a cookbook, automated install, and/or build from RA-1/RI-1 requirements and specifications. CNTT `RI-1 Chapter 3 <../ref-impl/cntt-ri/chapters/chapter03.md>`__ documentation provides the matching RI-1 requirements for the build. .. _9515-expectation-2-execute-the-rc-1-test-suite: 9.5.1.5 Expectation 2: Execute the RC-1 Test suite ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ CNTT will provide the participants with the community RC-1 Test suite, found at this Link: `https://github.com/cntt-n/CNTT/blob/master/doc/ref_impl/cntt-ri/chapters/chapter04.mdTable 1: Survey/Questionnaire example
.. _9517-expectation-4-providing-trials-results: 9.5.1.7 Expectation 4: Providing Trials Results ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ As a community, CNTT is concerned with the privacy of participant data. CNTT abides by the LFN anti-trust policies ( `https://www.linuxfoundation.org/antitrust-policy/Figure 3: Field Trial Approach
Success Indicators ^^^^^^^^^^^^^^^^^^ 1) Agreement secured on the use of trials results data, including: a. Level of data detail required to validate the results b. Acceptable data values indicating valid results c. Level of data detail that will be published 2) VI-1 Labs are successfully deployed in all target environments - Vendor (NFVI, VNF, VIM, 3rd Party) - Community (OPNFV) - LaaS (e.g. UNH) 3) Engaged vendors successfully configure their VI and run the RC-1 test suite and are able to provide expert feedback 4) Engaged vendors are able to validate that they can instantiate and run rudimentary validation of VNF functionality on more than one conformant cloud infrastructure (NFVI) .. _9527-initiation: 9.5.2.7 Initiation ~~~~~~~~~~~~~~~~~~ .. _95271-objectives-of-ri-1rc-1-trials: 9.5.2.7.1 Objectives of RI-1/RC-1 Trials ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The object is to quantitively and qualitatively assess and evaluate the following CNTT requirements, methods, and support processes: - RI-1 Specifications - VI-1 implementation support methods ( i.e. cookbooks, installation manuals, how to guides etc.) - RC-1 Test Suite - TC Traceability - Test Pass Criteria - Benchmark Data - Other criteria to be determined at commencment or during the execution of the trial Overall, feedback from the trials and issues and gaps found shall be used to enhance and improve the CNTT approach. Enhancements to future releases will/shall be identified accordingly. .. _95272-trial-participant-interaction-with-the-community: 9.5.2.7.2 Trial Participant Interaction with the Community ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The focus of the field trials is on the test suites and CNTT methods, not on the systems under test. A process is being developed to identify issues and gaps and managing how they are reported. CNTT will work very closely with field trial partners (NFVI vendors, VNF vendors, or system integrators) and agree on labs that will be used for the trial. CNTT will take all necessary measures to protect the intellectual property rights (IP rights) for all partners involved in those trials. All Reports and findings will be vetted carefully and only published after being approved by all parties concerned. No test results or records will be kept in any public records without the consent of the participants. The targeted repositories for this information are: CNTT GitHub - GitHub Code - GitHub Projects - GitHub Issues OPNFV - (Where?) .. _95273-test-case-identification: 9.5.2.7.3 Test Case Identification ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Specific test cases for the field trials will be documented and provided to the participants based upon the CNTT RI-1 and RC-1 work streams requirements. The technical testing methods, procedures and documentation shall be provided by these work streams. .. _95274-vendor-solicitationcommitment: 9.5.2.7.4 Vendor Solicitation/Commitment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Vendor members will be solicited for participation in the trials. The vendors will be required to commit fully to the assessment and evaluation processes. As previously mentioned, additional discussion is needed to define what results data and at what level of detail is acceptable to be shared. .. _95275-deliverable: 9.5.2.7.5 Deliverable ^^^^^^^^^^^^^^^^^^^^^ The Initiate Field Trial Stage will deliver execution and assessment plans including: - A high-level check list of the tasks each participant will need to complete shall be provided. - The plan will contain all the key milestones and activities the participants will expected to perform. .. _9528-execution-stage: 9.5.2.8 Execution Stage ~~~~~~~~~~~~~~~~~~~~~~~ .. _95281-objectives-of-the-execute-stage: 9.5.2.8.1 Objectives of the Execute Stage ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The objective of Execute Stage is participants implementing field trials tasks and record/assess outcomes CNTT will assemble the Trials team to fully develop the action plan including resource assignments, materials requirements, and timelines. Activities include the deployment and configuration of VI and execution of the RC-1 test cases. Vendor community members that commit to the trials will build/setup/prep labs for the trials per the instructions: 1. Secure appropriate environment space (pre-existing, new build, LaaS) 2. VI per published RI-1 Specifications 3. RC-1 Test suite will be provided to the participants 4. Trial Participants ensure a complete understanding of the test suite actions and expected outcomes. .. _95282-running-the-field-trial: 9.5.2.8.2 Running the Field Trial ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The field trial will run the Test Suite for 3 Iterations. For each iteration: - Vendor RC-1 test results are documented. Vendor provide feedback to CNTT - OPNVF RC-1 test results are documented. OPNFV provides feedback to CNTT The Community shall review Issues/Gaps during the evaluate stage and do one of the following: - Accept the Issue/Gap, and accordingly modify the RI/RC - Not-Accept the Issue/Gap and document the condition of non-conformance while maintaining the privacy of participants .. _9561-resources-and-roles: 9.5.6.1 Resources and Roles ^^^^^^^^^^^^^^^^^^^^^^^^^^^ CNTT will staff the plan by soliciting volunteers from the participants. The list below is suggested list of roles to be staffed: - Overall Field Trial Lead - Technical Field Trial Steering Lead - Vendor lead from each supplier - SME(s) for RC1 supporting suppliers - SME(s) for RI1 supporting suppliers - OPNVF lead for RI1/RC1 - Other support roles such as Governance, technical writers, etc. The participants that volunteer for the roles will be expected to provide the appropriate amount of time to support the trials initiative. .. _95284-deliverables: 9.5.2.8.4 Deliverables ^^^^^^^^^^^^^^^^^^^^^^ The deliverables of the execute stage will be: - Implemented Participant RA-1 Labs which have been tested. - RC-1 Test cases are run. .. _9529-assessment: 9.5.2.9 Assessment ~~~~~~~~~~~~~~~~~~ The Assess stage shall utilize data collected during the execute stage. Participants will assess their experience using the methods used by CNTT accordingly to quantitatively and or qualitatively measure: Required Assessments ^^^^^^^^^^^^^^^^^^^^ - VI Implementation methods and procedures (cookbook, etc) - RI-1 Specifications - RC-1 Test Suite - TC Traceability - Test Pass Criteria - Benchmark Data - Other? Optional (Pre-Launch Trials only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ NF Instantiation '''''''''''''''' - Smoke test the level of verification and validation - Non-functional - Stand up with only key operations working OPNFV will also assess their experience of the methods used by CNTT to assess the following operational areas: 1. Mechanism for Reporting Issues / Receiving Status 2. Results Collation and Presentation, 3. Support Availability - SME (Human) - Materials 4. Release Notes 5. Other? .. _95291-measuring-outcomes: 9.5.2.9.1 Measuring Outcomes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. _95292-qualitative-outcomes: 9.5.2.9.2 Qualitative Outcomes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Participants and project teams will be provided a questionnaire based upon a set of User stories related to the field trail. Questionnaire responses will be used in the Evaluate phase. .. _95293-quantitative-outcomes: 9.5.2.9.3 Quantitative Outcomes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Technical outcomes i.e. technical test results will be collected and managed by RI-1/RC-1 work streams based upon participants privacy preferences. .. _95294-deliverables: 9.5.2.9.4 Deliverables ^^^^^^^^^^^^^^^^^^^^^^ - Feedback is provided from the participants on their outcomes to CNTT. - Completed Questionnaire and test case results (Participant and OPNFV) .. _95210-evaluation-stage: 9.5.2.10 Evaluation Stage ~~~~~~~~~~~~~~~~~~~~~~~~~ Proving the ‘right’ value to the operator and vendor community is ultimately what will ensure adoption of CNTT requirements. These field trials are intended to verify and validate the requirements and methods developed by CNTT so that adjustments can be made to ensure the intended value is being delivered. CNTT shall evaluate all feedback and test results to understand whether CNTT methods and measures are meeting intended objectives. If a method or measure is not meeting its intended purpose, it shall be identified as a gap or an issue for resolution. Determinations if and when adjustments or adaptations are needed shall be made to by the CNTT community. .. _952101-deliverables: 9.5.2.10.1 Deliverables ^^^^^^^^^^^^^^^^^^^^^^^ All identified gaps and issues shall be captured in the CNTT GitHub repository. Decisions and determinations will be captured and logged accordingly. .. _95211-closeout-stage: 9.5.2.11 Closeout Stage ~~~~~~~~~~~~~~~~~~~~~~~ To close out the Field Trial, CNTT shall summarize its evaluation of the Field Trial and actions to be taken to address any adaption needed. .. _952111-final-deliverable---end-of-trial-report: 9.5.2.11.1 Final Deliverable - End-of-Trial Report ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Upon completion of field trials, CNTT shall develop an End of Trial Report which summarizes the overall conclusions based on the evaluation, to include: - Successes - What went activities well both generally or specifically? How did it compare to the past or alternative results? - Challenges - What didn’t go well overall? What impact could these challenges have to adoption? - Discoveries - What are key discoveries/strategic learnings about any CNTT approaches or methods? Other? - Decisions and Recommendations - Identification of key decisions made and list of what corrective actions shall be taken. What to enhance, maintain, or discontinue? - Next Steps - Indication of proposed steps and activities to be undertaken by the community