The entirety of the commit → review → test → integration → build → release process

Introduction

This document provides a clearing house to provide high level guidance to the TF TSC for delivering more detailed requirements for the entire release process, from code commit, to review, through testing, and final build and release.

Questions to Answer / Outcomes to Accomplish (please add)

• What is the release cadence?
  • JG I think anything faster than 6 months is not realistic, maybe we should start with 9 months.
  • DAH How will we communicate features / APIs that get deprecated?
• Are there multiple types of release? e.g. experimental, daily, stable, production?
  • JG Let's start with having CI in place and just one release (doesn't need to be called anything). Small steps to start with, we can go faster if we feel comfortable later.
• Will we have LTS releases or is that a problem for vendors making distributions?
  •  JG No need for LTS releases to start with IMHO
• Is there variance in the release cadence for the different types of release?
• How are these releases tested differently? For example:
  • Experimentals get basic sanity tests that are hours in length
  • Weekly stable releases get multi-day sanity/burn in tests
  • Production releases get month long scalability, performance tests, and synthetic transaction testing with VNFs or other workloads
  • Edward Ting For community releases, maybe it will be ok to have 1 ~ 2 weeks of soak test instead of month long.
• How do we determine what goes in a release?
  • JG From the blueprints
  • Edward Ting we need to able to prioritize the blueprints going to the releases within the 1st month of the release cycle.
  • Edward Ting suggest to have close the gate if a blueprint is not done or stable by a certain time before the release date, says 1 month.
• Are releases tagged? Are features tagged? Are tagged features included in a release?
  • JG Absolutely releases should be tagged. Big features can have their branches and then those branches get merged back.
• Or do we operate in branches?
• What infrastructure is required to support the release plan?
  • Where is it hosted?
  • Who is contributing?
  •  JG These are important questions that need to be answered. We need CI/CD in place. Without it it is nearly impossible to have stable releases. We need to also have performance testing as part of CI. We do need a contributors list too. Both developers and devops that run the whole CI engine.
• What are the quality criteria for release?
  • Edward Ting I have proposed to start with 3 use cases, CentOS, Kubernetes, and Ubuntu, using ansible-deployer, setting up 1 control node (K8S master) and 2 compute nodes (K8S slaves). Plus one DPDK performance. (TF OpenLab already has such setup.) These are the absolute minimum baseline. Community members can contribute different use cases.
  • Edward Ting At the 2nd phase, we should add HA scenarios repeating the non-HA 3 use cases above. Plus we should increase # of compute nodes to something meaningful, says, 10, 20, 30,...
• What are the security criteria for release?
  • Edward Ting I have authored security blueprint. Suggest to review that blueprint as a reference. 
• What are the documentation criteria for release?
  • Edward Ting Release documentation should be automatically generated. Features are the blueprints and fixed issues can be retrieved from gerrit, same as the known issues. The is no point to duplicate the efforts. The author of the blueprint should also provide the engineering documentation.
    
    

Requirements

1. Releases MUST have a web/wiki page that explains exactly what is in the release
   1. The page SHOULD be automatically generated if at all possible
2. Releases MUST be in the form of docker images stored in DockerHub
   1. Releases MUST be architected as microservices and be Kubernetes compatible
3. Releases MUST have a regular cadence
   1. Ideally releases WOULD happen every month
   2. The minimum cadence velocity MUST be 3 months, not 6 months
4. Releases MUST have some form of labels, tags, or branching scheme that allows for mapping included features and bug fixes into a CHANGELOG and RELEASE NOTES
   1. See #1 above
   2. The scheme MUST be documented here in the Wiki
5. There MUST be a variety of release types for various purposes to be determined; a starting point might be:
   1. Experimental releases for nightly or high interval testing looking for build breakages
      1. Lightly tested builds
   2. Stable regular builds for doing downstream integration testing into commercial distributions
      1. Standard testing
   3. Production builds that can be used for official releases
      1. Extensive scalability and performance testing (as possible)
6. SHOULD develop a scheme for LTS builds based on production builds (above)
   1. QUESTION: what does an LTS build mean for an open source project that has no official support?