On Change Advisory Boards

This topic repeatedly comes up with the homies. I want to collect all my excerpts & notes in one place, ie here.

From Accelerate

From what I can tell, these conclusions are coming from primary research.

In large organizations, we often see change management processes that take days or weeks, requiring each change to be reviewed by a change advisory board (CAB) external to the team in addition to team-level reviews, such as a formal code review process.

We found that external approvals were negatively correlated with lead time, deployment frequency, and restore time, and had no correlation with change fail rate. In short, approval by an external body (such as a manager or CAB) simply doesn’t work to increase the stability of production systems, measured by the time to restore service and change fail rate. However, it certainly slows things down. It is, in fact, worse than having no change approval process at all.

  • Lightweight peer-approval process
    • Pair Programming
    • Intra-team code reviews
  • Deployment pipeline to catch & reject bad changes
  • Having sensible-defaults defined can also be useful here.

Finally, a related antipattern is the gatekeeper pattern. Having humans who perform gatekeeping activities creates very odd social dynamics, and is rarely a great use of a human’s time. When at all possible, build systems with sufficient isolation that you can allow most actions to go forward. And when they do occasionally fail, make sure that they fail with a limited blast radius. - Will Larson - An Elegant Puzzle