Making DevOps Valuable

A presentation at DevOps Vision in December 2024 in Florida, USA by Sasha Czarkowski (Rosenbaum)

Slide 1

Slide 1

Making DevOps Valuable Sasha Rosenbaum @DivineOps COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 2

Slide 2

Sasha Czarkowski (Rosenbaum) @DivineOps Dev Ops Product Sales Management

Slide 3

Slide 3

Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 4

Slide 4

Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 5

Slide 5

Once upon a time Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 6

Slide 6

Traditional IT dev ops wall of confusion Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 7

Slide 7

Dev Ops Speed Reliability

Slide 8

Slide 8

DevOpsDays Ghent 2009 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 9

Slide 9

Jez Humble and Dave Farley: 2010 COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 10

Slide 10

@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 11

Slide 11

“DevOps” Google Search Trend 2009-Present Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 12

Slide 12

State of DevOps Report 2019 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 13

Slide 13

@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 14

Slide 14

We have the technology… Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 15

Slide 15

Just move faster! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 16

Slide 16

You cannot CI/CD Without Testing Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 17

Slide 17

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 18

Slide 18

Did we get DevOps wrong? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 19

Slide 19

The future is already here. It’s just not evenly distributed ~ William Gibson Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 20

Slide 20

CALMS Sasha Czarkowski Culture Automation Lean Measurement Sharing COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 21

Slide 21

CALMS Sasha Czarkowski Culture Automation Lean Measurement Sharing COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 22

Slide 22

Culture Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 23

Slide 23

Popular Perception of Culture Change Assumptions Sasha Czarkowski Values COPYRIGHT 2023. ALL RIGHTS RESERVED. Artifacts (behavior) www.ergonautic.ly

Slide 24

Slide 24

“Anyone wanting to change a culture needs to define the actions and behaviors they desire, then design the work processes that are necessary to reinforce those behaviors.” - John Shook, MIT Sloan Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 25

Slide 25

Culture Change (Schein) Artifacts (behavior) Sasha Czarkowski Values COPYRIGHT 2023. ALL RIGHTS RESERVED. Assumptions www.ergonautic.ly

Slide 26

Slide 26

Motivation Matching: Help people do what they already want to do. - BJ Fogg Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 27

Slide 27

Automation Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 28

Slide 28

@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 29

Slide 29

Naive automation Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 30

Slide 30

This new tool is going to solve ALL of our problems! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 31

Slide 31

COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 32

Slide 32

Automation Drift @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 33

Slide 33

The irony of manual takeover Source: Ironies of Automation. https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 34

Slide 34

All systems are sociotechnical Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 35

Slide 35

Entropy always wins @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 36

Slide 36

Inevitably, the friction with the machine is getting blamed on human error Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 37

Slide 37

Let’s talk about the forgotten pillar Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 38

Slide 38

Measurement Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 39

Slide 39

The aim of the system must be clear to everyone in the system. Without an aim, there is no system. The aim is a value-judgment. - W. Edwards Deming Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 40

Slide 40

We want to measure Value Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 41

Slide 41

Measuring Value delivery is Hard Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 42

Slide 42

Just measure things! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 43

Slide 43

Metrics Done Wrong ● Subjective ● Imbalanced ● Only a Proxy Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 44

Slide 44

Velocity ● Subjective ● Was there a story-point inflation? ● Imbalanced ● Did we sacrifice quality? ● Only a Proxy ● Did we deliver more units of value? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 45

Slide 45

What gets measured, gets gamed Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 46

Slide 46

COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 47

Slide 47

You want a balanced system of metrics Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 48

Slide 48

Process Quality Creativity Reuse Speed

Slide 49

Slide 49

Let’s all use the 4 Golden Signals! Can you connect them to revenue, costs or customer experience? @ergonauticly Latency Traffic Errors Saturation COPYRIGHT 2023. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 50

Slide 50

Google: 500ms of latency dropped traffic by 20% Amazon: 100ms of latency dropped sales by 1% https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-latency-cost-them-1-in-sales/

Slide 51

Slide 51

Ergonautic milliseconds of latency do not matter!

Slide 52

Slide 52

The narrative must be meaningful to your business @ergonauticly COPYRIGHT 2023. ALL RIGHTS RESERVED. Ergonautic

Slide 53

Slide 53

The true goal Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 54

Slide 54

Measuring flow of Value through the system Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 55

Slide 55

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 56

Slide 56

What is Reliability? @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 57

Slide 57

Reliability vs Availability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 58

Slide 58

Everything is 200 OK! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 59

Slide 59

The five nines 99.999% @ergonauticly 5.26 mins / year COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 60

Slide 60

Will your users even notice? @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 61

Slide 61

The ISP background error rate is 0.01% - 1% @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 62

Slide 62

100% availability is… ● Unattainable ● Unnecessary ● Extremely expensive Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 63

Slide 63

Will your users care? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 64

Slide 64

COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 65

Slide 65

Reliability is a Feature Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 66

Slide 66

Reliability Is my service doing what the users expect it to do? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 67

Slide 67

SLA, SLI, SLO Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 68

Slide 68

SLA = Financially-backed Availability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 69

Slide 69

Monthly downtime > 1.5 days means 100% refund COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 70

Slide 70

SLAs are about aligning incentives between Vendor & Customer @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 71

Slide 71

SLA usually includes a single metric For financial and reputational reasons, companies prefer to underpromise and overdeliver @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 72

Slide 72

SLI @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 73

Slide 73

SLI = Actual Reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 74

Slide 74

Monitoring @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 75

Slide 75

Without monitoring, you have no way to tell whether your service even works! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 76

Slide 76

Good Monitoring @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 77

Slide 77

Without good monitoring, you don’t know that the service does what users expect it to do! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 78

Slide 78

SLO @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 79

Slide 79

SLO = target reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 80

Slide 80

SLO = Business-approved reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 81

Slide 81

SLOs are about explicitly aligning incentives between Business & Engineering @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 82

Slide 82

Error Budgets @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 83

Slide 83

Error Budgets Acceptable level of unreliability Error budget = 1 - SLO EB = 1 – 99.99% = 0.01% ≃ 13 mins /quarter @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 84

Slide 84

Error budgets are about aligning incentives between Dev & Ops @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 85

Slide 85

If developers are measured on the same SLO, then when the error budget is drained developers shift focus from delivering new features to improving reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 86

Slide 86

What happens when you blow the Error Budget? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 87

Slide 87

Nothing @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 88

Slide 88

@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 89

Slide 89

SLO = Targeted Reliability Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 90

Slide 90

Define SLOs to be the best proxies of user value Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 91

Slide 91

Reliability is a Feature Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 92

Slide 92

Start with SLIs Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 93

Slide 93

Without good monitoring, you don’t know that the service does what users expect it to do! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 94

Slide 94

Information -> Insight Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 95

Slide 95

Slide 96

Slide 96

What’s the action? What’s the action?

Slide 97

Slide 97

ODIM Objective, Decisions, Insights, Measurements Working backwards from intended outcomes Given [Objective] what decisions do we need to make? Given the need to [Decision] what insights do we need have to improve it? Given the need for [Insight] what do we need to [Measure]? By: Larry Maccherone, https://medium.com/@lmaccherone/odim-12d80823222 @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 98

Slide 98

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 99

Slide 99

Desired State Current State Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 100

Slide 100

Models the System Control Loop Controller Decision Information Sensor Actuator Acts on the System Measures the System Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 101

Slide 101

Improving Productivity: Controller

  • Better Information - Better Decisions - Better Actions Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 102

Slide 102

It is control loops all the way down Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 103

Slide 103

Making your DevOps valuable Align Measurements and Incentives to delivering Value to the user Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 104

Slide 104

The journey of a thousand miles begins with a single step Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 105

Slide 105

The future is already here. It’s just not evenly distributed ~ William Gibson Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 106

Slide 106

Thank you! Sasha Czarkowski (Rosenbaum) COPYRIGHT 2024. ALL RIGHTS RESERVED.