From Hype to Help: Intelligent GenAI Adoption

A presentation at AI for the Rest of Us in October 2024 in London, UK by Sasha Rosenbaum

Slide 1

Slide 1

From Hype to Help: Intelligent GenAI Adoption Sasha Czarkowski (Rosenbaum) @DivineOps COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 2

Slide 2

Sasha Czarkowski (Rosenbaum) @DivineOps Dev Ops Product Tech Sales Management

Slide 3

Slide 3

Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 4

Slide 4

Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 5

Slide 5

How about you? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 6

Slide 6

GenAI is the most Transformative Technology Of the last 17 years Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 7

Slide 7

Have you ever asked an LLM to generate an image? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 8

Slide 8

Slide 9

Slide 9

Slide 10

Slide 10

Slide 11

Slide 11

Slide 12

Slide 12

2 hours later… Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 13

Slide 13

Intermittent reinforcement Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 14

Slide 14

We work harder for intermittent rewards Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 15

Slide 15

Slide 16

Slide 16

Slide 17

Slide 17

GenAI is an accidental experiment in Intermittent reinforcement Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 18

Slide 18

Your “feels” might be wrong: “While Copilot did not necessarily improve the task completion time or success rate, most participants preferred to use Copilot in daily programming tasks” Source: Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 19

Slide 19

“What I’m telling you is, productivity is meaningless unless you know what your goal is” Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 20

Slide 20

Your choice of productivity metrics will determine your outcomes Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 21

Slide 21

Source: https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/

Slide 22

Slide 22

Source: https://www.theregister.com/2024/10/16/ibm_insurance_industry_bosses_keen/

Slide 23

Slide 23

Source: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/

Slide 24

Slide 24

Source: https://resources.uplevelteam.com/gen-ai-for-coding

Slide 25

Slide 25

Source: https://www.upwork.com/research/ai-enhanced-work-models

Slide 26

Slide 26

More Robots @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 27

Slide 27

Slide 28

Slide 28

We have been here before Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 29

Slide 29

The Tesla Robot Revolution ● Tesla’s 2017 Model 3 assembly line built with > 1000 robots ● Musk said the line will produce 20,000 cars a month by EOY ● Tesla makes only 2,425 in the 4th quarter of 2017 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 30

Slide 30

Slide 31

Slide 31

But wait, there is more Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 32

Slide 32

The GM Robot Revolution ● In 1980s, GM spends > $45 Billion on industrial robots ● CEO Roger Smith lauded as a “visionary 21st century futurist” ● By 1990, GM earnings per US car sale fall from $588 to $47 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 33

Slide 33

Ergonautic

Slide 34

Slide 34

But wait, there is more Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 35

Slide 35

The origins of the term “Socio-Technical”: studies by Eric Trist in 1950 Britain of mechanization of Coal Mining Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Slide 36

Slide 36

Mostly Manual Giant Machine vs Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 37

Slide 37

The expected productivity of the new mechanized approach did not develop Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Slide 38

Slide 38

The Irony @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 39

Slide 39

February 17 2024 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 40

Slide 40

Source: Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 41

Slide 41

Production to Evaluation • Output overload • Complacency • Reduced situational awareness Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 42

Slide 42

Loss of Feedback No social commentary No ratings Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 43

Slide 43

Task Interruptions Cognitive costs Broken Flow Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 44

Slide 44

Task Complexity Polarization Automation can make easy tasks easier and hard tasks even harder Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 45

Slide 45

Inevitably, the friction with the machine is getting blamed on human error Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 46

Slide 46

Slide 47

Slide 47

The Transformation @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 48

Slide 48

xkcd.com/1425/

Slide 49

Slide 49

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 50

Slide 50

COPYRIGHT 2024. ALL RIGHTS RESERVED.

Slide 51

Slide 51

Slide 52

Slide 52

Ergonautic Is AI winter coming?

Slide 53

Slide 53

Serverless GenAI? Cloud IoT Continuous Delivery Big Data DevOps Lean Agile Microservices

Slide 54

Slide 54

Workflow Design Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 55

Slide 55

Person Sasha Czarkowski Product www.ergonautic.ly

Slide 56

Slide 56

Personal [work] flow @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 57

Slide 57

“The Death of the Junior Professional” Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 58

Slide 58

What skills do “seniors” have? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 59

Slide 59

Recognizing what “good” looks like Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 60

Slide 60

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 61

Slide 61

Desired State Current State Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 62

Slide 62

Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 63

Slide 63

Models the System Control Loop Controller Decision Information Sensor Measures the System Sasha Czarkowski Actuator Action COPYRIGHT 2024. ALL RIGHTS RESERVED. Acts on the System www.ergonautic.ly

Slide 64

Slide 64

Improving Productivity: Controller

  • Better Information - Better Decisions - Better Actions Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 65

Slide 65

It is control loops all the way down Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 66

Slide 66

Creator to Curator Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 67

Slide 67

Controller You are the loop! Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 68

Slide 68

July 23 2024 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 69

Slide 69

Humans are better at understanding complex tasks after they are decomposed into simpler subtasks Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 70

Slide 70

Human-Designed Problem GenAI Proposal GenAI Decomposition Human Repair Unit Test Evaluation Initial Experts Coding Problem GenAI Proposed Solution Heuristic Vanilla Solution Evaluation “NonExperts” Assistive COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 71

Slide 71

Controller Experts, “non-experts”, LLMs are better at “repairing” the well-decomposed LLM proposal Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 72

Slide 72

Productivity @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 73

Slide 73

Better Metrics @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly

Slide 74

Slide 74

Generative AI enables us to create MORE - More code - More text - More media Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Slide 75

Slide 75

If you ask for more [X] You will get more [X] Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 76

Slide 76

COPYRIGHT 2023. ALL RIGHTS RESERVED.

Slide 77

Slide 77

Process Quality Creativity Reuse Speed

Slide 78

Slide 78

Market Fit Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 79

Slide 79

Market Fit - Absolute Revenue - Relative Revenue - Cash Flow Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Slide 80

Slide 80

Market Fit - Customer Lifetime Value - Customer Acquisition Cost - Churn Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Slide 81

Slide 81

Slide 82

Slide 82

Slide 83

Slide 83

What’s the action? What’s the action?

Slide 84

Slide 84

All systems are socio-technical Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 85

Slide 85

Improving Productivity: Controller

  • Better Information - Better Decisions - Better Actions Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Slide 86

Slide 86

Thank you! Sasha Czarkowski (Rosenbaum) COPYRIGHT 2024. ALL RIGHTS RESERVED.