From Hype to Help: Intelligent GenAI Adoption Sasha Czarkowski (Rosenbaum) @DivineOps
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Slide 2
Sasha Czarkowski (Rosenbaum) @DivineOps
Dev Ops Product Tech Sales Management
Slide 3
Ergonautic Stop Chasing Buzzwords Start Working Better
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 4
Ergonautic Stop Chasing Buzzwords Start Working Better
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 5
How about you?
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 6
GenAI is the most Transformative Technology Of the last 17 years
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 7
Have you ever asked an LLM to generate an image?
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 8
Slide 9
Slide 10
Slide 11
Slide 12
2 hours later…
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 13
Intermittent reinforcement
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 14
We work harder for intermittent rewards
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 15
Slide 16
Slide 17
GenAI is an accidental experiment in Intermittent reinforcement
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 18
Your “feels” might be wrong: “While Copilot did not necessarily improve the task completion time or success rate, most participants preferred to use Copilot in daily programming tasks”
Source: Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models.
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 19
“What I’m telling you is, productivity is meaningless unless you know what your goal is”
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 20
Your choice of productivity metrics will determine your outcomes
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
More Robots
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 27
Slide 28
We have been here before
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 29
The Tesla Robot Revolution ● Tesla’s 2017 Model 3 assembly line built with > 1000 robots ● Musk said the line will produce 20,000 cars a month by EOY ● Tesla makes only 2,425 in the 4th quarter of 2017
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 30
Slide 31
But wait, there is more
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 32
The GM Robot Revolution ● In 1980s, GM spends > $45 Billion on industrial robots ● CEO Roger Smith lauded as a “visionary 21st century futurist” ● By 1990, GM earnings per US car sale fall from $588 to $47
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 33
Ergonautic
Slide 34
But wait, there is more
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 35
The origins of the term “Socio-Technical”: studies by Eric Trist in 1950 Britain of mechanization of Coal Mining Sasha Czarkowski
COPYRIGHT 2024 ALL RIGHTS RESERVED.
Ergonautic
Slide 36
Mostly Manual
Giant Machine
vs
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 37
The expected productivity of the new mechanized approach did not develop
Sasha Czarkowski
COPYRIGHT 2024 ALL RIGHTS RESERVED.
Ergonautic
Slide 38
The Irony
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 39
February 17 2024
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 40
Source: Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 41
Production to Evaluation • Output overload • Complacency • Reduced situational awareness
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 42
Loss of Feedback
No social commentary No ratings
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 43
Task Interruptions
Cognitive costs Broken Flow
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 44
Task Complexity Polarization
Automation can make easy tasks easier and hard tasks even harder
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 45
Inevitably, the friction with the machine is getting blamed on human error
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 46
Slide 47
The Transformation
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 48
xkcd.com/1425/
Slide 49
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 50
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Slide 51
Slide 52
Ergonautic
Is AI winter coming?
Slide 53
Serverless
GenAI?
Cloud
IoT Continuous Delivery Big Data DevOps Lean Agile
Microservices
Slide 54
Workflow Design
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 55
Person
Sasha Czarkowski
Product
www.ergonautic.ly
Slide 56
Personal [work] flow
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 57
“The Death of the Junior Professional”
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 58
What skills do “seniors” have?
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 59
Recognizing what “good” looks like
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 60
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 61
Desired State
Current State
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 62
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 63
Models the System
Control Loop
Controller Decision Information
Sensor
Measures the System
Sasha Czarkowski
Actuator
Action
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Acts on the System
www.ergonautic.ly
Slide 64
Improving Productivity: Controller
Better Information - Better Decisions - Better Actions
Decision
Information
Actuator
Sensor
Action
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 65
It is control loops all the way down
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 66
Creator to Curator
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 67
Controller
You are the loop!
Decision
Information
Actuator
Sensor
Action
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 68
July 23 2024
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 69
Humans are better at understanding complex tasks after they are decomposed into simpler subtasks
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 70
Human-Designed Problem
GenAI Proposal
GenAI Decomposition
Human Repair
Unit Test Evaluation
Initial Experts Coding Problem
GenAI Proposed Solution
Heuristic Vanilla
Solution Evaluation “NonExperts”
Assistive
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 71
Controller
Experts, “non-experts”, LLMs are better at “repairing” the well-decomposed LLM proposal
Decision Information
Actuator
Sensor
Action
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 72
Productivity
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 73
Better Metrics
@ergonauticly
COPYRIGHT 2024. ALL RIGHTS RESERVED.
www.ergonautic.ly
Slide 74
Generative AI enables us to create MORE - More code - More text - More media Sasha Czarkowski
COPYRIGHT 2024 ALL RIGHTS RESERVED.
Ergonautic
Slide 75
If you ask for more [X] You will get more [X]
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 76
COPYRIGHT 2023. ALL RIGHTS RESERVED.
Slide 77
Process Quality
Creativity
Reuse
Speed
Slide 78
Market Fit
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 79
Market Fit - Absolute Revenue - Relative Revenue - Cash Flow Sasha Czarkowski
COPYRIGHT 2024 ALL RIGHTS RESERVED.
Ergonautic
Slide 80
Market Fit - Customer Lifetime Value - Customer Acquisition Cost - Churn Sasha Czarkowski
COPYRIGHT 2024 ALL RIGHTS RESERVED.
Ergonautic
Slide 81
Slide 82
Slide 83
What’s the action?
What’s the action?
Slide 84
All systems are socio-technical
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 85
Improving Productivity: Controller
Better Information - Better Decisions - Better Actions
Decision
Information
Actuator
Sensor
Action
Sasha Czarkowski
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Ergonautic
Slide 86
Thank you! Sasha Czarkowski (Rosenbaum)
COPYRIGHT 2024. ALL RIGHTS RESERVED.