From Hype to Help: Intelligent GenAI Adoption

Sasha Czarkowski (Rosenbaum) @DivineOps Dev Ops Product Sales Management

Your “feels” might be wrong: “While Copilot did not necessarily improve the task completion time or success rate, most participants preferred to use Copilot in daily programming tasks” Source: Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Source: https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/

Source: https://www.theregister.com/2024/10/16/ibm_insurance_industry_bosses_keen/

Feb 2023 Source: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/

Apr 2024 Source: https://resources.uplevelteam.com/gen-ai-for-coding

Source: https://www.upwork.com/research/ai-enhanced-work-models

Tesla Robot Revolution ● Tesla’s 2017 Model 3 assembly line built with > 1000 robots ● Musk said the line will produce 20,000 cars a month by EOY ● Tesla made only 2,425 in the 4th quarter of 2017 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

GM Robot Revolution ● In 1980s, GM spends > $45 Billion on industrial robots ● CEO Roger Smith lauded as a “visionary 21st century futurist” ● By 1990, GM earnings per US car sale fell from $588 to $47 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Ergonautic

vs Video: Safety in Longwall Mining - https://youtu.be/85gg_kymDxg

Ergonautic

Timeless Ergonautic

GitLab Duo Generate: ● Documentation ● Test Cases ● Release Notes ● Code Reviews ● Incident Summaries ● External Codebase Analysis Source: https://about.gitlab.com/blog/2024/05/20/developinggitlab-duo-how-we-are-dogfooding-our-ai-features/ Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

xkcd.com/1425/

Ergonautic Is AI winter coming?

Serverless GenAI? Cloud IoT Continuous Delivery Big Data DevOps Lean Agile Microservices

Person Sasha Czarkowski Product www.ergonautic.ly

Improving Productivity: Controller

Better Information - Better Decisions - Better Actions Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Human-Designed Problem GenAI Proposal GenAI Decomposition Human Repair Unit Test Evaluation Initial Experts Coding Problem GenAI Proposed Solution Heuristic Vanilla Solution Evaluation “NonExperts” Assistive COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Assistive Decomposition High Assistive Value decomposition created through an iterative process involving human and LLM decomposition refinement Sasha Czarkowski COPYRIGHT 2024 ALL RIGHTS RESERVED. Ergonautic

Human-Designed Problem GenAI Proposal GenAI Decomposition Human Repair Unit Test Evaluation Initial Experts Coding Problem GenAI Proposed Solution Heuristic Vanilla Solution Evaluation “NonExperts” Assistive COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Assistive Decomposition Effects ● ● ● ● Accuracy improved by ~30% for both groups Had more benefits on hard problems Improved LLM ability to supervise itself Was preferred by human participants Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Controller Experts, “non-experts”, and LLMs are better at “repairing” the well-decomposed LLM proposal Decision Information Actuator Sensor Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic

Source: https://www.software.com/reports/code-time-report

https://xkcd.com/303/ COPILOTING

Process Quality Creativity Reuse Speed

Improving Productivity: Controller

What’s the action? What’s the action?

Conversational recruiting platform using AI to automate administrative tasks and improve the hiring process

https://xkcd.com/327/

Improving Productivity: Controller