This webinar examines why the feeling of "using more AI increases productivity" doesn't translate well into organizational outcomes, identifying structural reasons first. Based on Flex team's own experimental failures (in measurement, sequencing, and organizational adoption), it presents an AX design strategy that starts from the last mile and solves bottlenecks (SSOT, evaluation environments, validation, access control/security) step by step. In short: you need to change bottlenecks, verification, decision-making, and collaboration structures - not just increase output volume - to see real results.
1. "We'll Talk About Organizational AX Strategy, Not Product Demos"
The webinar begins with Flex team's CCPO Kim Tae-eun explaining they prepared a more broadly applicable discussion since applicants came from many different industries. The core message: Organizational AX that leads to business outcomes cannot be solved by AI adoption that only increases individual productivity.
2. "AI Is Definitely Faster, So Why Aren't the Metrics Moving?" - The Structure of Performance Illusion
Many organizations keep increasing AI budgets and deploying tools, yet performance metrics barely change. Individual task time reduction and output increases do happen, but connecting that to organizational goal achievement is "a different story." When one point gets faster, other points can't keep up, creating new bottlenecks. The real problems AI should solve are often in communication, collaboration, and decision-making systems - not code or document output volume.
3. Three Traps Flex Encountered Firsthand: Measurement, Sequencing, and Organizational Adoption
3-1. Trap 1: Measurement - "Mistaking Usage and Output for Performance"
The most naturally tracked metrics after AI adoption are token usage, frequency, and output counts. But these are just signals of "effort," not business performance. Even running parallel agents didn't always reduce total work time compared to sequential execution, and code review/verification processes remained bottlenecks despite increased production.
3-2. Trap 2: Sequencing - "Starting from the Easy Parts Only Speeds Up Pre-Bottleneck Stages"
Applying AI where it's easiest without addressing bottlenecks means only stages before the bottleneck get faster while outcomes don't change.
3-3. Trap 3: Organizational Adoption - "The Moment You Share AI Tools, Security and Access Problems Explode"
Individual AI use is fast and convenient, but organizational use triggers completely different problems: data access, external communication, and exposure to untrusted content. A real case where an AI sent 500+ spam messages after being given messenger access. Despite major companies banning usage, surveys show 20% of employees still use AI without approval.
4. The Core Engine of Organizational AX: Validation and Harness Engineering
The presenter emphasizes AI output validation as the key. When AI code volume increases but the same code review process remains, review burden explodes. The team shifted to building environments for validating AI output - test code, E2E tests written by AI, and even screen recording for visualization. Only after establishing this did results start connecting.
Harness Engineering is introduced as "meta-engineering that puts reins and saddles on powerful agents to make them run safely in the desired direction" - consisting of context, architecture constraints, feedback loops, self-validation, and garbage collection.
5. Four AX Design Strategies That Lead to Results
5-1. Strategy 1: "Start from the Last Mile" + Re-examine Bottlenecks
Start AX at the last mile (where customer value is actually confirmed) so AI effects connect directly to measurable performance metrics. Examples of last mile: QA stage, customer-facing CS teams, operations, sales/consulting, internal helpdesk. Flex achieved 70% problem resolution, 2.2 billion won cost savings, 83% AI customer satisfaction, and 60% CX headcount reduction through Intercom deployment.
5-2. Strategy 2: Without "SSOT + Agent Evaluation Environment," You Can't Build the Organization's Brain
SSOT (Single Source of Truth) is not just "data dumped together" but an organizational agreement ensuring the most current, meaningful data. When Notion specs, Figma specs, and code specs differ, SSOT must answer "which is the truth." The evaluation system includes prompt version management, LLM performance/cost monitoring, and continuous feedback loops.
5-3. Strategy 3: "Redefine Roles and Organization" - AI Is Not a Tool but a Change in How We Work
When using AI to solve collaboration bottlenecks, job boundaries naturally blur and more rules and agreements become necessary. Flex's PM no longer waits for developers - they set up evaluation foundations, manage prompts, and share source code, reducing hypothesis-to-verification cycles from weeks to hours. A PM (not an engineer by background) completed 10 PRs, 4,000+ lines of code, 75 changed files, and reduced a 4-hour knowledge migration to 3 minutes.
5-4. Strategy 4: "Start from Public Knowledge and Expand Gradually While Solving Access Control/Security"
Trying to expand AI across the entire organization at once gets blocked by authorization and security issues. Start with publicly accessible information (billing/payroll specs, labor law searches, company policies), then gradually expand to role-specific information. Flex built a multi-agent system with 15 HR categories and automated classification, eventually reaching toward enterprise-wide agentic AI where employee, organization, and goal data connect in real-time.
6. "What You Can Start Tomorrow" Checklist
- Find the last mile where AI should be applied
- Identify collaboration problems and bottleneck points rather than "output"
- Set measurable performance metrics connected to the last mile
- Invest in AI output evaluation environments