In a recent video, the creator shares the experience of adding an AI agent feature to Ellie, a scheduling app they built. The AI acts like a personal assistant, planning the user's day hour by hour or batch-editing multiple tasks. But shipping this feature turned out to be far more difficult than expected.

"There's so much people don't tell you about shipping an AI product. This video is about those things."

1. Real Problems Not Covered in AI App Tutorials

This video isn't a tutorial on building basic AI agents -- there's a separate video for that. Instead, it shares the real-world problems encountered when integrating AI into a service: cost, security, design, and more.

"Building AI features is a bit different from traditional software development. There are cost issues, security issues, design issues, and more..."

2. Cost: Far More Expensive Than Expected

The first unexpected issue was cost. During development, costs were only loosely estimated, but as launch approached, monthly spending exceeded $30. With app subscriptions at $10/month, even the creator's own usage meant a $20 monthly loss.

"The app subscription is $10 a month, but my own usage was costing $30. That's a $20 loss every month."

Cost Optimization Steps

1. System prompt optimization

   • Handling edge cases during development caused the system prompt to grow to 8,000 tokens
   • This long prompt was sent with every message, inflating costs
   • Trimmed the prompt to 3,000 tokens, with plans to reduce further

   "I didn't realize the system prompt had gotten so long. Even a simple greeting sends that entire prompt along with it."

2. Conversation history optimization

   • Testing involved only a few messages, but real users exchanged 50+ messages over 2-3 days
   • Sending the full history each time caused costs to skyrocket
   • Introduced a "window" approach sending only the most recent 10 messages, drastically cutting costs
   • Considering summarizing older conversations when needed

   "In real use, people leave the chat open for days and exchange 50+ messages. Sending all of that is enormously expensive."

3. Abuse Prevention: Even Unintentional Misuse Is a Problem

Preventing abuse was also critical. Whether intentional or accidental, a single user pasting an entire book into the chat or sending hundreds to thousands of messages means the developer absorbs all the costs.

Abuse Prevention Measures

1. Message size limit

   • Capped individual messages at 10,000 tokens

   "If someone pastes an entire book into the chat, that single message could cost $20."

2. Per-user rate limits

   • 100 messages per day, 1,000 per month
   • Based on actual usage patterns, this seemed sufficient

   "This app isn't meant for thousands of daily conversations like ChatGPT -- it's for sending commands, so 100 is plenty."

3. Remote kill switch

   • If a specific user generates abnormal costs, their AI features can be remotely disabled
   • Analytics tools like PostHog monitor per-user usage

   "If someone uses too much, I can flip a switch to turn off just that person, then reach out directly to ask why."

4. Analytics system

   • Built a system to track token usage and costs at both app-wide and per-user levels

   "Surprisingly, many apps don't include this kind of analytics from the start."

4. Don't Reinvent the Wheel: Use Existing Libraries

Initially, everything was built from scratch, but after publishing, many people pointed out well-established libraries. A prime example was the Vercel AI SDK, which proved far more stable and concise.

"100 lines of handwritten code became 10 lines with the Vercel AI SDK."

• Streaming, tool calls, conversation state management, and more were already well-implemented
• Open-source and free to use
• Building from scratch was educational, but ultimately leveraging proven tools is more efficient

5. Multiple Models Must Be Combined

Originally expected to use one AI model, but reality required combining multiple models.

• GPT-4o Mini outperformed Gemini Flash on most tasks, but both struggled with time zone-related work
• Grok excelled at time zone processing, so time-related tasks were routed to Grok separately
• Added a model selection layer that first analyzes user input with a cheap model to decide which model to use

"Simple tasks go to a fast, cheap model; only complex tasks get sent to an expensive model."

• Expects to combine even more models going forward

6. Small Tips from Real-World Usage

(1) Consider the Form Factor

• Initially developed for web only, but in practice, mobile voice commands were used heavily
• Realized that speaking commands on mobile is far more convenient

"When grocery shopping, saying 'add eggs, bacon, paper towels' instantly creates tasks. That's way more natural on mobile."

(2) AI App Personalization

• Traditional software offers settings via dropdowns and toggles, but AI apps can enable much finer personalization through free-text input
• For example: "I exercise in the morning, handle personal tasks after work, and want 15-minute breaks between meetings" -- the AI incorporates this into the schedule

"It's amazing how much more personalized software can become this way."

(3) The Strength of Specialized Apps

• People ask "What happens when ChatGPT or Claude adds this feature?" but in practice, specialized apps are far more convenient
• For example, adding a schedule in ChatGPT requires multiple confirmations, while Ellie processes it in one step

"When you compare a general app and a specialized app, the specialized one almost always solves the problem better. The same applies to AI."

7. Conclusion: Unexpected Challenges and Advice

Launching an AI app is much harder than expected, requiring diverse considerations: cost, security, form factor, model selection, existing tool adoption, and differentiation.

"If you're building an AI product, track costs from day one. Include abuse prevention systems, be prepared to use multiple models, and think about what environments your users will be in."

Finally, the creator mentions that their AI agent should be live by the time the video goes up, and asks viewers to share their own problems and tips from building AI products.

"If you've run into problems or found tips while building AI products, please leave a comment. I read every single one!"

Key Concepts Summary

• Cost optimization: Prompt/conversation history management, analytics systems
• Abuse prevention: Message/usage limits, remote kill switch, monitoring
• Leveraging existing libraries: Vercel AI SDK, etc.
• Combining multiple models: Selecting the optimal model per task
• Form factor considerations: Mobile/voice and other real usage contexts
• AI app personalization: Natural language-based settings
• Strength of specialized apps: Differentiation from general AI services

This video candidly shares the real-world problems and solutions encountered when bringing AI apps to production, offering invaluable insights for anyone looking to build AI products.