Where AI & Humans Still Struggle

The point of this page is credibility. A team that only lists wins isn’t trusted; a team that names its limits is. Below, what AI still struggles with, then what we (humans/process) still struggle with.

Where AI still struggles

Deep domain and business judgment

AI drafts plausible requirements and code, but it doesn’t understand the business. It can’t reliably tell a critical edge case from a trivial one, or know which “rule” is sacred. Human domain expertise remains the bottleneck and the safeguard.

Large, messy, legacy codebases

On big brownfield systems with implicit conventions and years of context, AI assistance degrades. It loses the thread across many files, misses cross-cutting concerns, and suggests changes that look right locally but break globally. Most of our wins are still in greenfield or well-bounded modules.

True architectural reasoning

AI proposes designs, but trade-off decisions under real constraints (cost, compliance, team skill, existing tech debt) still need a human Architect. AI will confidently produce a clean design that ignores a constraint it wasn’t told - and it won’t ask.

Correctness it can’t verify

For anything where being subtly wrong is expensive - concurrency, security boundaries, financial calculations, data migrations - AI output needs the same rigor as junior work. It’s a confident generator, not a verifier.

Stable, repeatable behavior

Model updates shift output quality between sprints. Reproducibility - central to engineering discipline - is still weak. We mitigate with pinned versions and shared prompts, but it’s a real friction.

Knowing when it doesn’t know

AI rarely says “I’m not sure.” It fills gaps with confident guesses, which is the root cause of most failure modes in Problems & Workarounds.

Where we (humans & process) still struggle

Calibrating trust

The hardest skill is knowing when to trust AI and when to scrutinize. Too much trust → escaped defects; too little → we lose the speed benefit. Calibration is experience-based and uneven across the team.

Review load shifting, not shrinking

AI generates more code faster, which moves the bottleneck to review. Quality Engineers and senior developers can become the new constraint. We’re still tuning PR size and review staffing for this.

Measuring real impact

Proving AI net helps - beyond “feels faster” - is hard. We track accept/reject rates, defect-escape, cycle time, and review round-trips, but attribution is noisy. Honest answer: directionally positive, precisely uncertain.

Skill development for juniors

Juniors who lean on AI can plateau on fundamentals. We’re balancing “use AI to go fast” with “learn the craft,” and haven’t perfected it. Pairing and “explain why” usage help.

Prompt and pattern maintenance

Good prompts are now team assets that need curation, versioning, and updating as models change - a new category of maintenance work we’re still formalizing.

Tool sprawl and cost

New AI tools appear constantly. Choosing, integrating, governing, and paying for them - without context-switching overhead - is an ongoing management challenge.

Honest bottom line

flowchart LR
    AI[AI: fast drafts,<br/>broad coverage,<br/>tireless review] --> COMBO((Best results))
    H[Humans: judgment,<br/>domain, accountability,<br/>verification] --> COMBO
    COMBO --> OUT[Faster delivery<br/>at maintained quality]

AI has measurably accelerated drafting, review, and testing across every role. It has not removed the need for skilled engineers - it has shifted where their effort goes: less typing, more reviewing, deciding, and verifying. Our biggest open problems are calibrating trust, managing the new review bottleneck, and proving impact rigorously. We treat all three as active work, not solved problems.