Treating artificial intelligence as a capable but inexperienced developer
Core Idea: Conceptualizing AI as an eager but inexperienced junior developer—fast and capable in narrow tasks but requiring supervision, guidance, and verification.
Key Elements
-
Capabilities and Limitations
- Excels at generating boilerplate and routine code
- Produces solutions quickly but often incomplete
- Handles common patterns well but struggles with novel scenarios
- May appear confident while being subtly wrong ("an overconfident intern with a great memory that is able to read really fast")
- Learns from examples but lacks deep understanding
- Works tirelessly but needs direction
- Produces quantity but needs quality control
- Can answer complex questions instantly but fail at seemingly trivial ones
-
Supervision Requirements
- Mandatory review of all AI-generated code
- Verification of security practices and edge cases
- Validation against project-specific requirements
- Checking for alignment with architectural patterns
- Testing of non-obvious failure modes
- Verification of performance characteristics
- Review of external dependencies introduced
- "Vibe coding + human review = 👍; Vibe coding + no human review = 👎"
- Essential to maintain a technical human-in-the-loop for complex projects
-
Guidance Approaches
- Clear specifications through effective prompts
- Providing context about existing systems
- Setting boundaries for acceptable solutions
- Breaking complex tasks into manageable chunks
- Offering examples of desired patterns
- Setting explicit constraints and requirements
- Iterative feedback on initial attempts
- Defining expectations clearly (specify if you want responses short, technical, etc.)
- Giving specific roles and tasks rather than vague instructions
- Breaking down abstract tasks into smaller, concrete steps
-
Integration into Team Dynamics
- Establishing when to leverage AI assistance
- Creating workflows that include AI verification steps
- Setting expectations for AI contributions
- Addressing concerns about skill replacement
- Measuring impact on team productivity
- Creating shared best practices for interaction
- Developing team-specific AI usage patterns
- Selecting the right AI model for specific tasks (e.g., GPT 4.5 for writing, Sonnet 3.7 for complex coding)
- Being mindful about when to delegate tasks to AI versus handling them personally
Additional Connections
- Broader Context: Trust but Verify Principle (fundamental to this approach)
- Applications: The 70% Problem (understanding AI's partial solutions)
- See Also: Knowledge Paradox (explains why AI needs supervision), LLM Weaknesses and Strengths (detailed breakdown of capabilities)
References
- Developer surveys on AI code trust levels
- Research on AI code quality and correctness
- Case studies of AI usage in development teams
- Industry best practices for AI integration
- Karpathy, A. Introduction to LLMs [Video]. YouTube. (Recommended for understanding how LLMs work)
#ai #softwareengineering #teamdynamics #supervision #collaboration
Connections:
Sources: