1 of 29

Use AI�and check its work

Session 5 of 8 · AI-Assisted Development

2 of 29

© The Build Fellowship 2024

You've made the codebase faster. Now we let AI in.

psychology

We are past whether we should use AI and now to how we use AI.

Today we generate code with AI, then review it as critically as we would an engineer's pull request.

3 of 29

  • 10 min: AI tools, good prompting, and the trap of cognitive offloading
  • 5 min: Good prompts versus bad prompts
  • 15 min: Refactor a god function in `apps/api/src/routes/reports.ts`, with a comprehension-preserving system prompt
  • 10 min: Generate tests for `apps/api/src/services/auth.service.ts`, then prove you understand them
  • 10 min: Code-review the AI output against a quality rubric
  • 10 min: Group discussion: where AI helped, where blind acceptance would have caused issues

© The Build Fellowship 2024

Agenda

4 of 29

AI in Engineering

© The Build Fellowship 2024

5 of 29

© The Build Fellowship 2024

Three modes of working with AI

keyboard

Autocomplete

Copilot, Cursor, your IDE. The AI suggests the next few lines as you type. Low-friction, high-volume. Great for boilerplate. Easy to over-trust because it feels harmless.

forum

Chat

Claude, ChatGPT, Cursor's chat mode. You ask, it generates a chunk of code or explanation. Higher leverage per interaction, but the quality of the prompt matters a lot more. This is what we use today.

smart_toy

Agents

Autonomous execution. You define a high-level goal, and the AI plans, executes, and debugs. Extremely high leverage, but requires careful monitoring and verification as it acts independently.

6 of 29

What good prompting actually looks like

The most common mistake is treating the AI like Google.

The second most common mistake is treating it like a senior engineer who already knows your codebase.

Treat it like a new contractor who can read everything you show it.

1

Give context

Paste the existing code, and mention the libraries or design conventions you follow.

2

Be specific

Explicit directions like "rewrite this to use early returns" always beat generic requests like "make it better."

3

Ask for validatable output

Request outputs that are easy to verify programmatically or visually, such as tests, types, or before/after diffs.

4

Iterate the prompt, not the code

When the AI's output is wrong, the prompt itself is usually wrong. Refine your constraints instead of manually editing the output.

© The Build Fellowship 2024

7 of 29

The cost

© The Build Fellowship 2024

8 of 29

© The Build Fellowship 2024

Cognitive offloading

The thing to watch for

Offloading is when you hand the thinking to a tool and stop building the mental model yourself.

Understanding is the ultimate bottleneck in software. When you offload comprehension, the true cost comes later.

When it works fine

A calculator does it for arithmetic, GPS does it for navigation. These tools handle execution where deep conceptual mastery isn't the primary goal.

With AI writing code, it's different

It happens fast, it happens invisibly, and the output looks like you wrote it yourself.

You read code that works, you accept it, and you never form the understanding that writing it yourself would have forced.

9 of 29

Why this is a real problem, not a vibe

Typing was never the bottleneck in software. Understanding is. When you offload the understanding, the bill comes due later.

Individual Risks

• Blind Spots: You cannot debug, review, or extend code you never understood. The first incident finds the gap.

• False Security: Over-trust ships subtle bugs: the code reads as correct, so nobody looks hard.

Team & Long-Term Impact

• Skill Atrophy: The reps you skip today are the judgment you lack next year.

• Compounding Gaps: On a team, the gaps compound. Nobody owns the parts the AI wrote and nobody read.

© The Build Fellowship 2024

10 of 29

One question tells

you if you

offloaded

© The Build Fellowship 2024

The Golden Question

"Could you re-derive this, or explain it to a teammate, without the AI in front of you?"

The Diagnostic

• If yes: You offloaded the typing, which is fine.

• If no: You offloaded the comprehension, which is the trap.

Code you do not understand is a liability you have not noticed yet.

11 of 29

Build comprehension into the system prompt

The system prompt is the highest-leverage place to fight offloading, because it shapes every answer in the session. Instead of asking the AI to just emit code, instruct it to keep you in the loop. You will write your own version of this in the activity.

Core Instructions

• Explain Reasoning: Not just the result.

• Surface Assumptions: And the tradeoffs it accepted.

• Offer Alternatives: Say why it picked this one.

Interactive Verification

• Ask Clarifying Questions: Have the AI ask you instead of guessing.

• Quiz the User: Prompt you to explain the output so you have to articulate why it works.

© The Build Fellowship 2024

12 of 29

© The Build Fellowship 2024

Habits that keep the understanding yours

The system prompt sets the AI's behavior. These habits set yours.

01 Read and understand before you accept. No exceptions for code that "looks right."

02 Rewrite the AI's explanation in your own words. If you cannot, you do not understand it yet.

03 Review AI code like an engineer's PR: assume nothing, check everything.

04 Write or understand the test first, so you can judge the code against intent.

05 Modify the output without AI to prove it is yours. If you cannot change it, you do not own it.

13 of 29

After the AI answers, run the loop.

Three questions, every time the AI produces something you intend to keep. This is the loop you will practise in today's activity.

01 Why does this work?

Say it out loud or write it down.

02 What breaks if an assumption changes?

For example, an input is empty, a client times out, or the schema shifts.

03 What tradeoff did we accept?

Consider why we did it this way instead of the alternative.

© The Build Fellowship 2024

14 of 29

© The Build Fellowship 2024

What bad prompting produces

THE LAZY APPROACH

"make this function better."

Lacks critical context. Produces a generic, low-quality response that often misses the actual goal.

THE CAREFUL PROMPT

  • Include the target file & project conventions
  • Define the exact test we want to pass
  • Request output as a clean, unified diff

Visibly superior quality. Guided directly by engineering intent.

The ultimate takeaway: Prompts are first-class code. Treat them with the same precision and care.

15 of 29

Refactor�a real function.

© The Build Fellowship 2024

16 of 29

© The Build Fellowship 2024

This is the target

Open apps/api/src/routes/reports.ts

The Complex Monolith

~130 lines doing ten things at once

•Queries tasks and hydrates assignees

•Computes rates and applies timezones

•Buckets results into a CSV report

•Dispatches final exit notification

The Refactoring Plan

Deconstruct safely using AI

1. Split It Apart Ask AI to break the function into single-responsibility, modular helpers.

2. Verify Integrity Run the existing test suite immediately to confirm nothing broke during the split.

17 of 29

Refactor `reports.ts` with AI help

© The Build Fellowship 2024

1. Preparation

Follow along in your own clone.

System Prompt

Set the comprehension-preserving prompt from active-comprehension-prompt.md

Template

Use the template in refactor-prompt.md

2. AI Refactor

Paste Code

Paste the contents of reports.ts into the prompt.

The Goal

Ask for a refactor that splits the function into smaller, modular helpers.

Apply

Apply the AI's suggestions to your file.

3. Verification

Run Tests

Verify with test suite. If failed: fix regression & update prompt.

Comprehension Check

Name each helper and its function from memory without re-reading.

Record

Document your results in your logbook.

18 of 29

Generate tests�for code that has none.

© The Build Fellowship 2024

19 of 29

© The Build Fellowship 2024

Zero tests

Open�auth.service.ts

This file is absolutely critical for security, yet it currently has zero coverage. Today you will change that with AI help.

Why test this file?

It is responsible for critical authentication mechanisms: hashing passwords, signing JWTs, and validating tokens.

The Prompt Template Playbook

Feed the AI the core components to get robust, accurate tests:

  • Provide the target source file.
  • Provide your testing conventions.
  • Request cases covering both happy paths and error cases.

20 of 29

​

Write tests for auth.service.ts with AI

© The Build Fellowship 2024

01 / SETUP & RUN

We do this together, system prompt still on. Use the template prompt:

test-gen-prompt.md

Generate and run the tests.

02 / ITERATE & FIX

They probably won't pass on the first try. AI often:

  • Gets imports wrong
  • Invents APIs
  • Makes false type assertions

Fix what's broken and log it.

03 / VERIFY

Comprehension Check

Change an assertion to something you know is wrong.

Predict the failure before running. If the actual failure surprises you, re-evaluate!

21 of 29

Now grade�its work.

© The Build Fellowship 2024

22 of 29

© The Build Fellowship 2024

Grade the AI output against four axes.

01 / CORRECTNESS

Does it pass the existing tests?

Does the behavior match the spec?

02 / MAINTAINABILITY

Would a teammate be able to read and modify this six months from now?

03 / COVERAGE

Does the AI handle edge cases, or only the happy path?

04 / IDIOM

Does it match the conventions of the surrounding code?

23 of 29

© The Build Fellowship 2024

Review your AI output

Together, for each piece of AI output we applied (refactor + tests), we grade it on all four rubric axes.

Capture *specific* examples: "the AI didn't handle the empty-tasks case" beats "coverage was okay."

👉 Record your grades and justifications in your logbook as we go.

24 of 29

When AI tends to go awry

A pattern of failures you'll see across many tools.

01 / HALLUCINATIONS

Confident hallucinations of behaviour that doesn't

02 / EDGE CASES

Edge cases silently ignored

03 / TYPING

"Plausibly typed" but actually wrong TypeScript: types compile but lie.

04 / MISALIGNMENT

Code that solves a slightly different problem than the one you asked about.

© The Build Fellowship 2024

25 of 29

Group�discussion

© The Build Fellowship 2024

26 of 29

© The Build Fellowship 2024

​

Where did AI help, and where would blind acceptance have bitten you?

01 / SHARE EXPERIENCE

Around the room, share one place where AI saved you time, and one place where accepting output blindly would have caused problems

02 / SPOT THE GAP

Name a specific moment from today's comprehension checks where you initially thought you understood the code, but actually did not.

03 / LOG FOR POLICY

Record these near-misses in your logbook. These real-world learnings directly shape the final engineering policy we establish in Session 7.

27 of 29

© The Build Fellowship 2024

What this means for you

Engineers who use AI well are pulling ahead

01 / DIRECT THE TOOL

Hiring managers look for engineers who can direct AI without being directed by it. The core skill is judgment: what to ask for, what to accept, and what to discard.

02 / PROTECT UNDERSTANDING

Judgment is built entirely on deep understanding. Blindly offloading tasks to AI erodes comprehension, taking away your ability to evaluate the output critically.

03 / MULTIPLY VELOCITY

The engineers pulling ahead use AI to move faster and keep their mental model intact. That is the compounding practice that separates high performers.

28 of 29

© The Build Fellowship 2024

SESSION 06 / METRICS

Experimentation and measuring impact

You have a measurably faster pipeline, a better developer experience, and AI-assisted improvements. Now, prove it with metrics.

01 / BASELINES

Revisit your Session 1 baselines and compare them directly against your deployment pipeline performance today.

02 / DORA METRICS

Learn the key DORA metrics—deployment frequency, lead time, MTTR, and change failure rate—that engineering leaders care about.

03 / ROLLOUT PLAN

Design a comprehensive rollout plan for successfully introducing your optimized engineering changes to a broader development team.

29 of 29

Questions?

© The Build Fellowship 2024