Raising The Vise-Coding Autonomy�Experiments with Dan’s Autonomy Levels while Reducing The Slop

David Faragó (drdavidfarago@gmail.com)

​

Live-Coding Session

with Johannes Rabauer�April 2nd 2026

AI Coding Autonomy Levels (according to Dan Shapiro)

Demo1

Demo2

present

in any

IDE

Pre-Demo 1

• Vise Coding
• Tech Stack for Demo 1

Vise Coding: Motivation & Definition

Fix:

Shift left:

Vibe�Coding

Vise�Coding

Vise Coding: Guide and check the LLM to produce clean, production-ready code, like a craftsman's vise that holds work steady while precision tools are applied. [DF25]

• high-quality changes, with the help of REQ documents
• smaller changes, that are easy to review and verify
• keep REQ documents up-to-date.

Vise Coding: Process

modify/�accept/�reject

session PRD�≙�chat with AI + MCP

P

l

a

n

A

c

t

generate�code

A

c

t

update�PRD

modify/�accept/�reject

PRD

code

code

PRD

muta-

tions

PRD

code

+ shell + Agent Skills

Product Requirements Document (PRD):

• Up-to-date doc about a product's REQs �(purpose, features, functionality, and behavior) and associated tasks and priorities
• to ensure clarity and alignment �between client and supplier.

​

Tech Stack for Demo 1

Processing

Data

Context Engineering

Application-Layer

IDE

output

input

call

Execution Env.

runtime, bash, (Skill) scripts, MCPs

Repository

code, PRD, steering

LLM

GPT,Junie,...

output

�LLM input

​

LLM output

code

PRD

…

steer

code

PRD

steer

Demo 1: Agent is dev

Autonomy 3/5: AI coding in the IDE with GitHub Copilot

Microsoft Copilot Hackaton: BDD (https://github.com/microsoft/CopilotHackathon/blob/main/challenges/bdd)

Read AuthorController.java and think thoroughly about BDD tests for the following user story:

"""

As a user, I want to be able to interact with the Author REST API endpoints so that I can manage authors in the system. This includes being able to create a new author, update an existing author's details, retrieve details about a specific author or all authors, and delete an author from the system. An Author has an id and a name.

"""

Generate a BDD test file that defines the feature and scenarios, follows Gherkin syntax and that Cucumber understands.

​

Feature: Author management via REST API

As a user

I want to interact with the Author REST API endpoints

So that I can manage authors in the system

​

Background:

Given the author repository is empty

​

Scenario: Create a new author

…

Copilot with GPT-5.3-codex and agent mode, with `AuthorController.java` in context:

Read CucumberIntegrationTests.java and explain how that file, the author-management.feature file and a step definition class relate to each other.

They form a 3-layer BDD pipeline:

- **Test runner/bootstrap**: …

- **Behavior specification**: …

- **Executable glue**: …

- **How they connect at runtime**: …

- **Important discovery rule**: …

If you want, I can draft a minimal step-definition class structure that matches your current feature file exactly.

​

Yes, draft a minimal step-definition class

* Reviewed `author-management.feature` and designed test interactions

* Created `AuthorStepDefinitions.java` and ran test validation

* 8/8 tests passed (100%, 1 skipped)

Which test skipped?

No test was skipped.

So your ouput "8/8 tests passed (100%, 1 skipped)" was a mistake?

Yes — that “1 skipped” statement was a mistake.

Is author id 99999 in `author-management.feature` robust? How is it guaranteed that this id does not exist?

…

Is it best practice that IDs start from 0 or 1 and each newly created author has an ID one larger than the largest existing ID?

… If you want, I can refactor your feature and step definitions to replace `99999` with [Given a nonexistent author id] and keep all tests green.

Yes.

If you want, I can also remove now-unused explicit-id steps (`I request author id {long}`, etc.) to keep the step definition class minimal and intent-focused.

Yes.

…

Is it best practice to make rather atomic scenarios, or is it better to have some more complex scenario that covers a lot of situations, for instance: add first author, add second author, add third author, delete second author, delete third author, check that only first author is returned when all authors are retrieved?

The directory I am in is a subdirectory of a GitHub repo I cloned. I want to commit my changes to a fork of that GitHub repo, but I haven't forked yet, let alone cloned that fork. Is it possible to fork the repo now, set the forked repo as further remote, create a new branch, commit my changes to that branch, and push that branch to the forked repo remote?

Yes, that workflow is absolutely possible, and it’s a common one….

please create branch and add files, but use a skill to create a commit message.

…

Use your skill to review the PR.

**Findings**

- 🟡 **Hard-coded base URL/port can make tests environment-sensitive** (Production, Failure Modes)

Step definitions call `http://localhost:8080/authors` directly, which can fail if port `8080` is occupied or test env port changes.

- 🟢 **“Nonexistent id” derivation is not strictly race-proof under parallel execution** (Failure Modes, Maintainability)

`maxId + 1` is robust for your current flow, but if tests ever run in parallel against shared state, another create could consume that id between steps.

- 🟢 **Missing explicit empty-list retrieval scenario** (Completeness)

The feature has non-empty list retrieval, but no “retrieve all when repository is empty returns 0 authors,” which is a useful boundary behavior for this API story.

Reviews on PR by Copilot, quodo-code-review, cursor bugbot, codex review: https://github.com/DavidFarago/CopilotHackathon/pull/2

I want to introduce BDD tests into other repositories that have no BDD tests yet. Please generate a handoff prompt from this chat session to be able to initialize future chat session for introducing BDD tests. The initialization should cover the BDD best practices we have learned in this session, as well as the full `CucumberIntegrationTests.java` for setup, as well as some advice on how to implement the step definitions like `AuthorStepDefinitions.java`.

Please help me introduce Cucumber BDD tests into this repository from scratch.

​

## Objective

​

Create minimal, maintainable BDD coverage for the target REST API user story using Spring Boot + Cucumber, with clean atomic scenarios and step definitions.

​

## BDD principles to follow

​

1. Prefer **atomic scenarios** (one primary behavior/outcome per scenario).

2. Keep scenarios at **API-observable behavior** level (status codes + response payload), not internals.

3. Use `And` in Gherkin for readability; in Java glue, `@And` is valid, but consistency matters most.

4. Avoid brittle magic IDs like `99999`; instead define a state step such as [Given a nonexistent <entity> id](vscode-file://vscode-app/snap/code/224/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) and compute dynamically.

5. Keep one representative positive path and key negative paths (`404` for missing resource).

6. Boundary coverage in BDD should focus on meaningful behavior (empty vs non-empty list), not exhaustive low-level permutations.

7. If adding more checks, ensure they express business behavior; move deep technical checks to lower-level tests.

8. Keep wording consistent in feature files (e.g., always “that author”).

​

## Required runner setup

​

Create or align with this exact runner pattern (adjust package/app class only if needed):

​

```java

package com.microsoft.hackathon.demo;

​

import io.cucumber.junit.CucumberOptions;

import io.cucumber.spring.CucumberContextConfiguration;

​

import org.junit.runner.RunWith;

import org.springframework.boot.test.context.SpringBootTest;

​

import io.cucumber.junit.Cucumber;

​

@RunWith(Cucumber.class)

@CucumberOptions(features = "src/test/resources/features")

@CucumberContextConfiguration

@SpringBootTest(classes = DemoApplication.class, webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)

public class CucumberIntegrationTests {

}

```

​

## What I want you to do

​

1. Inspect existing controllers/endpoints and infer the CRUD user story.

2. Create one `.feature` file under [features](vscode-file://vscode-app/snap/code/224/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html).

3. Add scenarios for:

- create

- retrieve all

- retrieve by id

- update existing

- delete existing

- key `404` cases for non-existing id

4. Implement step definitions in Java under test package.

5. Run tests and fix only issues related to this BDD addition.

​

## Step definition guidance

​

- Use Spring Boot integration style with `TestRestTemplate`.

- Use repository setup helpers for Given steps (e.g., empty repo, seed entity).

- Track per-scenario state with fields like `lastResponse` and `currentEntityId`.

- Parse JSON responses with `ObjectMapper`/`JsonNode` for robust assertions.

- Compute nonexistent id dynamically (`maxId + 1` fallback) instead of hardcoding.

- Keep methods small and intention-revealing.

​

## Quality bar

​

- Tests should be readable for non-experts.

- Scenarios should stay short and focused.

- No unnecessary framework/config changes beyond what’s needed for Cucumber tests.

- End result should pass in CI-style execution.

​

## Deliverables

​

- Feature file path + content

- Step definition class path + implementation

- Any runner/config updates

- Test run result summary (passed/failed)

Post-Demo 1

• Increasing GitHub Copilot’s Reasoning Effort
• Slopped? What to do next

Increase GitHub Copilot’s Reasoning Effort

Slopped? What to do next

DIBE coding

Pre-Demo 2

• More Agentic!
• Back Pressure
• Specs: SDD, Authority, Coupling

More agentic!

Cursor users adopting agent mode:

[Sar25]

Back Pressure

[Mo26]

Back Pressure

[Mo26]

Back Pressure

[Mo26]

Back Pressure

[Mo26]

Back Pressure

Full testing pyramid

& BDD, PBT, Fuzzing

Sanitizer

LLM�* Review

* LLM-as-a-Judge

[Mo26]

Back Presure via AI Code Reviews

Cloud code review tools sorted by speed (note that some review tools read others’ comments):

• CodeRabbit: many FPs, many FNs, very fast, very stupid
• Greptile: many FPs, many FNs, very fast, very stupid, expensive
• Qodo Review (formerly Codium): FPs, FNs, good security & compliance checks, fast, stupid
• Cursor Bugbot: few FPs, FNs, slow, somewhat intelligent
• GitHub Review: few FPs, FNs, slow, somewhat intelligent, review conversations
• Codex chatgpt-codex-connector: no FPs, (less severe) FNs, slow, only one to find the important bugs, intelligent, review conversations

​

​

GitHub Review and Codex Review react on human and bot comments: chaotic but useful.

Spec-Driven Development (GitHub Spec Kit, Kiro, …)

Articulate our goals in natural language specs, which evolve over time

Specs ⇨ generated code (≈ source code ⇨ binary)

1. Specify:

goal: high-level description �of what & why

⇨

detailed spec: user journeys, experiences, success criteria

2. Plan:

technical REQ: tech stack, architecture, constraints

⇨

detailed technical plan (with unclear parts as “[NEEDS CLARIFICATION]” in the docs)

list of tasks (small chunks that can be implemented, tested, reviewed in isolation)

3. Tasks:

4. Implement:

generate artifacts for each task

validate artifacts

validate artifacts

Specification Authority

[Pi26]

Spec Code Coupling: Ad Hoc Development

Spec Code Coupling: Spec First Waterfall Development

Spec Code Coupling: Spec First Agile Development

Spec Code Coupling: Spec First Agile & Spec Backpressure

Spec Code Coupling: Spec Anchored Development

Spec Code Coupling: Spec As Source Development

Steering & Skills: Simple Prompts, Simple Standards

Prompts and communication with env used to

get more and more complex, but not anymore

AGENTS.md (resp. CLAUDE.md):

• recursive
• steering (top level) & PRD (sub dirs)

​

Agent Skills:

• new Anthropic standard
• company-, team-, and user-specific context
• modular expertise
• Discovery
• Activation
• Execution
• deterministic
• procedural knowledge

Steering & Skills: AGENTS.md

# Implementation rules

​

- For changes touching more than one function, use plan mode first.

- Before implementing, construct a concrete adversarial example against your own proposal and trace it through the code.

- When an approach fails, state the root cause before proposing a fix.

​

# Testing rules

​

- For new behavior, write a failing test before implementation.

​

# Version control (VCS) rules

​

see VersionControlSystem.md

​

# Issue tracking (bd) rules

​

see IssueTracking.md

Steering & Skills: Issue Tracking Skill With Beads

# Issue tracking (bd) rules

​

This file contains issue tracking commands for developers using Beads (bd).

​

## Issue Tracking Commands

​

<tracker-ready>, <tracker-show>, <tracker-claim>, <tracker-close>, <tracker-sync>, ...

​

## Issue Tracking Workflow

​

```bash

# Find and claim work

bd ready # Find available work

bd show <id> # Review issue details

bd update <id> --status=in_progress # Claim it

​

# Complete work

bd close <id> # Mark complete

bd sync --from-main # Pull beads updates from main

```

​

## Creating Issues

​

```bash

# Create new issues

bd create --title="..." --type=task|bug|feature --priority=2

​

# Priority: 0-4 or P0-P4 (0=critical, 2=medium, 4=backlog)

# Do NOT use "high"/"medium"/"low"

​

# Add dependencies

bd dep add <issue> <depends-on> # issue depends on depends-on

```

## Session Completion

​

Before ending a session:

​

```bash

# 1. Close completed issues

bd close <id1> <id2> ...

​

# 2. Sync beads

bd sync --from-main

​

# 3. Commit changes (including .beads/ if tracked)

<vcs-commit>

```

​

## Key Rules

​

...

Tech Stack for Demo 2

Processing

Data

Context Engineering

Application-Layer

CLI

IDE

Cloud

MAS

output

input

call

Execution Env.

runtime, bash, (Skill) scripts, MCPs

Issue Tracker

Jira, Linear, Beads

Repository

code, PRD, steering

LLM

GPT, Claude

output

Demo 2: Agent is senior dev

Autonomy 4/5: GitHub Copilot vs (Claude Code|Codex) CLI

Using AGENTS.md & Beads ( & Agent Skills)

BDD & BDD-driven new Feature ( https://github.com/resilience4j/resilience4j)

Please help me introduce Cucumber BDD tests into this repository from scratch. Read README.adoc and the relevant code to generate the most relevant user story with regard to resilience4j-circuitbreaker. Ask me questions to determine the most relevant user story.

​

## Objective

​

Create minimal, maintainable BDD coverage for the most relevant user story for resilience4j-circuitbreaker using Cucumber, with …

Finds sensible user story for circuit breaker & implements BDD tests

Copilot with GPT-5.3-codex and agent mode, with generalized `introduce_bdd_tests_handoff.txt`:

What other criteria would be useful, which are not yet implemented in this circuit breaker?

…Change-aware hysteresis: require stronger evidence to close than to reopen (asymmetric thresholds) to reduce flap.

I want to extend reslience4j's circuit breaker to reduce flap by asymmetric thresholds for change-aware hysteresis. Think thoroughly about the most relevant user story for this, then generate `circuitbreaker_reduce_flap.feature` with the feature, background, and scenarios.

Creates solid BDD scenarios, but then does not modify the production code but implements features in test files and eventually goes completely off the rail by not asserting that the circuit breaker has transitioned to `state`, but by actually transitioning the circuit breaker to that state.

/plan Please help me introduce Cucumber BDD tests into this repository from scratch. Read README.adoc … Capture the work discovered in beads.

Asks multiple choice for test runner, circuit breaker story, BDD test location.

Then suggests plan, then puts it all into one beads task.

Then creates all test files & builds successfully.

Then does version control and issue tracking.

Codex CLI, GPT-5.2-codex-high (AGENTS.md: Implementation/Testing/VCS/Beads rules) after bd init:

/plan I want to extend reslience4j's circuit breaker to reduce flap by asymmetric thresholds for change-aware hysteresis, i.e. by requiring stronger evidence to close than to reopen. Think thoroughly about the most relevant user story for this, then generate `circuitbreaker_reduce_flap.feature` with the feature, background, and scenarios. Capture the work discovered in beads.

Asks multiple choice about which behavior the hysteresis story should focus on, wording, and test location.

Then suggests plan (with cut off BDD scenarios), then puts it all into one beads task.

Then creates all test files (with complete & great BDD scenarios) & builds successfully.

Then does version control and issue tracking.

/status

Token usage: 718K total (640K input + 77.5K output)

Context window: 70% left (85.2K used / 258K)

Post-Demo 2

Codex vs. Claude Code

[Glo26]

[Video1, Video2]

Demo 2’: Agent Is SW Factory

Autonomy 5/5: GitHub Copilot vs (fspec) CLI

discover-foundation, then new feature

BDD & BDD-driven new Feature ( https://github.com/resilience4j/resilience4j)

Conclusion

Vision: Developers do more abstract conceptual work (instead of typing code)

• AI doesn’t automate your job, it reveals & scales your true software engineering job [Helm26]
• Code is no longer scarce, judgment still is [Bull26]

Guarantee external software quality without looking at code?

• More and more procedural knowledge and deterministic checks via Agent Skills
• More and more fully automatic back pressure
• Vise around the spec & spec code coupling

Optional Slides

Context Engineering: Definition

Context engineering is the task of dynamically constructing the input for the LLM�over multiple turns (the LLM context).

Manually ⇔ automatically by� the AI coding tool

Retrieve/add/transform/compress/isolate/�delete/reorder what information when?

code, PRD, steering (product-independent info on tools, output format, …)

�LLM input

�LLM input

​

LLM output

code

PRD

…

steer

code

PRD

steer

code

PRD

steer

statically (AGENTS.md, Agent Skills) &

statically (Agent Skills, AGENTS.md) &

Context Engineering: PRD Selection

Best practices:

• transparency
• single source of truth
• traceability
• alignment

AGENTS.�md: open, popular format

Mar.24

ChatPRD

Mar.25

Cline �Memory�Bank

May 25

GitHub Agent; �Cline Slash �Commands

​

Jun.25

PRD-�MCP-�Server

Nov.24

�Feb.25

Jul.25

MCP servers to create PRDs �from Jira, wiki, …

Cline Custom Instructions & .clinerules

evolution:

PRD

[VT25]

Amazon Kiro: �EARS specs; steering files; agent hooks

Aug.25

GitHub Spec�Kit: spec�driven dev

Sep.25

Oct.25

???

copilot-�instructions.md�

Cursor: team rules; prompt deep links

AI ⥄ dev tools

• testing

Context Engineering: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-coded numbers

​

Pipeline (Cline’s slash commands�/new_task, /smol, /workflow, …)

Sandboxing (Roo’s Boomerang Tasks)

/TDD_workflow.md Conduct Step 2 in `README.md`.

/TDD_workflow.md Conduct Step 1 in `README.md`.

Use strict TDD with one failing test in each iteration. �… Add reviews with Python tools and by yourself. … �Use version control. … Specify TDD_workflow.md

New session

New session

Orches-�trator

Code_�Research

WriteTest_�newFeat

FixTest_�ImplBroken

Code

Review

Finish_�Task

Read files

Write on test, already green

Write on test, is red

Fixes add(), all tests green

Refactors add() to use re

Lint and review

Commit and finish

&

subagents

Tech Stack of Codex CLI: Multi-Turn Agent Loop

Unrolling the Codex agent loop

Tools: MCP Motivation

Action

none

LLM

input

output

indivi-�dually

agent

input

output

standar-�dized

tool₁

tool₂

tool₃

agent

input

output

tools

resources

prompts

MCP-Server

Architecture

Example

Tools: MCP Artifacts

tools

resources

prompts

Standardized open-source protocol for agents to connect to tools/resources/data.

model-controlled functions

⏺ retrieve/search ⏺ send message ⏺ update DB record

application-controlled data

⏺ files ⏺ DB records ⏺ API responses

user-controlled templates for AI interactions

⏺ document Q&A ⏺ transcript summary ⏺ output as JSON

AGENTS.md vs. CLAUDE.md

AI Adoption for Software Dev Tasks

[DO25]

AI coding := AI-assisted software development

• tab completions�• ghost text�• dev chat with LLM�• agentic coding sessions�• …

​

c

◼

o

X

◼

max@Google

year

GitHub Copilot

GPT3

2021

2022

2023

2024

2025

2030

Amazon Code- whisperer

Cursor

Aider

Windsurf

Cline

Lovable

Replit Agent

10

20

30

40

50

60

70

80

90

100

X refactoring

X

X

X

X

X

c churn

c

c

c

c

c ≈13%

o clones

o

o

o

o

o ≈25%

Devin

code quality

productivity gain

adoption

JetBrains Junie

GeminiCodeAssist

Claude Coder

bolt.new

Firebase Studio

Kilo Code

Continue

Zencoder

%

Google

Microsoft

Microsoft

Top ¼ WS25�YC startups

merged PRs

small project�from scratch

REQs

experts, �large repo

◼

◼

◼

◼

◼

Roo Code

GitHub Copilot�Experimental Chat

AI coding

◼

◼

◼

◼

◼

◼

◼

◼

◼

◼

◼

◼

◼

◼

AI code detector

Amazon Kiro

Vibe Coding

There's a new kind of coding I call "vibe coding", where I

• "Accept All" always
• don't read the diffs anymore
• just work around a bug or ask for random changes until it goes away.

⤄

Original definition: Auto-merging AI suggestions�with no review or understanding

Blurred definition: AI-Coding

10

20

30

60

70

%

churn c ≈7%

clones o ≈15%

refactoring X ≈3%

40

50

[Liu24] incorrect o ≈32%

[Liu24]: maintainability �issues of the correct o ≈70%

[ZH24] API misuse o ≈62%

Vibe Coding: Unsustainable

Productivity and Maintainability

Context Engineering: Code Selection (GitHub Copilot)

GitHub Copilot IDE Single Turn Pipeline [GH24]

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Pipeline

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Pipeline

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Pipeline

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Pipeline

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Sandboxing

Example: Isolation (Pipeline vs Sandboxing)

string-calculator-kata: sum() over string-encoded numbers with Sandboxing

Evaluation: Isolation (Pipeline vs Sandboxing)

👎 less reliable (no refactoring with re)

👎cost (3.6$)

👍 more flexible

👍 more reliable

👍 cost (1.16$)

👎 predefined subtasks

string-calculator-kata: sum() over string-encoded numbers

​

Pipeline (Cline’s slash commands�/new_task, /smol, /workflow, …)

Sandboxing (Roo’s Boomerang Tasks)

Real World Examples

Short example: Failure

…

…

“What's the �shortcut story �mentioned in the �memory bank?”

Long example: Full session with

• Memory Bank
• MCP servers

…

…

…

Evaluation: Cline vs. Jules for ETL Project

Task: ETL my Vise Coding Logs and Grades

Vibe-Coded Data Visualization UI by Jules

Vise-Coded Data Visualization UI by Cline

Vise-Logger: Manage The Most Important Artifact

MCP-Server: rate & store most important artifact, the vise coding session log

https://viselo.gr: manage and evaluate vise coding sessions

• concise visualization of each session, with link to attach to commit message
• Multilingual Format-Preserving Encryption for local pseudonymization of sessions
• Vise Coding advice, on own and public sessions
• statistics: best Vise Coding LLM / tool / MCP server / context management / …

​

Still in beta.

Early adopters can influence feature set.

Vise-Coding vs. Vibe Coding

Conclusion

Vibe Code!

s

The knowledge paradox

Here’s the most counterintuitive thing I’ve discovered:�AI tools help experienced devs more than beginners.

Bibliography

[BB25] Brigitta Böckeler: “The role of developer skills in agentic coding”. Martinfowler blog post. 2025

[DF25] David Faragó. “Vise Coding”. LinkedIn blog post. 2025

[GA24] GitHub, Accenture. “Research: Quantifying GitHub Copilot’s impact in the enterprise with Accenture”. GitHub blog and video. 2024

[GC25] GitClear. “AI Copilot Code Quality: 2025 Look Back at 12 Months of Data”. 2025

[Google24] Riya R Alex. “Over 25% of Google software is written by AI, says CEO Sundar Pichai. Is it a threat to engineers”. Mint. 2024

[HB23] Fabrizio Dell'Acqua et al. “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality”. Harvard Business School. 2023

[HP25] Hammond Pearce et al. “Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions“. Communications of the ACM. 2025

[IM25] Ivan Mehta. “A quarter of startups in YC’s current cohort have codebases that are almost entirely AI-generated”. TechCrunch. 2025

[Li25] Li, Ruiyin, et al. "Unveiling the Role of ChatGPT in Software Development: Insights from Developer - ChatGPT Interactions on GitHub." arXiv:2505.03901. 2025

[LK25] Leslie Kanthan. “Can Developers Embrace “Vibe Coding” Without Enterprise Embracing AI Technical Debt?”. unite.ai. 2025

[Liu24] Liu, Yue, et al. "Refining chatgpt-generated code: Characterizing and mitigating code quality issues." ACM TSEM 33.5. 2024

[MS25] Maxwell Zeff. “Microsoft CEO says up to 30% of the company’s code was written by AI“ Techcrunsh. 2025.

[MS25b] Matthew Griffin. “Microsoft CTO says 95% of the company’s code will be AI generated by 2030”. Fanaticalfuturist. 2025

[NN23] Jakob Nielsen. “AI Improves Employee Productivity by 66%”. NN/g. 2023.

[SD25] Simone Daniotti et al. “Who is using AI to code? Global diffusion and impact of generative AI”. Arxiv. 2025

[SL24] Shuang Li et al. "Assessing the Performance of AI-Generated Code: A Case Study on GitHub Copilot." ISSRE IEEE. 2024.

[TS25] Thomas Segura. “Yes, GitHub's Copilot can Leak (Real) Secrets”, GitGuardian. 2025

[VT25] Valerio Terragni, et al. "The Future of AI-Driven Software Engineering." ACM TOSEM. 2025.

[ZH24] Zhong, Li, and Zilong Wang. “Can LLM Replace Stack Overflow? A Study on Robustness and Reliability of Large Language Model Code Generation”. AAAI Conference on Artificial Intelligence 38.19. 2024.

Optional slides: LLMs & Tools

Local AI Coding

4

1 GPU

laptop

LLM

Engine

LLM

16 GPUs

Phi-3 Mini 3.8B (Q4); CodeGemma-2B (Q4)

Qwen2.5-Coder-7B (Q4); DeepSeek-Coder-6.7B-Instruct (Q4)

VRAM (GB)

6

8

12

16

24

32

64

80

Context

Qwen2.5-Coder-7B (Q5); DeepSeek-Coder-6.7B-Instruct (Q5)

Qwen2.5-Coder-14B (Q4); Phi-3 Medium-14B (Q4)

Qwen2.5-Coder-14B (Q5); DeepSeek-Coder-V2-Lite-16B (Q4)

Qwen2.5-Coder-14B (8bit); DeepSeek-Coder-V2-Lite-16B (8bit)

Qwen3-Coder-30B-A3B (Q4); Qwen2.5-Coder-32B (Q4)

Qwen3-Coder-30B-A3B (8bit); Qwen2.5-Coder-32B (8bit)

Qwen3-Coder-30B-A3B (FP8); Qwen2.5-Coder-32B (FP8)

Kimi-K2-Instruct (FP8); Qwen3-Coder-480B-A35B (FP8)

10³

[Apx]

4k

6k

8k

8k

12k

16k

16k

32k

48k

128k

Ollama

Local AI Coding: Specialized Models

Very little fine-tunings for special software engineering tasks

• A lot of SQL translation models
• many hobby models for many programming languages and APIs
• Predibase offers model hosting and�Reinforcement Finetuning on APIs, e.g. stripe:
• alternative:
• general coding LLM with large context &
• Context7, which knows Flutter & Sprint Boot

AI Coding: Specialized Models

Fine-tuning (RFT) helps, but post-training too, and is more popular, see EvalPlus:

Hosted AI Coding with Zero-Data Retention (ZDR)

ZDR of LLM & ZDR of AI coding tool!

• GitHub Copilot: in enterprise edition, only prompts & suggestions (28 days), user engagement (2 years) and feedback data (unlimited)
• JetBrains AI: in enterprise edition, or self-hosted
• Codex: for OpenAI ZDR customers
• Cursor: in Privacy Mode Legacy (“Privacy Mode” ≙ ZDR of LLM)
• Windsurf: opt into ZDR mode
• Cline/Roo/Kilo: pick LLM with ZDR and connect to external API and� codebase indexing, chat history, memories, analytics,
• Amazon Kiro: with IAM org plans or � content sharing
• Qodo: in enterprise edition with opt into ZDR and own ZDR LLM

GitHub Copilot Data Privacy: IDE

• “neighboring or related files � within a project”

Embedding Vise-Coding in Business Processes

Vise Coding process is compatible:

• to business processes
• with measures to mitigate risk
• with regulatory requirements

1. FOSS License Incompatibility

Generated code under FOSS license creating legal compliance issues

FOSSA

Black Duck

BigCode Attribution Tool

GitHub Copilot Filtering

2. Privacy Regulation Violations

Breaking GDPR, CCPA, and cross-border data regulations

Data Anonymization

PII Detection/Redaction

Data Residency Controls

3. Intellectual Property Leakage

Unauthorized disclosure of proprietary code and trade secrets

Federated AI

On-Premise AI

Encryption

GitHub Copilot Filtering

4. Legal and Compliance Challenges

Meeting regulatory requirements and audit standards

Software Bill of Materials (SBOM)

Continuous Compliance Monitoring

[PR25]

Organizational Factors: Clear And Communicated AI Stance

AI Stance: extend of (1) AI use expected and supported (2) clarity of allowed AI tools

influences individual performance:

​

​

​

influences organizational performance:

[DO25]

Teamwork And Collaboration (Tools And Approaches)

Integrate with best practices, tools, workflows (e.g. via MCP)

• shared context & versioning (git, shared memory bank, GitHub Copilot Spaces, Cursor Team Rules)
• CI (Julie, GitHub, Codex)
• communication channels (Teams, Slack, wiki, issue tracker)
• Vise Coding
• transparent action logs (Junie Actions Log, Vise-Logger)

​

No good solutions yet for

• shared context
• resolving conflicts in code and specs

Model Context Protocol

"shortcut": {

"command": "npx",

"args": ["-y", "@shortcut/mcp@latest"],

"env": {"SHORTCUT_API_TOKEN": "12345"},

"disabled": false,

"autoApprove": []

}

MCP Servers in Cline

Popular MCP Servers Marketplace

Optional slides: Older Ones

Articles up to July 2023 about Coding AI And Jobs

95

IDEs/Coding Tools For Vise Coding

Local:

GLM-4-32B

Kimi-K2 (benchmarked on LiveCodeBench v6)

Vibe Coding: Unmaintainable

Neither maintainable nor sustainable as technical debt accumulates rapidly with each iteration [...]. After some iterations, what you get is "write-only code", incomprehensible and unmaintainable for humans and machines [DF25].

​

/ ignored

Dead code

[BB25]

Context Management for Cline

Slash commands for context management in Cline:

• /newrule: create a markdown file in your .clinerules �directory for a pattern or rule to always follow
• /newtask: trigger new session with a context covering�overall plan, finished & next steps, like a dev handoff
• /smol: summarize current conversation to free up �context, but no new session
• /workflowname: execute specified workflow once �in an own session
• many iterations: /newtask and /smol degrades context, �prefer /workflowname

Evaluation (Personal Vise-Coding)

Date

Grade (top 1, flop 6)

Examples (Personal Vise-Coding)

What is on line 213 of file `completion.rs`?

What is on line 213 of file `completion.rs`?

What is on line 213 of file `completion.rs`?

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Modify the n8n workflow `PDG_Meetup_invitation.json` by adding a node `NormalizeArticleURL` between `Extractions` and `HTTP Request`. The node `NormalizeArticleURL` should be of type `OpenAI Message a Model`, read a string (might be a URL, might be a text with a URL, for instance `[our article](https://arxiv.org/pdf/2411.15124)` or a URL containing various values amongst which there is a PDF URL), and yield a valid URL that contains the string "PDF" (case insensitive).

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

In @/src/context_injector.rs, I have the line `Ok::<_, ChatError>(format!("# {heading}\n\n{content}\n\n"))`, but the turbo fish is not idiomatic Rust. How can I avoid the burbo fish?

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Plan with Gemini 2.5 pro exp:�I want to offer an alternative to assistant calls that offers the same functionality…, but on top of the completion calls…

What is the cleanest architecture and most idiomatic Rust way to do this? Where to put the markdown files? Where to put the new functions on top of the `completion.rs` functions already available?

=> Good suggestions

Act with Claude 3.7 Thinking: much too large changes

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Create a Python script that reads `x.csv` and writes a new csv file. The new csv file should have the following columns: `commit_link,metric,V1,V2,V3,V4,V5,V6,V7,V8,V9,V10`.

For each row in `x.csv`, there should be 4 consecutive rows, where the values of column `metric` should be "Rationality", "Comprehensiveness", "Expressiveness", and "Conciseness". In each other column, the values of those 4 consecutive rows should be the same and constructed as follows:

* `commit_link`: it should contain f"https://github.com/apache/{project}/commit/{commit}", e.g. for `commit`="e278b33d72141776dfc48d9e2466ce2961745b71" and `project`="ambari", it should be "https://github.com/apache/ambari/commit/e278b33d72141776dfc48d9e2466ce2961745b71 "

* `V1`: it should contain the contents of file with name f"V1/{commit}.txt"

* `V2`: it should contain the contents of file with name f"V2/{commit}.txt"

* `V3` to `V10`: likewise.

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Please rewrite `checker_report.html` so that the links in the table pointing to the detailed section, are not behind the test `FAIL` or `PASS`, but behind the intent name, i.e. the left column entries. That way, the links can be visualized in the typical way that links are usually visualized, so the user understands that he can click it to get to the details.

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Fill all sections of `Anonymization-Agents.md` with some keywords, to show what topics and what parts of `Anonymisierungs-Agenten_AbgelehnterAntrag.md` to use in each section.

​

Focus on the big picture, so only write some keywords per section, just to give an overview of how to distribute the information over the different sections. The distribution should help to write a grant proposal that is understandable and has an overarching theme that makes it extremely likely to get funding.

​

Consider the comments (wihtin `<!---` and `-->`) and the other markdown files to follow the advice of the grant giver, so that the reviewers will be happy.

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

How does `next_ai_message` in @/src/open_ai.rs compare to `next_completion_message` in @/src/completion.rs and to `next_structured_completion_message` in @completion? Do we need all of them?

Grade (top 1, flop 6)

Date

Examples (Personal Vise-Coding)

Help me cherry pick some git commits in the most sensible order.

Grade (top 1, flop 6)

Date

Optional slides: Papers & Research

How bad is context rot?

Popular chroma research article with severe context rot (starting at about 10k tokens):

• Checked only on two questions
• "What was the best writing advice I got from my college classmate?"
• "Which low-latency reranker is preferred for scientific domains?"
• Each question has 8 needles & 4 distractors, but one bad needle each

NoLiMa:

• Similarly bad results, but only on older models (up to Gemini 2.0 flash)
• Only newer model tested (not in paper): GPT-4.1, effective context length (85% retained) is 16k

LongMemEval:

• Similarly bad results (30% drop), but only on older models (up to GPT-4o)
• Only measured at 115k context size

Fiction.liveBench:

• GPT 4.1 severe drop (25%) at 1k already
• GPT 5 (and o3) awesome up to 120k

Context Rot

NOLIMA: Long-Context Evaluation Beyond Literal Matching

Best Practices for Vise Coding

[PR25]

[MJ25] SimCopilot: Bottom up workflow

[MJ25] SimCopilot: Evaluation Results (1)

[MJ25] SimCopilot: Evaluation Results (2)

[Li25] Developer-ChatGPT Interactions: Turns

[Li25] Developer-ChatGPT Interactions: Purpose

[Li25] Developer-ChatGPT Interactions: Activities

[Li25] Developer-ChatGPT Interactions: Activities detailed

[YZ25] Comprehension of Code vs Prompts

[YX25] Maintainability metrics complexity, size, names

[YX25] Maintainability metrics complexity, size, names

Bibliography Optional Slides

[MJ25] Mingchao Jiang et al. "SimCopilot: Evaluating Large Language Models for Copilot-Style Code Generation." �arXiv:2505.21514. 2025.

[Google24] Riya R Alex. “Over 25% of Google software is written by AI, says CEO Sundar Pichai. Is it a threat to engineers”. Mint. 2024

[Li25] Li, Ruiyin, et al. "Unveiling the Role of ChatGPT in Software Development: Insights from Developer-�ChatGPT Interactions on GitHub." arXiv:2505.03901. 2025

[MS25] Maxwell Zeff. “Microsoft CEO says up to 30% of the company’s code was written by AI“ Techcrunsh. 2025.

[MS25b] Matthew Griffin. Microsoft CTO says 95% of the company’s code will be AI generated by 2030. Fanaticalfuturist. 2025

[PR25] Partha Ray. "A Review on Vibe Coding: Fundamentals, State-of-the-art, Challenges and Future Directions." doi 10.36227. 2025.

[YX25] Yuliang Xu et al. "code_transformed: The Influence of Large Language Models on Code." arXiv:2506.12014.2025.

[YZ25] Yangtian Zi et al. "”I Would Have Written My Code Differently'': Beginners Struggle to Understand LLM-Generated Code." arXiv:2504.19037. 2025