What 105 AI experiments taught me

What the data actually say, and what I didn't expect to find

The bug wasn't new. Claude reintroduced in v30 exactly what it had just fixed in v29.

This tracking tool, the one behind every number in this post, isn't code I wrote. I asked Claude to build it, ran each version against my real files, and reported back what broke. That's the whole loop: it writes, I test, I tell it what's wrong, it revises. Iteration by iteration.

Somewhere in that loop, one bug turned up three separate times wearing three different disguises. I'd added a column to my tracking spreadsheet, and the code had three different places that needed to know about it: one to read data, one to write it, one to figure out which version of the spreadsheet it was looking at. Claude fixed the first. Then the second. It took a third pass, a week later, to catch the last one.

Same bug, three disguises, three iterations before it was actually squashed. Claude didn't remember catching it the first two times. I did.

I've been running AI experiments since April 2025, systematically enough to track them. As of June 2026, I've got 105 projects across 7 AI tools: Claude, ChatGPT, Gemini, Copilot, NotebookLM, Otter, Perplexity.

Most of what gets written about AI productivity is anecdote dressed up as insight. I wanted my own numbers. Here's what they say.

More than coding

That opening bug story is a coding story, and it would be easy to assume this whole dataset is about coding. It isn't. 105 projects sounds like a big number, but the mix behind it might be the more interesting one. Coding is the largest single category, at 20 projects, but it's less than a fifth of the total.

Type	Count	Example
Coding	20	Transcript Editing Tool (ChatGPT, 40 iterations)
Document editing & review	16	Firepit project (ChatGPT, 3 iterations)
Blog editing	11	Blogpost - Don't Wing It (Claude, 5 iterations)
Admin	10	Convert Bank Statements (Claude, 11 iterations)
Report	10	Life Insurance Options (Claude, 22 iterations)
Research	8	AI Impact on Jobs (Claude, 2 iterations)
Creative	7	Birthday eCard Video Game (Claude, 24 iterations)
Presentation	6	Alaska Tour (ChatGPT, 8 iterations)
Persona/voice projects	4	Ask Grandpa chatbot (NotebookLM, 9 iterations)
Personal	3	Credo, Values & Goals Update (ChatGPT, 13 iterations)
Product comparison	3	Dishwasher Comparison (Claude, 4 iterations)
Advising	3	Interview Practice (ChatGPT, 2 iterations)
Analysis	2	Chat & Project History (Gemini, 6 iterations)
Health	2	Health Data Conversion (Claude, 3 iterations)
Total	105

That range is its own finding. AI as a co-pilot shows up everywhere I let it in: a Christmas vCard, a fund proposal, a mentoring analysis, alongside the coding work. The rest of this post drills into a few specific examples, coding especially, because that's where the iteration counts get interesting. But the breadth matters before the detail. Most of what I do with AI isn't code.

What this dataset is, and isn't

One honest caveat before the numbers: this is one person's work.

105 is every AI project I did and saved, not a sample I picked from a larger pool. The selection happens upstream of that: which tasks I decided were worth bringing AI into in the first place, and whether the session left a file behind. A quick one-off question I didn't save doesn't exist in this dataset.

I'd treat these numbers as one careful person's pattern, not a population average. Your mileage will vary with what you use AI for and how you work.

The number that surprised me

The table above gives a glimpse of the range project to project, but the full spread is wider than any single row shows. Before I ran this, if you'd asked me the typical number of AI iterations to get something usable, I'd have guessed high. AI is capable, but it's incomplete, and getting from draft to done takes work.

Iterations ranged from 1 to 68.

That range is the real finding. A one-shot win and a 68-round project both count as "a project" in this dataset, and no single average captures what that spread means. The median is 4: half of my 105 projects wrapped in 4 rounds or fewer. The mean is 9.4, pulled upward by a handful of long projects.

Coding is where the range opens up. Coding projects average about 21 iterations. Everything else averages about 7. The clearest run-check-revise example is a Transcript Editing Tool, built with ChatGPT, to catch the repetitive fixes in my weekly Data4Good transcripts: correcting member names, product names, the same edits week after week. It took 40 rounds to complete.

So, the first thing the data says: it depends on what you're asking AI to do. Most knowledge work converges fast. Coding can be a long haul.

Why coding is different

That range doesn't come from bad prompting or a bad model. It's what coding actually requires: run it, test it, report back what's wrong, get a revision, uncover the next problem, repeat.

There's also a memory problem. AI memory is scoped to a single chat. As long as I stay in the same project chat, it can recall earlier versions. Start a new chat, even the next day on the same project, and I'm re-briefing from scratch. Even carrying context over myself means copying and pasting from the old chat into the new one.

I have to re-brief it. That's exactly what happened with the column-offset bug from the opening: I was the one who remembered catching it before, not Claude.

That's just where coding assistance is right now.

Working across AIs

There's something the iteration counts miss. Projects don't always stay in one AI. Looking back at the data, I combine tools within a single project in a few distinct ways.

•       Playing to strength. Different AIs are better at different things, and I'll use more than one in the same project. My Christmas video card is the clearest example: image generation in ChatGPT, storyboarding in PowerPoint with Claude, 18 iterations combined.

•       Critique and improve. I'll take one AI's code or output and hand it to another AI to review, tighten, or catch what the first one missed.

•       Switching when one struggles. Sometimes an AI just isn't working out on a project, so I'll move the whole thing to a different tool.

•       Cross-checking. Feeding one AI's answer to another is a form of fact-checking, or at least assumption-checking. It's a habit I recommend to my students. A second opinion is one prompt away.

None of this shows up in a single project's iteration count. It's part of the real workflow behind these numbers.

The time question

Here's the finding people pushed back on hardest when I shared an early draft: in my own rough sense of it, many AI-assisted projects took 3 to 4 times longer than if I'd just done them myself, without AI.

I want to be honest about that number. I didn't use a stopwatch for every session. It's an impression from watching myself work, not a precise measurement, and I'd treat it as directional.

But if it's even roughly right, it says something. For a lot of routine tasks, iterating with AI can take longer than just doing the work yourself. A key example is project #56, the Transcript Editing Tool.  By the time it took to get the program right (40 iterations, to be precise), I could have edited the doc a dozen times. In the end, I abandoned it: it rarely got all the edits right, and I've been correcting the transcripts by hand ever since. I've filed that one under learning experiences, not productivity wins.

Where AI earns its keep for me is capability. Coding is where this shows up clearly. I'm not a developer. I don't write Python code. Many of the project results exist because AI built them with me. That's capability I didn't have, and it's worth more to me than any hours saved.

Time-saver versus capability-extender. That distinction shapes what I hand to AI and what I keep for myself.

AI use compounds

One thing I didn't expect going in: using AI more makes you use it more.

My project pace nearly tripled over the 14 months I've tracked, from about 4 projects a month in early 2025 to 10 to 12 a month by early 2026. Part of that is speed. Most of it is habit. I got better at spotting when AI was the right first move, and the better I got, the more often I reached for it.

I wrote about this a while back as the conversational approach to AI [1]: treat it like a working session with a colleague, not a query to a search engine. The skill grows with practice, and practice compounds.

The human never left

105 projects. Zero ran start to finish without me in the loop.

I don't think the next model release closes that gap, because none of this is really a model-capability problem. AI can't see my file system unless I give it access, and I'm not willing to give it that kind of blanket access. That's my boundary, regardless of how good the model gets. It can't check its own facts against sources I haven't given it. It can't hold context across sessions without a re-brief. And it can't tell me when a result is wrong in ways specific to my world: a formula error in a financial model, a missing source in a research summary, a chart with the wrong axis, a blog post that reads like AI wrote it. That takes knowing my world. No model upgrade changes that.

The human is in the loop because that's where the judgment lives.

AI as co-pilot is still the right frame for me. AI makes me sharper. But I'm still the one who decides where we're going and when it's right.

Will Agentic AI change any of this?

I want to flag this one as speculation, not data. Nothing in my dataset covers agentic AI yet, because I haven't run enough agentic projects to say anything real.

My hypothesis: coding projects built on a run-check-revise loop, run it, test it, report back what's wrong, revise, repeat, will look the same whether the AI acts autonomously or waits for me at each step. The loop doesn't change. Only who triggers each round does.

I'll write about this properly once I have real agentic projects in the dataset instead of a hypothesis.

What I'd tell someone starting out

Here are my top 6 recommendations based on my 105 trials:

1.     Start with projects that need a lot of iteration: coding, detailed reports, financial models, multi-round research. These expose limitations fast and build your instincts faster than one-shot tasks ever will.

2.     Version your output files from day one. v1, v2, v3. It's a record of how the work evolved, and it happens to make iteration counting automatic if you ever want to track this the way I do. That also means asking for output files as you go, so you keep the history and the code you tested.

3.     Track your experiments, even in a plain spreadsheet. You can't see the patterns until you look at them together, and that's where the real learning is.

4.     Expect to iterate. Across 105 projects, iterations ranged from 1 to 68, with a median of 4. AI isn't a one-shot machine, and depending on what you're building, it might not be a four-shot machine either. Treating the first draft as the final one is the mistake I often see.

5.     Treat every session like a conversation. Pose the problem, see what comes back, add context, correct what's wrong, push on the weak spots. AI gets more useful as the conversation goes on, same as a colleague or research assistant does. [1]

6.     Match the tool to the task. If a project needs more than one AI can offer, use more than one. When the stakes are high, get a second AI's take before you trust the first answer.

One more thing

The dataset I just described was built using AI. The Python, the PowerShell file indexer, the fuzzy matching logic: all of it developed iteratively with Claude, ChatGPT, and Copilot. This tool is, in a sense, all the other 104 projects looking back at themselves.

That's the part I find most interesting about running an experiment like this on myself. You're not just using AI. You're watching, from the inside, how it actually works on something that matters to you.

If you're doing anything like this yourself, tell me about it in the comments. And if you'd like a copy of the dataset, leave a note there too.

[1] See "You Are Your Career Pilot. AI Is Your Copilot," Blogpost, Feb. 26, 2026, https://eghapp.blogspot.com/2026/02/you-are-your-career-pilot-ai-is-your.html

Full disclosure: I used Claude to help draft this post, drawing from my AI Experiments Analysis Report (v5, June 2026) and 14 months of project notes. I provided the outline and edited the final copy. Another collaborative use of AI.

The postings on this site are my own and don't necessarily represent positions, strategies or opinions of any of the organizations with which I am associated.