A Time I Trusted AI Too Much
I made a mistake recently that cost me days of work.
It was for a side project for my wife’s business: a web app for order tracking.
With side projects there’s never enough time. 10 hours of uninterrupted time is a luxury. I might get that once a month.
I’m always in a rush, and I made THE big mistake:
I trusted AI too much.
My wife runs an interior design business. It’s her first year, and it’s been very successful, but we’re still figuring things out.
We desperately need to know:
What furniture items have been shown to the client?
What have they approved?
What’s been ordered/shipped/charged/…?
You get the idea. It’s a headache.
In a sense, there’s an easy solution…
Studio Designer is a web app that basically does what we need.
BUT it’s tough to pay $1,068 / year / user when you know you can create the tool yourself and host it for pennies on Firebase.
Even more importantly, if I write it myself, I have direct access to the database, and I can add whatever features I want.
It’s at least worth a shot.
So what was my mistake?
The web app needs to track lots of lists:
Vendors, products, clients, projects, selections, orders, invoices, expenses, etc.
First I wrote VendorList…check.
Next wrote ProductsList…check.
Then, to speed things up, I created a CommonList that could be re-used for everything.
So far so good. The problem was that I let AI write CommonList.
On the surface, this seems to be a reasonable request.
I already had two examples of what a list should look like. AI is good at generalizing across examples.
The problem is that, while it seems simple, this task is actually too hard for AI.
Why?
This is the “correct” solution that I wrote later:
EditableListItem: Edit item fields or delete list item
EditableList: Add or delete list items
ListPage: Database loading and navigation
VendorListPage and ProductListPage: Supply title, navigation links, and database methods
Here’s the key (sorry for the nitty gritty details):
Pop-up modals for adding, editing, and deleting all need to live at the lower levels, but the actual editing of the database needs to be done at the highest level. Changes to the database need to be propagated back down.
If you do it differently, you’ll find yourself in React Render Loop Hell.
That is where AI took me.
Thinking about it…
On the surface, generalizing across two list components seems super simple.
And honestly, for a human, it is.
But as you can see, it requires a non-trivial software design.
How good it AI at software design?
Not great yet.
GPT-4o does not have the power to perform this design correctly in the 10 seconds it takes to spit-out an answer.
GPT-o1 may do better. (Mr. Altman, can we talk about the naming?…)
AI will continue to improve. In the meantime, we need to understand its limits.
I love using ChatGPT for coding, but I did want to make sure I’m showing the bad as well as the good.
It takes discipline to use AI effectively.
I need to know myself and the AI tool well enough to only give it tasks that it can handle.
If you got this far, I’m impressed!
Thanks for taking the time.
I’ll continue to post what I learn about using AI for coding.
My goal is to learn alongside other developers willing to share their experiences with AI.