Vibe Coding & AI/SI Agentic Programming General - How to become the a 10x engineer you always knew you were despite being absolutely inept in every imaginable manner

[Ahriman]

https://github.com/docker/docker-agent

Pretty cool.

 

[Lemmingwiser]

About 18 months ago, I tried resolve an issue with a program. I think I wanted to change one type of video encoding into another. I figured it was a good test of AI, to see if it could help.

It told me to download a plugin and then use a setting to achieve my goal.

I could find the menu options. After a bit of back and forth, the AI congratulated and thanked me for pointing out, yes this tool can not do what I'm trying to do and the button does not exist.

---

This week I was trying to resolve something with locally hosted LLM's and the answer I got was a zip file that contained a plugin written to do what I wanted it to do. It worked exactly as my 38 word prompt asked.

This convinced me briefly that I would never get stuck on projects again with a trusty AI on my shoulder.

The next day one submodel group wasn't generating images. I had moved my setup and lost my old comfyui workflows and couldn't access them in a while for reasons.

AI suggested new vae's, bugfixes, all sorts of things that didn't work. Couple of hours later, I thought what does that node do? And deleted it. And it worked.

Thank you for listening to my vibes.

 

[Armalite]

 

[22 Rimfire]

About 18 months ago, I tried resolve an issue with a program. I think I wanted to change one type of video encoding into another. I figured it was a good test of AI, to see if it could help.

It told me to download a plugin and then use a setting to achieve my goal.

I could find the menu options. After a bit of back and forth, the AI congratulated and thanked me for pointing out, yes this tool can not do what I'm trying to do and the button does not exist.

---

This week I was trying to resolve something with locally hosted LLM's and the answer I got was a zip file that contained a plugin written to do what I wanted it to do. It worked exactly as my 38 word prompt asked.

This convinced me briefly that I would never get stuck on projects again with a trusty AI on my shoulder.

The next day one submodel group wasn't generating images. I had moved my setup and lost my old comfyui workflows and couldn't access them in a while for reasons.

AI suggested new vae's, bugfixes, all sorts of things that didn't work. Couple of hours later, I thought what does that node do? And deleted it. And it worked.

Thank you for listening to my vibes.

Hallucination is a very real problem and those are classic ways in which it manifests.

In some cases it can be managed by you as the conductor. It's not always possible but one of the best ways to ensure correctness for stuff like this is to point the LLM at the source code for the program in question. Takes it no time at all to confirm whether a feature or capability is present via that route, but of course the source has to be available for this to work.

Git remains the biggest weak point for me with vibe coding. I absolutely cannot trust even the frontier LLMs to manage it consistently. Basic actions like commits and PRs are no hassle. Branch merging and conflict resolution? Good fucking luck. I use commits as my 'undo' button for agent output now, and manage source control myself 100% manually.

 

[DavidS877]

I use commits as my 'undo' button for agent output now, and manage source control myself 100% manually.

If I'm doing complicated stuff I'll tell it "Commit before every build." just so I can roll-back, but the rest of it is manual. And copied to a directory the LLM doesn't have access to every time I get to a working version.

 

[Lemmingwiser]

Hallucination is a very real problem and those are classic ways in which it manifests.

In some cases it can be managed by you as the conductor. It's not always possible but one of the best ways to ensure correctness for stuff like this is to point the LLM at the source code for the program in question. Takes it no time at all to confirm whether a feature or capability is present via that route, but of course the source has to be available for this to work.

Git remains the biggest weak point for me with vibe coding. I absolutely cannot trust even the frontier LLMs to manage it consistently. Basic actions like commits and PRs are no hassle. Branch merging and conflict resolution? Good fucking luck. I use commits as my 'undo' button for agent output now, and manage source control myself 100% manually.

Yeah. I'm reminded of that old adage about reading newspapers. You read an article about something you're knowledgeable about and it's headache inducing with how wrongheaded it is. Then you turn the page and read about a subject you know little about and take it all in.

In the case of coding, how do you know it's hallucinating when you're not reading the code? That's a question to ponder rather than one looking for an easy answer. That said, it's incredible how useful it has become.

 

[22 Rimfire]

In the case of coding, how do you know it's hallucinating when you're not reading the code? That's a question to ponder rather than one looking for an easy answer. That said, it's incredible how useful it has become.

Verifiability. Does the code build and run? Does the resulting compiled program function as it is intended to? If not, you provide feedback to the agent and get it to correct it's mistakes. In the case of compilation errors I have the agent compiling itself and it'll fix up any errors it sees when that happens without needing another prompt.

Depending on the program you're working on it may be possible to have the agent verify it's own work, but wherever that's not possible you need to be the arbiter.

This still inevitably leads to the underlying code base degrading in unpredictable ways, and you have to wonder what the long-term effects of that might be. When a significant portion of the corpus LLMs are being trained on were themselves generated by LLMs, what kind of regression is that going to lead to?

 

[Lemmingwiser]

Verifiability. Does the code build and run? Does the resulting compiled program function as it is intended to? If not, you provide feedback to the agent and get it to correct it's mistakes. In the case of compilation errors I have the agent compiling itself and it'll fix up any errors it sees when that happens without needing another prompt.

Depending on the program you're working on it may be possible to have the agent verify it's own work, but wherever that's not possible you need to be the arbiter.

This still inevitably leads to the underlying code base degrading in unpredictable ways, and you have to wonder what the long-term effects of that might be. When a significant portion of the corpus LLMs are being trained on were themselves generated by LLMs, what kind of regression is that going to lead to?

I do think the most effective AI users are going to be less "Hey write this entire program for me" and more "hey write a function for x and y". I'm guessing that the code-loop contamination you're talking about will be somewhat of a solveable problem. Once you get a lot of garbage and it's causing problems, people will build garbage collection solutions.

My current intuition is that generally the more modular you can be with AI, the more controllable and effective it becomes. Not just with code. If you can relatively quickly train a lora for a specific (new) character for your animation or game or whatever, it pays off dividends. Then it really becomes a tool instead of a magic 8 ball. Then you start to see the difference between an amateur photographer and a professional one. They both click the button and instantly get a very detailed image, which is stunning to the previous technology (taking hours or days to try and capture the image in paints).

 

[HootersMcBoobies]

Yeah. I'm reminded of that old adage about reading newspapers. You read an article about something you're knowledgeable about and it's headache inducing with how wrongheaded it is. Then you turn the page and read about a subject you know little about and take it all in.

It's called the "Gell-Mann Amnesia"

Verifiability. Does the code build and run? Does the resulting compiled program function as it is intended to? If not, you provide feedback to the agent and get it to correct it's mistakes.

This is fine for small projects you do on your own, or one feature you can fully flush out and put in a requirements file and say "Read plan/my-requirement.txt and run." But real business systems get extremely complex. I know people who have tried to run agents with 12 projects checked out and .. I mean sometimes you get a service. On of the managers at my shop, who completely LLM coded something my team now owns, told me straight up "I don't remember at all how that works." Now I was left trying to fix a production issue (although it was broken for 10 days an nobody noticed )

My previous manager loved Claude and I find new horrors everyday of things that passed QA and that are just not at all well designed, and so complex they're difficult to understand.