Does it work? Well, sort of.
I’m an old man. I am a trained systems analyst who loves to program. Back when I was working in a top level development group, we talked about the different types of programmers.
- Systems Analyst—A person capable of looking at how systems interact and designing systems.
- Analyst-A person that could look at a program and design programs
- Programmer-A person that could implement a design
- Coder-A person that could turn pseudocode into real code
Yes, I put myself at the top of that list. Not because I say so, but because my peers say so.
I enjoy every part of that stack. My family used to say that if I wasn’t programming, I wasn’t happy.
I still love programming. It still makes me happy.
Weaknesses
Today, it isn’t enough to know a language. You need to also know the libraries and the frameworks. And there are 1000s of libraries doing 1000s of things, some better than others.
To write good software today, you need to know your frameworks. But more than that, you need to know the patterns that work within that particular framework.
I just do not spend enough time in that world to know all the frameworks and all the patterns for those frameworks. Instead, I pick a framework and go for it.
My AI Assistant, Some Context
My development environment is Emacs with gptel. This environment gives me complete control of the AI’s context.
But what is “context”?
Context is all the information that is sent with your prompt. Which begs the question, “What is a prompt?”
When you are talking to an AI, everything you say to the AI is your prompt. “Who is the best chef?” can be a prompt. So is “In the fetch methods you are referencing self.client. This no longer exists; instead, we do client = GiteaClient() because GiteaClient is a singleton.”
The AI receives the prompt and evaluates it based on its training. Training is fixed. Every time you start a new conversation, you start with no history, just the trained AI. There is no doubt in my mind that any AI would be able to give some type of answer to “Who is the best chef?”
No AI, with just trained data would be able to answer the second prompt. It would have no references.
Context is all the extra stuff that gets sent with your prompt. That can include any files you explicitly sent. It will include your conversation, both what you sent as prompts and what it sent in reply.
It is that context that allows you to have a conversation that tracks.
My interface to my AI assistant allows me to add files as context. When I’m coding, the AI is sent the current version of my code files with each prompt.
Each buffer (Window) can have a different set of context files. In general, each code file buffer adds the other code files. All the code file buffers also add my master prompt for coding.
Finally, I have a conversation buffer. This is a buffer that has all code files and the master prompt as part of the context load.
Every time I send a prompt in the conversation buffer, everything above my cursor and all the file context is sent to the AI, which then responds.
My AI Assistant, What Should Work
What I want to happen is to be able to have the AI rewrite parts of code. The way this should work is that I highlight a section of code, start the rewrite process, I will be asked for a prompt: “Rewrite this if block to negate the test.” This is sent to the AI and the AI replies with pure code to replace what I had highlighted.
My tool shows me the new text. I can accept the new text, reject it, compare it, use a special model that allows me to easily examine differences and choose the old version or the new version. Or I can iterate, changing the prompt to get better results.
When I was testing with a fictional novel, this worked wonderfully. The one time I tried it with Python code, it attempted to replace the entire file. I rejected that and moved to other methods.
Most of that is that conversation buffer.
Regardless, when I get the rewrite tuned, that will be a game changer for me.
My AI, Dumb As A Rock
The thing to know about generative AI, is that they are dumb; they produce results based on what they think you want and what they see most often.
As an example, I’m coding in the Qt GUI system using PySide 6. This is a powerful framework. It gives me a graphical user interface that just works.
I asked Grok for a starting point that did certain things. It gave me a single file result. It did what it thought I asked it to do. That starting point saved me hours. My UI looked like what I wanted from the start.
Then the problems started. The ISO date format is very specific. YYYY-MM-DDThh:mm:ss+0.0 is an example. Notice the timezone is specified in that date, the +0.0 (which could be shown as ‘Z’). When Grok proposed its first version, it just stripped the time and timezone from the time date timezone string and said “That’s good enough”.
It wasn’t. I had to drive it to do it right.
The answer comes down to: this thing programs like a lazy student who waited until 2 hours before the due date to start coding.
You have to stay on top of it every step of the way.
And unlike a human, it will happily give you working code with tech debt and never tell you. Or blindly do it “the simple way” just because it doesn’t think about “doing it right”.
I’m always going back to Galaxy Quest, “I have one job on this ship, and it’s stupid, but I’m going to do it.” It sounded like a crazy thing, to have a crewman whose only job was to talk to a computer that understood and spoke English.
I have lots of jobs, some of them stupid, but I’m talking to this damn computer because you can’t trust its output, and too many don’t understand.