What is the difference between the models writing code “well” and their performance in this context? Are we referring to readability?
Genuine question. If we use agents to read, edit, and review code, why do we care about readability? That’s a human constraint. Unless attempting to do those three is not effective and thus requires human attention to correct issues which would justify readable code. If that’s the case; why use the agent to edit the code in the first place?
“Writing code well” includes several relavant things:
Security implications
Stability and edge case handling
Performance
Quality of output (ie. for UI/UX)
Readability (Agents need to read to make changes too! Violating DRY/SOLID/etc. could still cause issues!)
As far as I’m aware you do currently still need a human to ensure stuff like this is followed. People use agents because they don’t care about the above, or because they can get close enough and intervene to fix any issues that appear.
Two, there is a decent body of evidence to suggest that this method does not actually speed up development unless you go full lights out software factory which… Why would you want to do that?
EDIT:
As an aside this actually reminds me of the Xerox park study into efficiency gains for keyboard heavy workflows vice mouse heavy workflows. Keyboards were perceived as faster by subjects but when actually measured the mouse was faster.
You joke but that’s basically the conclusion of both the newer agent studies and the older Xerox park study on peripheral use. Language and logic are stored in an easier to access location in the brain than the kinds of skills used in architecture and planning.
To be clear, I’m no rube. I recognize that these are powerful tools. But I suppose my take here is that if an LLM is necessary to get rid of a lot of the boilerplate and setup for a task that indicates we should explore how we’re doing the task.
What I want to see is using these tools not to delegate our problem solving but finding ways to enhance and accelerate it and I don’t think writing spec sheets is the answer.
What is the difference between the models writing code “well” and their performance in this context? Are we referring to readability?
Genuine question. If we use agents to read, edit, and review code, why do we care about readability? That’s a human constraint. Unless attempting to do those three is not effective and thus requires human attention to correct issues which would justify readable code. If that’s the case; why use the agent to edit the code in the first place?
“Writing code well” includes several relavant things:
As far as I’m aware you do currently still need a human to ensure stuff like this is followed. People use agents because they don’t care about the above, or because they can get close enough and intervene to fix any issues that appear.
Agents for generation, humans for review
One, that sounds truly miserable.
Two, there is a decent body of evidence to suggest that this method does not actually speed up development unless you go full lights out software factory which… Why would you want to do that?
https://ide.mit.edu/insights/ai-productivity-and-roi/
EDIT: As an aside this actually reminds me of the Xerox park study into efficiency gains for keyboard heavy workflows vice mouse heavy workflows. Keyboards were perceived as faster by subjects but when actually measured the mouse was faster.
That’s just because I need to stop and appreciate my efficiency for 10 seconds every time it saves me 10 seconds :-)
You joke but that’s basically the conclusion of both the newer agent studies and the older Xerox park study on peripheral use. Language and logic are stored in an easier to access location in the brain than the kinds of skills used in architecture and planning.
To be clear, I’m no rube. I recognize that these are powerful tools. But I suppose my take here is that if an LLM is necessary to get rid of a lot of the boilerplate and setup for a task that indicates we should explore how we’re doing the task.
What I want to see is using these tools not to delegate our problem solving but finding ways to enhance and accelerate it and I don’t think writing spec sheets is the answer.
I think you’re correct, the amount of rigor in your prompts seems orthogonal from whether you’re using them for architecture or implementation.