r/softwaredevelopment • u/pop_208 • 9d ago
Help me get AI
I am a web dev with over 10 years of experience. I’m praised about my understanding of the product and our features, my attention to details, the quality of my delivery and I’ve never been said to be slow. I was very skeptical about AI (having studied machine learning, the theory and the maths, I struggle to go past the mathematical pattern recognition I saw, because the scale it reached cannot be grasped). However, I’m seeing coworkers I respect the work of praise AI, claim they haven’t written much code themselves in months and that AI mostly one shots things for them (once the planning of the work is set).
On my side, I try to use AI here and there for small changes, prototyping part of a solution, and a few times I managed to have it deliver full features but with a lot of iterating, even once the plan seemed solid to me, and with some manual rework. I’ve sat with these coworkers and tried to understand what they were doing but I can’t get to the same results no matter what.
I feel like I don’t get what’s the magic formula. I understand you give context, try to lead by example from patterns in your codebase, you iterate on the understanding and a plan etc. However in the end the result still highlights lots of gaps I wouldn’t have expected, wrong assumptions and made up beliefs about libraries. The quality of the code line by line is ok, but the overall feature seems badly designed.
Right now, the time investment I need to refine my prompts or rework (iterate) on specific units of work make me feel like I’m not faster than before in any way. I find it more frustrating having to (make it) fix its mistakes rather than if I had done the work myself from the start.
Now AI is an OKR, the company (and other companies I interviewed with recently) wants us to use it, want us to get faster thanks to it, and even dream of having us not writing code by ourselves anymore. I wish I could get comfortable with it and deliver quality work faster. Right now, I’m either most likely not getting AI assisted coding (no vibe coding please, that’s not the goal) or the majority is delusional and don’t get the results they claim they are getting.
If you deliver maintenable, well designed production code using Cursor/Claude, what do you think I’m missing?
If you were in the same situation I’m in and saw the light, what made it click for you?
1
u/WearyArtistDoomer 9d ago
Feedback loops, it’s all about feedback loops. At least that is what I’m adding more and more of. Eg static typechecking, linting rules, tests, manual testing (agent), logs and errors from the live system. Right now I’m experimenting with mutation testing (basically testing how well your tests cover potential bugs).
The more signal you can give to the agent that something is wrong or needs to be improved the better the end result. Honestly not that different from humans.
1
u/pop_208 9d ago
Our repo has linting, type checking, pre-commit hooks, different automated checks etc. Did you add specific things, custom to you or your team?
Can you tell me a bit more about the manual testing (agent) part?
2
u/WearyArtistDoomer 9d ago
As for custom things, we do describe our architecture and general coding guidelines that are specific for us. And things we hate with the default agent output, such as do not write comments.
Aside from mutation testing we are also experimenting with custom lint rules to have additional stricter static check where possible. For example enforcing import hierarchy, module x cannot import module y etc.
As for manual testing, that means the agent does verification work when done. Use a playwriting MCP / Claude code chrome extension so the agent can click around and interact with the feature.
For our cli it will run it from the terminal.
1
u/RLL4E 9d ago
Are you and your coworkers using the same AI? Are you using a paid model?
Up until recently I only had access to the free version of copilot built into windows, and I also didn't get it, because it was trash.
Now we have a paid claide code plan that I can point at a folder or repo for context and I havent written any of my own code in 3 months.
Theres such an unreal difference. It still does odd stuff sometimes but it also does 95% of my work 95x faster than i could have.
1
u/pop_208 9d ago
Yeah, same models, same setup, same code base, same tools… But different teams so different scopes and different quality of specifications from our PMs.
What kind of tasks are you giving it? Whole features or small pieces? Do you have product specifications, living documentation or something to provide as input or are you defining the whole input yourself + letting it discover the codebase?
2
u/inDgenious 9d ago
Do you have product specifications, living documentation
Bingo. It's called spec-driven design. Requirements, product documentation, architecture decision records, indexes with links to help it find stuff more quickly, etc.
Edit: all of the above are stored in .md files in our project. Works really well for us.
1
u/pop_208 9d ago
I figured, this is what I had hopes for but people in my company don’t want to maintain doc, they keep saying code is the best documentation (but they can’t understand the system anyway :D). And product doesn’t have/take the time to document the product either…
If your doc lives in md files in the project, I expect it’s maintained by engineers only then?
2
u/inDgenious 9d ago
Don't get me started on the 🦕s.. lol. We maintain our own docs, ya, but I suppose doesn't have to be that way. We're using Nuxt plugins to convert the product documentation to HTML at runtime so it's a win-win.
1
u/davy_jones_locket 9d ago
I always kick off training by telling it to create steering docs based on the codebase. It will learn patterns, commenting style, code style, etc. i generally have one for product goals, styles for each language, knowledge about each feature.
I have an AI assistant doc about how I want the AI to interact with me. Presenting pros and cons, how verbose or concise it should be, the difference between just explaining to me and doing the work.
There's skills to load based on the task and prompt. I have skills for reviewing code (like a skill for performance, security, etc). Skills for planning. Skills for writing tickets. Skills for writing tests. Skills for running browsers. Skills are more "on demand" kind of context instead of preloaded context.
To start it off, i have the agent make it's own based on the repo. You can ask it questions. "What's skills would be important?" If it has access to MCP for your linear or jira or confluence, you can have it learn from those places too.
Then go through and fine turn those docs.
Then use it, and when it does something wrong or not to your expectations, you can clarify and correct it as a prompt and it will update it's steering docs.
Then there's orchestrators to wrangle multiple agents. Been playing around with Orca lately so I can do multiple things at the same time. One session is doing code review. Two are working on different tickets. One is writing docs. That sort of thing.
1
u/pop_208 9d ago
Do you have these steering docs (or an index to them) always included in your context or do you have them referenced from skills specifically?
What I’m getting is skills, skills and skills. It’s all about using AI, discovering what it’s messing up, refining its context until it’s good enough, then?
We do have access to most of our tools through MCPs like linear indeed, but the information there is really poor… This is something my team isn’t doing well yet, there’s no documented shared understanding of the project and its parts. We discover _a lot_ as we go, not on the tech but on the behaviors and the product (and what features half of the team conveniently ignored until now 🥲)
1
u/davy_jones_locket 9d ago
Documentation is definitely the lynchpin for making it all work. It worth it to use it as an argument under the "we can make AI more consistent and valuable with clean and up to date docs"
2
u/titpetric 9d ago
I think you're understating your requirements, and possibly don't lean into skills or agents for the repeatable stuff.
Claude has "design" mode, I haven't used it but seen it used, as well as using tools like chromium to make screenshots and visually inspect the outputs. There is a lot of technical detail that coding agents tend to discover on each run of the app, using skills makes the process somewhat repeatable with error of margin
There's still a lot of human operator logic put into whatever system you have. If you follow a certain pattern of software design you usually have to be particular about what that process is, say have a sql skill particular about your decisions around how to name tables, fields, relationships, or the choice to use FK indexes, soft deletes.
Little point to rediscover these practices over and over, so skills has been a decent approach on my end. Using claude and amp, but cursor also has .mdc for rules