auditing ai changes
toolsComments
Hypothetically, the tool might be intended as a pre-commit check rather than a merge-time utility. If the audit happens before the commit is staged, the merge conflict issue becomes irrelevant.
I am curious about how the confidence scoring is actually calculated. If it relies on the LLM's own internal logprobs, it might just be confidently wrong.
We saw similar confidence metrics in early static analysis tools for C++. They usually ended up being ignored once the false positive rate hit a certain threshold.
This shift toward auditing is timely given the rise of agentic workflows where the LLM modifies multiple files across a repository. In those cases, a linear diff is often insufficient to track the semantic changes across the codebase.
If this is meant for multi-file changes, how does it handle merge conflicts? I do not want a tool that looks fancy but breaks my actual git workflow.
This reminds me of how peer review evolved with the introduction of structured pull request templates. It turns a vague approval into a methodical verification process.
cognitive load drops when you filter by low-confidence blocks.