01 / The problemDT · 03
The problem
An AI coding assistant can generate plausible code without enough repository context. Letting it write changes automatically increases the cost of a wrong assumption.
← Back to selected work
CASE 03 — LOCAL AI / RAG / CONTROLLED EXECUTION
GGUF Orchestrator
A local-first coding assistant that retrieves repository context and lets the developer decide which generated changes may reach disk.
PythonFastAPIPostgreSQLpgvectorRAGWebSocketsGGUF models
EXECUTION MAPDT / 03
01
RequestReceive a coding request through the local interface.
02
RouteSelect an appropriate role/model for the request.
03
RetrieveSearch codebase embeddings stored in PostgreSQL + pgvector.
04
ProposeGenerate a diff and stream the result to the interface.
05
ReviewAccept or reject the proposed change before any write to disk.
SYSTEM DESIGN / INDEPENDENT BUILD
THE BUILD
Problem → decisions → proof.
A concise view of what was built, why the system is structured this way, and which engineering choices mattered most.
02 / The implementationDT · 03
The implementation
A FastAPI backend orchestrates local GGUF models, WebSocket streaming and codebase retrieval through PostgreSQL + pgvector. Project context and conversation state are persisted.
03 / The key decisionDT · 03
The key decision
Generated changes are proposals. The developer must accept a diff before it touches disk. This is a product-level control, not just a prompt telling the model to be careful.
04 / What this demonstratesDT · 03
What this demonstrates
Local model orchestration, retrieval-augmented generation, persistence, API design and a deliberately controlled execution path.
DESIGN PRINCIPLEAI output should be treated as a proposal with context and a review boundary—not as permission to execute.
MORE WORKInterested in the