This internship was for a sensitive government use case. The system had to run on an air-gapped network, so there were no external APIs, no cloud services, and no internet connection to fall back on. Everything had to be self-hosted. I worked on it on my own, from the first design through to deployment.

The offline RAG pipeline

The core of the work is an end-to-end RAG (retrieval-augmented generation) pipeline over the organization’s own data. It runs fully offline on a local machine.

It works in a few stages:

• Ingestion: the organization’s documents are loaded into the pipeline on the local machine. Nothing is sent anywhere else.
• Embeddings: the content is turned into vector embeddings and stored in ChromaDB, which acts as the local vector store.
• Retrieval: when someone asks a question, the pipeline searches ChromaDB for the content most relevant to that question.
• Answering: the question and the retrieved content are passed to a locally hosted LLM, which writes the answer from that content.

Because every stage runs on the same offline setup, the data never leaves the network.

The pluggable model layer

I didn’t want the pipeline to depend on one particular model. So I designed the model layer to be plug-and-play: any local LLM can be plugged in and swapped out without changing the rest of the pipeline. Ingestion, embeddings and retrieval stay the same, and only the model behind the answering step changes.

Exploring knowledge graphs with Neo4j

Alongside the main pipeline, I explored knowledge graphs and graph-based RAG as a possible extension of the offline retrieval. I used Neo4j for the graph and learned it while doing this.

To be clear, this was exploration, not a shipped feature. The pipeline that was built and deployed is the ChromaDB-based one described above.

Built with: ChromaDB, locally hosted LLMs, and Neo4j (explored).