Building Semantic Memory with RAG
Most AI tools forget you the moment you close the tab
If you have ever used an AI writing tool for more than a few days, you have probably noticed the same limitation. The model is impressive in the moment, capable of generating fluent prose, answering complex questions, and even mimicking specific tones when prompted correctly. But it has no understanding of your past work, your writing style, or the documents you already created. Every session starts fresh, as if you are meeting for the first time.
This is not a model limitation in the traditional sense. Modern language models are remarkably capable. The problem is system design. Large language models are stateless by default. Without additional architecture to persist and retrieve information, they have no mechanism to remember anything beyond the current prompt window. For creators working on projects that span weeks or months, this statelesness quickly becomes a serious bottleneck. You end up spending more time re-explaining context than actually creating.
Why standard AI cannot retain creator context
Large language models generate responses based on the input they receive within a fixed context window, typically ranging from a few thousand to over a hundred thousand tokens depending on the model. Once that window is exceeded or the session ends, the information disappears entirely. There is no persistence layer. The model does not know what happened yesterday, last week, or in any previous conversation.
In practice, this architectural limitation creates several recurring problems for creators. The model does not remember previous drafts, so you cannot reference edits you made earlier. Reference documents and research must be pasted repeatedly into every new session. Writing style and tone reset across conversations because the model has no memory of how you prefer to communicate. Long documents and complex projects exceed context limits, forcing you to manually chunk and summarize your own work. Prompt engineering can mitigate some of these issues, but it does not solve them. The system needs an additional layer that can retrieve relevant information dynamically at runtime instead of relying on users to manually reconstruct context every time.
Using RAG to give AI a working memory
At Plura, we address this limitation using Retrieval Augmented Generation, commonly known as RAG. RAG is an architectural pattern that allows AI systems to fetch relevant context at inference time rather than requiring everything to be stuffed into a single prompt upfront.
Instead of asking the model to remember everything you have ever written, we let it read only what matters for the current task. When you write a query or make a request, the system embeds your input, searches a vector database for semantically similar content from your past work, and injects the most relevant chunks into the prompt before generating a response. This approach keeps generations fast, contextually grounded, and directly relevant to your actual content rather than generic training data. The model reads from your personal knowledge base, not just its pre-trained weights.
How Plura reads and understands user documents
When a user uploads a document, whether a PDF, text file, or other supported format, Plura does not simply store it as raw text. The document goes through a processing pipeline designed to make its contents semantically searchable.
First, the document is parsed and split into smaller semantic chunks. Chunking strategies vary depending on document structure, but the goal is to create passages that are meaningful in isolation while remaining small enough to fit into context windows efficiently. Each chunk is then converted into a high-dimensional embedding using a language model specifically trained for semantic similarity tasks. These embeddings capture the meaning of the text in a format that allows for mathematical comparison. The embeddings are stored in Qdrant, our vector database of choice, which enables fast approximate nearest neighbor search across millions of vectors. When you write or ask something in the studio, we embed your query using the same model and retrieve the chunks with the highest semantic similarity. Only those relevant chunks are passed into the language model as context, allowing the AI to reference your prior work naturally without exceeding token limits or introducing unrelated noise.
System architecture behind Plura's memory layer
Plura is built with a deliberate separation between product logic and AI orchestration. This architectural decision keeps the system modular, maintainable, and scalable as workloads grow.
The frontend and user-facing APIs are handled by Next.js, providing a responsive interface and efficient server-side rendering. AI-specific workloads, including document processing, embedding generation, and retrieval orchestration, run on a separate FastAPI service. This separation allows us to scale AI compute independently from web traffic. Qdrant serves as our vector database, storing embeddings and handling similarity search queries with low latency. Supabase provides the relational database layer for metadata, user data, workspace state, and document records. Cloudflare R2 handles object storage for uploaded files, offering cost-effective storage with zero egress fees. This architecture ensures that each component can be optimized and scaled independently, and it provides clear boundaries for testing, deployment, and future iteration.
Managing context windows without sacrificing accuracy
One of the most significant challenges in building RAG systems is context selection. Including too little context leads to shallow, generic responses that miss important nuances from your work. Including too much context increases latency, consumes more tokens, and introduces noise that can confuse the model or dilute the relevance of retrieved information.
Plura addresses this tradeoff through several mechanisms. We limit retrieval to a small number of highly relevant chunks, typically between three and eight depending on the task complexity. Retrieved chunks are ranked by similarity score, and we apply a minimum threshold to filter out marginally relevant content. We also scope retrieval to the active workspace and current document whenever possible, ensuring that results reflect the immediate project context rather than unrelated work from other areas. Additionally, we experiment with re-ranking strategies and metadata filtering to further refine result quality. These techniques keep responses focused and ensure that AI output reflects your actual work rather than generic patterns from training data.
Why this is more than a ChatGPT wrapper
The term "wrapper" gets used dismissively in AI discourse, and for good reason. Many products simply pass user prompts to an API and return the result without adding meaningful value. That approach works for simple use cases but fails entirely for sustained creative work.
A system like Plura is fundamentally different. We manage memory, relevance, and continuity across time. We process your documents, generate embeddings, store them persistently, and retrieve them intelligently when needed. We scope context to your workspace. We handle chunking, ranking, and injection behind the scenes. The difference is not visible in a single response, but it becomes obvious over repeated use. When AI understands what you have already written, what research you have collected, and what you are currently trying to build, the quality of collaboration changes completely. This is the layer most tools skip, and it is the layer we believe matters most for creators who work on complex, evolving projects.
Closing thoughts
RAG is not a buzzword or a marketing term. It is a practical, well-established architectural pattern that solves one of the biggest limitations of modern AI tools: the lack of persistent memory. By treating context as a first-class concept rather than an afterthought, Plura moves beyond one-off generation and toward long-term creative collaboration where the system genuinely understands your work.
For technical readers interested in diving deeper, the concepts covered here, embeddings, vector databases, chunking strategies, and retrieval pipelines, are well-documented in the broader AI engineering community. We will continue sharing more about our implementation decisions and lessons learned as Plura evolves.