Your AI knowledge base needs an expiry date
An AI assistant can quote a document perfectly and still give the wrong answer.
Imagine a customer-support product that retrieves last quarter’s return policy instead of the current one. The answer has a citation. The wording matches the source. The customer still receives an outdated promise.
That is a knowledge-lifecycle problem, not necessarily a model problem.
On September 25, 2026, Cohere announced a private beta for Compass Cloud, its managed retrieval offering. One detail deserves a founder’s attention. Cohere says Compass retention policies can expire content automatically and prevent deleted documents from being resynced.
The lesson is broader than one platform. If your AI product depends on changing documents, you need rules for when those documents stop being valid evidence. Build that before adding a larger knowledge library.
A relevant document is not always an authoritative one
Retrieval-augmented generation, or RAG, means finding source material and giving it to a model to help produce an answer. It can make your product more useful. It does not decide which policy your customer should follow.
Consider a hypothetical support assistant for a small equipment supplier. Its library contains an approved warranty policy, an older policy, and a draft update. All three discuss the same products. All three could look relevant to a search for warranty coverage.
Your domain knowledge tells you which one applies. Similar wording does not.
Start by defining the authoritative source for one workflow. Ask the customer who approves it, how changes are announced, and whether different customers operate under different versions. The newest upload is not automatically the governing document. Someone may have uploaded an old policy today.
Microsoft’s Azure AI Search documentation explains that vector queries can filter on nonvector fields, including suitable document metadata. That provides a mechanism for narrowing eligible evidence. You still have to supply the business rules that make the filter meaningful.
Give each source a useful life
An expiry date does not mean deleting every document after an arbitrary number of days. It means deciding when the product must stop treating a source as current without review.
For your first pilot, keep a small source register in a database or managed knowledge tool. Each approved document should have enough information to answer these questions:
- Who owns the document and approves changes?
- Which version is this?
- When does it become effective, and when does it stop applying?
- When must someone review it again?
- Is it approved, draft, superseded, or withdrawn?
- Which document replaces it?
Separate a review deadline from an effective date. A policy can remain valid while awaiting routine review. Another policy can be approved today but take effect next month. Decide how the product handles both cases with the customer.
Also separate answer eligibility from storage retention. You may need to preserve an old document for historical reference while excluding it from answers about today’s policy. Keeping an archive and using it as current guidance are different decisions.
OpenAI’s retrieval guide supports file attributes and filters, including date comparisons applied before semantic search. Use that capability, or an equivalent mechanism in your chosen stack, to restrict the evidence before it reaches the model.
A prompt saying “prefer recent documents” is not a substitute for that rule.
Test retirement across the whole product
Removing a file is not the same as proving the product has stopped using it.
OpenAI’s documentation explicitly says removal from a vector store is eventually consistent. Search results may still include removed content for a short period. That is a documented limitation of that service, not a guarantee about every retrieval platform.
Your own product may also have saved answers, summaries, or conversation history derived from an earlier source. Design how those copies lose authority when the source changes.
Run a simple test with synthetic documents before involving customer data:
- Publish a sample policy that allows returns for 30 days. Ask the assistant about returns and inspect its source.
- Approve a replacement policy that allows 14 days. Make it effective immediately and retire the old version.
- Repeat the question in both a new conversation and the original conversation.
- Test while the replacement is still being indexed. Confirm the product waits, escalates, or explains that current evidence is unavailable rather than reverting to the retired policy.
- Delete the retired source and run a connector sync. Check that it does not reappear as eligible evidence.
Those numbers are test fixtures, not policy recommendations.
Inspect retrieved passages as well as the final answer. A correct answer can conceal an incorrect source selection that fails on the next question.
If the evidence is missing or contradictory, define a visible fallback. For this pilot, that could mean asking the document owner to resolve the conflict and telling the user why the answer is paused. Do not let the assistant silently choose whichever version sounds plausible.
Sell one maintained workflow, not a document chatbot
For an experienced professional, the product opportunity is specific. You understand which documents govern a decision and which changes create real work.
Interview a few potential buyers about the last time an outdated document caused confusion. Ask what happened, who corrected it, and how they now check the current version. Look for a recurring problem with a named owner, not a theoretical concern about AI accuracy.
Then offer a narrow pilot. For the equipment supplier, that might be an assistant that answers one category of warranty questions using approved, effective policies and routes uncertain cases to a reviewer.
Agree on the evidence that would justify paying for continued use. Track outdated-source incidents, review time, and whether staff keep using the workflow. Record the effort required to maintain the library as well. That work belongs in your delivery costs.
You can prototype the workflow with no-code tools, but verify that your platform exposes the source metadata, filtering, and update behavior you need. A document-upload button alone does not establish those capabilities.
If you later raise funding, evidence that customers pay for a maintained workflow is a stronger story than the number of files your chatbot can ingest. Your professional experience becomes valuable when it defines what the product should trust, when that trust expires, and who renews it.
AI Product Accelerator is a 12-week, mentor-led program for turning your experience into a product you build, launch, and own. Explore the program if you want a structured path to test your idea and plan your first pilot.