Local by design.
Filtering runs through a local cross-encoder. No LLM API key is needed for the filtering step.
ms-marco-MiniLM-L-6-v2Uma
Context optimization
for your LLM.
Keep the sentences that answer your question.
Leave the rest behind.
Uma
Context optimization for your LLM.
UmaTHE SPACE BETWEENLESS IN. MORE FOCUS.
Keep your stack.
Refine what flows through it.
Retrieval finds the documents. Uma finds the sentences that matter to your question, before you send them to the model.
01 / RETRIEVE
→Your retriever
Bring your existing documents.
02 / REFINEUma
→Uma
Query-aware sentence filtering.
03 / GENERATE
Your LLM
Keep the model you already use.
THE FILTER, UP CLOSEINTERACTIVE ILLUSTRATION
Less noise.
The same source.
Move the threshold to see how sentence filtering works. These example scores are illustrative, not live model output.
Keep moreFilter more
2 / 6sentences retained
A score ranks relevance. It is not a probability.
THE QUESTION
What does Uma do to retrieved context?
CONTEXT SENT TO YOUR MODEL
Original wording preserved4 sentences removed
SMALL ENGINE. THOUGHTFUL CONTROLS.
Your context budget.
Set a relevance threshold or an optional token limit. Retained sentences return in their original order.
threshold=0.5 · max_tokensMeasure the tradeoff.
Uma Calibrate tests thresholds against your benchmark so you can compare retained context with answer quality.
uma calibrateResults depend on your workload. Filtering can omit useful context and adds its own inference time. Calibrate before relying on a threshold.
BUILT TO FITA smaller prompt.
A smaller prompt.
A familiar workflow.
Use Uma in Python or through LangChain. MCP clients connect to the same filtering engine.
Read the documentation ↗quickstart.py
A FEW THINGS WORTH KNOWINGBuilt for context.
Built for context.
Clear about the tradeoffs.
Does Uma rewrite my documents?
No. Uma keeps or removes whole sentences. The sentences it retains keep their original wording. Document metadata is preserved.
Do I need a different model or retriever?
Uma sits between your retriever and your model. Pass retrieved documents through Uma, then send the resulting context to your existing LLM.
What stays local?
The cross-encoder runs locally and needs no LLM API key for filtering. Benchmarking and answer generation use the provider you configure.
How should I choose a threshold?
Start with the default, then evaluate on your own questions. Uma Calibrate compares thresholds against a benchmark. Higher thresholds can remove information needed for an answer.
Will every request become faster or cheaper?
Results depend on the input and your provider. Filtering adds its own inference time. Fewer input tokens can reduce token-based charges, but latency improvements are not guaranteed.