RAG chatbot — FOSS / self-hosted platform shortlist¶
Internal evaluation of the open-source / self-hostable RAG platforms on GitHub that we would consider deploying and managing for a client as an alternative to Microsoft Copilot, Glean, or AWS Q Business.
This is the third path referenced from the off-the-shelf comparison: neither "buy SaaS" nor "build from scratch" but "take a mature FOSS platform, host it for the client, configure it, wrap it in a managed service".
Currency of information. Star counts and last-commit recency in the tables below are best-estimate as of mid-2026. Verify on GitHub before any client commitment — this space moves fast. Capability and licence claims are stable enough to act on.
What "good" looks like for our use case¶
A platform we would deploy for a client should tick all seven of these boxes, not just the first three:
- Self-hostable on Docker / Kubernetes. Not a SaaS-only product.
- Pluggable LLM backend — point at frontier APIs (Anthropic, OpenAI, Google) and at local models (Ollama, vLLM, llama.cpp).
- Multiple document source connectors — SharePoint, Google Drive, Confluence, Notion, S3, web crawl, local files.
- Vector pipeline — handles chunking, embedding, vector DB (pgvector / Qdrant / Weaviate / Chroma).
- Role-based access control — users / groups, with per-document or per-collection permissions, ideally inheriting the source system's ACLs so an HR document the user cannot see in SharePoint does not surface in chat either.
- Channel integrations — Microsoft Teams, Slack, Google Chat, web widget, REST API.
- Resale-friendly licence — Apache 2 / MIT / BSD without awkward clauses. AGPL or source-available licences with commercial restrictions are problems we don't want to inherit.
Plus two operational must-haves:
- Active maintenance. Recent releases, healthy issue activity.
- Documentation good enough to onboard a new engineer in a week.
Tier 1 — Purpose-built enterprise RAG¶
Onyx (formerly Danswer)¶
The closest FOSS analogue to Glean. Purpose-built enterprise search and RAG chatbot.
| Attribute | Detail |
|---|---|
| Repo | github.com/onyx-dot-app/onyx (~13k stars) |
| Licence | MIT (core) + source-available enterprise edition |
| Self-host | Yes — mature Docker Compose and Helm chart |
| LLM | Anthropic, OpenAI, Azure OpenAI, Bedrock, Vertex, Ollama, vLLM, any OpenAI-compatible endpoint |
| Connectors | 40+ — SharePoint, OneDrive, Google Drive, Confluence, Jira, Notion, Slack, Teams, GitHub, GitLab, S3, Salesforce, Zendesk, web crawl, local files, Dropbox, Box |
| RBAC / per-doc | Yes — inherits source-system ACLs per user; document sets with group-level access |
| Channels | Slack bot, Teams bot, web UI, REST API; Google Chat via API |
| Recency | Weekly releases through 2025-2026 |
| Verdict | Green — lead recommendation |
The per-document ACL inheritance is the differentiator. Almost no other FOSS option does this properly.
RAGFlow (InfiniFlow)¶
Deep-document-understanding RAG engine with strong OCR and layout parsing.
| Attribute | Detail |
|---|---|
| Repo | github.com/infiniflow/ragflow (~30k stars) |
| Licence | Apache 2.0 |
| Self-host | Yes — Docker Compose; GPU optional |
| LLM | OpenAI, Anthropic, Bedrock, Azure, Ollama, Xinference, local |
| Connectors | Local files, S3-compatible, web crawl, Confluence (recent), Notion (recent). Weaker on SharePoint / Drive — needs n8n glue |
| RBAC / per-doc | Partial — team / tenant workspaces and knowledge-base level permissions; not per-document ACL inheritance |
| Channels | Web UI, REST API, embeddable widget; Slack / Teams via custom integration |
| Recency | Extremely active |
| Verdict | Green — for complex-document specialists |
Best-in-class at parsing messy PDFs, scans, and tables; the trade-off is weaker connectors than Onyx.
Dify¶
LLMOps platform with agents, RAG, and a polished workflow builder.
| Attribute | Detail |
|---|---|
| Repo | github.com/langgenius/dify (~60k stars) |
| Licence | Apache 2.0 with commercial clauses — no white-label / no multi-tenant SaaS resale without separate licence. Read before reselling. |
| Self-host | Yes — Docker Compose, Helm |
| LLM | Every major frontier provider + Ollama, vLLM, LocalAI |
| Connectors | Local files, web crawl, Notion, Firecrawl, Jina; SharePoint / Drive via API nodes |
| RBAC / per-doc | Workspace + role (admin / editor / normal); per-knowledge-base not per-document |
| Channels | Web app, embed widget, REST API; Slack / Teams / Discord via webhook nodes |
| Recency | Extremely active |
| Verdict | Amber — use internally, careful as resale |
Outstanding builder UX. Use as our internal build platform for bespoke agent + RAG work; flag the licence to leadership before any white-labelled client resale.
Tier 2 — Single-tenant / SMB-friendly¶
AnythingLLM (Mintplex Labs)¶
| Attribute | Detail |
|---|---|
| Repo | github.com/Mintplex-Labs/anything-llm (~30k stars) |
| Licence | MIT |
| Self-host | Yes — Docker, also a desktop app |
| LLM | Anthropic, OpenAI, Gemini, Ollama, LM Studio, vLLM |
| Connectors | Local files, web crawl, Confluence, GitHub, YouTube, S3; paid connector hub for SharePoint / Drive |
| RBAC / per-doc | Workspace-level user permissions; no source-inherited ACLs |
| Channels | Web UI, embed widget, REST API, Discord; Teams / Slack via community plugins |
| Recency | Very active |
| Verdict | Amber — good for SMB single-team deployments |
Easiest to demo. Insufficient for clients with strict per-role document segregation.
LibreChat¶
| Attribute | Detail |
|---|---|
| Repo | github.com/danny-avila/LibreChat (~22k stars) |
| Licence | MIT |
| Self-host | Yes — Docker, Helm |
| LLM | Best-in-class pluggability — every provider including Bedrock, Vertex, Ollama |
| Connectors | File upload + web search; RAG via separate Python sidecar |
| RBAC / per-doc | User / group roles; no per-document ACL |
| Channels | Web UI, REST API; no native Teams / Slack |
| Recency | Very active |
| Verdict | Amber — Copilot-style chat UI, not enterprise search |
Best paired with Onyx for retrieval.
Open WebUI¶
| Attribute | Detail |
|---|---|
| Repo | github.com/open-webui/open-webui (~80k stars) |
| Licence | Modified BSD-3 with a branding / attribution clause added in 2025 — effectively source-available at resale scale. Flag for legal review. |
| Self-host | Yes — Docker, widely deployed |
| LLM | Anthropic, OpenAI, Ollama, any OpenAI-compatible |
| Connectors | File upload, web search, basic knowledge bases; no native SharePoint / Drive |
| RBAC / per-doc | Users / groups, knowledge-level permissions; not source-inherited |
| Channels | Web UI, REST API; Teams / Slack via "pipelines" plugins |
| Recency | Extremely active |
| Verdict | Amber — great chat shell, licence and connectors stop it being turnkey |
Note: we already run Open WebUI for internal use (app 26).
Tier 3 — Frameworks and builders (build-your-own, not products)¶
| Project | Licence | Verdict | Note |
|---|---|---|---|
| Haystack (deepset) | Apache 2.0 | Green for bespoke builds | Python framework |
| LlamaIndex | MIT | Green for bespoke builds | Framework + create-llama starter |
| Flowise | Apache 2.0 + commercial clauses | Amber | LangChain-on-canvas |
| Langflow | MIT (DataStax owned) | Amber | Increasingly product-like |
| Cognita (TrueFoundry) | Apache 2.0 | Amber | Lower momentum in 2026 |
| Quivr | Apache 2.0 | Amber | Pivoted toward agents, less RAG focus |
| Verba (Weaviate) | BSD-3 | Red as product | Demo-grade reference app |
| Khoj | AGPL-3.0 | Red — licence blocker | Otherwise capable |
| PrivateGPT | Apache 2.0 | Amber / red | Momentum dropped sharply in 2025 |
| ChatGPT-Next-Web (NextChat) | MIT | Out of scope | Chat UI, no real RAG |
Emerging entrants worth watching¶
- Morphik (
morphik-org/morphik-core, MIT) — multimodal RAG, strong PDF / image grounding. Small but rising. - R2R (SciPhi) — Apache 2.0 "production RAG engine" with graph + hybrid retrieval and built-in auth. Positions directly against Onyx for developer-led teams.
- Kotaemon (Cinnamon AI, Apache 2.0) — clean RAG UI with citation-first UX.
- n8n + pgvector / Qdrant templates — given we already run n8n (app 25) this is a credible fourth option for bespoke client builds, not a product.
None of these has overtaken Onyx as the de-facto enterprise FOSS answer as of mid-2026.
Licence quick-reference (resale risk)¶
| Project | Licence | Resale risk |
|---|---|---|
| Onyx core | MIT | Low |
| RAGFlow | Apache 2.0 | Low |
| AnythingLLM | MIT | Low |
| LibreChat | MIT | Low |
| Haystack / LlamaIndex | Apache 2.0 / MIT | Low |
| Dify | Apache 2.0 + commercial clause | Medium — read terms |
| Flowise | Apache 2.0 + commercial clause | Medium |
| Open WebUI | Modified BSD-3 + branding clause | Medium |
| Khoj | AGPL-3.0 | High — avoid for client resale |
Any "medium" or higher licence should go past leadership and ideally a brief legal review before being baked into a proposal.
The three platforms we should pilot¶
- Onyx — the flagship Copilot / Glean replacement for clients with SharePoint + Confluence + Slack / Teams who need real per-document ACL inheritance. Lead recommendation. Stand up an internal Onyx instance pointed at our own SharePoint and Slack as a reference deployment.
- RAGFlow — the complex-document specialist for regulated clients, contract-heavy estates, scanned PDFs, and tabular data, where retrieval quality on messy material matters more than connector breadth.
- AnythingLLM — the SMB / single-team quick-win for clients who do not need source-inherited RBAC and want a one-week deployment. Useful as a low-cost entry-point that can grow into an Onyx engagement later.
Honourable mention: Dify as our internal build platform for bespoke agent + RAG engagements, with the licence carve-out reviewed by leadership before any white-labelled resale.
How this changes the quote¶
When the client situation fits one of the three pilots above, the quote guide one-off implementation phase changes:
- Document pipeline (Phase 1) drops from 5-20 days to 2-8 days — the platform's connectors do most of the work.
- RAG engine (Phase 2) drops from 5-15 days to 1-3 days — retrieval, ranking, citations are built in.
- UI / integration (Phase 3) drops from 5-25 days to 2-8 days — the Teams / Slack / web UI ship out of the box.
- Guardrails & audit (Phase 4), UAT & tuning (Phase 5), Handover (Phase 6) roughly unchanged — these are about this client's corpus and people, not about the platform.
Total person-day band typically shifts from 27-101 days (build) down to 12-45 days (FOSS + configure). Monthly opex shifts too: the software is free, but a sustaining engineer relationship is not, so the managed-service line is more honest than "we put it on a VM and walked away".
This middle path — Onyx (or similar) under a managed-service wrap — is often the right answer for clients between 200 and 2,000 staff who need real per-document permissions and a UK / EU residency posture, where Copilot is the wrong shape and a full custom build is over-investment.