Ragscore MCP Server

Generate QA datasets & evaluate RAG systems in 2 commands 🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

People who work with evaluation, ollama and privacy and want it reachable from Claude, Cursor, VS Code, or another MCP client. The project is written in Python.

VERIFIED ACTIVE

LAST COMMIT 2026-05-29 · ★ 15 · #62 OF 106 MAINTAINED AI & ML · VERIFIED 2026-09-18

Apache-2.0 · Python servers · how we verify → /methodology

01 · Install Ragscore

before you install - you'll need

Set OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK, RAGSCORE_WORK_DIR, RAGSCORE_NO_TELEMETRY before connecting.

Claude Code

claude mcp add hzyai-ragscore -- uvx ragscore

Claude Desktop / Cursor / VS Code - add to config

{
  "mcpServers": {
    "hzyai-ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ]
    }
  }
}

Same JSON for Cursor. For VS Code, rename the top-level key from `mcpServers` to `servers`.

Using another client? Same JSON, different key

Claude Desktop · mcpServers

Cursor · mcpServers

VS Code · servers

Windsurf · mcpServers

Zed · context_servers

Cline · mcpServers

Roo Code · mcpServers

Continue · mcpServers

LibreChat · mcpServers

Gemini CLI · mcpServers

Codex CLI · mcp_servers

Full setup guides: every client.

02 · Evidence

Security posture

What to check before giving this server access to your agent - from the registry, GitHub, and our own probes. We don't score safety; we show what's verifiable.

runs as local process (stdio) - runs on your machine with your user's permissions

license Apache-2.0 - declared in the repository

pypi package ragscore - check the name against the project README before installing (PyPI has no namespace ownership)

registry namespace io.github.HZYAI is GitHub-verified and matches the repo owner

03 · What Ragscore can do

Prose above is summarized from the project's README and registry record - no invented capabilities.

Latest releases

v0.7.0 · 2026-02-15

What's New · Detailed Multi-Metric Evaluation · Add detailed=True to get 5 diagnostic dimensions per answer - same single LLM call: · Correctness - semantic match to golden answer · Completeness - covers all key points…

04 · Who maintains Ragscore

RAGScore is maintained by hzyai. It's the only MCP server we track from this author; the repo dates to Dec 2025.

05 · Facts

repository: github.com/hzyai/ragscore
category: AI & ML - ranked #62 of 106 actively-maintained AI & ML servers as of 2026-09-18.
registry: io.github.HZYAI/ragscore (active, first published 2026-02-11 · 3 versions)
packages: pypi:ragscore

06 · Ragscore FAQ

What is Ragscore?

Generate QA datasets & evaluate RAG systems in 2 commands 🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

Is Ragscore still maintained?

Yes - as of 2026-09-18, its last commit was 2026-05-29. We re-verify nightly.

How do I install Ragscore?

Run `uvx ragscore`. The README documents 5 environment variables (OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK…) to set first. Set OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK, RAGSCORE_WORK_DIR, RAGSCORE_NO_TELEMETRY before connecting. You can also paste the ready-made client config above.

Does Ragscore run locally?

Yes - it's a stdio server: it runs on your machine (via uvx) with your user's permissions. Your data stays local unless the server itself calls external APIs.

07 · Alternatives to Ragscore

Alternatives to Ragscore

Maintained AI & ML servers if Ragscore isn't the fit.

Pairs well with

Servers that cover what Ragscore doesn't - only shown when the pairing reason fits the companion.

More AI & ML MCP servers · Mcpbridge · Meta Prompt MCP · 3D Visualizer · Lynx · Droste

More Python MCP servers · Grad Agent · Tidal MCP · Disney Lorcana TCG Collection · T-Bank MCP · IdentArk Gateway · see all