dsh-plugin-abtest
Paired A/B testing framework and deterministic promotion gates for DeepSeek Harness plugins.
Install
$ dsh plugin --profile web add 'github:Morriaty-The-Murderer/dsh-plugin-abtest#<full-commit-sha>'Plugin Overview & Capabilities
AI-assisted organization based on the public repository snapshot. The content must be grounded in source evidence and does not replace compatibility or security verification.
Key Capabilities
- Execute paired benchmark evaluations between control and candidate plugin versions in isolated DSH runners
- Evaluate task success, token consumption, P95 latency, and tool error rates with blind comparisons
- Generate deterministic promotion decisions (PROMOTE, REVIEW, REJECT, INCONCLUSIVE) using manifest-defined rules
- Export comprehensive auditable evidence packs including JSON, Markdown, and static HTML reports
- Integrate into DSH profiles via Cordis extension manifest or execute standalone via CLI
Useful For
- Validating prompt, tool, or code changes in a DSH plugin before shipping to production
- Benchmarking latency and token efficiency regressions across plugin updates
- Implementing CI/CD promotion gates for LLM agent plugins with auditable offline evidence
Who It Fits
- DeepSeek Harness plugin developers
- Agent evaluation and QA engineers
- LLM application platform maintainers
Documented Limitations
- Does not automatically mutate production profiles or publish plugins; all promotion recommendations are offline only
- Requires Node.js ^22.19.0 or >=24.0.0 and DSH 0.1.0-rc.7 contract compatibility
- Model-facing tool entry points disallow test commands containing arbitrary command executions
DSH Compatibility
Version-specific runtime evidence collected by DSH Plugin. A missing result means we have not tested that combination yet.
Security Signals
Objective signals discovered from package metadata and source inspection. These are not a guarantee that a plugin is safe.
package.json declares 4 runtime, 9 development, 2 peer, and 0 optional dependencies.
package.json declares DSH bundle metadata.
GitHub reports the repository license as MIT.
A root package.json was captured and can be inspected by the registry.
Public GitHub source metadata is available for this registry snapshot.
Source & Registry Notes
Public provenance, Registry classification, and the latest source check for this entry, kept separate from runtime verification.
- Source repository
- Morriaty-The-Murderer/dsh-plugin-abtest
- Registry source
- GitHub · dsh-plugin topic
- Registry classification
- Plugin
- Source checked
- bfff76f · 2026-08-28
This project is independently indexed from public source information. DSH Plugin is not affiliated with DeepSeek or the plugin author. Always check the author repository before installation.
Repository Activity
- GitHub stars
- 0GitHub stars
- Forks
- 0
- Open issues
- 0
- Last commit
- 2026-08-26
- Last release
- 2026-08-26
Related DSH Plugins
Ranked by overlapping capabilities, use cases, plugin type, categories, and DSH profile.
Vision routing and pixel-level visual tool suite for DeepSeek Harness agents with built-in free fallbacks.
View plugin →DeepSeek Harness research bundle for industry maps, public-source tracking, company cards, and traceable reports.
View plugin →Second-model AI auto-review for DeepSeek Harness approval requests with fail-closed safety and session audit.
View plugin →The most comprehensive DeepSeek Harness plugin market — refreshed daily, sourced across the Internet, reviewed before publishing.
View plugin →