Research area

AI search, skill retrieval, NL2SQL benchmarks, LLM serving systems, and capability governance are treated as system surfaces that should be inspected before they are trusted.

Publication record

Authors

Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan

Venue

arXiv preprint arXiv:2606.10388

Year

2026

Area

LLM Agents

Problem

In agent skill libraries, many skills expose similar capability descriptions. A retrieval system can pick the correct capability family and still surface a sibling skill that differs on execution-controlling contracts such as resources, preconditions, procedures, or artifacts. The paper names this failure mode same-capability risk-exposure retrieval.

Benchmark

SameCapRisk-Bench contains 940 skill-risk units and 1,327 query cases. Units are either single-query (553, including 69 development units) or paired (387 units contributing 774 queries that exchange helpful and risky roles across queries). Evaluation uses a held-out split of 871 units and 1,258 queries, and the release records provenance and leakage checks.

Public evidence

On the held-out split, neural retrievers reach Recall@3 of 0.723 to 0.936 while exposing a risky sibling at HSR@3 of 0.386 to 0.402 (RRF 0.723/0.387, SkillRouter 0.910/0.386, SkillRet 0.928/0.389, R3-Skill 0.925/0.396, SkillSight 0.936/0.402). Applying a released family selector to their score orders cuts HSR@3 to 0.110 to 0.154 at a Recall@3 cost (0.781 to 0.851), and an auxiliary scorer-plus-selection control reports 0.731 Recall@3 with HSR@3 0.136. Retrieving more items raises exposure further (HSR 0.684 to 0.951 at K=20). These are benchmark measurements, not production deployment results.

Takeaway

Capability matching and same-family risk exposure should be reported together. HSR@K serves as a targeted exposure certificate for a fixed skill library, complementing the auditable-retrieval line on inspecting interfaces before they become production dependencies.