Agent Directory Service

MAS Data Discovery ADS implements a two-level mapping system that enables efficient discovery of Multi-Agent System components through a distributed hash table architecture. This approach separates capability-based discovery from content location, providing both scalability and flexibility in agent retrieval.

Skill Taxonomy Effective agent discovery in multi-agent systems requires sophisticated organization of capabilities and skills. ADS employs a hierarchical skill taxonomy that serves as the foundation for efficient search and discovery operations across the distributed network.

The Challenge of Capability Search Traditional keyword-based search approaches face significant limitations when applied to agent discovery: Vocabulary Fragmentation: Different publishers may describe similar capabilities using varying terminology. For example, "sentiment analysis," "opinion mining," and "emotional classification" may all refer to similar agent capabilities, leading to search results that miss relevant agents due to terminology mismatches. Scale Complexity: As the number of agents in the ecosystem grows, exhaustive search across all records becomes computationally prohibitive. Without structured organization, every query potentially requires examining every agent record, leading to poor performance characteristics. Semantic Relationships: Many agent capabilities have natural hierarchical relationships that flat keyword systems cannot capture. An agent capable of "named entity recognition" is inherently relevant to searches for broader "text analysis" capabilities, but keyword matching alone cannot establish these connections.

Taxonomy-Driven Search Optimization ADS addresses these challenges through a structured hierarchical taxonomy that provides several critical optimization benefits: Search Space Partitioning: The taxonomy enables efficient partitioning of the search space. When processing a query for "computer vision" capabilities, the system can immediately focus on the relevant taxonomy branch, eliminating the need to examine agents in unrelated categories like natural language processing or mathematical reasoning. Index Structure Optimization: The hierarchical organization allows the distributed hash table to create specialized indices for different taxonomy branches. Rather than maintaining a single massive index, the DHT can distribute indexing responsibility across nodes, with each node specializing in specific capability domains. Query Semantic Expansion: The taxonomy enables intelligent query expansion where searches automatically include semantically related subcategories. A search for "text analysis" can transparently include results from "sentiment analysis," "entity extraction," and "text classification" without requiring users to explicitly enumerate all relevant subcategories. Standardized Vocabulary: By providing a canonical taxonomy, ADS reduces terminology fragmentation. Publishers are encouraged to tag their agents using standardized skill categories, improving search precision and recall across the ecosystem.

Two-Level Mapping Architecture The discovery system operates through two distinct mapping layers:

Skill Taxonomy for Search Optimization ADS employs a hierarchical skill taxonomy to optimize search performance and enable efficient capability-based discovery. Taxonomies provide several critical advantages for agent discovery systems: Search Space Reduction: Rather than performing exhaustive searches across all agent records, taxonomies allow the system to quickly narrow the search space to relevant categories. When a user queries for "natural language processing" capabilities, the system can immediately identify the subset of agents tagged with NLP skills without examining agents focused on computer vision or mathematical reasoning. Hierarchical Organization: Skills are organized in a tree-like structure *that reflects natural relationships between capabilities. For example: This hierarchy enables both specific queries ("sentiment analysis agents") and broader capability searches ("all natural language processing agents") while maintaining efficient indexing structures. Query Expansion and Refinement: Taxonomies support automatic query expansion *where searches for parent categories can include relevant child categories. A *query for "text analysis" can automatically include agents tagged with *"sentiment analysis," "named entity recognition," and "text classification" *without requiring users to know all specific subcategories. Semantic Consistency: Standardized taxonomies reduce ambiguity and improve *search precision by providing consistent terminology across the ecosystem. This *prevents fragmentation where similar capabilities are described using different *terms by different publishers. Scalable Indexing: The hierarchical structure enables efficient distributed *indexing where different DHT nodes can specialize in specific taxonomy *branches, distributing both storage load and query processing across the *network.

Additional Taxonomies While skills form the primary taxonomy for capability-based discovery, ADS supports multiple parallel taxonomies to enable rich, multi-dimensional agent classification and search.

Domain Taxonomy The domain taxonomy organizes agents by their application domains, representing the broader problem spaces or industries where agents are designed to operate: Domain classification enables users to discover agents that are specifically tuned for their operational context, even if those agents share similar underlying skills with agents from other domains.

Feature Taxonomy The feature taxonomy categorizes agents by the integration frameworks and architectural patterns they support, facilitating the discovery of agents compatible with specific system architectures:

Multi-Dimensional Search The parallel taxonomy system enables sophisticated multi-dimensional queries that combine criteria across different classification axes. All searches must include at least one skill criterion as the mandatory foundation, with domain and feature taxonomies providing additional filtering dimensions:

Skill + Domain: "Find natural language processing agents specialized for healthcare applications"
Skill + Feature: "Discover computer vision agents that support MCP integration"
Skill + Feature + Domain: "Locate natural language processing agents with observability features for manufacturing applications"

Skills as Search Foundation: The skills taxonomy serves as the primary index structure in the DHT, making skill-based criteria mandatory for efficient query resolution. This design ensures that:

Query Performance: All searches leverage the optimized skills-to-CID mapping as the starting point, providing consistent performance characteristics
Result Relevance: Domain and feature filters are applied to skill-based result sets, ensuring functional capability remains the core selection criterion
Index Efficiency: The DHT can optimize storage and lookup patterns around the skills taxonomy while supporting supplementary filtering through domains and features

Domain-only or feature-only queries are not supported, as they would bypass the primary indexing structure and provide results that may not have the functional capabilities required by the requesting system. This multi-taxonomic approach provides the flexibility to support diverse use cases while maintaining efficient indexing and search performance across all dimensions.

Skills-to-CID Mapping The first level maps agent capabilities and skills to their corresponding Content Identifiers (CID): This mapping enables queries such as "find all agents capable of natural language processing" to quickly resolve to a set of content identifiers without needing to search through individual agent records.

CID-to-PeerID Mapping The second level maps Content Identifiers to the Peer IDs of nodes that store the corresponding agent records: This separation allows the system to: - Optimize for capability queries without requiring knowledge of data locations

Enable dynamic content replication as peer availability changes
Support multiple storage strategies for the same content across different peers

DHT-Based Discovery Process The Distributed Hash Table stores and maintains both mapping layers across the network: ADS uses [Kad-DHT] for server and content discovery by using the libp2p implementation that constitutes the IPFS core DHT .

Skill Registration When agents are published to the network:

Capability Extraction: The system parses OASF records to extract skills, domains, and capabilities
DHT Updates: Skills-to-CID mappings are distributed across DHT nodes using consistent hashing
Location Registration: Peer nodes register themselves as providers for specific CIDs

Discovery Query Resolution Agent discovery follows a three-phase process:

Capability Resolution: Query "agents with skill X" resolves to a list of relevant CIDs via DHT lookup
Location Resolution: For each discovered CID, query DHT to find peer nodes storing the content
Result Aggregation: Combine capability matches with location information to produce actionable discovery results

Additional Tanomoxies

Content Distribution via OCI Protocol Once discovery identifies the relevant CIDs and their hosting peers, the actual agent records are retrieved using the OCI distribution protocol:

Peer-to-Peer Synchronization The discovered list of CIDs enables efficient content synchronization between peers:

Content Negotiation: Requesting peer queries hosting peers for available agent records
OCI Pull Operations: Standard OCI registry pull commands retrieve agent artifacts and metadata
Incremental Sync: Only missing or updated content is transferred, reducing bandwidth requirements
Verification: Content integrity is verified through cryptographic hash validation during transfer

Distribution Strategies The system supports multiple distribution patterns, each with distinct trade-offs and operational considerations: On-Demand Retrieval: Records are pulled from remote peers only when specifically requested, minimizing local storage requirements. Trade-offs: While this approach minimizes storage costs and ensures fresh content, it introduces several challenges:

Query Latency: Each request requires network round-trips to locate and retrieve content, increasing response times
Network Cost: Spurious or exploratory requests generate unnecessary network traffic and computational overhead
DoS Vulnerability: The system becomes susceptible to denial-of-service attacks where malicious actors can trigger expensive content retrieval operations by flooding the network with requests for non-existent or rarely-accessed agents
Scalability Limits: Performance degrades as the network grows due to increased query coordination overhead

Proactive Caching: Popular or frequently accessed agents are automatically replicated to improve query response times. Trade-offs: This strategy offers significant scalability benefits but requires sophisticated management:

Performance Gains: Dramatic reduction in query latency for popular content, enabling sub-second response times
Scalability: Can handle high query volumes efficiently once popular content is cached locally
Popularity Measurement: Requires implementing metrics collection and analysis to identify which agents warrant caching. This includes tracking query frequencies, download patterns, and usage statistics across the network
Storage Requirements: Needs sufficient local storage capacity to maintain cached copies of popular records
Cache Management: Must implement cache eviction policies, freshness validation, and synchronization mechanisms
Administrator Oversight: Proactive caching policies must be configured and monitored by agent directory node administrators to balance storage costs with performance benefits

Strategic Replication: Critical agents can be replicated across multiple peers to ensure high availability and reduce single points of failure. Trade-offs: This approach addresses availability concerns but introduces subjective complexity:

High Availability: Ensures critical agents remain accessible even during peer failures or network partitions
Reduced Single Points of Failure: Distributes risk across multiple storage locations
Subjective Criticality: The definition of "critical" or "useful" agents varies significantly between users, organizations, and use cases. What constitutes strategic value for financial services may be irrelevant for manufacturing applications
Administrative Burden: Requires agent directory node administrators to make strategic decisions about which agents warrant replication, considering factors like:
- Organizational priorities and business requirements
- Compliance and regulatory considerations
- Cost-benefit analysis of storage versus availability
- Community consensus on agent importance
Resource Allocation: Strategic replication consumes storage resources that could otherwise be used for proactive caching of popular content

Administrative Management: Both Proactive Caching and Strategic Replication require active management by agent directory node administrators. Administrators must:

Configure caching policies based on local network characteristics and storage capacity
Monitor popularity metrics and adjust caching strategies accordingly
Define strategic replication criteria aligned with organizational objectives
Balance resource allocation between different distribution strategies
Implement governance policies for content lifecycle management

This architecture provides a scalable foundation for MAS data discovery that can efficiently handle large networks of distributed agents while maintaining low latency for capability-based queries.

Agent Directory Record Examples The following examples illustrate the structure of OASF-compliant agent records stored in the directory: These examples demonstrate how the DHT indexing system extracts skills and domains from agent records to populate the Skills-to-CID mappings, enabling efficient capability-based discovery across the distributed network.

Security Model The OCI-based architecture provides multiple layers of security that address the unique challenges of distributed agent directories:

Cryptographic Integrity ADS leverages the OCI layer's built-in cryptographic mechanisms to ensure data integrity: Automatic Hash Computation: The OCI registry layer automatically computes SHA-256 hash digests for all stored artifacts. These content identifiers are generated transparently during the push operation, ensuring that every agent record has a cryptographically verifiable fingerprint without additional overhead. Tamper Detection: Content addressing ensures immediate tamper detection through cryptographic hash verification. Any modification to an agent record—whether malicious or accidental—results in a different hash digest, making unauthorized changes immediately detectable during retrieval operations. End-to-End Verification: Clients can independently verify that received content matches its advertised identifier, providing built-in protection against data corruption during transmission or storage without trusting intermediate network components.

Content Provenance and Digital Signatures ADS integrates with Sigstore, a security framework for OCI storage, to provide comprehensive content provenance and authenticity guarantees: Sigstore Integration: The system leverages Sigstore's security framework to provide verifiable proof of when and by whom agent records were signed. This creates an immutable audit trail that cannot be retroactively modified, enabling forensic analysis of agent deployment history. Keyless Signing: Sigstore's keyless signing approach eliminates the complexity and security risks associated with long-lived cryptographic keys:

Identity-Based Authentication: Uses OpenID Connect (OIDC) tokens from trusted identity providers (GitHub, Google, Microsoft) to authenticate publishers at signing time
Short-Lived Certificates: Issues ephemeral signing certificates valid only for minutes, reducing the window of potential key compromise
Automatic Key Rotation: Eliminates the need for manual key management, distribution, and rotation procedures
Scalable Trust: Publishers don't need to maintain or distribute public keys, making the system accessible to individual developers and large organizations alike

Transparency and Verification: All signatures are stored directly in OCI storage alongside the agent artifacts and public keys, providing:

Public Auditability: Anyone can verify the signing history of agent records stored in accessible registries
Non-Repudiation: Publishers cannot deny having signed records that are cryptographically linked to their identity
Supply Chain Security: Enables detection of compromised or unauthorized agent publications

Trust Boundaries and Isolation Organizational Isolation: Separate registries maintain security boundaries between different organizations, allowing each entity to control their own agent ecosystem while still participating in the broader federated network. Content Verification: Nodes can validate artifact integrity and signature authenticity without trusting transport layers or intermediate storage systems. This zero-trust approach ensures security even when using untrusted storage infrastructure. Reputation Systems: The cryptographic foundation enables the development of reputation systems based on verifiable evidence rather than subjective claims. Publishers with consistent signing practices and high-quality agents can build measurable trust over time.

Threat Mitigation The security model addresses several key threats to distributed agent directories: Supply Chain Attacks: Sigstore integration and transparency logs make it difficult for attackers to inject malicious agents without detection, as all publications are cryptographically signed and publicly auditable. Data Integrity Attacks: Automatic hash verification prevents tampering with agent records during storage or transmission, ensuring users receive authentic content. Identity Spoofing: OIDC-based keyless signing prevents attackers from impersonating legitimate publishers without compromising their identity provider credentials. Availability Attacks: The distributed nature of the system, combined with content replication across multiple registries, provides resilience against denial-of-service attacks targeting individual nodes.

Access Control

Registry-level permissions control who can publish and retrieve agent records
Fine-grained policies can restrict access to specific agent categories or capability types
Audit trails leverage existing registry logging capabilities to track access patterns

Trust Boundaries

Organizational isolation through separate registries maintains security boundaries
Content verification allows nodes to validate artifact integrity without trusting transport layers
Reputation systems can build on cryptographic proofs of past agent performance

Performance Optimizations The architecture incorporates several optimizations for the specific requirements of agent discovery:

Bandwidth Optimization

Incremental updates use OCI layer semantics to transmit only changed portions of agent records
Content compression reduces storage and transmission costs for large agent definitions
Selective replication based on query patterns minimizes unnecessary data transfer

Scalability Architecture The system scales horizontally through several mechanisms:

Registry sharding distributes storage load across multiple OCI registry instances
Index partitioning in the DHT allows query load to scale with the number of participating nodes
Lazy loading defers retrieval of detailed agent specifications until actually needed

This architecture provides a robust foundation for a decentralized agent directory that can scale to support the growing ecosystem of AI agents while maintaining the security and reliability requirements of production systems.