Add a native glob tool for agent file discovery #82

New Issue

hugo · 2026-08-30T20:48:45Z

hugo commented

2026-08-30 20:48:45 +00:00

Problem

The agent can search file contents with search and inspect one directory with list, but it cannot recursively find files by name or path pattern. It therefore falls back to Bash commands such as find, fd, or rg --files for a read-only operation that should be available as a native, permission-confined agent tool.

Goal

Add a read-only glob tool that finds regular files by basename or relative path pattern without launching a shell or any external process.

This is filename/path discovery only. Content matching remains the responsibility of search, and single-directory inspection remains the responsibility of list.

Tool contract

Expose glob through the existing generated tool schema and validation path in src/agent.rs.

Parameters:

pattern (required string): non-empty filename or relative-path wildcard.
path (optional string, default .): directory under an existing readable root from which to search.
max_results (optional integer, default 100, range 1–500): maximum number of paths returned.

Pattern behavior:

A pattern without / matches basenames recursively, e.g. Cargo.toml or *.rs.
A pattern containing / matches the path relative to the selected search root, e.g. src/*test*.rs.
Reuse the wildcard behavior already used by the search tool's glob filter (* and ?) so the two tools do not develop incompatible matchers. Document that * follows the existing matcher and can span path separators.
Match paths using normalized / separators.
Return regular files only, not directories.
Files with spaces or Unicode names must work without panics.

Example calls:

{"pattern":"*.rs"}
{"pattern":"agent*.rs","path":"src"}
{"pattern":"**/SKILL.md","max_results":50}

Because the existing matcher treats consecutive * characters like *, the last example remains valid without introducing separate ** semantics.

Required behavior

Implement this in Rust using the existing filesystem/tool infrastructure; do not invoke Bash, find, fd, rg, or another subprocess.
Reuse Tools::existing_path and inside_readable_root. The tool must have exactly the same readable-root policy as read, list, and search: project files, enabled skill roots, and visible managed Dev Brain content.
Reject an invalid search root, a non-directory root, parent traversal, or a path outside the permitted roots with the normal structured tool error.
Do not follow symlinks. A symlink must never allow traversal or results outside a readable root.
Always exclude .git. Preserve the existing hidden-entry policy for project, skill, and Dev Brain traversal rather than inventing a second policy.
Reuse/refactor the existing recursive file collector and wildcard matcher where practical. Do not add a new crate solely for this tool.
Check the tool cancellation flag during recursive traversal and return the normal cancellation error promptly.
Sort matching paths lexically before applying max_results, so output is deterministic.
Return paths relative to the selected search root, one per line. Do not read file contents or exclude large/binary files.
If there are no matches, return an explicit No matches result.
If more matches exist than max_results, return the first max_results entries and an explicit truncation summary that tells the agent to narrow the pattern or raise the limit.
Update the tool description/instructions so the model chooses glob for filename discovery before resorting to Bash.

Implementation location

Keep the change focused in the existing agent tool implementation unless a small traversal-helper refactor is needed:

add a ToolHandler::Glob variant;
add the glob ToolSpec;
dispatch it through execute_validated;
implement the handler using the existing readable-root and recursive traversal helpers.

No UI or server API changes are required.

Tests

Add focused tests that prove:

the generated schema exposes glob with the required and optional fields and validation rejects missing/invalid arguments;
exact basename, *, ?, and relative-path patterns find nested files;
results are deterministic, root-relative, and truncated at max_results with a visible truncation notice;
no-match behavior is explicit;
large and binary files are returned based on their names without reading their contents;
.git entries, symlinks, .., and out-of-root paths cannot escape confinement;
readable skill/Dev Brain roots follow the same visibility rules as existing file tools;
cancellation interrupts traversal;
filenames containing spaces and Unicode are handled.

Acceptance criteria

An agent can locate files recursively by name/path pattern with glob and does not need Bash for that task.
The implementation is fully native, read-only, deterministic, bounded in output, cancellable, and confined to existing readable roots.
Existing read, list, and search behavior remains unchanged.
No new dependency is added for traversal or wildcard matching.
cargo fmt --all -- --check, cargo clippy --all-targets --all-features -- -D warnings, make bundle, and cargo test --all-features pass.

## Problem The agent can search file contents with `search` and inspect one directory with `list`, but it cannot recursively find files by name or path pattern. It therefore falls back to Bash commands such as `find`, `fd`, or `rg --files` for a read-only operation that should be available as a native, permission-confined agent tool. ## Goal Add a read-only `glob` tool that finds regular files by basename or relative path pattern without launching a shell or any external process. This is filename/path discovery only. Content matching remains the responsibility of `search`, and single-directory inspection remains the responsibility of `list`. ## Tool contract Expose `glob` through the existing generated tool schema and validation path in `src/agent.rs`. Parameters: - `pattern` (required string): non-empty filename or relative-path wildcard. - `path` (optional string, default `.`): directory under an existing readable root from which to search. - `max_results` (optional integer, default 100, range 1–500): maximum number of paths returned. Pattern behavior: - A pattern without `/` matches basenames recursively, e.g. `Cargo.toml` or `*.rs`. - A pattern containing `/` matches the path relative to the selected search root, e.g. `src/*test*.rs`. - Reuse the wildcard behavior already used by the `search` tool's `glob` filter (`*` and `?`) so the two tools do not develop incompatible matchers. Document that `*` follows the existing matcher and can span path separators. - Match paths using normalized `/` separators. - Return regular files only, not directories. - Files with spaces or Unicode names must work without panics. Example calls: ```json {"pattern":"*.rs"} {"pattern":"agent*.rs","path":"src"} {"pattern":"**/SKILL.md","max_results":50} ``` Because the existing matcher treats consecutive `*` characters like `*`, the last example remains valid without introducing separate `**` semantics. ## Required behavior - Implement this in Rust using the existing filesystem/tool infrastructure; do not invoke Bash, `find`, `fd`, `rg`, or another subprocess. - Reuse `Tools::existing_path` and `inside_readable_root`. The tool must have exactly the same readable-root policy as `read`, `list`, and `search`: project files, enabled skill roots, and visible managed Dev Brain content. - Reject an invalid search root, a non-directory root, parent traversal, or a path outside the permitted roots with the normal structured tool error. - Do not follow symlinks. A symlink must never allow traversal or results outside a readable root. - Always exclude `.git`. Preserve the existing hidden-entry policy for project, skill, and Dev Brain traversal rather than inventing a second policy. - Reuse/refactor the existing recursive file collector and wildcard matcher where practical. Do not add a new crate solely for this tool. - Check the tool cancellation flag during recursive traversal and return the normal cancellation error promptly. - Sort matching paths lexically before applying `max_results`, so output is deterministic. - Return paths relative to the selected search root, one per line. Do not read file contents or exclude large/binary files. - If there are no matches, return an explicit `No matches` result. - If more matches exist than `max_results`, return the first `max_results` entries and an explicit truncation summary that tells the agent to narrow the pattern or raise the limit. - Update the tool description/instructions so the model chooses `glob` for filename discovery before resorting to Bash. ## Implementation location Keep the change focused in the existing agent tool implementation unless a small traversal-helper refactor is needed: - add a `ToolHandler::Glob` variant; - add the `glob` `ToolSpec`; - dispatch it through `execute_validated`; - implement the handler using the existing readable-root and recursive traversal helpers. No UI or server API changes are required. ## Tests Add focused tests that prove: 1. the generated schema exposes `glob` with the required and optional fields and validation rejects missing/invalid arguments; 2. exact basename, `*`, `?`, and relative-path patterns find nested files; 3. results are deterministic, root-relative, and truncated at `max_results` with a visible truncation notice; 4. no-match behavior is explicit; 5. large and binary files are returned based on their names without reading their contents; 6. `.git` entries, symlinks, `..`, and out-of-root paths cannot escape confinement; 7. readable skill/Dev Brain roots follow the same visibility rules as existing file tools; 8. cancellation interrupts traversal; 9. filenames containing spaces and Unicode are handled. ## Acceptance criteria - An agent can locate files recursively by name/path pattern with `glob` and does not need Bash for that task. - The implementation is fully native, read-only, deterministic, bounded in output, cancellable, and confined to existing readable roots. - Existing `read`, `list`, and `search` behavior remains unchanged. - No new dependency is added for traversal or wildcard matching. - `cargo fmt --all -- --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `make bundle`, and `cargo test --all-features` pass.

hugo added the enhancement label 2026-08-30 20:48:45 +00:00

hugo referenced this issue

2026-08-30 20:58:56 +00:00

Make F1 open a complete, up-to-date User Guide #83

hugo commented

2026-08-31 08:41:57 +00:00

Implemented and pushed as 5c74739.

Added the native, read-only glob(pattern, path, max_results) tool through the existing generated schema, validation, and execution path. Basename and root-relative patterns reuse the search wildcard matcher, including * spanning separators and repeated-star behavior.
Reused the recursive filesystem collector without adding a dependency or subprocess. Results contain regular files only, use normalized root-relative paths, sort lexically before limiting, include explicit no-match/truncation feedback, and include large/binary files without reading their contents.
Applied the existing project, enabled-skill, and visible Dev Brain readable-root policy; .git and Dev Brain hidden entries remain excluded as before. Traversal and selected roots do not follow symlinks, parent/out-of-root/non-directory paths fail normally, and cancellation returns the standard interrupted error.
Updated the User Guide to distinguish native filename discovery from content search and one-directory listing.

Verification covers required/optional schema fields and invalid arguments; exact, , ?, relative-path, repeated-star, spaces, Unicode, deterministic truncation, no-match, large/binary, .git, symlink and confinement cases; skill and Dev Brain visibility; and cancellation. Manual local-model verification selected glob with pattern=agent.rs and path=src, returned src/agent.rs, and did not use Bash. This is a new DS4Server agent feature with no corresponding DS4 functionality; existing DS4 transport plus read/list/search regressions remain green.

Commit gates passed: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (216 passed, 16 ignored).

Implemented and pushed as 5c74739. - Added the native, read-only glob(pattern, path, max_results) tool through the existing generated schema, validation, and execution path. Basename and root-relative patterns reuse the search wildcard matcher, including * spanning separators and repeated-star behavior. - Reused the recursive filesystem collector without adding a dependency or subprocess. Results contain regular files only, use normalized root-relative paths, sort lexically before limiting, include explicit no-match/truncation feedback, and include large/binary files without reading their contents. - Applied the existing project, enabled-skill, and visible Dev Brain readable-root policy; .git and Dev Brain hidden entries remain excluded as before. Traversal and selected roots do not follow symlinks, parent/out-of-root/non-directory paths fail normally, and cancellation returns the standard interrupted error. - Updated the User Guide to distinguish native filename discovery from content search and one-directory listing. Verification covers required/optional schema fields and invalid arguments; exact, *, ?, relative-path, repeated-star, spaces, Unicode, deterministic truncation, no-match, large/binary, .git, symlink and confinement cases; skill and Dev Brain visibility; and cancellation. Manual local-model verification selected glob with pattern=agent*.rs and path=src, returned src/agent.rs, and did not use Bash. This is a new DS4Server agent feature with no corresponding DS4 functionality; existing DS4 transport plus read/list/search regressions remain green. Commit gates passed: cargo fmt --all -- --check; cargo clippy --all-targets --all-features -- -D warnings; make bundle; cargo test --all-features (216 passed, 16 ignored).

hugo closed this issue

2026-08-31 08:42:05 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: hugo/DS4Server#82