SC-4: file enumeration and recursive directory traversal engine

.gitattributes

@@ -0,0 +1 @@
+* text=auto eol=lf

README.md

@@ -4,21 +4,18 @@ An rsync-inspired file synchronization tool written in Go.
 
 ## Status
 
-**CLI skeleton only — no sync logic yet.** The command-line interface parses
-arguments and flags and prints what it received, but no files are actually
-read, compared, transferred, or deleted. `internal/sync` and
-`internal/transport` are currently empty packages reserved for that work.
+CLI parsing and file enumeration are implemented; data transfer is not.
+`internal/sync` builds a sorted file list (`sync.Walk`), but nothing calls
+it yet — the CLI only echoes parsed flags — and `internal/transport` is
+still empty.
 
 ## Build
 
-Requires Go (see [go.mod](go.mod) for the version this module targets).
-
 ```sh
 go build ./cmd/grsync
 ```
 
-This produces a `grsync` (or `grsync.exe` on Windows) binary in the current
-directory. To run without building a binary:
+Or run directly without building a binary:
 
 ```sh
 go run ./cmd/grsync <source>... <destination> [flags]
@@ -30,68 +27,53 @@ go run ./cmd/grsync <source>... <destination> [flags]
 grsync <source>... <destination> [flags]
 ```
 
-At least one `source` and exactly one `destination` are required
-(the last positional argument is always the destination), matching rsync's
-own `SRC... DEST` grammar rather than restricting to a single source.
+At least one `source` and exactly one `destination` are required; the last
+argument is always the destination.
 
 | Flag | Shorthand | Description |
 |---|---|---|
-| `--archive` | `-a` | archive mode (equivalent to common rsync defaults) |
-| `--verbose` | `-v` | increase output verbosity |
-| `--compress` | `-z` | compress file data during transfer |
+| `--archive` | `-a` | archive mode |
+| `--verbose` | `-v` | increase verbosity |
+| `--compress` | `-z` | compress data during transfer |
 | `--recursive` | `-r` | recurse into directories |
-| `--dry-run` | `-n` | show what would be transferred without transferring |
+| `--dirs` | `-d` | list directories without recursing into them (implied by `-r`) |
+| `--dry-run` | `-n` | show what would be transferred |
 | `--delete` | | delete extraneous files from destination |
 | `--progress` | | show progress during transfer |
-| `--exclude PATTERN` | | exclude files matching PATTERN (repeatable) |
-| `--include PATTERN` | | include files matching PATTERN (repeatable) |
-| `--filter RULE` | | add a file-filtering RULE (repeatable) |
+| `--exclude PATTERN` | | exclude matching files (repeatable) |
+| `--include PATTERN` | | include matching files (repeatable) |
+| `--filter RULE` | | add a filter rule (repeatable) |
 
-`--exclude`, `--include`, and `--filter` are collected into a single ordered
-rule list, not three independent lists — their relative order on the command
-line is preserved regardless of which of the three flags produced each rule.
-This matches rsync's first-match-wins filter semantics, where rule order is
-significant.
+`--exclude`/`--include`/`--filter` share one ordered rule list — their
+relative order on the command line is preserved, matching rsync's
+first-match-wins semantics.
 
-At this stage, running the command only echoes the parsed values back — it
-does not transfer any files:
+## File Enumeration
 
-```sh
-$ go run ./cmd/grsync ./src1 ./src2 ./dst -av --exclude "*.log" --include "keep.log"
-sources:     [./src1 ./src2]
-destination: ./dst
-archive:     true
-verbose:     true
-...
-filters:     exclude:*.log, include:keep.log
-```
+`sync.Walk`, in the `internal/sync` package, recursively lists a source tree
+into a sorted `[]FileEntry` (path, size, mtime, mode, uid/gid, symlink
+target). Symlinks are captured via `Lstat`, never followed.
+
+`-r`/`-d` control how far it descends:
+
+| Recursive | Dirs | Result |
+|---|---|---|
+| off | off | directories skipped entirely |
+| off | on | directories listed, not descended into |
+| on | any | full recursion |
+
+On Windows, `UID`/`GID` are always `0` — there's no POSIX ownership concept
+to read, so `0` means "unavailable," not a real value.
 
 ## Architecture
 
-grsync follows the same conceptual split as upstream
-[rsync](https://rsync.samba.org/): a **sync layer** that decides *what* needs
-to change (comparing source and destination file trees, applying
-include/exclude/filter rules) and a **transport layer** that handles *how*
-data actually moves (local copies today, with room for remote transports
-later). The codebase mirrors that split:
-
-- `cmd/grsync` — CLI entrypoint (binary wiring only).
-- `internal/cli` — argument/flag parsing (built on
-  [spf13/cobra](https://github.com/spf13/cobra)); owns no sync or transport
-  logic.
-- `internal/sync` — (placeholder) will own comparison and change-planning
-  logic, analogous to rsync's file-list generation and delta algorithm.
-- `internal/transport` — (placeholder) will own how bytes are actually moved
-  between source and destination.
-
-grsync's goal is full feature parity with upstream rsync, not just
-similarity of CLI flags — including protocol- and format-level
-interoperability with the C implementation where specified (for example,
-batch mode's `--write-batch`/`--read-batch` file format is required to be
-interoperable with the C rsync implementation). None of that is implemented
-yet: the wire protocol, delta-transfer algorithm, and batch file format are
-still to come in later tickets. This README will be updated as each piece
-lands.
+- `cmd/grsync` — CLI entrypoint.
+- `internal/cli` — flag/argument parsing (built on cobra).
+- `internal/sync` — file-list generation today; comparison/delta logic later.
+- `internal/transport` — (placeholder) data movement, local and remote.
+
+Goal: full feature parity with upstream rsync, including protocol/format
+interoperability where specified (e.g. batch mode's file format).
 
 ## License
 

internal/cli/root.go

@@ -42,6 +42,7 @@ type options struct {
 	verbose     bool
 	compress    bool
 	recursive   bool
+	dirs        bool
 	dryRun      bool
 	delete      bool
 	progress    bool
@@ -95,6 +96,7 @@ func NewRootCmd() *cobra.Command {
 	flags.BoolVarP(&opts.verbose, "verbose", "v", false, "increase output verbosity")
 	flags.BoolVarP(&opts.compress, "compress", "z", false, "compress file data during transfer")
 	flags.BoolVarP(&opts.recursive, "recursive", "r", false, "recurse into directories")
+	flags.BoolVarP(&opts.dirs, "dirs", "d", false, "include directories themselves without recursing into their contents (implied by --recursive)")
 	flags.BoolVarP(&opts.dryRun, "dry-run", "n", false, "show what would be transferred without transferring")
 	flags.BoolVar(&opts.delete, "delete", false, "delete extraneous files from destination")
 	flags.BoolVar(&opts.progress, "progress", false, "show progress during transfer")
@@ -131,12 +133,13 @@ func run(cmd *cobra.Command, sources []string, destination string, opts *options
 			"verbose:     %t\n"+
 			"compress:    %t\n"+
 			"recursive:   %t\n"+
+			"dirs:        %t\n"+
 			"dry-run:     %t\n"+
 			"delete:      %t\n"+
 			"progress:    %t\n"+
 			"filters:     %s\n",
 		sources, destination,
-		opts.archive, opts.verbose, opts.compress, opts.recursive, opts.dryRun,
+		opts.archive, opts.verbose, opts.compress, opts.recursive, opts.dirs, opts.dryRun,
 		opts.delete, opts.progress, rules.String(),
 	)
 

internal/sync/uidgid_unix.go

@@ -0,0 +1,21 @@
+//go:build !windows
+
+package sync
+
+import (
+	"os"
+	"syscall"
+)
+
+// lookupUIDGID extracts the owning user/group ID from a Lstat'd
+// os.FileInfo. On POSIX platforms this is available via the
+// syscall.Stat_t underlying the info; ok is false only if the platform's
+// Sys() doesn't actually return that type (unexpected, but Sys() is
+// documented as platform-dependent, so it's not guaranteed).
+func lookupUIDGID(info os.FileInfo) (uid, gid uint32, ok bool) {
+	stat, ok := info.Sys().(*syscall.Stat_t)
+	if !ok {
+		return 0, 0, false
+	}
+	return stat.Uid, stat.Gid, true
+}

internal/sync/uidgid_windows.go

@@ -0,0 +1,14 @@
+//go:build windows
+
+package sync
+
+import "os"
+
+// lookupUIDGID always reports unavailable on Windows: os.FileInfo.Sys()
+// there returns *syscall.Win32FileAttributeData, which carries no POSIX
+// uid/gid concept at all (Windows uses SIDs and ACLs instead). Callers must
+// treat ok == false as "not populated", not as "owned by uid/gid 0" —
+// zero is not a meaningful default here.
+func lookupUIDGID(_ os.FileInfo) (uid, gid uint32, ok bool) {
+	return 0, 0, false
+}

internal/sync/walk.go

@@ -0,0 +1,142 @@
+// Package sync builds the file list ("flist") that grsync compares between
+// source and destination before any data transfer happens — the same
+// planning phase upstream rsync performs before its delta algorithm runs.
+package sync
+
+import (
+	"io/fs"
+	"os"
+	"path/filepath"
+	"sort"
+	"time"
+)
+
+// FileEntry describes a single file, directory, or symlink discovered under
+// a source root. Path is always relative to that root, using "/" as the
+// separator regardless of host OS — rsync's wire protocol and file lists
+// are always "/"-separated, and grsync targets protocol-level
+// interoperability, so paths are normalized at collection time rather than
+// left OS-native and converted later.
+type FileEntry struct {
+	Path    string
+	Size    int64
+	ModTime time.Time
+	Mode    fs.FileMode
+	UID     uint32
+	GID     uint32
+	// OwnershipAvailable reports whether UID/GID were actually populated.
+	// A real uid/gid of 0 (root) is a valid value, so callers must check
+	// this rather than treating a zero UID/GID as "unavailable" — see
+	// uidgid_windows.go, where it is always false.
+	OwnershipAvailable bool
+	LinkTarget         string
+	IsDir              bool
+}
+
+// WalkOptions controls how far Walk descends, mirroring rsync's own
+// -r/--recursive and -d/--dirs flags:
+//
+//   - Recursive=false, Dirs=false (rsync's default with neither flag):
+//     directories are skipped entirely — not listed, not descended into.
+//     Only regular files/symlinks directly under root are collected.
+//   - Recursive=false, Dirs=true (-d): directories are listed (so they can
+//     be created on the receiving end) but their contents are not — Walk
+//     does not descend into them.
+//   - Recursive=true (-r): full recursion into every subdirectory,
+//     regardless of Dirs — this matches rsync, where -r makes -d redundant.
+type WalkOptions struct {
+	Recursive bool
+	Dirs      bool
+}
+
+// Walk collects a FileEntry for every entry found under root, not including
+// root itself, subject to opts. Paths are relative to root and
+// "/"-separated.
+func Walk(root string, opts WalkOptions) ([]FileEntry, error) {
+	var entries []FileEntry
+
+	err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if path == root {
+			// The root itself isn't part of its own file list.
+			return nil
+		}
+
+		if !opts.Recursive && d.IsDir() && !opts.Dirs {
+			// Neither -r nor -d: rsync skips directories entirely rather
+			// than creating them empty, so this one is neither listed nor
+			// descended into.
+			return filepath.SkipDir
+		}
+
+		rel, err := filepath.Rel(root, path)
+		if err != nil {
+			return err
+		}
+
+		// os.Lstat, not os.Stat: a symlink must be reported as itself, not
+		// resolved to whatever it points at. WalkDir's own DirEntry is
+		// already lstat-derived (it never follows symlinks when deciding
+		// what to recurse into), but calling Lstat explicitly here makes
+		// that guarantee obvious at the call site rather than implicit in
+		// WalkDir's internals.
+		info, err := os.Lstat(path)
+		if err != nil {
+			return err
+		}
+
+		entry := FileEntry{
+			Path:    filepath.ToSlash(rel),
+			Size:    info.Size(),
+			ModTime: info.ModTime(),
+			Mode:    info.Mode(),
+			IsDir:   info.IsDir(),
+		}
+
+		// lookupUIDGID is platform-specific (see uidgid_unix.go /
+		// uidgid_windows.go): on Windows it always reports unavailable,
+		// leaving UID/GID at their zero value. OwnershipAvailable carries
+		// that ok flag through so callers can't mistake the zero value for
+		// a real uid/gid of 0 (root) — see uidgid_windows.go for why.
+		entry.UID, entry.GID, entry.OwnershipAvailable = lookupUIDGID(info)
+
+		// info.Mode()&fs.ModeSymlink is only ever set by Lstat (Stat
+		// resolves through it), which is exactly why Lstat was required
+		// above: this branch would never be reachable otherwise.
+		if info.Mode()&fs.ModeSymlink != 0 {
+			target, err := os.Readlink(path)
+			if err != nil {
+				return err
+			}
+			entry.LinkTarget = target
+		}
+
+		entries = append(entries, entry)
+
+		if !opts.Recursive && d.IsDir() {
+			// Whether or not -d caused this directory to be listed above,
+			// without -r its contents are never descended into.
+			return filepath.SkipDir
+		}
+
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+
+	// filepath.WalkDir only guarantees lexicographic order within each
+	// directory, not across the whole tree (a directory's contents are
+	// interleaved with sibling entries in traversal order, not sorted
+	// globally). Sorting the flattened list by full path afterward is what
+	// actually gives deterministic, rsync-comparable ordering; plain Go
+	// string comparison is byte-wise, matching C's strcmp, which is what
+	// rsync's own pathname sort relies on.
+	sort.Slice(entries, func(i, j int) bool {
+		return entries[i].Path < entries[j].Path
+	})
+
+	return entries, nil
+}