Skip to content

Find duplicate files with dupes: - #192

Open
OmarGonD wants to merge 3 commits into
omacom:mainfrom
OmarGonD:find-duplicates
Open

OmarGonD wants to merge 3 commits into
omacom:mainfrom
OmarGonD:find-duplicates

Conversation

@OmarGonD

Copy link
Copy Markdown
Contributor

Adds dupes: to the search (/), after Everything's duplicate finder. It lists the files here and below that have an identical twin, each set together and the biggest first. Files are compared by size, then their first 64 KB, then every byte (SHA-1). Folders, symlinks and empty files are skipped. The rest of the query still narrows what's compared (dupes: ext:jpg).

Includes a test and a README line. bin/test passes.

🤖 Generated with Claude Code

Like Everything's duplicate finder: "dupes:" lists the files here and
below that have an identical twin, compared by size, then the first 64 KB,
then every byte, each set of twins together and the biggest first.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 3/5

[Medium impact] The changes affect folder search within Folio's file manager.

The PR is not safe to merge while read failures can label different files as duplicates and the trash search misses matches across drives.

Findings

  1. P1 Read failures look like duplicates. ▶
  2. P1 Trash misses duplicates across drives. ▶
  3. P2 Folder counts stay blank ▶
  4. P2 Canceled searches keep reading files. ▶

Summary

Adds dupes: searches for identical files and reuses completed search results when the user narrows a query.

  • Folder search finds files with identical contents and groups them together.
  • Narrower searches reuse completed results from the same folder and filter scope.

Reviews (2) · Last reviewed commit: "Merge incremental-search, and make a dup..." · Reviewed by Greptile

Comment thread src/foldermodel.cpp
Comment on lines +444 to +451
const QByteArray chunk = file.read(left < 0 ? 1 << 20 : qMin<qint64>(left, 1 << 20));
if (chunk.isEmpty())
break;
hash.addData(chunk);
if (left > 0)
left -= chunk.size();
}
return hash.result();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Read failures look like duplicates. hashOf treats a failed read like the end of the file and returns a valid hash of whatever it read. If equal-size files fail before returning any bytes, they all receive the hash of empty content and appear as duplicates despite having different contents. The same problem occurs when reads fail after a shared prefix. Reject failed reads instead of returning their partial hash.

Comment thread src/foldermodel.cpp
Comment on lines +570 to +571
if (wanted.dupes) {
entries = duplicatesAmong(entries, limit, stop);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Trash misses duplicates across drives. The new duplicate check runs inside search, but listTrash calls search separately for each drive. A file in the home trash and its identical twin in another drive's trash are therefore both omitted. Each call also restarts group, so unrelated sets from different drives can mix together when sorted. Gather candidates from all trash folders before finding and numbering the duplicate sets.

Comment thread src/foldermodel.cpp
Comment on lines +466 to +469
const auto split = [&](const QVector<int> &rows, qint64 bytes) {
QHash<QByteArray, QVector<int>> sets;
for (int row : rows) {
const QByteArray hash = hashOf(files[row].path, bytes);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Canceled searches keep reading files. split never checks stop between files, and hashOf never checks it between chunks. Leaving a duplicate search during a full-file comparison still reads every file in that set, which can take a long time for disk images or videos. These jobs share the eight workers used to list folders, so abandoned searches waste disk work and can delay later listings. Check stop between files and between read chunks.

OmarGonD and others added 2 commits November 18, 2026 13:39
Typing on ("beac" to "beach") or adding an ext: keeps the matches that
still fit instead of walking the folder again, when the last search wasn't
cut short by the limit. Widening, or a change of folder, looks again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Comment thread src/foldermodel.cpp
Comment on lines +1000 to +1002
setLoading(false);
emit loaded();
return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Folder counts stay blank. Narrowing a completed search returns here without restarting countFolders. If the user types while the previous folder counts are still running, reload advances m_generation and those counts are discarded. The kept folders still have itemCount == -1, so their counts stay blank until another search or refresh.

Call countFolders(generation) before returning so unfinished counts restart for the kept rows.

@greptile-apps

greptile-apps Bot commented Nov 18, 2026

Copy link
Copy Markdown

Want your agent to iterate on Greptile's feedback? Try greploops.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant