Perceptual Image Hashing
Two files can look identical yet share nothing a computer can match on: different filenames, different URLs, different byte counts after recompression. What is a perceptual hash? It is a compact fingerprint of how an image looks, not what it is named or how many bytes it contains. Perceptual image hashing uses those fingerprints to group near-identical pictures that string matching never catches.
Bulk Image Downloader From URL List computes hashes locally in your browser through the Perceptual Duplicate Finder on the options page.
Step 1: perceptual hash versus cryptographic hash
A cryptographic hash like SHA-256 changes completely when a single byte changes. Re-save a JPEG at 90% quality instead of 95% and the hash is unrecognizable even though the picture looks the same. A perceptual hash shrinks the image to a coarse pattern of brightness or structure and produces a short code. Similar-looking images produce similar codes even after resize, recompression, or a format swap from JPEG to WebP.
Closeness is measured as Hamming distance—how many bits differ between two hashes. A small distance means near-identical appearance; a large distance means unrelated images.
Step 2: common hash types you will see
- dHash records gradient direction—how brightness changes across the image
- aHash compares each pixel block to average brightness
- pHash uses a frequency transform to capture structure that survives compression
No single hash catches every variant, which is why production tools combine several signals instead of trusting one algorithm.
Step 3: why URL and filename matching fail
Exact URL deduplication only catches identical strings. Real scrapes repeat the same photo as hero.jpg, banner-final-2.jpg, and two different CDN paths. Byte comparison fails too because resizing rewrites file contents. Perceptual hashing compares appearance, so a thumbnail and its full-size original can land in the same group.
Step 4: how the duplicate finder uses perceptual hashing
The Perceptual Duplicate Finder blends 15+ visual signals—dHash, aHash, and pHash alongside histogram comparison, dominant colors, texture (LBP), sharpness, and shape descriptors. Pick a sensitivity preset:
- Strict — fewer false positives, may miss heavily edited variants
- Balanced — default middle ground
- Aggressive — catches more near-matches, review groups carefully
Licensed PRO scans up to 600 URLs per run; Try PRO is capped at 1 URL or source per scan. Use the All Tasks scope on the Duplicate Finder tab to scan across every task at once. Tune individual signal weights when you need finer control.
Step 5: review groups before you delete
Treat similarity scores as strong evidence, not absolute proof. Heavy cropping, large rotation, or strong recoloring can push two versions of the same image apart; very plain graphics can occasionally over-match. Review flagged groups, pick a keep rule (first, largest, shortest URL, or manual), then remove duplicates from task lists.
Install the extension when filename matching leaves visually identical files behind.
Where the duplicate pass sits on the screen
Find duplicates opens on top of Home rather than in its own window, which tells you what it is for: deciding about the download you are about to save, not investigating links. You keep one image and drop the copies against the actual list, then start the task. Filter images behaves the same way.
Contrast that with the link tools. Check redirects and Find broken links open in a separate window because they examine the list itself. Knowing which tools overlay Home and which open apart saves a step every time you are unsure where a result will appear.
Continue reading: What Is Image MIME Type.