BUG: Same image gets indexed multiple times when directory contains symlinks/hardlinks

[roushnisareen]

Is there an existing issue for this?

• I have searched the existing issues

What happened?

Describe the bug

When a library includes symbolic links (or hardlinks) to photos, PictoPy appears to index the same underlying image multiple times instead of treating it as one media item.

Steps to reproduce

1. Create folder A with 20 images.
2. Create folder B containing symlinks to the same 20 images from A (or hardlinks).
3. Add both folders in PictoPy settings.
4. Start scan/indexing.

Expected behavior

Each physical image should be indexed once (deduplicated by inode/path-resolve/hash strategy), with a single media record in the UI.

Actual behavior

Duplicate entries appear for the same image content, and indexing/AI tagging runs repeatedly for linked copies.

Impact

• Slower scans and higher CPU usage
• Duplicate thumbnails/search results
• Larger metadata DB than necessary

Suggested fix

• Normalize file identity before insert (resolve symlink + stable file identity check).
• Add a unique constraint / duplicate guard in ingestion pipeline.
• Skip AI pipeline when file identity already exists.

Record

• I agree to follow this project's Code of Conduct

[github-actions]

👋 Thanks for opening this issue! The maintainers will review it shortly. Till then, do not open any PR. This is a strict rule we like to follow around here :)

[Riddhi8077]

Hey @roushnisareen and other Mentors,

I’ve analysed the ingestion flow and it looks like images are currently treated as unique based on their path, without normalizing the underlying file identity.

Because of this, symlinks or hardlinks pointing to the same physical file are indexed as separate media entries and go through the AI pipeline again.

I’d like to work on resolving file identity before insertion (e.g., via realpath/inode strategy) and adding a duplicate guard in the ingestion layer to prevent re-indexing of the same physical file.

Please assign this issue to me if this approach aligns with the expected solution.

[codewith-shivansh]

Hi @rahulharpal1603
I’d like to work on this issue.
I understand that due to symlinks or hardlinks to the same underlying image, PictoPy currently indexes the same file multiple times.
My plan is to investigate the current indexing pipeline, add normalization of file identity (e.g. resolving symlinks or using a stable identifier), and add a guard to prevent duplicate entries.
I will also include tests for folders with linked images to ensure correctness.
Before I start, could you please confirm if there are any preferred methods (like inode vs. hash) for identifying unique files in the index?
Thank you!

[Riddhi8077]

Hi @everyone,

I commented on this issue a couple of weeks ago after analysing the ingestion flow and noticing that images are currently treated as unique based on their path, which causes symlinked or hardlinked files to be indexed multiple times.

Since there hasn’t been activity on the issue for some time and no PR appears to be linked yet, I’m planning to work on a fix that normalizes file identity (e.g., resolving real paths / inode checks) before inserting into the media database to prevent duplicate indexing.

I’ll open a PR shortly so the approach can be reviewed.