The Idea of Chaining
The core idea of chaining
DBSCAN (and connected components, which is basically the same mechanism) never asks "are these two images actually similar?" It only ever asks "is there a path of small, similar steps connecting them?" Think of stepping stones across a river. I only ever check that the next stone is within reach, never that the far bank resembles the near bank. If frame 1 is close to frame 2, frame 2 is close to frame 3, and so on for a hundred frames, the algorithm links all hundred into one group, even if frame 1 and frame 100 look nothing alike. No single comparison ever directly checked frame 1 against frame 100. The link is purely transitive.
Why my data was the perfect condition for this to go wrong
Medical video is a slow, continuous walk through embedding space. The scope moves a little, the embedding moves a little, frame after frame. That is exactly the "river of stepping stones" scenario. Every consecutive step is small and safe on its own, but a long enough run of small steps can drift the endpoints very far apart without any single step ever looking risky.
How I proved it, in three independent ways
1. A direct numeric proof on my real 10 frame matrix. Frame 1 vs. frame 10 similarity was 0.931, clearly below any threshold I was considering (0.95). Directly compared, nobody would call them duplicates. But connected components at threshold 0.95 still merged all 10 into one group because the chain in between never broke: 1→6 (0.955), 6→7 (0.989), 7→8 (0.983), 8→9 (0.982), 9→10 (0.993). Every single link cleared the bar. The merge happened entirely through intermediate stones, never through a real comparison of the two ends.
2. A numeric signature in my real 9,013 image sweep. Going from eps=0.01 to eps=0.02, the number of distinct clusters barely moved, from 711 to 754, only a 6% increase. But the largest cluster nearly doubled, from 76 to 155, a 104% increase. That was the tell. DBSCAN was not discovering new duplicate groups as the threshold loosened. It was swallowing more frames into groups that already existed. That is chaining showing up as a number, not just an argument.
3. Visual confirmation in the UMAP plots. Distinctive looking content, such as a clear fold or a specific angle, formed tight, isolated trails that filled in early (eps≈0.01) and then simply stopped changing. They were well behaved and self contained. But the crowded central region, almost certainly the bland, generic frames with the least distinguishing visual signal, stayed mostly unclustered until eps pushed past 0.015, then rapidly fused together in exactly the same window where the largest cluster exploded. I could actually see chaining happening and identify exactly where it occurred. It was not in the clean content but in the ambiguous, low signal content that had the least visual structure to anchor it.
Why this makes DBSCAN structurally wrong for this problem, not just mistuned
The deeper lesson was not that I picked a bad eps. It was that no single global eps can work because the two regions of my data need opposite things. The clean, distinctive trails need a looser threshold to fully saturate their legitimate duplicate runs. The bland, ambiguous region needs a stricter threshold to avoid false merges across genuinely unrelated moments. One global number is forced to compromise between two contradictory requirements, and my sweep showed that compromise breaking down visibly in exactly the region I would expect, where the content is least distinctive and stepping stones are easiest to find.
What I learned to replace it with
The fix was not a better clustering algorithm. It was abandoning the "path of small steps" logic entirely. The greedy walk I moved to only ever compares a new candidate against the actual set of frames already kept, never through a chain of intermediate dropped frames. On the same 10 frame example, it kept three representatives, frames 1, 7, and 10, instead of collapsing everything to frame 1. It correctly preserved the real drift in the run instead of erasing it. That is the practical difference. Chaining silently collapses information through transitivity. Direct comparison against what is actually retained does not allow that to happen.