Re-identification of patterned species in open-set scenarios by matching keypoints and lines
ReMatch identifies individual animals from photographs of their natural patterns — lizard scale mosaics, zebra stripes, seal pelage, shark spots, human fingerprints alike. It segments the individual, matches keypoints and line segments between every pair of images with GlueStick, and turns those correspondences into a calibrated probability that two photographs show the same animal.
Its distinguishing feature is open-set operation. Unlike approaches that assume a fixed set of known individuals, ReMatch supports the detection of previously unseen individuals through a learned identity-level threshold, with minimal manual annotation — so a query can be answered "this is a new animal" rather than being forced onto the closest known match.
Across five heterogeneous datasets (Balearic wall lizards, plains zebras, Saimaa ringed seals, whale sharks and human fingerprints) ReMatch achieves an average closed-set Top-1 accuracy of 88.23 % (Top-5: 90.22 %) and an average open-set accuracy of 85.78 %. Performance is pattern-dependent: ReMatch excels on fine-scale local patterns such as scale mosaics and fingerprints (up to 98.51 %) and remains competitive under viewpoint variation, while other deep-embedding models retain an advantage on datasets dominated by global cues, such as whale sharks.
Paper — Alcaraz, Amores, Villa, Marcos, Tavecchia, Igual & Rotger, ReMatch: Re-identification of patterned species in open-set scenarios by matching keypoints and lines, Pattern Recognition 114744 (2026). doi:10.1016/j.patcog.2026.114744
- Try it first
- How it works
- Installation
- Data layout
- Training a model
- Identifying new images
- Adapting to a new species
- Configuration
- Running on a cluster
- Project structure
- Citation
- License
Four notebooks run the method end to end on small datasets bundled in
data/. demo-1 and demo-2 use 12 Balearic wall lizards in the
gallery and a query batch of 20 photographs, seven of which show animals the
model has never seen. demo-3 repeats the whole thing on a different species —
8 plains zebras — to show what adapting to your own animal actually involves, and
demo-0 shows what happens to a photograph before any of it starts.
demo-0-image-preparation.ipynb |
How a raw photograph becomes something matchable — both routes side by side: lizard (YOLO, then pattern extraction) and zebra (Grounded-SAM, no pattern step) |
demo-1-training.ipynb |
Match every pair, train the classifier, meta-model and threshold |
demo-2-query.ipynb |
Identify the new batch, reject the unseen animals, score the result |
demo-3-new-species.ipynb |
Start here for your own data. A second species (plains zebras) end to end, segmented with Grounded-SAM instead of YOLO — or with no segmentation at all |
uv venv --python 3.11
uv pip install -r requirements.txt
uv run jupyter labMatching every pair of the bundled 48 images is 1 128 comparisons, about a
quarter of an hour on a CPU. So the demos load precomputed match tables from
data/precomputed/ by default and each runs in a few minutes; set
RUN_MATCHING = True at the top of a notebook to compute them yourself. See Installation if you
do not have uv.
They read their configuration from params/, their helpers from utils/, and
write into results/ — the same paths the pipeline scripts use, so the demo is
the pipeline rather than a parallel implementation. No model weights are needed:
GlueStick and SuperPoint download themselves, and the bundled images are already
pattern crops. See data/README.md.
The images come from BalearicLizard, published separately:
Alcaraz, R., Albalat-Oliver, B., Villa, A. et al. A long-term photographic dataset for individual identification of the Balearic wall lizard. Scientific Data (2026). doi:10.1038/s41597-026-07411-z · Kaggle
Training (green) and deployment (red) share the first three steps: (1) image preparation, to extract the region of interest; (2) pattern matching between image pairs with GlueStick; and (3) aggregation of the point and line matches into features. Step (4) trains a classification model on the training set, and applies it to each new photograph to return its ten most likely candidates. An asterisk marks an optional step.
Five stages, common to every dataset:
- ROI segmentation — isolate the animal from its background.
- Wireframe extraction — describe it with SuperPoint keypoints and LSD line segments, merged into a single wireframe per image.
- Pair-level feature aggregation — match two wireframes with GlueStick, drop
matches scoring below 0.2, and reduce what survives to four numbers:
num_nonzero_points,mean_point_prob,num_nonzero_lines,mean_line_prob. - Pair-level classification — Logistic Regression, Random Forest, XGBoost and
CatBoost are compared over image-disjoint rolling folds; the best mean F1 wins.
Output: a probability
pthat a pair shows the same individual. - Identity-level calibration — for a query and a candidate identity,
pis aggregated over all that identity's photographs into[p, max, mean]and passed through a logistic-regression meta-model. If the best score falls below the threshold τ*, the query is reported as a new individual.
One stage is optional. Pattern extraction (SAM + structured edge detection) further isolates fine-scale texture and is worth enabling for species patterned like lizards; the other four datasets in the paper match on the segmented ROI directly.
Stages 4 and 5 are retrained per dataset. That is a design choice, not a limitation — the models are small and the matching stack above them never changes.
ReMatch is installed with uv, a single self-contained binary that manages both the Python interpreter and the packages. You do not need conda, and you do not need Python already installed — uv fetches the interpreter itself.
# Install uv (macOS / Linux)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows PowerShell:
# powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
git clone https://github.com/RoberAlcaraz/ReMatch.git
cd ReMatch
uv venv --python 3.11 # creates .venv, downloading Python 3.11 if needed
uv pip install -r requirements.txtTwo pins are not interchangeable with their obvious alternatives:
opencv-contrib-python (not opencv-python) because pattern extraction
calls cv2.ximgproc, and scikit-learn==1.7.2 because trained models are
stored as scikit-learn pickles.
requirements.txt installs everything, segmentation included. One exception:
Grounded-SAM also needs GroundingDINO, which is not on PyPI and so cannot be
pinned there. Install it only if you set SEGMENTATION_MODEL = "GroundedSAM":
CUDA_VISIBLE_DEVICES="" uv pip install --no-build-isolation \
"git+https://github.com/IDEA-Research/GroundingDINO.git" "transformers<5"All three parts of that command matter. --no-build-isolation: GroundingDINO's
setup.py imports torch at build time without declaring it as a build
dependency, so in an isolated build environment it fails with
ModuleNotFoundError: No module named 'torch'; the flag lets it see the torch
you already have. transformers<5: GroundingDINO wraps BERT through
bert_model.get_head_mask, which transformers 5 removed, so an unpinned install
imports fine and then dies with
AttributeError: 'BertModel' object has no attribute 'get_head_mask' the moment
you load the model. CUDA_VISIBLE_DEVICES="" stops setup.py from trying to
compile GroundingDINO's CUDA attention kernel — which, against the torch this
project pins, cannot be compiled at all. Its ms_deform_attn_cuda.cu hands
value.type() to AT_DISPATCH_FLOATING_TYPES, which wants a c10::ScalarType,
and torch has since removed the implicit conversion from
at::DeprecatedTypeProperties, so nvcc stops with no suitable conversion function ... exists and pip aborts the entire install. setup.py only
attempts the build when torch reports a visible GPU, so hiding the GPUs for the
duration of the install is enough to skip it. It changes nothing about the
installed package: this is not a CPU-only build.
That is why Failed to load custom C++ ops. Running on CPU mode Only! at import
time is expected, and not a sign of a broken environment — nobody gets that
kernel with this torch, GPU or no GPU. ReMatch notices and substitutes
GroundingDINO's own pure-PyTorch one, which runs on a GPU as happily as on a
CPU — a little slower than the fused op, and the same masks. Do not take that
warning at its word and force DEVICE = "cpu": on CPU, SAM's ViT-H encoder
makes segmentation about 5 s per photograph.
Left to itself GroundingDINO ignores what was built and dispatches on where the
tensors are, so on a machine that has a GPU it takes the compiled path anyway
and dies mid-forward-pass with NameError: name '_C' is not defined
(#2). If you hit that, you
are on a version of this repository from before the fix — git pull.
The demo notebooks ship pre-extracted pattern crops, so neither the segmentation packages nor GroundingDINO are needed to try ReMatch out.
Before SAM segments the individual scales, the pipeline upscales the image 2x.
The published version used ISR's RDN(weights="noise-cancel"), which denoised
as it upscaled. REMATCH_UPSCALER selects the model: edsr (the default, and
the only one needing no extra install), rdn, bicubic or none.
rdn reproduces the published crops exactly, but ISR pins a TensorFlow too
old to install. The pin is the only stale part, so skip it:
uv pip install --no-deps ISR
uv pip install "tensorflow>=2.16"A CUDA-capable GPU is strongly recommended. Everything falls back to CPU, but matching every pair of a real database is slow without one — on CPU, budget roughly 1 s per pair after the one-off wireframe pass, and remember the pair count grows quadratically: 48 images is 1 128 pairs, 500 images is 124 750.
Memory is dominated by the wireframe cache, at roughly 3 MB per image (the point–line associativity matrix is quadratic in the number of keypoints). The matching step loads the whole cache at once, so budget about 1.5 GB per 500 images, plus ~1 GB for the models themselves.
Place these in models/:
| File | What for | Source |
|---|---|---|
sam_vit_h_4b8939.pth |
SAM ViT-H — segmentation and pattern extraction | segment-anything releases |
groundingdino_swint_ogc.pth |
GroundingDINO — text-prompted detection | IDEA-Research/GroundingDINO |
model.yml.gz |
Structured edge detection — pattern extraction only | OpenCV contrib |
yolo-segmentation.pt |
Optional: fine-tuned YOLO detector for lizards | Hugging Face |
GlueStick and SuperPoint weights download themselves on first run. Behind a
TLS-intercepting proxy that can fail; fix your trust store, drop the weights into
resources/weights/ by hand, or as a last resort set REMATCH_INSECURE_SSL=1.
Training images go in one directory per individual — the directory name is the identity label:
data/images/
├── individual_001/
│ ├── IMG_0001.jpg
│ └── IMG_0002.jpg
└── individual_002/
└── …
Each batch of new photographs to identify goes in its own directory:
data/new/Batch1/
├── IMG_1001.jpg
└── IMG_1002.jpg
Run from the repository root, in order. Each step reads what the previous one wrote.
# 1. Segment every image, and optionally extract patterns.
python scripts/P1-image_preparation.pyWrites data/images-segmented/, results/unique_ids.txt, and — with
STEP_1B = True — data/images-pattern/.
Review the segmentations before continuing; a bad mask wastes the whole matching run.
# 2. Compute wireframes and match every pair.
python scripts/P2-pattern_matching.pyWrites results/precomputed_wireframe.h5 and results/matches.lmdb/. This is
the expensive step — cost grows quadratically with database size. Both caches
are resumable, so rerunning skips work already done.
# 3. Aggregate matches into the pair-level feature table.
python scripts/P3-feature_aggregation.pyWrites results/processed_matches.parquet. This step also drops near-duplicate
photographs — pairs matching above 900 keypoints are almost certainly the same
image twice, and one copy of each is removed. demo-1 and demo-3 show you
which ones, side by side, before they go.
# 4. Train the classifier, meta-model and threshold.
python scripts/P4-model_training.pyWrites results/best_classification_model.pkl,
results/logistic_regression_model.pkl, results/scaler.pkl and
results/threshold.txt. These four files are your model.
export NEW_IMAGES_NAME="Batch1" # which directory under data/new/ to process
# 1. Segment the new images.
python scripts/Q1-image_preparation.pyReview data/new/Batch1_checks/ and delete failures from
data/new/Batch1-pattern/ before continuing.
# 2. Match against the database and rank candidates.
python scripts/Q2-pattern_matching.py
python scripts/Q3-feature_aggregation.py
python scripts/Q4-model_application.pyWrites results/top10_results_Batch1.csv — the ten best candidate identities per
query, with a calibrated probability each. Queries scoring below τ* are marked
new.
Review that file: confirm the matches and assign identities to genuinely new animals.
# 3. Merge the reviewed batch into the database.
python scripts/Q5-add_results_to_db.pyAdds the new images, wireframes and matches to the database, so the gallery grows as you work.
The two review checkpoints are deliberate. ReMatch ranks candidates for an expert; it does not silently commit identity decisions.
- Start with Grounded-SAM. Set
SEGMENTATION_MODEL = "GroundedSAM"andCLASSES = ["your animal"]inparams/image_preparation_params.py. It is text-prompted, so it needs no per-species training — the right thing to try first. The paper uses this route for zebras, seals and sharks. - If segmentation struggles, fine-tune a YOLO segmentation model on a few
hundred annotated images, point
YOLO_SEGMENTATION_MODELat it and setSEGMENTATION_MODEL = "YOLO". A bounded, one-time annotation cost — this is the route used for lizards. - Pattern extraction (
STEP_1B) isolates fine-scale texture with SAM and structured edges. Leave it on for lizard-like scale mosaics; turn it off if the whole segmented animal is the pattern, as with stripes or spots. - Retrain. Run the four
Pscripts on your labelled images. Only the two small models change — the matching stack never does.
Everything lives in two files:
params/params.py— GlueStick and wireframe settings,IMAGE_HEIGHT_RESIZE(670 px; the models are calibrated against it, so changing it means retraining),RANDOM_SEED, and every data and result path.params/image_preparation_params.py—SEGMENTATION_MODEL("GroundedSAM"or"YOLO"),CLASSES, detection thresholds, and the step togglesSTEP_1A(segmentation) andSTEP_1B(pattern extraction).
NEW_IMAGES_NAME is read from the environment and selects which batch the Q
scripts operate on.
STEP_1B decides what the matching stage reads. With it on, pattern
extraction writes data/images-pattern/ and matching uses that — the lizard
route. With it off nothing is written there and matching runs on
data/images-segmented/ directly, which is what zebras, seals and whale sharks
do. params.MATCHING_IMAGES_FOLDER resolves to whichever applies; the scripts
read it rather than hardcoding either path.
The scripts are plain Python and take no arguments, so a job script is three lines. For SLURM:
#!/bin/bash
#SBATCH --job-name=rematch-match
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=16
#SBATCH --mem=64G
#SBATCH --time=12:00:00
source .venv/bin/activate
export PYTHONPATH="${PWD}:${PYTHONPATH:-}"
export NEW_IMAGES_NAME="${1:-Batch1}" # only needed for the Q scripts
python scripts/P2-pattern_matching.py
python scripts/P3-feature_aggregation.pyGive step 2 the most time and CPUs — it dominates. Everything else is minutes.
Always run from the repository root: paths in params/ are relative to it.
ReMatch/
├── scripts/ P1–P4 training, Q1–Q5 query — the entry points
├── params/ Configuration
├── utils/ Wireframes, matching, segmentation, pattern extraction
├── gluestick/ Vendored GlueStick (MIT) — see gluestick/NOTICE.md
├── data/ Your images, plus the bundled lizard and zebra demo sets
├── models/ Model weights — downloaded, not tracked
├── results/ Pipeline outputs and trained models — not tracked
├── licenses/ Third-party license texts
├── demo-0-image-preparation.ipynb What a photograph goes through first
├── demo-1-training.ipynb Match every pair, train, calibrate
├── demo-2-query.ipynb Identify a batch, reject unseen animals
└── demo-3-new-species.ipynb A second species — your own data starts here
@article{alcaraz2026rematch,
title = {{ReMatch}: Re-identification of patterned species in open-set
scenarios by matching keypoints and lines},
author = {Alcaraz, Roberto and Amores, Angel and Villa, Alejandro and
Marcos, Marta and Tavecchia, Giacomo and Igual, Jos{\'e} Manuel and
Rotger, Andreu},
journal = {Pattern Recognition},
year = {2026},
pages = {114744},
issn = {0031-3203},
doi = {10.1016/j.patcog.2026.114744}
}Please also cite GlueStick, on which the matching stage is built — see
THIRD_PARTY_NOTICES.md.
MIT — see LICENSE. Bundled and runtime third-party components carry
their own terms, including vendored GlueStick code (MIT), the SAM-derived mask
generator (Apache-2.0) and Ultralytics YOLO (AGPL-3.0, with obligations of its
own if you redistribute a service built on it). All documented in
THIRD_PARTY_NOTICES.md.
