The main objective of my work was to develop and validate a spatio-temporal AI algorithm to track, quantify, and analyze changes in skin and hair over time, particularly focusing on how these changes may correlate with grooming routines and product interactions. Specifically, this research focused on:

Unlike conventional tracking algorithms, which assume static signals (like tracking an airplane against a shifting sky), hair and skin analysis involves a unique scenario: the tracked "signal" (hair) evolves with time, while the "background" (skin) is generally more stable.

My work aimed to tackle this novel tracking problem using advanced AI techniques, creating algorithms capable of identifying and quantifying skin and hair changes over time in response to grooming products and treatments.

The main objective of my work was to develop and validate a spatio-temporal AI algorithm to track, quantify, and analyze changes in skin and hair over time, particularly focusing on how these changes may correlate with grooming routines and product interactions. Specifically, this research focused on:

1. Validating and improving an existing prototype for tracking hair and skin changes.
2. Developing new algorithms for detailed analysis of skin and hair changes in response to grooming interventions (e.g., shaving, IPL treatment)

1. Locate hairs in a pre-shave image using a Yolo model
2. Use the hair as a template and search the neighbourhood at the same location in the post-shave image
3. Iterate over different sizes to use as the template
4. Record the best match that corresponds to the maximum value of the correlation peak using correlation filters

I experimented with a variety of correlation filters including Normalized Cross Correlation, Phase-Only Correlation, and kth Law Nonlinear Matched Filters. I also attempted to preprocess the image using different techniques such as KNN, Vornoi transformation, and Binary Masking, to assist the performance of the correlation filters.

I experimented with a variety of correlation filters including Normalized Cross Correlation, Phase-Only Correlation, and kth Law Nonlinear Matched Filters. I also attempted to preprocess the image using different techniques such as KNN, Vornoi transformation, and Binary Masking, to assist the performance of the correlation filters.

One critical shortcoming of the initial approach was the lack of a definitive validation. The peak correlation value tells how strong the match was but a strong match might not mean a correct match. Furthermore, low contrast or high background noise might not yield very strong peaks and multiple matches of similar strength may confuse. I experimented with Signal-to-Noise Ratio (SNR) and Peak-to-Sidelobe Ratio (PSR) and while there was some success with PSR, it was not sufficiently reliable.

One critical shortcoming of the initial approach was the lack of a definitive validation.

The peak correlation value tells how strong the match was but a strong match might not mean a correct match.

I experimented with Signal-to-Noise Ratio (SNR) and Peak-to-Sidelobe Ratio (PSR) and while there was some success with PSR, it was not sufficiently reliable.

1. If post and pre images are perfectly aligned on top of each other, then the global location of the same hair should approximately be the same in both images.
2. Assuming optimal camera setup, with a perfect alignment, the only way for hairs to not be in the same location would be by the captured skin stretching.
3. In case of skin stretching, hair location translation across post and pre should generally follow a quadratic pattern for an image.
4. Deviation from a quadratic model signals an alignment or matching error.

1. Template Matching: Use cropped post-image as a template for normalized correlation against the pre-image.
2. Image Alignment: Adjust pre-image based on overlapping post-image regions.
3. Hair Localization: Identify hairs in the pre-image using a YOLO model.
4. Hair Matching: Use the hair as a template and search a small neighborhood at the same location in the post image

1. Distance Calculation: Measure translation ("distances") between same hair locations between pre and post images.
2. Curve Fitting: Perform linear regression for quadratic curve fitting on distance.
3. Thresholding (ROC curves):
   1. Filter faulty alignments based on thresholded R² scores.
   2. Assess hair matching accuracy based on distance absolute square difference between distance and predicted distance.

With the new pipeline, I was able to create a validation metric, and obtain a matching accuracy of 95% on filtered signals.