Whereas fingerprint analysis is widely used in forensic science, the rest of our hands equally retains a wealth of anatomical information. Bringing together anatomy and biometrics, ERC grantee Sue Black seeks to understand variability in the human hand though images contributed by ‘citizen scientists’. In search of uniqueness, she uses multimodal biometrics to visualise hand variation, and therewith develop a forensic identification system that might become as reliable as DNA. 

The H-Unique project has arisen from Sue Black’s expertise in the forensic identification of individuals from images of their anatomy in child abuse cases. “In 2004, I worked on a case of a young girl who alleged that her father came into her bedroom at night, and sexually assaulted her. Her mother refused to believe her. Therefore, the girl left the camera on her computer running at night in near infrared light mode. She took that footage to the police: at half past 4 in the morning, we see her laying on her bed, and we see a hand and a forearm coming into the vision of the camera, and do exactly what she said was happening to her.” 
“The police asked me if I could see anything in the image that might be of use as evidence”, Black continues. “But how do you identify someone from a hand and a forearm?  I could see the superficial vein pattern on the offender. When near infrared light shines onto the skin, it interacts with the deoxygenated blood so the veins stand out. I told the police: I can mark those, and we compare them with dad. If they are different, I can say with a 100 percent certainty that, whomever this is, it is not dad. However, if the vein patterns are the same, I cannot tell you what that means. We don’t know how variable vein patterns are in the forearm and the hand because that research has not yet been done.”
 

Vein patterns
 

Using artificial intelligence, Sue Black and her team have developed a method of identification that could become as reliable as DNA. “Hand anatomy is not solely based on genetics”, she explains. “It also depends on how you developed as a fetus because your vein pattern is set long before you were born, and it does not change. Then you have the effect of limbs interacting with the outside world, whether in your pigment pattern because you go out in the sun, or your scars or burn marks.  The pattern of skin creases on the back of the knuckles are different on every finger, different on both of your hands, and different in identical twins. “
Computers are trained using high quality images to recognise anatomical features such as veins and tendons, and to distinguish for instance pigment patches and a skin creases. A second database contains images of hands contributed anonymously by ‘citizen scientists’, taken with their mobile phone camera. “We wanted the public to be involved because public engagement with science is just so important to me”, Black explains. “People ask the most wonderful questions that make us test our own assumptions, for instance: I have got vitiligo, is that going to be a unique pattern of pigment?”

Real-world data
 

The hand-photos sent in by the public also give Sue Black and her team a dataset of less than optimal images. The computers learn through comparison of such real-world data rather than data collected in lab-based conditions. “Once we have developed an algorithm to extract and effectively compare vein patterns, we can take an image of one such pattern, go to the public citizens’ database and ask the computer: extract all the vain patterns that you see; how many of them match? The answer should be: none of them.” 
A third dataset containing 3-dimensional images and videos of hands adds another real life dimension. “It is not mannerism or movement that we are interested in”, says Black. “We look at how anatomy changes as the hand dynamic changes. If a hand goes into different poses, how much of the anatomy is lost and how much can we still detect?  What you then develop, factoring in different anatomical traits, is a multimodal biometric. Once you move into multimodal biometrics your reliability and your accuracy increases exponentially.” 
“With this knowledge, we can say with confidence in the courtroom that about 15 percent of anatomical detail is lost using a poor quality mobile phone camera, and another 10 percent is lost because the hand is not in an optimal position”, Black continues. “Then we can explain why there is no perfect match. In addition, if we have a multimodal biometric for an image, law enforcement agencies can run that algorithm through police databases anywhere in the world. Potentially, a perpetrator whose image has been held in different countries can be found, whereas before they were unable to bring those cases together.” 

Between disciplines
 

For Sue Black, the interdisciplinarity of her research project has been a rewarding experience. “Magic happens at thresholds between disciplines”, she says. “Anatomy is very much a science of form and function, and it is about variation in that form and function. Biometrics is oriented in computer science and mathematics: what is important there, sometimes has no bearing on anatomic logic. We have welded biometrics and anatomy together to create something that can solve particular problems in a real life forensic world.” 
The project’s contribution to forensic investigations has already been remarkable. “We have helped the UK police to secure at least 32 life sentences and over 500 years of prison sentences that would perhaps not been achieved if we had not been able to do this kind of work”, Sue Black estimates. With the algorithms becoming more refined, Black sees her work as an anatomist becoming redundant quickly. “Bryan, my senior researcher, phoned me up one day and said: we have a bit of a problem because on one of the images, the computer has identified some anatomy that you have missed. At first, I did not believe him. Then we went to the lab, and he showed me the images. He was right: the computer had seen part of a vein pattern that I did not see. I got so excited: the results of the research are living up what we really hoped for: now we have a computer system that is better at extracting veins than I am.”

Biography
 

Baroness Professor Dame Sue Black is a forensic anthropologist. She is the Pro Vice-Chancellor for Engagement at Lancaster University and is the current President of the Royal Anthropological Institute of Great Britain and Ireland.  From 2003 to 2018, she was Professor of Anatomy and Forensic Anthropology at the University of Dundee. She is President elect of St John's College, Oxford. In 2001, Black was appointed Officer of the Order of the British Empire (OBE) for her services to forensic anthropology in Kosovo. She was promoted to Dame Commander of the Order of the British Empire (DBE) in the 2016 Birthday Honours for services to forensic anthropology.