ARTICLE: "Losing Face: How a facial recognition mismatch can ruin your life," by Ava Kofman, published by The Intercept on October 13, 2106. (Ava Kofman (a journalist based in Brooklyn. This article was reported in partnership with The Investigative Fund at The Nation Institute.)
GIST: "It was just after sundown when a man knocked on Steve Talley’s door in south Denver. The man claimed to have hit Talley’s silver Jeep Cherokee and asked him to assess the damage. So Talley, wearing boxers and a tank top, went outside to take a look. Seconds later, he was knocked to the pavement outside his house. Flash bang grenades detonated, temporarily blinding and deafening him. Three men dressed in black jackets, goggles, and helmets repeatedly hit him with batons and the butts of their guns. He remembers one of the men telling him, “So you like to fuck with my brothers in blue!” while another stood on his face and cracked two of his teeth. “You’ve got the wrong guy,” he remembers shouting. “You guys are crazy.” Talley was driven to a Denver detention center, where he was booked for two bank robberies — the first on May 14 and the second on September 5, 2014, 10 days before his arrest — and for assaulting an officer during the second robbery. Surveillance footage from a robbery that occurred on May 14, 2014, at a U.S. Bank in Denver, Colorado. After surveillance camera images of the September robbery were publicly distributed, three of Talley’s acquaintances called in with tips to the police hotline, noting similarities between Talley’s appearance and the robber’s. A detective then showed photographs of both the May and September robber to Talley’s estranged ex-wife. “That is Steven,” she told him. “That is my ex-husband.” The identifications justified Talley’s detention, even though he claimed he had been at work as a financial adviser for Transamerica Capital when the May robbery took place. Talley said he was held for nearly two months in a maximum security pod and was released only after his public defender obtained his employer’s surveillance records. In a time-stamped audio recording from 11:12 a.m. on the day of the May robbery, Talley could be heard at his desk trying to sell mutual funds to a potential client. Nine miles north, a white male wearing a black baseball cap, red athletic jacket, white shorts, and black sneakers entered a U.S. Bank, where he threatened the teller, hid $2,475 in his shirt, wrestled with an off-duty officer, and jumped down a flight of 10 stairs to the parking lot. At the same time as Talley was trying to close a deal, parking lot surveillance tapes show the robber tumbling with the officer, escaping his grip, and jogging away. Talley was released in November, and the charges were apparently dropped. In the months that followed, a series of medical exams revealed that Talley had sustained several injuries on the night of his arrest, including a broken sternum, several broken teeth, four ruptured disks, blood clots in his right leg, nerve damage in his right ankle, and a possibly fractured penis. “I didn’t even know you could break a penis,” he told me. But while voice recordings had exculpated Talley, an appeal to other, seemingly objective markers of his identity would soon be used to implicate him again. Nearly a year after his release from jail, Talley was arrested a second time on December 10, 2015, and charged with the aggravated bank robbery that had taken place the morning of September 5, 2014. This time around, Denver prosecutors obtained what looked like damning forensic evidence of their own. The detective assigned to Talley’s case, Jeffery Hart, had requested that an FBI facial examiner manually compare stills from the banks’ grainy surveillance videos to several pictures of Talley — a tall, broad-shouldered white man with short blond hair, mild blue eyes, and a square jaw. The FBI analysis concluded that Talley’s face did not match the May robber’s, but that he and the September robber shared multiple corresponding characteristics, including the shape of the head, chin, jaw line, mole marks, and ear features. “The questioned individual depicted” in the September images, the report concluded, “appears to be Talley.” Except that it wasn’t. Again. Steve Talley is hardly the first person to be arrested for the errors of a forensic evaluation. More than half of the exonerations analyzed by the Innocence Project have involved cases where forensic experts cited flawed or exaggerated evidence, and in 2009 a landmark paper by the National Academy of Sciences stated what many had long suspected: Apart from DNA testing, no other forensic method could reliably and consistently “demonstrate a connection between evidence and a specific individual or source.” The report launched the forensic science community into a crisis of interpretation, with many questioning whether its methods should be deemed “sciences” at all. Last year, the FBI announced that virtually all of its hair analysis testimony had been scientifically indefensible, while the Texas Forensic Science Commission recently recommended banning bite-mark evidence from court. In September, the President’s Council of Advisors on Science and Technology issued a report that firmly concluded that forensic techniques relying on visual patterns fell short of scientific standards and relied on the subjective opinions of law enforcement. But while the accuracy of other visual, pattern-matching methods like blood-splatter analysis has been subject to vigorous public debate, the fallibility of facial comparison, or facial identification, has received less attention. This may be because comparing facial images can seem like an easy or even intuitive task. “We assume, wrongly, that we are good at recognizing faces,” said David White, an Australian psychologist who researches facial perception. In reality, however, we are for the most part terrible at comparing photographs, video stills, and composite images of unfamiliar faces — and this remains the case even with high-quality, full frontal images. The photographs of Talley and the suspects were sent to the FBI’s Forensic, Audio, Video and Image Analysis Unit, where trained forensic examiners manually compare points of similarity between faces to help investigators confirm or eliminate the identities of potential suspects. After selecting frames from the video, they work back and forth between evidence from the crime scene and images of their suspect to develop a conclusion regarding the type and number of similarities. While the FBI has been comparing facial images since at least the 1950s, FAVIAU was formed in 2000 to merge video and analysis units spread across the agency into a main office in Quantico, Virginia. Examiners there compare not only faces but also the voices and heights of suspects submitted by law enforcement for terrorism, homicide, armed robbery, and financial fraud cases, among others. As of 2012, the FBI unit had around five employees comparing faces, all of whom had undergone a two-year training program, but according to a source familiar with the agency’s workings, the unit has since expanded. The methods used by the FBI and other independent examiners typically follow the ACE VR method — which stipulates that examiners analyze, compare, and evaluate the (known) images from the crime scene and the images in question. They then verify and peer review the analysis. “It is dangerous for a video examiner to tell the court that the person on video is the defendant. If it were that easy, there would be little need for trials in a surveillance society and that’s a frightening thought.” But even this method — undertaken in ideal conditions — remains vulnerable. No threshold currently exists for the number of points of similarity necessary to constitute a match. Even when agencies like the FBI do institute classification guidelines, subjective comparisons have been shown to differ greatly from examiner to examiner. And the appearance of differences, or similarities, between faces can often depend on photographic conditions outside of the examiner’s control, such as perspective, lighting, image quality, and camera angle. Given these contingencies, most analysts do not ultimately provide a judgment as to the identity of the face in question, only as to whether the features that appear to be present are actually there. “You step back and let the argument be made by the prosecutor,” explained Grant Fredericks, a video analyst who teaches widely and has worked with the Texas Forensic Science Commission. “It is dangerous for a video examiner to tell the court that the person on video is the defendant. If it were that easy, there would be little need for trials in a surveillance society and that’s a frightening thought.” “As an analytical scientist, whenever someone gives me absolute certainty, my red flag goes up,” said Jason Latham, who worked as a biochemist prior to becoming a forensic scientist and certified video examiner. “When I came from analytical sciences to forensic sciences, I was like some of these guys are not scientists. They are voodoo witchcraft.” Forensic reports generally provide few details about the methods they use to arrive at points of similarity. But in Talley’s case, the FBI examiner’s report displayed a high degree of certainty. George Reis, a facial examiner who has testified more than 50 times for state, federal, and military courts throughout the country on forensic visual comparisons, pointed out that the report on Talley’s case was vague. “It is generally considered best practice to be specific in reports and to point out features of similarity, as well as differences, in any comparison illustration or chart,” Reis noted. “In the Talley case no such markings exist. The video frames that were used in the FBI illustration were of poor quality and limited value.”......... Such a drastic re-evaluation of the accuracy of current systems would spell bad news for the FBI’s digital systems, which, according to the most recent numbers available, provide a correct match 85 percent of the time. The agency’s face recognition software has access to 411 million images as part of its Next Generation Identification system, a decade long effort to build the world’s largest database of human identifiers. State, local, and federal law enforcement agencies can search the faces in the database for a range of cases — from DMV fraud to missing persons to immigrations claims. And as researchers have repeatedly shown, the potential for false matches only increases with the size of the dataset: The more faces there are to search, the more prone they are to appear similar and be mistaken for one another. Human examiners are currently not required to adopt a higher threshold of similarity to justify their decisions when looking at candidate lists — even when the likelihood of such similarities increases. The automated ranking of candidates may also subtly bias examiners, as a study with fingerprint examiners working with automated systems has shown. Itiel Dror has recommended agencies consider randomizing the candidate lists so that examiners will not be biased when comparing faces ranked at the top of the list.
Despite these concerns about accuracy, official standards for
human-computer collaborations have yet to be developed. Nicole Spaun, a
former FBI image examiner who has published several leading studies on
forensic facial identification, has been surprised by how many police
departments have installed face recognition systems without also
instituting the proper training to go along with them. To this end,
Spaun is currently working at MorphoTrak, a biometrics vendor, to
develop training so that the users of any computer system are a priority
rather than an “afterthought.” “There’s a lot of people being thrown at face recognition systems who
may know how to use a camera but don’t know anything about the science
of imaging,” she explained. While the temptation to make a definitive
identification is strong, Spaun says that most of her courses involve
explaining to people that certainty is rare. “Looking at my own driver’s
license photo I barely see the moles that I know are there,” she said.
“What I find myself saying to a lot of people is, ‘No you’re not going
to be able to positively identify a person.’” According to an FBI spokesperson, FAVIAU does not plan to replace
humans with automated facial recognition searching, but it may use
systems to locate “better potential candidate matches than the current
subject of the examination, which could provide support for eliminating
the current subject from consideration.” But even with automated facial recognition technology in place, false
matches will, according to Jain, continue to be “a valid concern.”
“Biometrics systems can make errors so we should be open to someone
complaining that they are put in the wrong place at the wrong time,” he
said. “The ‘fingerprints don’t lie’ attitude has to change. If someone
is claiming that they have a perfect system, that attitude needs to be
corrected.” Jennifer Lynch, an attorney at the Electronic Frontier Foundation who
works on face recognition, is concerned that human biases may even
exacerbate the errors of technology. The seemingly unassailable
combination of human expertise and technology may create legal
situations where the burden of proof shifts onto the defendant, she
explained. False matches end up forcing citizens to prove that they
aren’t who examiners (and, increasingly, their algorithmic partners) say
they are. In other words, what happened to Steve Talley could happen to
others again and again."
The entire article can be found at:
The entire article can be found at:
PUBLISHER'S NOTE: I am monitoring this case/issue. Keep your eye on the Charles Smith Blog for reports on developments. The Toronto Star, my previous employer for more than twenty incredible years, has put considerable effort into exposing the harm caused by Dr. Charles Smith and his protectors - and into pushing for reform of Ontario's forensic pediatric pathology system. The Star has a "topic" section which focuses on recent stories related to Dr. Charles Smith. It can be found at: http://www.thestar.com/topic/
Harold Levy. Publisher; The Charles Smith Blog.
