Transport for London’s (TfL) reliance on information from Uber’s facial verification-based driver ID system is under increased scrutiny following multiple instances of misidentification,. The drivers and their union say that, despite the technology’s questionable veracity, TfL has been overly reliant on it when making license revocation decisions, severely impacting drivers’ livelihoods if they cannot legally vehicles.
TfL’s unchallenging acceptance of the evidence provided by Uber’s Microsoft-based verification system has raised numerous legal challenges by drivers who say they have had their licenses unfairly revoked.
Earlier this(5 July 2021), for example, a Magistrate’s Court ordered TfL to reinstate the license of an Uber driver, which it found was revoked without notice or appeal following a failed facial verification check.
Thestraight to revoke the driver’s private hire license on identity fraud grounds and relied solely on evidence from Uber’s facial verification software without conducting its investigation, as a licensing authority, into the incident.
“The Court noted that at no stage had there been any risk to the public and that TfL had proceeded straight to revocation, despite other sanctions being available to them,” wrote the driver’s solicitor, Abbas Nawrozzadeh of, in his notes of the judgment.
“They had relied on the actions of a body that fell outside of their licensing power and revoked the license without any investigation into actual events on that day. The Court described the decision as perverse and unreasonable, stating that as a licensing body, TfL should be ashamed of bringing this case to Court as it represented a waste of public funds.”
Speaking to Computer Weekly, Nawrozzadeh added that he had been “dealing with a raft of these cases” since Uber introduced its.
In the case of 5 July, the misidentification occurred while the driver, a member of the App Drivers & Couriers Union (ADCU), workedfood delivery arm, Uber Eats. The error, however, resulted in the company deactivating his Uber Eats and driver’s accounts.
Under London’s transport regulations, Uber must inform TfL when a driver is dismissed via a, which Computer Weekly understands should trigger a “fitness assessment” by the regulator to decide whether the driver should continue to hold a private hire license.
This process would also typically entail TfL notifying the driver in writing that it had received an adverse report about them so they can respond and appeal if TfL decides to revoke their license.
According to the general secretary of the ADCU, James Farrar, drivers can usually continue working. At the same time, a decision is appealed, except in more severe cases with a perceivedor fraud. It should be noted that Uber was allowed to continue operating while it appealed its .
“Then they can immediately revoke,” he told. “You can continue to , but you can’t work.” Following the driver’s misidentification by Uber’s Facial-verification system and the subsequent deactivation of his accounts, TfL was notified and immediately revoked his license, Farrar added. This is a of action it usually takes in response to serious allegations, such as sexual or physical violence.
He further claimed the TfL decision was based on little data and that the regulator only asked Uber to substantiate the allegation when the driver challenged TfL’s decision. “Even if he had engaged in any kind of identity fraud, which he didn’t [as the court decision found], he would only have done it as a delivery driver, but TfL revoked the [private hire] license anyway,” he said. “There was no question he did anything wrong while acting as a licensed driver.” Computer Weekly asked Uber if it would like to comment on why the Eats and driver’s accounts were deactivated, but it received no response.
As a result of the Court’s findings, the driver was also awarded his legal costs against TfL, which Farrar described as “extraordinary”. “Usually they’re immune to cost claims because, if you’re a licensing authority in the interest of public safety, only under exceptional circumstances will a cost be awarded against you for an adverse licensing decision, even if it’s overturned,” he said.
a general response to Computer Weekly’s questions, a TfL spokesperson said: “The safety of the traveling public is our top priority, and where we are notified of cases of driver identity fraud, we take immediate action to revoke a driver’s license so that passenger safety is not compromised.” Uber was also asked if it would like to respond to the ruling, but Computer Weekly received no response.
Not an isolated incident
The plight of the ADCU driver, the Magistrates Court, heard about this month is far from an isolated incident concerning Uber drivers losing their licenses due toerrors.
In March 2021, the ADCU and its associated data trust, Workers Info Exchange (WIE), said it had identified a further seven cases of Uber drivers losing their jobs and having their licenses revoked by TfL because the company’s Real-Time ID Check system failed to recognize their faces.
Incidentally, when TfL was asked to comment on the July 2021 ruling, it made an abortive request for the driver’s name to substantiate it was commenting on the correct case, which indicates theit is involved with.
The ADCU did not wish to disclose the driver’s name to Computer Weekly out of a concern they couldrepercussions from TfL. As to why Uber’s facial-verification tool may be considered error-prone, research into Microsoft’s systems, as well as actions of Microsoft itself, suggest there is bias against particular groups, particularly people of color and women.
In 2018, research from MIT indicated that Microsoft’s facial recognition and detection systems – specifically the Face API being used by Uber – had gender and racial biases, finding it had.
“The substantial disparities in the accuracy of classifying darker females, lighter females, darker males, and lighter males in gender classification systems require urgent attention if commercialgenuinely fair, transparent, and accountable facial analysis algorithms,” said authors Joy Buolamwini and Timnit Gebru.
In June 2020, Microsoft – alongside Amazon and IBM –in response to several weeks of mass protests against the police murder of George Floyd on 25 May.
Microsoft President Brad Smiththat one of the challenges with the technology in its current form was that “it doesn’t work as well for women as it does for men, it doesn’t work as well for people of color”, adding that it was more likely to find errors, mismatch and generally “fail to identify” people from these groups.
A Microsoft spokesperson commented on the union’s allegations regarding Uber’s Real-Time ID Check system when they were first made in March, said the company was “committed to testing and improving Face API,to fairness and its accuracy across demographic groups”.
“We also provide our customers with detailed guidance for getting the best results and tools that help them to assess fairness in their system.” An Uber spokesperson added that the system was designed to protect the safety anddriver or courier is using the account.
“While no tech or process is perfect and there is always room for improvement, we believe the technology, combined with the thorough process in place to ensure a minimum of two manuala driver, is fair and important for the safety of our platform,” they said.
In an April 2021directly to the ADCU – which has also been shared with Computer Weekly – Microsoft told the union, “Errors in the human review process associated with Uber’s implementation of our facial recognition technology do not provide a basis for Microsoft to terminate its license to use the technology, especially when Uber has acknowledged the failure and is committed to improvement”.
As part of anfrom early March 2021, 14 Uber Eats couriers shared evidence with journalist Andrew Kersley that showed how the technology failed to recognize their faces, leading to threats of termination and account closure.
In terms of the impact on drivers, Farrar said, “They’ve either had to fall back on Universal Credit or try to find some alternative means of work and haven’t been able to get it – it’s had a devastating effect, and a lot of these guys are main breadwinners in their families as well”.
Computer Weekly asked Uber if it would like to comment on the claim that its facial-verification software has led to the wrongful deactivation of multiple driver’s accounts but received no response.
Other automated processes fire driver.
The ADCU is also appealing a series of other license revocation decisions by TfL at the Magistrate’s Court, which is similarly based on false information that led to accusations of fraud from Uber.
In April 2021, for example, Uber was(where Uber’s European headquarters is located) to reinstate six drivers because the decision to deactivate their accounts and terminate their employment was “based solely on automated processing, including profiling”, which goes against (GDPR).
London-based driver Abdifatah Abdalla, for example, claimed he was accused, without providing evidence, of sharing his account details with a third party when the app detected two sign-in attempts from different locations resulting in his deactivation.
TfL revoked his private hire license a, leaving him unable to drive for alternative ride-hailing apps like Kapten and Bolt. In the same month, the City of London Magistrates’ Court separately ordered TfL to reinstate Abdalla’s private hire license, concluding that “no investigation has taken place, ” further criticizing the regulator’s “willingness to accept” the evidence provided by Uber.
The drivers involved in the Amsterdam case have to separately appeal TfL’s licensing decisions in the UK Magistrate’s Court as the immediate revocation of a license means it cannot be appealed at the TfL level Farrar.
“As soon as TfL got these dismissal notices [from Uber], it took a very harsh view that these were safety threats and immediately revoked those licenses,” said Farrar. “Normally, immediate revocation would be for something like physical or sexual violence, but in these cases, TfL said they were immediate and those drivers and no notice – just ‘boom’, Kafkaesque, you’re deactivated from Uber, and you’ve lost your license.
“We [now] have to separately turn around and fight for the restoration of the licenses … appealing to the Magistrate’s to review the TfL decisions, but to do that; the bar is pretty high because you’ve got to prove the decision is wrong … it’s not just about presenting an alternative view, you have to reach a standard.”
Computer Weekly asked TfL if it would like to comment that multiple magistrates’ rulings have found it relied solely on evidence from Uber without conducting its investigation as a licensing authority into the actual events but received no response.
Rushed facial-verification implementation
The ADCU has previously claimed and maintains that Uber rushed to implement its identification system to win back its London operating license after TfL decided in late 2019 that it would not be renewed over.
Evidence given to TfL by Uber during its licensing appeal in September 2020 showed that it had started to roll out the system from April 2020 onwards because of its manual identification process failures.
“TfL has taken an active interest in [Uber’s] proposals about this. There are clear benefits to the product, and TfL supports any technology which increases passenger safety by ensuring the driver is licensed by TfL and permitted to use the Uber app,” wrote TfL’s director of licensing, regulation, and charging, Helen Chapman, in her .
“I consider the use of this product a step in the right direction, although clearly, its implementation is still at a very early stage,” said Chapman. “I, therefore, cannot meaningfully comment on the effectiveness of it at this stage.”
She added that TfL had received aImpact Assessment (DPIA) for the system from Uber in March 2020. DPIAs are of note because they are designed to increase awareness and allow them to comply with relevant laws and identify and fix any problems before harm is caused.
“A DPIA is not simply a rubber stamp or a technicality as part of a sign-off process. It’s vital to integrate the outcomes of your DPIA back into your project plan,” says the Information Commissioner’s Office (ICO) in its. “You should not view a DPIA as a one-off exercise to file away. A DPIA is a ‘living’ process to help you manage and review the risks of the processing and the measures you’ve put in place on an ongoing basis.”
Letter to the mayor
In March 2021, the ADCU, WEI, and digital rights group Big Brother Watch co-signed a letter toabout the case of another driver misidentified by Real-Time ID Check.
In it, theyTfL had “placed significant pressure on Uber, under threat of losing their license, to rapidly introduce facial recognition identity technology in London”, adding that it has also refused to share the DPIA with the DCU.
“It is our view that the mandatory use of such technology cannot be justified and is entirely disproportionate to the risk. In the evidence TfL presented at Uber’s licensing appeal, only 21 drivers were found toout of 90,000 analyzed over several years. This was only possible at the , which has since been rectified,” said the letter, seen by Computer Weekly.
“The use now of such intrusive, inaccurate, andsince there are other more proportionate means to achieving the same end. TfL’s promotion of Orwellian levels of surveillance of 100,000 Londoners working in the gig economy is not a reasonable or proportionate response for the failure of the gig employer to audit and test the
It added: “Checks, balances, and due diligence has failed at every level. TfL has bypassed all due process in its political rush to be seen as an aggressive enforcement authority.”
Given multiple reports of misidentification leading, in turn, to Uber’s deactivations and TfL license revocations, Computer Weekly asked the regulator if it still supported the roll-out of theon this point.
Computer Weekly also asked TfL whether it would like to comment on the ADCU’s claim that it had placed significant pressure on Uber to introduce facial-recognition technology but received no response.