Roger Clarke's Web-Site

© Xamax Consultancy Pty Ltd,  1995-2024
Photo of Roger Clarke

Roger Clarke's 'SmartGate'

SmartGate: A Face Recognition Trial at Sydney Airport

Roger Clarke

Principal, Xamax Consultancy Pty Ltd, Canberra

Visiting Fellow, Department of Computer Science, Australian National University

Version of 26 August 2003, with additions to 7 February 2004

© Xamax Consultancy Pty Ltd, 2003-04

This document is at http://www.rogerclarke.com/DV/SmartGate.html

See also my Release of 7 February 2004


Introduction

The term 'biometrics' is used to refer to a cluster of mechanisms whereby a person can be identified, or their identity can be authenticated, based on the person's physical characteristics. Most biometrics technologies have failed dismally. But in the security, law enforcement and national security communities, hope springs eternal. There are currently products available for purchase that rely on such parts of people's bodies as fingerprints, hands and irises, and the patterns of their voices.

This article considers a technology usually, but misleadingly, referred to as 'face recognition'. In the aftermath of the terrorist attack on the World Trade Center in New York and the Pentagon in Washington, face recognition products have been piloted in a number of airports, particularly in the U.S.A., but also in such countries as Germany, Iceland and Australia. This article focusses on a particular pilot that is running at Sydney's Kingsford Smith airport.


Biometrics Generally

A simple example of a biometric scheme involves a person presenting their thumb to a device outside a closed door. The device measures the shape of the thumb, and compares it to previous recordings of the thumb-shapes of people approved to pass through that doorway. If a match is found, the door opens; without a match, it stays closed.

More generally, a biometric scheme comprises two phases, which can be depicted as follows:

The processes involved are as follows:

Biometric systems may make decisions about:

The decisions made by a biometric system may be right, or wrong. Wrong decisions are of two kinds:

Fuller descriptions of biometrics are at Clarke (1994) and Clarke (2001), and in papers indexed in my bibliography.

Biometrics are intuitively appealing to the designers of systems who need to identify people, or to authenticate the identities that people present to them. In fact, the practical application of biometric technologies is fraught with enormous difficulties. Of particular concern are the conditions on which the particular technology depends in order to function effectively, the balance between false positives and false negatives, the consequences of the decision that the technology makes, and the risks of masquerade and identity theft that arise from a stored biometric reference measure. A catalogue of problems that arise in developing and applying biometrics technologies is at Clarke (2002).

Many privacy concerns arise in relation to biometrics. It strikes directly through an individual's multiple identities to the singular entity. This is examined in Clarke (2001). It represents a direct and dehumanising appropriation of something about the person, and therefore invades the privacy of the person. It imposes significant inconvenience and intrusiveness on people. It is also commonly associated with serious invasions of personal data privacy. The privacy impacts are examined at Clarke (2002).


Face Recognition in Particular

Biometric schemes need to be based on some aspect of a human being that is unique, and invariant. A very few technologies, such as fingerprints, are based on a fairly specific characteristic that is known, based on a great deal of empirical evidence, to be very close to unique, and relatively invariant. Even in the case of fingerprints, however, a substantial set of protocols has had to be developed, and continually improved, in order to protect the innocent.

Face recognition, on the other hand, is based on no such evidence. Rather, it claims validity based on some mathematical procedures applied to photographs of a person's face. Its proponents also claim that these mathematical procedures have the effect of overcoming the many minor variations in a person's facial appearance, such as make-up, glasses, beards and perhaps even some kinds of cosmetic surgery.

The procedure adopted is generally along the following lines:

The automated mathematical procedures are described by all suppliers in a highly abstruse manner, as evidenced by the explanations provided by a couple of suppliers. It all sounds highly sophisticated, which has the effect of sounding authoritative, confusing the reader, and making critical review difficult.

A number of applications have been piloted in a variety of settings. Following the terrorist assault on the World Trade Center, some twenty or so airports have trialled it, or expressed interest in it, or at least been mentioned in suppliers' media releases. This is unfortunate, because a few minutes' reflection makes clear that, even if they worked, the application of biometrics technologies in airports would not have prevented the terrorists from achieving their aims. See Clarke (2002).

The mathematics is meant to be clever enough to make it unlikely that two people will produce the same number. But the mathematics makes a huge number of assumptions about the real world. There are a host of practical problems that make it very likely that false-positives and false-negatives will both be very high.

The technology, and products using it, have been subjected to insufficient testing by independent laboratories. The performance of actual installations has not been subjected to critical evaluation by uninterested analysts, and results even of the limited tests performed have generally been hidden from view.

Such limited information as is available suggests that the technologies have performed atrociously badly (e.g. FRVT 2000, Smith 2001, ACLU 2002c, McCullagh & Zarate 2002, Thalheim et al. 2002, Smith 2002, Bray 2002).

These sources identify such serious deficiencies as heavy dependence on contextual factors (such as lighting, background, angle of head to camera, and expression), and susceptibility to tricks (perhaps not to still photographs, but to video clips; and to appearance-changing features such as glasses). These, combined with the lack of any underlying unique characteristic, result in very high false positives (i.e. acceptances of the wrong people) and fairly high false negatives (i.e. rejections of the right people). The implications were summed up in a Yahoo article as follows: "even under controlled conditions, the systems succeed in identifying individuals from a given database only a little more than half the time -- a rate which is diminished significantly in a typical airport environment" (Barkin N. (2001) 'Airport face-recognition systems no safety panacea' Reuters, October 18, 2001, originally at http://biz.yahoo.com/rf/011018/n18228094_1.html).

The government agencies, airport management companies, and corporations that have installed such systems have placed a great deal of trust in their supplier's assurances.


The SmartGate Pilot

SmartGate is a particular application of a particular face recognition product. It is the name being used for a trial funded by the Australian government through the Australian Customs Service (which is the government agency whose employees confront each person on their arrival in the country).

The pilot is being conducted at Sydney's Kingsford Smith airport at Mascot, and started in late 2002. Until late February 2003, the publicly available information appears to have been limited to two Media Releases in late 2002 and early 2003, and 1-1/2 pages of a conference presentation, and press reports which appear to be based in part on verbal briefings by the relevant agency and/or Ministerial advisors.

A flurry of criticism was directed at the scheme in mid-February. A request to Customs resulted in the issue of a 1-1/2-page Information Sheet, apparently dating to between 13 and 26 February 2003. An additional 2-page list of "Lines which can be attributed to the Minister" was faxed to one or more journalists on 21 February. Each document adds a small amount to the accumulated information available, but it is far from a comprehensive description.

Initial funding was suggested as being the substantial sum of $Aust 3 million. The adaptation of the technology has been contracted to a small Australian company, Nwork Pty Ltd, trading as Biometix, whose Principal is a biometrics consultant called Ted Dunstone. The equipment being used is the German-made Cognitec FaceVACS-SDK. The U.S. pilot schemes use other manufacturers' products, as do those in Berlin and at Schiphol (Amsterdam/Rotterdam).

The trial between November 2002 and February 2003 appears to be using a single device. A document issued by the Minister's office on 21 February said that "more than 13,000 live transactions" had been undertaken, whereas a press report of a few days later referred to 16,000. The Minister has said that it is being rolled out to other sites for further evaluation.

No clear description of the functioning of the scheme appears to be publicly available. In relation to the enrolment phase, there is some confusion in the available documents as to whether the reference measure(s) used is/are standard passport photographs or specially-captured photograph(s). It seems most likely that the process uses both the photograph in the passport and five additional photographs captured at the time.

There are several possible designs for the use phase. These include:

  1. capture a new image of the person, generate a number from it, and search for similar numbers in the existing database (a one-against-many identification approach, whereas the other alternatives listed below are one-to-one authentication);
  2. capture the passport number, capture a new image of the person, generate a number from it, and compare that number against that (or those) recorded in the database for that passport number. The passport number could be captured by keyboard, by optical scanning, or from a bar-code in the passport;
  3. optically scan the photograph in the passport, capture a new image of the person, generate numbers from both the photograph and the new image, and compare those numbers. This could also involve a comparison against that (or those) recorded in the database against matching images, or for the passport number if that is captured as well;
  4. acquire the pre-generated and pre-recorded number from the passport, capture a new image, generate a number from it, and compare that number against that (or those) recorded in the passport. This could also involve a comparison against that (or those) recorded in the database against matching images, or for the passport number if that is captured as well. This is not feasible with existing passports, and would involve the issue of a new form of passport, incorporating a chip and security features, and probably therefore in the form of a card, or a card inserted into a passport booklet.

One of more journalists were provided with "lines which can be attributed to the Minister" on 21 February. These stated that it "undertakes 'one-to-one' identity verification [sic] ... and it is not used for 'many-to-many' matching [sic - the author should have used 'one-to-many']". It therefore appears likely that it is an authentication design, most likely option 2.

The purpose of the pilot is claimed by the Minister to be "improved processing efficiency ... while maintaining Australia's high level of border security" (26 Aug 02). A more credible explanation is that the primary driver is the Australian Government's stance as an extremely loyal instrument of U.S. Government policy. The Australian Passports Office has been widely reported in the media as being required to upgrade Australian passports to include a biometric. This is because of the U.S. Administration's threat to create difficulties for people without them who seek to enter the U.S. (e.g. one story by Lebihan in ZDNet Australia, 22 March 2002 and another).

The pilot is being conducted on a small population of a few thousand QANTAS aircrew at Sydney airport. The conditions under which the additional reference measures are collected are able to be controlled. The population is compliant, partly because they are employees, and partly because they are motivated to achieve speedy matches in order to avoid delays. Queues and psychological pressure are non-existent, because aircrew have their own checkpoints separate from passengers. It is also entirely feasible for airlines to advise Customs in advance of the identities of aircrew that will pass through the checkpoint in any given couple of hours, which provides an additional and valuable security precaution.

Under such circumstances, statistics are on your side, because of the small population, the tightly-controlled conditions, and the high level of willing participation. This is fortunate, because statistics is the only basis on which the technology works. There is no stable and highly differentiated identifying biometric characteristic.

There are strong theoretical, practical and empirical reasons for believing that the quality will be very low, even with a controlled pilot like SmartGate. Apart from vague assertions ("The SmartGate system proved to be highly accurate during rigorous testing" - 29 Jan 03), the sponsors have failed to publish any evidence of the scheme's performance.

Unabashed, Customs told the Minister to declare the pilot a success, so that they could look for a strategic partner to further develop it. This issued a Request For Proposal in August 2003.


Misunderstandings About SmartGate

The Ministerial Media Releases provides evidence that neither the Minister nor his staff understand how the technology works. The unequivocal statement is made that "the system demands an exact match" (29 Jan 03). That is quite wrong. It actually involves large quantities of data massaging, and does not depend on equality between the numbers, but rather on a threshhold test. His releases keep using the word 'verification' to describe what it does, implying closeness to truth. Any such technology is subject to error (especially this one), and the appropriate word to use is 'authentication'.

The Customs executive responsible for the pilot is also under some misapprehensions. He refers to "unique physical characteristics" and "unique features of the face" (31 Oct 02), despite the fact that there is no such thing. The technology depends on very substantial mathematical processing of an image, in order to detect and exaggerate whatever optical images arrive in the camera's lens.

The same executive says that "if the process fails, the [person] can try again or proceed to a Customs Officer for manual processing" (31 Oct 02). This makes clear that the system is meant to be used without direct supervision of the behaviour of the person. And hence the person has the opportunity to use tricks to fool the system.

The Minister was also reported on 27 February 2003 as saying that there had been no 'false accepts' in about 16,000 transactions until 25 February. This is not a tenable statement, because it is of the nature of a false acceptance that you don't know that it's happened unless someone tells you. QANTAS aircrew are motivated to get to their aircraft, or get home, not to tell Customs that they've just cheated the system. All that the Minister can possibly say is that they are aware of no false acceptances.

The Minister's statement on 27 February was in response to the fact that two members of an international delegation had swapped passports, and walked through the system without their masquerades being detected. They were Japanese members of an IATA Simplifying Passenger Travel Committee, en route to a meeting in Sydney. What the Minister should say is that the only two people known to have attempted to beat the system have done so, and did so merely by appearing similar (at least to a non-Japanese) and swapping passports.

The Minister says that "photo matching will stop people travelling under a false identity". It is unclear what the Minister means by a 'false identity'. But all that this scheme could possibly do is check that the new photo is close enough to at least one of those that have been pre-recorded for that identity. But if the terrorist has his own photo on the file, it won't catch him out. 'Organised crime' is 'organised', and has no difficulty getting the government to issue identity documents with a photo of the appropriate person on the documents and on the database. 'Organised terrorist cells' use people who aren't on 'wanted lists', and who carry valid documents. Focussing on 'false identities' is of very little use in protecting people against terrorists, drug syndicates, or any other serious criminal.

The Minister thinks that the technology can be applied to people generally: "Customs will examine the possibility of expanding the SmartGate system for the general public. ... When introduced more generally ..." (29 Jan 03, my emphasis)

This is a very different proposition from the pilot, and vastly more challenging. The reasons include that the population is very large, is not capable of precise definition, is not subject to tight controls, and includes a proportion of people who are not disposed to cooperate. People will attempt to avoid and to undermine the scheme for various reasons. Some people will believe that the requirement is intrusive and unjustified. Some will reject the imposition of political power by the U.S. Government through a compliant Australian Government. Some will be miscreants or petty criminals who fear that they will be caught out. Some will have serious criminal intent, will be intent on not being caught, and will have the capability and the patience to circumvent the system.


Conclusion

The SmartGate pilot is mis-conceived, it has been misrepresented, and it uses technology that has not been and cannot be effective. Any large-scale implementation of such a system, based on such flawed technology, would be an utter fiasco.

It is understandable that the public would go through a brief period of great nervousness following a terrorist strike as spectacular as that of 11 September 2001. That does not justify executives stampeding to trial technologies that are highly privacy-invasive, designs that could never achieve the objectives set for them, and products that are both theoretically and demonstrably utterly ineffectual. It augurs ill for western democracies that a single strike could so dramatically undermine both managerial competencies and democratic freedoms.


Pointers to Resources


Some Suppliers' Explanations of Their Face Recognition Technology

Extract from Cognitec FaceVACS, which is the technology used in SmartGate, last accessed 22 February 2003:

"Feature extraction starts with local image transforms that are applied at fixed image locations. These transforms capture local information relevant for distinguishing people. [This is misleading in that the data are merely "local information", none of which possess any inherent value for distinguishing people.]

"The results are collected in a vector. A global transform is then applied to this vector. Using a large face-image database, the parameters of this transform were chosen to maximize the ratio of the inter-person variance to the intra-person variance in the space of the transformed vectors; i.e., the distances between vectors corresponding to images of different persons should be large compared to distances between vectors corresponding to images of the same person. The result of this transform is another vector that represents the feature set of the processed face image". [Put another way, the data is massaged in order to maximise the statistical differences between the measures. The outcome is therefore a self-fulfilling prophecy: the differences are maximised; but without any reference to the real world of people].

"For the creation of the reference set, several images are usually taken of each person during enrollment in order to better cover the range of possible appearances of that person's face. The reference set generated for a person consists of up to five feature sets, which are the centers of clusters obtained through a clustering process on the feature sets created from those images".

Extract from Imagis ID2000, last accessed 22 February 2003:

"At the very heart of Imagis' technology is a unique method of capturing facial data that is intrinsically more accurate. Whereas other solutions are limited through their reliance on outmoded facial recognition methods (namely local feature analysis and eigenface techniques), Imagis uses a combination of spectral analysis and 3-D modeling to locate and fit a face, identifying over 692 facial descriptors in the process. "Once a face has been identified, it is converted into a deformable surface model that takes into account the values of pitch, yaw, and roll of the head, and re-rendered using various light sources. This surface modeling allows the face detection to work accurately with an infinite number of face shapes (not just the "universal facial shapes" used in eigenface-based approaches). Unlike other solutions, ID-2000 works equally well with all races and genders and is not fooled by a change in hairstyles, or the growth/shaving of a beard. "Once a face has been captured and rendered, the software uses a proprietary algorithm to produce a wavelet that is unique to that image. It is this wavelet (compressed and encoded) that is used to make comparisons quickly in both one-to-one and one-to-many searches" (emphases in original).

From ZN Technologies, last accessed 22 February 2003:

"The enormous technical challenges of facial recognition are tackled with the help of neuronal structures: by means of the patented "Elastic Graph Matching" procedure, a highly complex mathematical calculation of the face pattern is carried out, which unlike the simple methodology of geometric face measuring is robust against variations and extremely flexible with regard to its application in various areas.

"For calculating the facial information, an elastic grid, the so-called graph, is put over the face. At every node within this graph, a feature vector is calculated from the specific local information. The facial similarities between the live feature and the graph model are established on the basis of the weighted sum of node similarities. All in all, the information of a total of 1,700 facial features forms the basis for comparison. Thanks to the great robustness and the flexible graph adjustment of the Graph Matching method, highest recognition performance is guaranteed, even in case of alterations to the outward appearance or the ageing process".



xamaxsmall.gif missing
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.

From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 65 million in early 2021.

Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916

Created: 21 February 2003 - Last Amended: 26 August 2003, with additions to 7 February 2004 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/DV/SmartGate.html
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2022   -    Privacy Policy