Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2024
Infrastructure
& Privacy
Matilda
Roger Clarke's Web-Site© Xamax Consultancy Pty Ltd, 1995-2024 |
|
|||||
| HOME | eBusiness |
Information Infrastructure |
Dataveillance & Privacy |
Identity Matters | Other Topics | |
| What's New |
Waltzing Matilda | Advanced Site-Search | ||||
Working Paper Version of 5 August 2023
Roger Clarke, Katina Michael and Roba Abbas **
© Xamax Consultancy Pty Ltd, 2023
Available under an AEShareNet 
licence or a Creative
Commons 
licence.
This document is at http://rogerclarke.com/DV/RDCS.html
Large-scale systems to administer social welfare payments face considerable challenges. This case study reports on an Australian Government project widely referred to as 'Robodebt'. The scheme's purpose was to save labour costs and increase recovery of overpayments. Welfare recipients were required to prosecute their innocence by producing documentation for income earned years earlier, failing which the agency took action to recover what it asserted to be overpayments. The scheme extended the agency's reliance on the digital persona the agency maintains for each client, and featured automated inferencing, automated decision-making and automated action.
Intended digital transformation turned into large-scale failure, due to poor-quality data-matching, faulty inferencing and inadequate legal foundations. The scheme was persisted with for three years after ample evidence had accumulated that the scheme was misconceived. Enormous harm was caused to hundreds of thousands of anxious clients, many thousands of them vulnerable. When the smoke cleared, affected clients received refunds and damages. The outcomes included net costs of about AUD 3 billion, demoralised staff, public opprobrium and substantial reputational damage to the agency and the public service.
One key lesson for large-scale transformational IT projects is the vital role of engagement, by the initiative's sponsors, with the organisation's data, IS and IT professional staff and their managers and executives, and with informed advocates for the stakeholder groups affected by the initiative. Effective whistleblower protections are needed, to ensure that wilful breaches of law, logic and commonsense by Ministers and senior executives are exposed before they do serious harm.
A further important message is that opaque inferencing processes must not be used as a basis for action by organisations without prior, careful testing of the fit to reality of the imaginary world created by 'big data' operated on by obscure mathematical and statistical processes. The case raises considerable doubt about any form of automated decision-making, far less automated action, in circumstances in which some people will be subjected to material harm, without explanation, and without recourse. Underpinning these factors is the need for the socio-technical view to permeate the conception, design and implementation of major interventions affecting people.
This paper contributes to understanding of the challenges involved in digitalisation in the public sector. It does this by reporting the results of a case study of a large-scale exercise in transformative IT applied to the administration of social welfare in Australia. The analysis challenges common preconceptions about the phenomena that fall within the field of view Information Systems (IS) practice and research. The authors contend that the scale and scope of 21st century IS, the increasing incidence of automation of decision-making and action, and the lack of transparency of some of the forms of information technology (IT) being applied, give rise to far more substantial impacts and implications than was the case during the second half of the 20th century, and that the breadth of vision of IS practitioners and researchers must change accordingly. This Introduction highlights some of the key elements of those changes.
Applications of information technology (IT) in business and government began in the 1950s with the automation of existing systems (retronymed 'manual' systems). The focus quickly shifted to the rationalisation of existing processes, in order to take advantage of new possibilities that technologies had created. Over time, the emphasis shifted to business process reengineering (Hammer 1990), and then to organisational and sectoral transformation (Venkatraman 1994). Most recently, the capacity of IT to achieve organisational disruption (Bower & Christensen 1995, Lyytinen & Rose 2003, Christensen et al. 2015) has been lauded.
In parallel, the marriage of computing and telecommunications enabled the scope of IT-supported systems to broaden from organisational sub-units, via cross-organisational functions, to entire organisations. It then leapt into inter-organisational systems (1-to-1), and to multi-organisational systems in star, chain and network configurations. An often-overlooked development was the emergence of extra-organisational systems (Clarke 1992b), which extended beyond a particular organisation's boundaries to individuals. At first, individuals had to be provided with access to network-connected devices. During the last decades of the 20th century, however, increasing numbers of people acquired desktop and laptop devices that could be harnessed into organisation-managed systems, and the first decades of the 21st century have seen the proliferation of handheld mobile phones and tablets, and their exploitation as an extension of organisations' own information infrastructure. During these stages of maturation, the technical challenges in system development came to be more than matched by issues of organisational and individual human behaviour, such that system sponsors needed to adopt a socio-technical view (Clarke & Davidson 2020, p. 489, Abbas & Michael 2022).
A concomitant development throughout the growth of such systems has been the digitisation of data, initially through purposeful capture of data specifically relevant to a particular purpose, and progressively as a by-product of other actions. Data capture has also been increasingly outsourced, so that organisations benefit from data-input by unpaid participants in systems, referred to in Abbas et al. (2022, p. 78) as ''customer work". By about the turn of the current century, the point had been reached where a large proportion of the data desired by organisations was 'born digital' rather than having to be converted from other paper-based formats.
The availability of vast quantities of data has enabled organisations to achieve efficiencies by no longer dealing directly with other entities, and particularly not with those most time-wasting of entities, people. Organisations find it far more economical to transact with, and impose surveillance on, a digital persona in preference to a person (Clarke 1994, 2014). The term 'datafication' was coined by Cukier & Mayer-Schönberger (2013) as part of those authors' inculcation of breathless excitement about 'big data' as the harbinger of a post-rationalist world. See also Lycett (2014) and Newell & Marabelli (2015). The steadier concept associated with the term 'digitalisation' describes the shift from the interpretation and management of the world by means of human perception and cognition of phenomena, to processes that are almost entirely dependent on digital data (Brennen & Kreis 2016). Where the entities being interpreted and managed are people, the term carries with it overtones of 'objectification', as the human aspects of interactions are abandoned.
For-profit business enterprises are largely unfettered by public policy considerations in their application of digitalisation. This reflects their role as drivers of growth, and the assumption of markets being sufficiently competitive that consumers have alternatives to choose among. Public sector organisations, on the other hand, operate in an environment in which there is an expectation that greater care will be taken about the direct impacts of transformative and disruptive technology-based interventions, and about their indirect implications or 'collateral damage'. Those impacts and implications are commonly felt by 'users', that is to say the people, inside and outside the organisation, who are direct participants in the relevant system. Too often overlooked are those who are indirectly but materially affected, usefully referred to as 'usees' (Berleur & Drumm 1991 p. 388, Clarke 1992b, Fischer-Huebner & Lindskog 2001, Wahlstrom & Quirchmayr 2008, Baumer 2015). Examples of usees include people who have entries in criminal intelligence and credit reporting databases, and those who are dependants of welfare recipients.
The contemporary context of IS therefore includes reach well beyond organisational boundaries. It involves substantial impacts on users and implications for usees, overlaid with reliance on digital personae and data analytics. This is combined with increasing degrees of automation of inferencing, decision-making and even action. We contend that the nature of systems is now such that broad interpretations are essential of the scope of IS, of IS practice, and of practice-relevant IS research. Understanding of large-scale systems is impossible if relevant personal, group, organisational, inter-organisational, extra-organisational, regulatory and policy factors are excluded from the field of view.
This paper presents a deep case study of substantial changes made to a large-scale government IS, with the purpose of throwing light on those propositions. The paper is therefore complementary to others that report on the same project. The paper commences with an overview of the context in which the intervention was imposed. Section 3 outlines the research method, and Section 4 provides a narrative description of the project. This lays the foundation for a thematic analysis of the salient aspects of the project's policy, design and conduct, presented in Section 5. We discuss broader issues in public sector digitalisation in Section 6, but with the degree of care essential with any single-case study. Our focus is on public sector projects in the area of transformative digitalisation involving relatively simple inferencing, but with a view also to extracting lessons for the application of more ambitious data-analytic techniques such as machine learning and generative AI.
Digitalisation depends on data not only being in digital form, but also being reliably associated with particular entities or identities. A considerable amount of public sector activity is accordingly concerned with the administration of relationships between government agencies and people, with governments striving to establish and maintain workable identification schemes (Michael & Michael 2006). Some countries have general-purpose identifiers, others use identifiers across a cluster of related functions, while others depend primarily on agency-level or program-level identifiers. In Australia, successive attempts to establish a national identification scheme have failed (Clarke 1987, Michael 2003). However, a multi-purpose identifier called a Tax File Number (TFN) was established in 1988, under the administration of the Australian Taxation Office (ATO, Clarke 1992a). It is in ongoing use in public sector financial programs, and in a range of private sector contexts that has continued to expand throughout its first 35 years.
Organisations began by gathering data for specific purposes, but have long since resorted to the consolidation and re-purposing of data gathered by many organisations for many different purposes, often with little account taken of incompatibilities among datasets. One use of the TFN is for data-matching schemes, which involve the comparison of machine-readable records from separate sources, which contain personal data that is purported to relate to the same person. Data-matching is a longstanding dataveillance technique much-used by government agencies in various countries since the mid-1980s, particularly to help in addressing financial waste and fraud (Clarke 1994b). In Australia, many such schemes are largely unregulated, and subject only to a non-binding guideline published by the data protection oversight agency. One series of very large-scale programs caused considerable public concern, however, and as a result was subjected to a modestly formalised regulatory environment, the Data-Matching Program (Assistance and Tax) Act 1990 (DMP).
Since about 1990, there has been a strong trend in Australia towards monolithic government, concentrating programs and agencies within super- or mega- 'portfolio agencies', but also with some degree of attention to 'purchaser-provider' relationships. Both are evident in the social welfare area, where policy has been separated from the operational aspects (Podger 2023). Social welfare transfer payments represent about 20 percent of the Australian Government's Budget. The major schemes assist 2.6 million people of working age (13 percent of those age-groups and 10 percent of the population) and 2.5 million people of Age Pension age (a further 10 percent of the population). "For the poorest 20% of Australian households, social security payments provide more than 70% of their income", and most recipients' earned incomes are highly variable across each year (Whiteford 2022).
Policy in relation to the more than 100 support programs continues to rest with specialist agencies, in a variety of portfolios. The primary policy agency, the Department of Social Services (DSS), is responsible for unemployment benefits (referred to using various terms, including Newstart and Jobseeker), the age pension, disability support and crisis payments in relation to fires, floods, the COVID pandemic, etc. Military service pensions policy rests with the Department of Veterans Affairs, and childcare subsidy was at the time with the Department of Employment and Workplace Relations, whereas student support (Austudy and Abstudy) rested with the Departments of Education and of Social Service, and farm household allowance policy with the Department of Agriculture, Water and the Environment (SA 2022).
Although policy responsibility vests in specialist agencies, the administration of social security is funnelled through a specialist agency called Centrelink, formed in 1997. It is a services-, data- and identifier-hub, handling all payments to welfare recipients, all interactions with them, and associated surveillance activities. In 2011, Centrelink and the national health insurance service, Medicare, were combined into a new super-agency, Department of Human Services (DHS), branded since 2019 as Services Australia. An enormous diversity exists, of programs, needs, clientele, and personal and regional circumstances. DHS delivers critical financial support to Australia's most vulnerable populations, such as people with significant physical disabilities or mental health conditions, and single parents. Some people are in multiple such categories. Those living in remote and rural Australia have limited access to local services.
The scores of complex, longstanding and frequently-modified welfare schemes are designed to protect the public purse perhaps as much as to address economic needs. The operation of Centrelink's distributed and now-outsourced call-centres, and its remaining physical service-centres, are beset with enormous challenges. Staff are expected to master an array of disparate details and interact with large numbers of needy and sometimes disturbed individuals. Diseconomies of scale and scope are in play. The complexities, ambiguities, inconsistencies, conflicts and discretions within written policies, and the diversity of circumstances, combine to give rise to misunderstandings and errors, and inevitably to recriminations, requests for review and appeals.
Broad scope exists for inefficiencies and errors. In addition, fraudsters are naturally attracted by the scheme, and some genuine recipients are tempted to take advantage of loopholes. Considerable effort is accordingly invested in discovering where overpayments have occurred, and in seeking restitution. Official estimates of the scale of overpayments are in the range AUD 1.5-4.0 billion p.a. (i.e. 1.0-2.5 percent of c.AUD 160bn p.a. in payments), but this is contested, with other estimates less than one-tenth of those of the agency. Successive generations of technology have been harnessed in support of Centrelink's work, and social services agencies have operated one of the nation's largest IT facilities since the 1960s.
This case study examines a project referred to internally as Online Compliance Intervention (OCI). The relevant government agency is referred to in the remainder of this document as the Department of Human Services (DHS). The project was intended by DHS as a form of transformative digitalisation. This study spans the period 2014-2023, from its conception, via the pilot in 2015, the launch in 2016 and the operational phase 2016-20, through to a series of official reviews and autopsies 2017-23. Letters were sent to more than 1 million clients and ex-clients, in most cases generated automatically, and in some cases with 4-10 follow-up letters. Debts were raised against 433,000 individuals. After the government was eventually forced to admit that the scheme was illegal, it undertook "reimbursement of [AUD] __746 million to some 381,000 aff__ected individuals ... writing off putative__ debts amounting in total to [AUD] 1.751 billion" (RCR 2023, p.xxix).
The previous sections have evidenced the complexity of the system operated by DHS. To gain insights into the conduct of a major digitalisation project in the area, it is necessary to examine events in their natural setting. The scale of activities and their dispersion across organisations and locations are such that fieldwork using observation-based methods would require enormous resources. In any case, the government agency involved was hostile to proposals that it should disclose information about its activities. For example, the final Report of a Parliamentary Committee on the matter concluded that the Government not only made many errors in relation to the program itself, but also on multiple occasions practised "the deliberate withholding of information" (SCA 2022, p. 17). The likelihood of observers being permitted inside the organisation appeared throughout to be very low.
The relevant research method to adopt was accordingly a case study based on published sources, seeking a multi-perspective view of the design, conduct and impacts of the project over time. We situate the project within a frame of the application of transformative digitalisation in a government agency, and adopt a narrative approach reflecting a blend of academic research and professional and consultancy practice expertise, followed by a thematic analysis and a discussion of the broader implications.
Although quantitative data needs to be used to the extent practicable, research of this nature is dependent largely on qualitative data. The research was not undertaken in the scientistic or positivist traditions (Benbasat et al. 1987, Yin 2017). Nor was a critical realism framework applied, whereby "a causal explanation for a given phenomenon is inferred by explicitly identifying the means by which structural entities and contextual conditions interact to generate a given set of events" (Wynn & Williams 2012). The approach adopted is interpretivist, because many stakeholders exist, with distinct points-of-view and conflicting values and interests, and an analysis can only achieve relevance if it reflects those differences (Walsham 1995).
Reflecting the motivations discussed above, the objective of the study was defined as:
To seek understanding of the factors that resulted in a large-scale public sector digitalisation project failing to achieve its objectives and causing enormous collateral damage, and document the dynamics by which those factors operated.
The nature of digitalisation is such that ICT itself is not the focus. Moreover, the scope is far greater than IS, in the narrow sense of just the underlying infrastructure, software and associated manual procedures. The case study of necessity includes activities beyond the organisational boundaries of the main players, and considers the institutional, economic, legal and social environments within which it operates.
Access to DHS records was not feasible, because its operations are not transparent, and, aided by the dysfunctionality of the country's Freedom of Information mechanism (Knaus & Bassano 2019, CPI 2022, Bajkowski 2023), it strenuously resisted requests for access to information by its clients, by lawyers acting on their behalf, and by a Parliamentary Committee. This study accordingly drew on multiple secondary sources of evidence, listed in Table 1. Preference was given to the authoritative sources, but news media have been cited where no better source could be found.
The first published report was on investigations by the Ombudsman (Omb 2017). It was limited to the purported debts raised, and excluded any consideration of any of the policy rationale, the debt raising and recovery program, procedural fairness, and the use of external debt collection agencies. The Ombudsman's Report was expressed in a manner noted by media to be very sympathetic to the agency and the project, despite the litany of problems that it uncovered. This was further evidenced by the delight apparent in the DHS response (Omb 2017, pp. 45-46), and use of the document by that agency as a major part of its defence against the subsequent Senate Committee Report (GR 2017). The Royal Commission found that "DHS and DSS [senior executives] engaged in behaviour designed to mislead and impede the Ombudsman in the exercise of his functions" (RCR 2023, p.208). A lengthy description of the two agencies' exercise in regulatory capture of the Ombudsman is on pp.208-226. The conclusion was that "... the circumstances ... give rise to a reasonable perception that the Ombudsman's Office conducted [that] investigation in a way which allowed DHS to influence the content of the resulting Investigation Report in order to further DHS's own interests, thus compromising the independence of the investigation" (p.585). The Ombudsman's Report did, however, provide some information that had previously been suppressed, and it did result in corrections to some of the many small-scale deficiencies.
The second authoritative source is a Senate Committee Report (SCA 2017). The Senate Committee had a much broader Terms of Reference than the Ombudsman, and encompassed the scheme's error rates, compliance with legal, regulatory and public policy requirements, and impacts. In Australia, the Government is formed in the lower House of Representatives, whereas the Committee reflected party representation in the Senate, with a mix of Government and opposition members and a chair from the cross-bench Greens Party (whose policies, other than in relation to environmental matters, can be depicted as being in the social democratic tradition). However, that first major Report elicited an entirely dismissive response from the Government (GR 2017). Subsequent sources, of less value for the present purposes, include a series of reports from a second Inquiry by the same Senate Committee, in 2020-22, including the final report (SCA 2022).
The third key source is the judgment of a senior court in a subsequent court-case (Pry 2021). In addition, some publicly-available sources from the agency itself were used, together with media reports that provided information that assists in understanding the business process and its outcomes. Following the demise of the scheme, and a change of Government, a Robodebt Royal Commission (RRC) was established. Transcripts and some additional documentation emerging from the RRC during 2022-23 shed additional light on a number of aspects, as did the Commission's final Report (RCR 2023).
Such peer-reviewed articles as have appeared were also checked for information and analysis (Hogan-Doran 2017, Carney 2019, Braithwaite 2020, Whiteford 2021, Graycar & Masters 2022). As the present article was nearing completion, a formal case study appeared in the refereed IS literature (Rinta-Kahila et al. 2022). That article adopts algorithmic decision-making (ADM) as a theoretical frame, and uses open coding followed by inductive coding. The method it adopts contrasts with the present work, with the two providing complementary assessments of the project's significance.
The orientation of our work is conditioned by our earlier observation that, with contemporary, large-scale, data-based and outward-facing IS, a strong focus on technical elements is inadequate, and socio-technical approaches are essential. Both our metatheoretic commitments and the data lead our analysis to place greater emphasis on human issues - firstly as 'human inputs' to both design and operation, secondly as 'human impacts' (on participants/users), and thirdly as 'human implications' (on usees). This case study identifies the root causes of the harm that the Robodebt scheme caused. Although some technical problems have been documented, it is through a socio-technical examination that we can see problems with the interaction of intentions, behaviour, well-known design principles, and program implementation.
The following section presents the relevant aspects chronologically, from 2015 to 2022, with the majority of the inferences and interpretations held over to a later section of the article. The narrative approach allows an anticipatory tone, thick description in explanation, and the scope to delve more deeply into particular elements of the story that shed light on the events. The chronological part of the paper will show that, as the project unravels, the hopeful application of technology to help the Government improve its promise to Australian taxpayers gives way firstly to questioning how fundamental flaws went unnoticed and/or unaddressed for so long, followed by emergent understanding of the staff toil and the human toll, and then widespread bemusement about the apparent absence of accountability for the fiasco.
This section traces the history of the Online Compliance Intervention (OCI) project. This term is used throughout the case study, noting however that the agency used a range of other project-names, including the Welfare Payment Infrastructure Transportation (WPIT), Strengthening the Integrity of Welfare Payments (SIWP), Employment Income Confirmation (EIC) and Check and Update Past Income (CUPI). An outline of the timeline is in Table 2. This section commences by summarising relevant predecessor activities. It then outlines the project's purposes and the design features intended to fulfil them. The implementation and operational phases are described, followed by evidence of the scheme's impacts, and - despite the Government's sustained denial of the scheme's misconception, its illegality and harmful effects - followed by its eventual demise and the subsequent repercussions. As previously explained, the scope of the case study is very broad, because an intervention of this scale cannot be understood by limiting the field of view to IT, nor to the intra- and inter-organisational elements of the IS, and instead needs to encompass all environmental factors that were material to the process and outcomes.
Data is gathered by DHS from its clients. DHS also has substantial powers to demand information from other sources, such as organisations for which welfare recipients perform work and from which they receive payments. In addition, using specific authority enacted in 1990, "DHS began data matching activities in 1991" (Omb 2017, p. 5, fn7), and it "has conducted [Pay As You Go] PAYG data matching activities with the [Australian Taxation Office] ATO since 2004 ..." (p. 5).
"Income support payments are subject to an income test which means that a recipient's fortnightly payment may be reduced once their income reaches a specific threshold" (SCA 2017, p. 13). The purpose of data matching with ATO is to check data provided to DHS by clients against that available through the taxation system: "payment recipients must report their income fortnightly ... Where it is found that a recipient has incorrectly reported their fortnightly income, and the correct amount would have affected their entitlement to a payment, a ... debt may be raised" (SCA 2017, p. 14). Discrepancies may indicate that overpayments have occurred, many of which are known to be minor. Many discrepancies result from errors, variously by the client, employers and DHS, and some are fraudulent. Prior to 2016, investigative work conducted by DHS staff had been giving rise to over 20,000 compliance notices annually, relating to less than 0.4% of DHS clients (Omb 2017, p.5). However, other evidence suggests that only about one-quarter of those clients were eventually found to be in debt (SCA 2017, p. 17), suggesting that the outcome was of the order of 5,000 enforced debt notices p.a., affecting <0.1% of clients.
Although the matching itself had long been automated, the process in use until 2015 continued to involve a considerable amount of manual work. In the spirit of digitalisation, DHS initiated a project with the intention of reducing the manual effort through further application of technology. At the heart of project was apportionment of data accumulated over a 12-month period into 26 equal parts, commonly referred to as 'income averaging'. The notion of 'averaging' was a key factor in the project and is further discussed below, in particular in s.5.1(c). In the Government's Budget of May 2015, the announcement of funding for the project declared zero tolerance for 'rorting' (Australian vernacular, meaning small-time cheating on tax or welfare) and net savings of AUD 1.5 billion. See Appendix 1-4-1. (For the remainder of this paper, important evidence that comprises lengthy quotations is provided in Appendix 1).
Evidence to the Royal Commission 7 years later showed that the legal branch of the policy agency, DSS, had provided advice as early as late 2014 that the use of income averaging in the manner proposed "would not be supported by law" (RRC 2022c, p. 1351). The Royal Commission found clear evidence that the policy and operational agencies were both pressured by the Minister ahead of the 2015 budget, and that the line of three levels of senior executives of the operational agency, DHS, contrived to make it appear that the income averaging approach was lawful. Further, it found that the executives and the Minister, in March 2015, had deliberately misled the Expenditure Review Committee of Cabinet (RCR 2023, pp.57-107).
The Government's Budget Media Release of 12 May 2015 declared that "The Government has zero tolerance for rorting of our welfare system. We will put a strong welfare cop on the beat ..." (MRR, 2015). However, what that meant in concrete terms only slowly became publicly apparent. According to the Ombudsman's Report, the main efficiencies were to be gained by:
A DHS Deputy Secretary stated in evidence to the Senate Committee that "we are trying to eliminate the need for anybody [on DHS staff] to [acquire information from the employer] so that it is easier [for DHS]" (SCA 2017, p. 16). The efficiencies were argued to encompass both reductions in labour costs significantly greater than the ongoing technology costs, combined with substantial improvements in the identification and recovery of overpayments. Together, these resulted in "forecast ... [AUD] 3.7billion worth of [net] savings ... over four years from 2016-17" (SCA 2017, p. 4). The interpretation of the Community and Public Sector Union was that: "[OCI] is a fairly obvious consequence of a department that no longer has the resources to provide effective services. The decision to replace the human oversight of debt recovery with automated data matching was absolutely based on a desire and an imperative to save money" (SCA 2017, p. 19).
A brief description of the core of the Online Compliance Intervention scheme was provided in the Ombudsman's Report (Omb 2017, p. 4):
[After] the OCI matches the [fortnightly] earnings recorded on a customer's Centrelink record with historical pay as you go (PAYG) income data from the Australian Taxation Office (ATO) [which was the total earnings for a lengthy period, in many cases a full year] ... customers are asked to confirm or update their income using the online system. If there is a discrepancy in the data match, this can result in a debt the customer must repay. Parts of the debt raising process previously done manually by compliance officers within DHS are now done using this automated process.
However, that description omits mention that the 'confirm or update' step requires the 'customer' to submit very specific evidence in relation to payments 2-7 years earlier, which in many cases needed to be first acquired from their then employer(s).
A simplified model of the primary process is in Figure 1. More detailed descriptions are provided of the manual system (Omb 2017, p. 31), the pilot (p. 32), the interim business process from 2015 (p. 33), the live process July 2016 to January 2017 (pp. 33-35), and the changes made in February 2017 (pp. 35-38).
Until July 2023, none of the authoritative sources made clear whether OCI applied to every means-tested benefits scheme administered by DHS (as could have been inferred from the expression "income support payments" (Omb 2017, p. 33), or only some of those schemes (as implied by Government statements in January 2017 that the scheme was to be later extended to age and disability pensions). DSS later provided a list of 15 payment types that were "at different times" subject to the scheme. "Newstart Allowance and Youth Allowance formed the bulk of the payments" (RCR 2023, p.9) - despite it being known that only 2-6% of those recipients had stable earnings flows. All types were subject to policy decisions by DSS, but two, Austudy and Abstudy, were managed jointly with Education.
To submit the information, recipients were required to register with an intermediary website called myGov (Omb 2017, p. 34). No telephone contact-point was provided. The notification included the statement that DHS "will update [customer] details using the enclosed employment income information" (p. 34). This represented an implied threat to use that data to compute a liability by the client to DHS, and demand return of the claimed overpayment. Further detail was later published (Deloitte 2023a, 2023b), but the description remained incomplete. The Royal Commission concluded that "There may be inaccuracies in the process maps and report leading to the possibility of inaccurate conclusions. If so, they are likely to be the product of DHS's haphazard and inconsistent documentation of its processes" (RCR 2017, p.471).
DHS rolled out the measure in three stages between July 2015 and September 2016. See Appendix 1-4-2. "The initial group ... included those who received income support payments in the 2010 to 2014 financial years" (Omb 2017, p. 33). The staged launch occurred during the first half of the July 2016 to June 2017 financial year, so the demands related to periods as far back as July 2009. No information has been found showing the distribution over those years. DHS was aware that many of the targets "were no longer Centrelink customers" (p. 33). No evidence has been found as to whether DHS estimated the proportion whose contact-details were no longer current and who would therefore quite probably not receive the letter, nor whether DHS monitored for evidence of non-receipt, nor whether DHS handled those cases any differently from current clients.
Each recipient was forced to think back to the relevant period(s), and search for specific, old documents. If they failed to find the documents, they needed to contact one or more employers and request copies. The relevant names of the documents varied depending on the employer. The recipients may have had no contact with those employer(s) for quite some time, and they had no standing to demand compliance by those employer(s). The minimum period they had to go back was 2-1/4 years, but the longest look-back required was 7-1/4 years, to July 2009 (i.e. the beginning of the 2010 financial year). Particularly where the elapsed time was long, some of the employers had ceased business in the meantime. It is unclear whether DHS had any evidence about, or did any studies using, for example, its own staff, or randomly-selected members of the public, let alone welfare recipients, in order to establish the extent to which the scheme may have demanded a level of intellectual capacity, online expertise, assiduousness in record-keeping and negotiation skills with ex-employers, that many benefit recipients do not have.
These notices were sent in an automated manner, without human (i.e. government employee or contractor) intervention. The number of debt notices skyrocketed from 20,000 per annum to 20,000 per week (Sarah 2016). Rather than causing alarm to the Government, it was trumpeted as a success that, after just four months of the scheme, in January 2017, 169,000 debt notices had been sent to some of Australia's most vulnerable individuals. It was credited by the Government as having already "identified close to [AUD] 300 million in overpayments to welfare recipients" (Belot & McGhee 2017) - although the Minister at the time referred to that figure, apparently erroneously, as having been recovered rather than merely identified (Belot 2017a). It was also announced that the Government was considering expanding the OCI beyond Jobseeker unemployment benefits to age and disability pensions (Belot 2017b).
Many recipients were unable to get through to the overloaded call-centres, with many reporting delays of 3-6 hours before a call was answered, and during some periods the telephone system was overwhelmed, and calls did not even reach the automated-response system. The 28 percent of the population who live in rural and remote areas, which includes about one-third of welfare-recipients, have to travel long distances to Centrelink offices. In regional and suburban offices alike, people had to wait in long queues to talk to under-resourced and under-informed counter-staff.
Where the alleged debtor was no longer a benefits recipient, DHS used private-sector debt collectors. The contracted firms included one that was soon afterwards successfully prosecuted for coercion, false representations and unconscionable conduct when dealing with a consumer (Bajkowski 2022). Data provided to the Royal Commission suggests that the "Total number of Robodebts referred to External Debt Collectors" was 305,977 (RCR 2023, p.507), of a total of 794,000 debts in all (p.402), or 38.5%. The total value of those was AUD 955m (54.5% of the AUD 1,751m total), with a recovery-rate of about 11.2%.
"In the 2016 calendar year, 126,571 [individuals] ... were sent debt notices under the OCI" (Omb, p. 4, fn5), with the vast majority sent in the 4 months from September to December. However, as early as the beginning of December, after only 2 months of the main rollout, complaints about the scheme were rife (TT 2016), and by late December it was public knowledge that the basis on which OCI was built was grossly flawed and the error-rate very high (Knaus 2016). Despite the annual summer slow-down from early December 2016 to late January 2017, media pressure built rapidly. The term 'Robodebt' was coined, and widely adopted by no later than mid-January 2017. The Ombudsman felt it necessary to commence a study, DHS felt the need for some token changes, and a Senate Committee lumbered into existence.
Some changes were commenced in late 2016 and early 2017, but they were limited to corrections of obvious blunders in the original detailed design, and not corrected until 6-9 months after rollout commenced. These related to the absence of the phone-number from the letter, exception-handling where DHS was aware that the letter has not reached the target-person, extension of the 21-day response-time to 28 days, acceptance of bank statements in lieu of payslips, fixing of errors in the UI/UX design, the pausing of recovery action while a matter was under internal review (not previously implemented, in breach of fair process), and easing of the circumstances in which a 10 percent debt-recovery fee was automatically applied (Omb 2017, pp. 35-38). The correction of greatest substance was the belated recognition that complex cases are unsuitable for automation and need to "fall out for manual intervention" (p. 36).
The Government drew further public opprobrium when, in an endeavour to discredit a complainant who had gone public, the Minister at the time, Alan Tudge, released personal data about that individual from DHS records. "[O]ne of Australia's leading criminal barristers [wrote] "it is reasonably clear that the minister or one of his office's staff has committed an offence ... punishable by up to two years imprisonment if proven in court" (Knaus 2017), but no enforcement action was ever taken.
By April 2017, the scheme's problems were well-understood and could be clearly explained to the Senate Committee (Barbaschow 2017). The Senate Committee's first report, published in June 2017, was scathing:
"Communication problems included letters not being received, trouble contacting the department via phone, difficulty in receiving intelligible income data used to calculate purported debts, hard to navigate online communication portals, difficult to understand correspondence and a lack of material translated into other languages" (SCA 2017, p. 41, with specifics documented on pp. 43-48 and 56-59). "The committee notes it is clear there is a significant communication problem when 65,000 from 300,000 people do not respond to requests from the department to engage" (p. 48).
Inadequacies under the heading 'Communication barriers' were listed as Vulnerability flags, Literacy, English as a secondary language, Disability-related communication barriers, Geographic barriers, and difficulties people faced with all three communication channels (pp. 48-55 and 59-69).
The Senate Committee concluded that the key concern was the outsourcing of the income checking process to individuals, thereby reversing the onus of proof. See Appendix 1-4-3. Inadequacies were also noted in the process to clarify or review a purported debt, the policy on handling queries and its impacts, the challenges posed by reversing the onus of proof, the impact on community legal centres, and even in the subsequent process improvements made by the department (SCA 2017, pp. 71-89). The Senate Committee also noted the absence of any obligation on DHS to comply with any guidance in relation to the use of debt collectors, or even to take any responsibility for malpractice by them (SCA 2017, pp. 29-32).
Remarkably, it took a further 2-1/2 years, until November 2019, before the Government accepted defeat, and ceased raising debts "where the only information we are relying on is our own averaging of Australia Taxation Office income data" (Farrell 2019). The government had ignored the multiple Tribunal decisions made against it, 10 of which were decided prior to March 2017, within 6 months of the scheme's commencement (RCR 2023, pp. xxviii, 239-240, and Appendix 9). DHS had also been assiduous in avoiding test-cases reaching the Federal Court. However, the agency inadvertently let two cases, Masterton and Amato (Amato 2019), slip through to the courtroom, resulting in a clear judicial statement of illegality (RCR 2023, pp.288-317). It then took a further 6 months, until May 2020, for the Attorney-General to concede that "all Centrelink debts raised using the 'income averaging' method were unlawful" (Henriques-Gomes 2020).
A class action was mounted. In breach of the agency's formal public policy obligation to be a 'model litigant', DHS resisted the action until immediately before the hearing commenced, whereupon it conceded the illegality of its actions, and all claims - although for the pitifully inadequate sum of AUD 100m, representing AUD 250 per member of the class action. The department agreed to settle the class action only in November 2020, with judicial approval further delayed until June 2021 (Pry 2021, SCA 2022 p. 6), and repayments and payments not completed until the end of September 2022.
The Senate Community Affairs Reference Committee had commenced a second review of OCI in July 2019. It published a succession of five interim reports from September 2020 onwards and a final report in May 2022. The last (SCA 2022) was entitled 'Accountability and justice: Why we need a Royal Commission into Robodebt', and used the headline 'A massive failure of public administration'.
The Coalition Government was defeated in the election of May 2022, and the new Labor Government quickly moved to establish a Royal Commission, in August 2022. It had been widely understood that the then Coalition Government had cancelled all OCI-generated debts in mid-2020. However, it was reported in the media in mid-October 2022 that the new Labor Minister had announced the cancellation of about 124,000 cases still under review and another 73,000 where potential debts had been identified but the clients had not been informed (Brookes 2022).
DHS had projected that the number of overpayment reviews processed each year would leap almost forty-fold from 20,000 to 783,000 (Omb 2017, p. 5). Yet the agency appears not to have realised that this would give rise to very large increases in the numbers of transactions conducted not only in the online system, but also at the call-centre and physical service-centres. It might have been expected that additional staff would have been put in place, particularly for the inevitably fraught transitional phase. In fact, it appears that the agency was forced to comply with Government-imposed reductions in headcounts, such that call-centre and front-counter staffing was lower than it had been prior to the new system being launched. If human resources had been involved in cross-checking notices, the volume would have been internally obvious, and precautionary reviews undertaken.
The staff union drew attention to the "classic false economy" and the "costly reverse workflows where staff are taken offline to deal with complex and difficult disputes over incorrectly raised automated debts." (SCA 2017, p. 19). Staff dissatisfaction, embarrassment, and stress were widespread, as a result of having to deal with ill-informed, alarmed, rude, depressed and disturbed clients (Belot 2017a). The union submitted that "[OCI] has been an absolute disaster for many Centrelink use[r]s and also for the workers charged with implementing a system they know to be deeply flawed and unfair" (SCA 2017, p. 18).
Both the Ombudsman's and Senate Committee's Reports detailed many personal accounts of the stress and distress caused to the benefits-recipients subjected to it, with repeated mentions of "feelings of anxiety, fear and humiliation". "Individuals had spent hours finding the required pay slips and bank statements, some dating back to 2010-11, often for Centrelink to find that no debt was owed" (SCA 2017, p. 38).
Media reports, court judgments, and the Royal Commission transcripts are littered with evidence of the harmful and in some cases devastating impacts on recipients of the mostly erroneous and in all cases non-lawful demands. Some of the tales were harrowing. For example, a Ms Miller gave evidence to the Royal Commission in relation to her son Rhys Cauzzo, who committed suicide on 26 January 2017 after being pursued by DHS and then a debt-collector: "he was very distraught when he made that phone call to me. And - and he said he was feeling suicidal. And so we probably spoke for about two hours on the phone, maybe longer". The five letters of demand induced him to draw "a face with a gun through there and dollar signs just coming out of his head. And it had 'debt life' written on it, I think" (RRC 2023a, pp. P-3256, 3259-60).
On the basis of smatterings of such little data as ever emerged, it appears that in only about 40 percent of the cases that DHS commenced did the agency pursue the debts in full, with about 20 percent of debts reduced, about 20 percent of debts deemed not worth pursuing, and about 20 percent withdrawn because they were clearly wrong. It later accepted that even the 60 percent that it pursued had to be repaid, because there was no legal basis for raising the purported debts. DHS appears not to have invested in extraction and analysis of evidence about the proportion of the purported debtors and purported debts that may have been actually legitimate. It appears possible that the proportion of over-payment due to errors and fraud is greater than the 0.1% estimated by advocates, and greater even than the 0.4% that DHS has claimed in the past. On the other hand, it appears that the undiscovered overpayments may be a great deal less than the 1.0-2.5% that the Government postulated when it announced the scheme, and the even larger proportion that it claimed to have discovered during the period the OCI operated.
The Senate Committee's Report noted that over AUD 1.7 billion in illegitimate debts were imposed on 433,000 people, of whom 381,000 individuals were pursued, many through private debt collection agencies, resulting in the repayment of over AUD 750m to the Commonwealth. The debts that had been raised were later withdrawn, the payments made in the meantime were eventually refunded, and a further AUD 100m was paid to individuals in damages and AUD 12m to the class action lawyers. Further losses to the public purse included admitted costs of implementation of AUD 600m to mid-2019 alone (Burgess 2019); but many elements of the total cost remain shrouded because the Government refused to disclose such information.
Beyond the financial damage, "The Income Compliance Program impacted hundreds of thousands of people and, for many, resulted in devastating emotional and psychological harm. It has undermined many people's financial security as well as their willingness to engage with and trust government services" (SCA 2022, p. 1). See also Braithwaite (2020).
The Federal Court judgment included statements of a kind not commonly seen in such documents, referring to financial hardship, anxiety and distress, including suicidal ideation and in some cases suicide. See Appendix 1-4-4. How many suicides Robodebt threats were associated with has been the subject of speculation. See Appendix 1-4-5. The Royal Commission Report discussed three that were brought to its attention (RCR 2023, pp.181-183).
This section builds on the narrative presentation above, by examining themes that emerged in relation to the OCI's prospects of success, public policy issues and/or impacts on people. The first group of themes has to do with the policy and designs aspects of the scheme, and the second group with its conduct.
This sub-section examines the following aspects of the policy intentions for OCI, and the design measures to implement those intentions, with the focus on those relevant to its dysfunctionality and harmfulness:
This is followed by a recapitulation of the key aspects.
"[DHS] provides the ATO with the identity information of Centrelink recipients which the ATO matches against their records. In order for the ATO to provide income information to the department, the ATO must identify a high confidence match between the identity information provided and the ATO's records" (SCA 2017, p. 14).
The Tax File Number (TFN) scheme was introduced in 1991, and has operated since then, for the express purpose of enabling efficient and reliable identification of fund flows relevant to income taxation processes. Since that time, the ATO and other agencies, in particular DHS, have used the TFN in multiple, ongoing data-matching programs. However, "the department has chosen not to use TFNs under the OCI program. This has meant that, unlike previous data-matching processes, the OCI program is not legally bound by the provisions of the Data-matching Act ..." (SCA 2017, p. 28). Instead, "[DHS] sends approximately 80 million identities to the ATO and receives approximately six million matches back from the ATO" (Omb 2017, p. 34). That seemingly very low match-rate raises questions about the accuracy and reliability of the expedient adopted by DHS, and whether it is significantly inferior to the match-rate that would have been achieved if the TFN were used.
It is very difficult to see any reason for the design decision to not use the TFN in the OCI scheme other than to avoid having to comply with the statutory provisions in the Data-matching Program (Assistance and Tax) Act 1990, and to be subject instead to a voluntary and weak set of guidelines issued by the data protection oversight agency (OAIC 2014).
The Senate Committee's Recommendation 4 was that the statutory provisions should be applied. See Appendix 1-5-1. However, the Government did not accept the proposition (GR 2017, p. 10). In the absence of any coherent explanation from DHS or the Government, it appears that the reason was that the program would not have been able to be performed effectively if it had to comply with that law, and may not even have been able to be performed at all.
A surprising finding of the Royal Commission, which only came to light in July 2023, was that "[DHS] no longer conduct any of their data-matching programs under the DMP Act ... since 2004" (RCR 2023, p.448). This is despite DHS continuing to use TFNs to match identities and extract data for provision to DHS under the Scheme (RCR 2023, p.618).
"[DHS] does not include the [TFN] because use of the TFN is restricted to data matching under the Data Matching (Assistance and Tax) Act 1990 (the Data Matching Act)" (Omb 2017, p.39 at 2.13), but "Once the DHS data file is received by the ATO, the ATO uses tax file numbers to extract information it holds about that individual (for example, name, address and date of birth)" (para. 2.15). The Royal Commission Report says that "the Ombudsman__s report ... disclosed that the ATO was using TFNs to match identities and extract data for provision to DHS under the Scheme" (p.618).
The reasonable inference is that DHS has subverted the intentions of Parliament as expressed in the DMA. It is unclear how that was achieved. One possibility is that the data-matching scheme was transferred from DSS to DHS without the DMA (1990 ss.3, 3A and/or 4) being amended to refer to the newly-relevant agency. Another possibility is that DHS contrived to achieve the effect of matching identities based on the TFN without triggering the relevant provision of the DMA. This may have been through the expedient of DHS providing data to ATO without the TFN included - forcing ATO to do an internal lookup of the individual's TFN, thereby enabling data-matching using the TFN, but avoiding triggering the condition under which the DMA would apply.
Further, DHS did not even comply with the unenforceable requirements of the self-regulatory arrangements. It retained personal data that it was required to delete. It failed to undertake manual review of discrepancies resulting from the data-matches. It failed to operate a compliant issues resolution scheme. The Royal Commission concluded that "there is a serious __question as to whether information was lawfully disclosed by the ATO to DHS for the purpose of data matching under the Scheme" (RCR 2023, pp.453-461).
The oversight agency also failed to fulfil its role of protecting the public interest. The Royal Commission noted that "[ATO's correspondence with OAIC on 10 January 2019] recorded the fact that TFNs were __appended to the input file__ but did not directly acknowledge __use__ of the TFNs in the data-matches the ATO conducted" (RCR 2023, p.624). It also found that the oversight agency, OAIC, failed its responsibilities firstly to question the ATO about that agency's obligation "to use TFN information only for a permitted purpose under taxation or personal assistance law" (p.625) and secondly to pursue an own-motion investigation at the very early stage of January 2016, when it became aware from a complaint that DHS may have unlawfully used TFN information (p.627).
DHS's subterfuge enabled it - and, it appears, continues to enable it - to avoid a formal regulatory mechanism and instead adopt the pretence of being subject only to a loose and entirely voluntary guideline arrangement. It is unclear why the Royal Commission failed to declare ATO and DHS to have been in breach of the law, and why it failed to make any Recommendation that the agencies be prosecuted, and required to comply with the DMA.
The data protection oversight agency (OAIC) identified multiple breaches of Privacy Act requirements in relation to data quality and steps to correct poor quality data (SCA 2017, p. 25). Data quality failures noted by the Senate Committee included:
The Australian Privacy Foundation argued that there were far more breaches involved than those identified by OAIC (SCA 2017, pp. 28-29).
The Royal Commission concluded that "The evidence before the Commission ... suggests that there may have been breaches of the APPs under the Privacy Act, in relation to disclosures/collections by the ATO and DHS for the purpose of data matching in the Scheme" (RCR 2023, p.461).
Despite these breaches being known since at least 2017, and the reinforcement of the message by the Royal Commission, no evidence has emerged of any enforcement measure being initiated.
The Ombudsman's Report provided an explanation of the averaging technique used by DHS in the OCI scheme. See Appendix 1-5-2. However, it is unclear over what period each client's income from each employer was averaged, because the only information that appears to be publicly available is that it was "over the period the employer told the ATO the customer worked for them" (Omb 2017, p. 34). However, " ... employers must only provide [to ATO] an annual figure paid during that financial year ... [and only about half] of records [from employers] were for a full year employment" (SCA 2017, p. 16). So it appears very likely that, in a great many cases, erroneous periods were used as a basis for the averaging calculation. The Senate Committee and a regional social services advocacy organisation (ACTCOSS) identified multiple further, specific examples of problems with 'averaging'. See Appendix 1-5-3. Another aspect came to light in the court-case that finally forced abandonment of the scheme: "A Freedom of Information request revealed that [the client] had actually been underpaid by AUD 480 for the period concerned. If the government were serious about 'the right payment', it would have sent the client a cheque, not a debt" (Whiteford 2021, p. 354).
Interpolation based on a scatter of data that became available suggests that inferring the income for a period by averaging will be wrong about half the time, and that the available data is inadequate to determine which category each case falls into. The averaging approach is in no way a suitable basis for any administrative decision-making, let alone where those decisions impose punitive measures, let alone in relation to vulnerable populations, let alone by automated means unchecked by any appreciation of reality and by 'common sense'. The Royal Commission found that "The consistent evidence was that [prior to 2015] income averaging was used relatively seldom, usually by agreement with the recipient, and in the context of other information which provided some assurance that it would give a reliable answer" (RCR 2023, p.16). Importantly, the DHS's own 'Operational Blueprint' acknowledged "difficulties", including "If employment is for a part of a year only" and "If income varied greatly during the year" (Omb 2017, p. 42). The agency also tacitly acknowledged that the "difficulties" were severe, in that, during the 30 years of use since the early 1980s, its use had been "limited to last resort situations" (Omb 2017, p. 42). Given that the agency was well aware that averaging was suitable only for "last resort situations", it is difficult not to infer culpability on the part of Ministers and DHS executives, at the very outset of the project, in adopting the technique.
Ultimately, it was established that "By December 2014, DSS had received internal advice, which was as they had expected: the proposal ... was unlawful" (RCR 2023, p.26), and evidence before the Royal Commission showed that the ATO had, in 2017, asked DHS to "cease and desist" using taxpayer data for the robodebt scheme, but was ignored (Wong 2022). The unlawfulness arises "because social security legislation [requires] a person's income support entitlement to be calculated on the basis of the income that they had actually 'earned, derived or received' in a fortnight. The average of someone's income over a period was no indication of what they had earned in any particular fortnight, unless there was also evidence that they had earned their income at a regular fortnightly rate over that period" (RCR 2023, p.37). The unlawful behaviour occurred because three senior executives suppressed the unwelcome advice during December 2014, because it would have de-railed a policy measure strongly desired by a rising-star Minister - who became Treasurer 9 months later, and Prime Minister 3 years after that (RCR 2023, pp.37-50).
The Federal Court judgment was scathing about the absence of a proper legal basis for asserting the existence of debts, and the victimisation of people with limited financial means (Pry 2021, pp. 3-4). See Appendix 1-5-4. The judgment (Pry 2021, p.2) affirmed that the Commonwealth was to:
The Royal Commission concluded that "The position that income averaging was a long-standing lawful practice was so entrenched within DHS that lawyers at all levels were unable to question it in accordance with their professional obligations" (RCR 2023, p.521). The oversight agency, the Office of Legal Services Coordination (OLSC), also failed to perform its function, in that "there was cause for alarm within OLSC that DHS considered there were no legal issues and that there was no legal advice pertaining to the Scheme" (pp.537-538), but it took no action.
"Previously, where a discrepancy was identified between [the data held by DHS and by ATO], the department manually checked the information for accuracy and contacted the recipient and/or their employer to clarify the information" (SCA 2017, p. 15). "Under manual data-matching arrangements in 2009-2010, approximately 25.5 per cent of identified discrepancies were resolved as the recipient or employer was able to provide information which confirmed the recipient had received the correct Centrelink payment" (SCA 2017, p. 17).
However, under OCI, the effort was outsourced to DHS's clients. See Appendix 1-5-5. The result was that "the significant reduction in workload for the department by this outsourcing, has allowed for a huge increase in the number of income discrepancy investigations that the department initiates" (SCA 2017, p. 17). In short, the agency abandoned its obligations to perform data quality assurance in order to increase its enforcement activities, and it did so without recognition of or concern that a large proportion of the additional enforcement actions were unjustified, because the putative debt was illusory.
Staff have powers that they can exercise, have sophisticated technological facilities at their disposal, know who to contact, have training, and perform the tasks frequently. Individual benefits recipients have none of those advantages. Some are incapable of working out what needs to be done, many lack the skills to execute the tasks, and many are cowed by DHS and/or by their employers or ex-employers.
The Senate Committee was emphatic that this was unjustifiable, with its Recommendation 6 being that "the department resume full responsibility for calculating verifiable debts (including manual checking) relating to income support overpayments, which are based on actual fortnightly earnings and not an assumed average" (SCA 2017, p. 109). This was ignored: "the Government rejects the central conclusions and recommendations of the [Senate Committee's] Report" (GR 2017, p.5). The agency inflicted ongoing harm to further hundreds of thousands of individuals until it was forced to abandon the scheme over 3 years after the mid-2016 launch, in November 2019.
Previously, DHS checked that the information being used for debt collection was accurate. Under OCI, the welfare recipient was forced to present counter-evidence to a vague debt collection notice. Many citizens lacked the capacity to respond adequately, if at all. The Senate Committee expressed concern about these features. See Appendix 1-5-6.
Despite the enormous challenges to individuals created by the reversal of the onus of proof, in 20 percent of cases "the individual has been able to provide clarifying information and this has resulted in no debt being owed" (SCA 2017, p. 33). Given the difficulties faced by benefits recipients, rehearsed throughout the Senate Committee Report, the 20 percent was clearly the tip of a much larger iceberg of instances in which the inference DHS had drawn was in serious error. The Committee noted multiple impacts of the unjustified reversal of the onus of proof. See Appendix 1-5-7.
In automating the decision and taking action without human review or even the capacity to intervene, OCI appears to have relied on s.6A(2) of the Social Security (Administration) Act 1999 (Cth), which states that "a decision made by the operation of a computer program under an arrangement made under subsection (1) is taken to be a decision made by the Secretary". Although that provision was inserted into the Act in April 2001, it appears that this project may have been one of the first occasions on which the automation of administrative decision-making has been conducted by an Australian government agency, at least on a large scale (Carney 2019). People's lives were placed at the direct mercy of a computational system, without human consideration of the matter prior to action being taken that was harmful to the people concerned. This suggests blind faith in technology, and excessive trust in the digital persona as an adequate proxy for the person, in digitalisation, and in transformative IT.
The algorithm used to automatically identify putative instances of overpayment was seriously flawed, and in many cases the inference automatically gave rise to a decision that a debt existed, which it appears could in turn give rise to an automated referral to a debt collection agency. The Senate Committee rejected the proposition that such substantial automation could be acceptable. See Appendix 1-5-8. The Royal Commission concluded that "... the Scheme ... involved a system of business rules with no ability to move outside of specific and defined action on the basis of the data received. It was extremely rigid; once the rules had been coded and set in place, the system itself would stay in place until the rules were changed by way of human intervention (RCR 2023, p.472). The automation extended from calculation of the entitlement, to calculation of the overpayment, generation of a debt, application of a 10% penalty, generation of a debt notice, and despatch of that debt notice" (p.473).
In addition, the lack of human intervention, coupled with the apparent absence of managerial supervision, meant decision-making was not only delegated to a machine, but the delegation remained in place for an extended period. Clarke (2019a, p. 426-427) identifies 7 degrees of autonomy. Three are of the nature of decision support systems, with human decision-makers remaining in control. The other three involve automated decision-making, in which the system respectively notifies a human supervisor of an impending action (enabling override or veto), acts and informs the human it has acted (enabling interruption, suspension or cancellation of the action), or acts without any scope for human intervention. Great care is essential in granting such delegations.
Errors are also made by engaged humans trained to deal with delicate cases pertaining to human welfare; but humans have the capacity to notice anomalies, to apply common sense, to feel discomfort, and to draw their superiors' attention to problems. The absence of a human-in-the-loop was alone fatal to the project - and to some of the agency's clients.
Reflecting the findings of the various reviews, the designers of OCI, driven by the Minister's and senior executives' focus on fraud, error and waste, lost sight of the agency's primary function of service. DHS perceived its 'customers' as mistake-makers and cheats, not as 'dependants' or 'clients'. The digitalisation meme encouraged blind belief in data and data analysis. The transformative IT meme stimulated automation of inferencing, decision and action without any equilibration mechanisms. The combination of those aspects with the sudden outsourcing of data-collection and reconciliation effort to people ill-equiped for the task, and the avoidance and breach of relevant regulatory mechanisms, resulted in a 'perfect storm' of intertwined defects that guaranteed failure.
Basic standards need to be enunciated and then respected. In a public statement issued early in the project, in January 2017, entitled 'The Imminent Threat of Automated Government', the Australian Privacy Foundation (APF 2017) laid out the reasonable public expectation that government agencies and business enterprises alike be under legal obligations to:
This section considers inadequacies in the process adopted by DHS, in the areas of:
As regards other agencies involved in the matter, "The ATO ... noted that it had not been consulted on the design or implementation of the system" (Omb 2017, p. 22). This was despite the Australian Taxation Office being the agency that performed the matching process and that provided data from its files on which the entire scheme depended. In addition, "DHS advised our office it did not consult with the [then Digital Transformation Office] DTO in the development and testing of the OCI" (Omb 2017, p. 24, fn.54). DTO, shortly afterwards re-invented as the Digital Transformation Agency (DTA), was the agency with nominal oversight over any government project that automated processes that were previously conducted by human government staff. The later involvement of DTA during 2017 was relatively minor.
The Royal Commission also concluded that "It does not appear that DHS engaged with the OAIC ahead of the commencement of the Scheme" (RCR 2023, p.614). Although OAIC, being the privacy oversight agency, "encouraged DHS to undertake a PIA for its data-matching activities" (p.614), it took no action when DHS failed to do so, and did not exercise its power to direct that a PIA be performed. It appears that OAIC failed to pursue a complaint that would have unearthed the unlawful collection and use of personal data (p.615), and in April 2017 it failed to undertake an own-motion investigation - a process that provides greater powers to acquire information (pp.618-619). It performed two unjustifiably narrow assessments commencing in December 2017 and August 2018, but they each whimpered to inadequate conclusions fully 18 months later, in June 2019 and July 2020, 3 and 4 years after the scheme began operation, and either side of the date the scheme was discontinued (pp.620-623). OAIC's review of the ATO's role was similarly ineffective (pp.624-625). The Royal Commission concluded that "the OAIC approach ... was too muted ... Its three assessments ... were of little real consequence. ... the data matching under the Scheme was not given the examination which it needed and which could, with the use of the OAIC'__s full investigative powers, have occurred" (p.627).
The extent to which all of the oversight agencies (the Ombudsman, OAIC and OLSC) fell into line with the desires of the Minister and the operational agency suggests that all of them are either toothless-by-design or the appointees are incapable of performing the functions for which they were appointed.
In addition, "[DHS Director] Jason Ryman ... told the Commission that DHS did not engage with non-government organisations in the development of the Scheme" (RCR 2023, p.373). Even the peak advocacy organisation, the Australian Council of Social Service (ACOSS) was excluded, "despite ACOSS's clear articulation of the desire for engagement in the stakeholder process" (SCA 2017, p. 22). The Royal Commission found that "In December 2016, [ACOSS CEO] Dr Goldie's letter to [Minister] Tudge represented one of the earliest and most detailed warnings to both the minister and his department about problems associated with the Scheme ... with striking accuracy, particularly given that, at the time it was written, there was very little information about the Scheme available publicly, and what was available often contained vague or opaque explanations. But perhaps the accuracy with which ACOSS was able to identify those issues reflected the fact that the identified flaws were glaringly obvious to anyone with experience in the social security system" (RCR 2023, p.169).
Neither did DHS engage with any associations that represent particular categories of recipient. Advocacy organisations that sought to convey concerns to the agency were not provided with any opportunity to do so, and were left with no other options than to pursue the matter through the media, to use their limited resources to support a few of the hundreds of thousands of affected individuals in making complaints and escalating matters to the tribunal and the court, and/or to present evidence to formal inquiries that were conducted after the damage had been done.
From a public policy perspective, DHS's behaviour was reprehensible, in clear breach of the norms applicable to impactful projects planned by government agencies. From an executive perspective, it was irresponsible behaviour that should be almost unknown in large government agencies, failing as it did the most basic principles relating to stakeholder analysis, and risk identification and assessment. Even the agency-friendly Ombudsman felt constrained to use firm expression. See Appendix 1-5-9.
DHS has had ample opportunity over many decades to familiarise itself with relevant advocacy organisations, establish a register, and run periodic conferences and as-required meetings on specific matters. A report prepared for the Royal Commission criticised the considerable reduction in external engagement in recent decades, and cited an earlier official review that had done the same (Podger 2023, pp. 14-15, citing Thoday et al. 2019). The Royal Commission concluded that "The lack of consultation with relevant advocacy groups, before and during the Scheme, exemplifies one of many instances in which a possible safeguard against the catastrophic results of the Scheme was rendered ineffective. It evidently suited the government's agenda in pursuing the Scheme to not engage with advocacy groups who might - and did - raise the fundamental failings of the Scheme" (RCR 2023, p.373).
Limited information is on the public record concerning the extent to which DHS took advantage of its own corporate knowledge in designing the scheme. However, the sources of relevant information that have been located are sufficient to suggest that the executives responsible for the project may be guilty of professional incompetence in this area as well:
The Royal Commission concluded that "One of the worst aspects of the Government__'s response to concerns raised was its resistance to the warnings from staff who could see what was happening" (p.658). It does not appear to have come to the Commission's notice, however, that DHS staff were given formal warnings by executives about the Code of Conduct (APS 2022). This occurred mainly by email, but some staff were subjected to a ritual reading of the Code, and others were required to sign an acknowledgement of having read and understood it. The relevant clause appears to be "an ... employee ... must not improperly use inside information ... to cause, or to seek to cause, detriment to the employee's Agency". However, one informant claimed that even the use of the term 'Robodebt' was declared to be a breach of the code, and that the code was "routinely used to end the employment ... of people at a lower ... grade" (Landis-Hanley 2023). Rather than drawing on the insights of relevant front-line staff, supervisors and managers, DHS executives abused the APS Code of Conduct by using it to protect executives who were behaving highly irresponsibly, and to impose repressive measures designed to chill the behaviour of staff-members and deter whistleblowing.
It is incumbent on Australian government agencies to recognise that many people are affected by powerful government agencies applying impactful information technologies, particularly when they embody automated inferencing, decision or action, and that, in addition to users, non-participant usees may need to be considered. In the context of ICO, usees included financial dependants of welfare recipients, non-dependent family members (particularly the parents of despondent clients), and the households in which DHS clients live. Many further categories are relevant, however, such as ex-clients who did not receive the notification, clients and ex-clients who lacked the technical means or expertise to respond and hence did not engage with the new sub-system, and the legal support centres and counselling services that felt a surge of demand. It was necessary for a Senate Committee and a Federal Court judge to draw to attention that families are economically dependent on DHS clients, and that mothers become distraught when an adult child is driven by public service ineptitude to take their own life.
Remarkably, it also appears to have been beyond the relevant executives' understanding, or perhaps just outside their frame of reference, that the scheme's success depended to a considerable extent on the capacity of the intended users to actually use the poorly-conceived and poorly-designed facilities. It appears to have been beyond the understanding of those executives that agencies can benefit from the use of participative or co-analysis and co-design of complex socio-technical systems. But, at the very least, it has to be expected that user interface and user experience design and testing would be undertaken, and that it would directly involve members of the targeted client segments. Within the organisation's boundaries meanwhile, it simply beggars belief that people employed as 21st century executives responsible for such systems would, during the analysis and design process of a compliance scheme, fail to take advantage of the expertise of the agency's own specialist compliance staff. This is further considered below.
The doggedness with which DHS pursued the agenda, and ignored the evidence of ineffectiveness and illegality, underlines the importance of safety-valves within the system. It is widely acknowledged that organisations need space to develop strategies into plans, without full transparency of the interim stages of discussion. For this reason, the various forms of 'sunshine' legislation generally exempt working documents from disclosure. It is also common for such protections as exist for 'whistle-blowers' to be available only after they have made genuine attempts to communicate issues internally. Legislation in Australia, on the other hand, strongly favours the government agency over the whistle-blower (Martin 2013). Several highly repressive prosecutions were in train during the Robodebt period, in circumstances where the disclosure was essential to the exposure of fraud and corruption.
The political commitment of a powerful Minister resulted in multiple senior executives suffering wilful blindness to the technical deficiencies, unlawfulness, and capacity for harm of a dramatic adaptation of a large-scale operational system. This was followed by wilful deafness to a great many sources of unwelcome news. These included information flows from clients through physical and IVR channels, incident reports, formal complaints, escalation of complaints to the tribunal, multiple tribunal decisions finding against the agency, abnormally clear and impassioned submissions from staff-members, increasingly precise and articulated submissions from advocates, gently negative comments from captive, cowed or incompetent oversight agencies, and substantive reports in the media that drew on specific individuals' cases and utilised inside information provided by one or more whistle-blowers.
It is commonplace to attribute greater harm to 'government cover-ups' than to the original actions of government that gave rise to the problem. The extent to which this maxim was ignored is remarkable. The earliest protestation found, to the effect that there was no problem, was less than 3 months after launch, in mid-December 2016: "[DHS is] confident in the online compliance system and associated checking process with customers" (Knaus 2016). In January 2017, it was reported that "[Minister] Tudge had rejected assertions people were being issued with unfounded debt notices or having difficulties in updating their details" (Anderson & Belot 2017), despite ample evidence of both. The Government's September 2017 response to the Senate Committee was dismissive: "the Government rejects the central conclusions and recommendations of the Chair's Report, especially the conclusion that the online system lacked procedural fairness" (GR 2017, p. 5).
As late as February 2019, a spokesperson for the then Minister, Keenan, declared that the automatic debt notice process is "reasonable, lawful and fair" (Medhora 2019). In all, it took more than 2 years before the tone of Ministerial statements changed - in the period between May 2019, when a court-case brought by Deanna Amato was heard, and November 2019, when the judgment was handed down. This concluded that it was "not open on the material before the decision-maker" to decide that the client had received social security benefits she was not entitled to, because there was "no probative material" that the average reflected Amato's actual income (Karp 2019).
The reluctance to accept the unlawfulness of the scheme continued far longer, however. "Government departments must at all times act with 'best practice', and in legal issues must also act as a 'model litigant'" (SCA 2017, p. 108. See also Appendix B of LSD 2017). In breach of that legal instruction, DHS was anything but a model, using multiple methods to delay proceedings, and to increase litigants' costs, capitulating only at the last possible moment in a conventional but immoral negotiating tactic to force the litigant to accept a substantially lower settlement amount.
The previous sections have provided a description of the OCI project, and identified the problems that gave rise to a great deal of harm to users and usees, and, despite a resolute project sponsor, project failure. This section considers what can be learnt from the debacle that is of relevance to digitalisation projects in the public sector generally. Considerable care is needed, because of the many dimensions across which contexts and projects vary, including cultural factors at organisational and societal levels, the nature of the government agency and its IT applications, and the purposes of systems sponsors. On the other hand, some commonalities exist as well, such as the technologies, the promotional techniques of technology providers, uncritical enthusiasm for technological innovation, energetic political commitments, and the categories of needy clients.
The themes addressed in this section are:
It was noted in the Introduction that the socio-technical view is essential to understanding large-scale systems that reach out beyond organisational boundaries and impact on people. It has transpired that inadequate attention to this aspect has been an underlying element of a great many of the deficiencies that feature in the failure of Robodebt. The technical aspects of the OCI sub-system were fairly minor and easily-understood elements. The issues emerged from the inadequacies of conception, engagement, testing and regulatory clearance, overlaid by Ministerial hubris and executive over-commitment to the fulfilment of Ministerial desires. We accordingly address a cluster of ideas in the socio-technical area as the first of the broader issues.
Socio-technical design builds on open systems thinking (von Bertalanffy 1950), and acknowledges that systems comprise technical components working in combination with social and/or human elements (Emery 1959, Abbas & Michael 2022). It offers a principled approach to the implementation of socio-technical interventions by seeking balance between humanistic values and technological capabilities (Cherns 1976, Cherns 1987, Trist 1981, Mumford 2000, Mumford 2006). It has been embedded in various methods (e.g. Checkland 1981, Avison & Wood-Harper 1990). Developers of public sector information systems need to adopt design methods that reflect socio-technical insights (Abbas, Michael et al. 2021; Abbas, Pitt et al. 2021).
DHS operates a still somewhat `high-touch' operation, with Interactive Voice Response (IVR) and call-centres complemented by a moderate intensity of footprints and counter-services. Under ongoing pressure to reduce staff and costs, DHS grasped at the promise of automation based on assumptions about datafication and digitalisation. In doing so, it left behind such of the basic tenets of human-centred design as it had previously respected (Norman 2013). The agency reverted to a merely technical vision of efficient-but-intolerant, IT-delivered inferencing, decision, and action. As a consequence, it lost sight of the need to recognise stakeholders and appreciate their interests.
A range of techniques exist that assist organisations to avoid such mistakes. The field of participative / participatory systems design has a decades-long history (Land & Hirschheim 1983), with occasional revivals in such forms as co-design (Liu et al. 2002, Piller et al. 2004) and co-creation (Zwass 2010). Involvement of users is challenging, not least due to their diversity and their lack of familiarity with design processes. For many organisations it has proven impractical, at least in part because of entrenched technocratic and xenophobic ('Not Invented Here') values within design teams. The increased emphasis in recent years on design science does not appear to have eased the predominance of organisation-centricity and technocracy. Participation is not non-existent, but it remains marginalised, with language tending to depict 'designers' and 'users' as being on opposite sides of conversations rather than collaborating as team-members. Clarke & Davison (2020) drew attention to the possibility of and need for 'participative design science': "Beyond asking 'What is a feasible and effective process for the design of a particular system or category of systems?' (cf. Guideline 3 of Hevner et al., 2004), research questions of the following form can be investigated: 'What is a feasible and effective process for reflecting the perspectives of all parties in the design of a particular system or category of systems?" (p.492).
Where direct involvement of users is not adopted, there remain other ways in which designers can gain some understanding, if not deep appreciation, of the perspectives of users and usees. These are addressed in the later section on engagement.
One of the most remarkable aspects of the Reports of the Ombudsman, the Senate Committee and the Royal Commission is the almost complete absence of mentions of the staff within DHS who were responsible for data management and data quality, and for process design and quality assurance.
The Royal Commission's 'Dramatis Personae' (RCR 2023, pp.xiii-xxi) lists 138 people, of whom not one among DHS staff has Data/IS/IT professional background or direct responsibility for the primary Branches where those staff work. Two individuals are mentioned in the text but omitted from the list. One was an unnamed "contractor [to DHS] who was working as a test manager in the __Enterprise Testing Branch", who reported that "__further robust/vigorous testing and analysis of more complex customer scenarios [need to] be undertaken for this task". His contract was revoked within a few hours of him delivering that report. The other was a Director (the most senior level of management outside the executive grades), Gary Clarke, "DHS director of Capability Delivery Management", who supported the contractor's report with empirical evidence (RCR 2023, pp.129-130). No information is available as to whether he retained his job; but, 6 years later, no entry for a person of that name is found within the executive levels of the Australian Public Service.
Further, among many scores of meetings whose participants were mentioned, it appears that none were ever present who had responsibilities for, or apparent professional background in, business analysis, or data, information or IT management. Nor was there any evidence of any participant making reference to information provided by any employee or contractor with professional background of that kind.
DHS is an agency whose business processes are immensely dependent on the quality of data, data modelling, requirements elicitation, process design, process quality assurance mechanisms, safeguards and controls, all of which are dependent on Data/IS/IT professionalism. The agency had been a world leader in computerisation of welfare payments for over 50 years. Yet it appears that the senior executives and policy executives in DHS had lost track of that fact, and that dependence. Moreover, this oversight by DHS was so comprehensive that the Royal Commission staff appear never to have thought to ask questions about the input of Data/IS/IT professionals or even their senior managers.
DHS executives also successfully avoided consultancy firms delivering information that was inconsistent with the Minister's desires and the agency's strategems. Only two relevant consultancies have come to light. "[CSIRO's Data61 Division conducted a brief assignment] to deliver general advice on optimal algorithms to be used for the outcomes intended by DHS ... [Of the 43-strong slide-set], only slides 25-29 [12%] ... were drafted by Data61, with the remainder [88%] being drafted by the department" (RCR 2023, pp.228-229). In addition, "[In mid-February 2017, DHS contracted PwC to conduct] an independent review of the compliance and fraud activities of__ DHS, culminating in identification of areas for improvement, together with a high-level implementation plan [for AUD 850,000] ... [DHS Dep Sec] Ms Golightly sent numerous, lengthy emails requesting changes to the presentation ... [By May 2017,] The report confirmed, in writing, that the Scheme as it existed was a failure. [It] had been the product of 34 previous drafts and spanned almost 100 pages, [but Secretary Campbell contrived to block the report, and it] was never formally received by DHS" (RCR 2023, pp.229-237).
The conception, requirements elicitation and analysis, design, coding, testing, implementation, review, and adaptation of IS that involve complex data and processes, require substantial competencies in a wide variety of IS and IT fields. Large organisations hire many staff and contractors, and let many consultancy assignments, to ensure access to the relevance competencies. Professional job designations such as business analyst, enterprise data analyst, enterprise architect, solutions architect, quality manager and risk manager are particularly relevant. Those staff are commonly managed by senior staff with at least some IS professional background. Designations for peak roles such as Chief Information Officer (CIO), Chief Data Officer (CDO) and Chief Information Security Officer (CISO) are commonplace in large organisations. If executives exclude those voices even from policy discussions relating to large-scale projects, but particularly from their development and operational stages, those executives are flying blind, making decisions without being adequately informed, and hence derogating their duty.
Another consideration relates to the ethical obligations of staff, contractors and consultancies. There is no professional association for government executives and senior managers. However, there are requirements of all employees of the Australian Public Service, under s.13 of the Public Service Act 1999 (Cth), documented in the APS Code of Conduct (APSC 2022). On the other hand, there is considerable public scepticism about the effectiveness of this framework, particularly given the apparent absence of training materials and jurisprudence. Further, it would appear that there is no institutionalised framework assigning substantial responsibilities to executives, and far less is there any visibility of enforcement of such a framework.
A special category of public sector employee is in-house legal counsel, who are subject to formal requirements as lawyers. That degree of independence is intended, or at least claimed, to weaken the power of agencies to give instructions about the substantive content of the legal advice that they provide. The Royal Commission's Report provided evidence of some compliance with those formal requirements, but also of a considerable number of quite gross breaches of them. It remains to be seen whether any disciplinary action will be forthcoming, whether by the Commonwealth or by the professional bodies whose registries enable the individuals concerned to practise law. Once again, there is considerable public scepticism as to whether any such outcomes will eventuate.
In the IS field, professional responsibilities are weaker, because of the lack of compulsory registration or licensing schemes. Nonetheless, the Codes of Ethics of relevant professional associations exist, are used in professional education, and are used as evidence in courts. The same applies to industry standards for a range of business processes within the IS and IT fields. These provide scant protection for those employees, and still less for those contractors, who perceive the need to deliver information that agency managers and executives are trying to avoid receiving. Nonetheless, Codes of Ethics and industry standards provide benchmarks that each such individual can use to test their own consideration of relevant principles and their applicability to situations that they are confronted with. Professionalism in Data and IS practice needs to be encouraged by the institution of the public service, and executives need to be required by law to consider information provided by those professionals.
At the heart of the failure of the OCI project was an inappropriate algorithm, whose dysfunctionality, even when it became impossible to ignore, did not trigger reconsideration. Different approaches to error management are appropriate depending on the means used to generate inferences from data. The OCI used what is usefully referred to as a 'genuinely algorithmic' inferencing technique - the simplistic algorithm being, in many cases, division by 26 of what DHS presumed to be an annual figure, to generate a putative fortnightly average.
More sophisticated, and far less transparent, approaches to inferencing bring with them higher risk that inappropriate decisions and actions will be remain undiscovered. Rule-based expert systems apply sets of rules to the facts of cases. A current vogue applies Machine Learning aspects of Artificial Intelligence inferencing techniques (AI/ML), in particular artificial neural networks, and uses 'training-sets' to produce a set of weightings that are then applied to data representing a particular instance. The new phenomenon of 'generative AI', based on Large Language Models (LLM), results in even more opacity and unexplainability, and grave risk exists that is will generate more errors than ever before, at the same time undermining the accountability of decision-makers, and thereby exposing even more members of the public to unremediable wrongs.
Attempts to produce a general set of guidelines for responsible inferencing that applies across all of those categories are confronted by considerable challenges. One categorisation of ways in which "automated assisted decision-making may well be productive of deficient reasoning" is provided by Hogan-Doran (2017, pp. 14-15), where the inferencing process may be defective for any one or more of many reasons, including where it:
For 'genuinely algorithmic' techniques, such as the trivially simple one used in Robodebt, analyses of sources of error exist, as do guidelines on how to avoid those errors (e.g. Clarke 2017). Such tools, in applying procedures to data in order to draw inferences, express a solution and at least imply a problem-definition. These approaches to inferencing therefore support the important feature of explainability. Rule-based expert systems, on the other hand, merely define a problem-domain. Inferences are drawn for individual circumstances, but there is no explicated problem or question. Whereas algorithmic schemes inherently embody a rationale, and enable expression of a humanly-understandable explanation, rule-based systems require investment to deliver explanations and thereby enable decision review.
The AI/ML and generative AI approaches, contrary to popular expression, are not algorithmic in nature. They are purely empirical, creating weightings on factors on the basis of whatever prior instances are fed into them. ML inferencing is a-rational, and does not provide a basis for explanations of how inferences were drawn. Variants of 'The established weightings were applied to the data, resulting in the output' are not usefully different from 'The computer says "no''' (Clarke 2019, pp. 425-426). The challenges involved in recognising error in AI/ML-based inferencing are accordingly far greater. Much effort has been invested into 'explainable AI' (XAI), but to date little evidence exists that any general solution will ever be feasible (Meske et al. 2022).
In discussions of errors in AI/ML applications, the misconceived term 'algorithmic bias' is much-used. Terms such as 'model bias', 'data bias' and 'socio-cultural bias' are also used as loose classifiers; but no comprehensive categorisation of problems appears to have been settled upon. See, however, AHRC (2020). Banks & London (2017), for whom algorithmic bias exists when a data process is "not merely a neutral transformer of data or extractor of information" (p. 1), distinguish the following elements:
Despite the remaining fog, comprehensive guidance on responsible application of AI/ML is available from a variety of sources, notably Zeng et al. (2019), Clarke (2019b) and Jobin et al. (2019). These crucially involve real-world checks of the efficacy of the inferencing technique used. No evidence was found that DHS applied the necessary professionalism to ensure that sources of error were examined and understood, and that design features were incorporated to deal with them. It is essential that organisations undertaking projects with substantial impacts on and implications for users and usees take advantage of the ample guidance available on error management.
Depending on the jurisdiction, there may be legal or merely moral constraints on the conduct of processes by government agencies that may have detrimental effects on people. Terms such as 'due process' and 'procedural fairness' are applied to constraints of this nature (Citron 2008). Key aspects of procedural fairness are symmetry between the parties in relation to information, power and resources; participation in the process; submission of evidence and argument by the parties; opportunity to respond to evidence and argument by the parties; decision-making based on evidence; and impartiality of the decision-maker.
The Senate Committee concluded that OCI embodied a fundamental lack of procedural fairness. See Appendix 1-6-1. For one specific example, see Appendix 1-6-2. This reflects an element of contemporary data capture designs that is in effect tyrannical: Mandatory-response items in web-forms force users to make an arbitrary choice among inapplicable options (hence breaching their obligation to provide accurate information), in order to get past the obstacle(s) and achieve submission of the form. In the hands of a skilled (and hence expensive, and hence perhaps inaccessible) lawyer, this alone may have been sufficient to sink the OCI program beneath the thin ice on which it had been constructed and over which it skated for more than 3 years.
Even if system sponsors are not subject to legal or public policy constraints in relation to procedural fairness, or are willing and able to ignore them, they are well-advised to consider the possibility that apparently weak stakeholders may in fact wield considerable power, or have it wielded by others on their behalf, in such a way as to deny the benefits that the agency seeks, or even destroy the project entirely. In this case study, the volume of individuals who were obviously being subjected to unreasonable behaviour reached critical mass, such that media reports and the efforts of several small community legal centres eventually defeated one of the nation's largest organisations.
The comfortable presumption could be made that 'rational management' precepts lead to standards and best practices being applied. This case study has shown that cannot be relied upon. Alternatives are to presume that executives will make decisions in the best interests of the organisation, or in the best interests of themselves, and hence all organisations and/or all executives need to be subject to sanctions if they breach laws, and if they intentionally breach standards or good practice and thereby cause material harm.
Arguments can also be made for adopting an ethical or perhaps humanistic perspective. For example, some advocates invoked the notions of human rights and human dignity. From the evidence provided to the reviews, little or no sense of empathy for DHS clients was apparent at any stage. The project's public announcement referred to a 'welfare cop on the beat' and 'welfare fraud and non-compliance', and language of this ilk dominated the whole life of the project, to the point of excluding any appreciation of the needs of the >95% of clients whose behaviour was not criminal or even non-compliant. In the case of suicide-victim Rhys Cauzzo, referred to earlier, indicators of the absence of empathy embodied in both the agency's information system and its organisational values included the following:
Two particularly critical elements of procedural fairness are the existence of a rationale and data underlying each decision, and the provision of clear information about them to each accusee. The Senate Committee asserted that "it is important recipients ... are given adequate information as to how their purported debt has been calculated" (SCA 2017, p. 39). For evidence from one welfare recipient, see Appendix 1-6-3.
This had broader implications, with community legal services that were "unable to understand how their client's purported debt has been raised [having to resort] to Freedom of Information (FOI) requests in an attempt to gather information relating to their client's purported debt" (SCA 2017, p. 39).
It is not common for legal rights to exist requiring explanations for decisions, in either the private or the public sector. As datafication takes hold, and as automation is imposed, calls are being articulated for such rights to be created, in order to enable the identification of administrative blunders, and to provide people subjected to them with some ability to fight back against poor systems design and poor decisions (Wachter & Mittelstadt 2019).
As discussed earlier in this section, explainability is a feature of 'genuinely algorithmic' inferencing techniques. The approach used in this project was explainable - and had the explanation been publicly exposed, it would have been entirely clear that the approach was seriously inadequate. It is more challenging to deliver explanations when using rule-based systems, and it is essentially infeasible to do so where AI/ML or generative AI are applied. Their inherent a-rationality leads to opaqueness instead of transparency. Use of AI-based inferencing therefore undermines the essential qualities of procedural fairness in administrative schemes, of replicability, of auditability, of correctability, and hence of accountability (Clarke 2019a, Michael et al. 2021). The current, widespread and largely blind adoption of AI-based inferencing raises the prospect of many future disasters for users and usees, the need for vigorous resistance by individuals and their advocates, social media vilification campaigns, critical media coverage, substantial reputational harm, and for some system sponsors outright defeat, IS project failure, and loss of the entire investment.
The Senate Committee's Report was crystal-clear on the responsibility of DHS to recognise the power imbalance and exercise its powers with due care. See Appendix 1-6-4. It appears unlikely that many jurisdictions make illegal the irresponsible exercise of government power. On the other hand, such behaviour may be inconsistent with public policy, and it may be harmful to the standing of government agencies with the public. Many government agencies in many countries may be impervious to the concerns of stakeholders, or to public opinion generally. Some may, however, find that the morality of the behaviour of members of the public in their dealings with the agency declines, perhaps precipitously, and perhaps uncontrollably, if the agency is seen to be arrogant and entirely unconcerned with its stakeholders' interests. Further, when a Government, while addressing inefficiency and waste, and investigating and prosecuting criminal behaviour, demonises welfare clients as 'dole-bludgers' and 'welfare cheats', it is very likely to lose credibility, resulting in a fall in the morality of public behaviour.
It is also possible that irresponsibility in dealing with vulnerable population segments may exacerbate negative impacts on the credibility of government agencies and of Governments. The OCI system's targets were people who depend on social security payments for housing and basic necessities like food and paying energy bills and who by definition lack assets, lack steady incomes, and work casually, reliant on sporadic work often at short notice, sourced in many cases from multiple employers, resulting in unpredictability in earnings. Even the agency-sympathetic Ombudsman's Report drew attention to "the risk of over-recovering debts from social security recipients and the potential impact this may have on this relatively vulnerable group of people" (Omb 2017, p. 8). The Senate Committee was "particularly concerned about the impact that debt repayments are having on income support recipients who are on very low incomes and former recipients who may be on very low wages" (SCA 2017, p. 97). In countries where large proportions of the population are dependent on social welfare, the scope exists for populist or extremist parties to attract a lot of votes by highlighting the Government's irresponsible behaviour.
The Royal Commission was particularly critical of the publication of personal data of a complainant, which was done with the express intention of chilling speech, leading to "less people speaking out in the media": "As a minister, Mr Tudge was invested with a significant amount of public power. Mr Tudge's use of information about social security recipients in the media to distract from and discourage commentary about the Scheme's problems represented an abuse of that power. It was all the more reprehensible in view of the power imbalance between the minister and the cohort of people upon whom it would reasonably be expected to have the most impact, many of whom were vulnerable and dependent on the department, and its minister, for their livelihood" (RCR 2023, p.179).
Robodebt used a simplistic, algorithmic inferencing mechanism, which was readily capable of being analysed. Much more serious concerns arise where automated decision-making and action are based on opaque inferencing methods, because they deny procedural fairness and natural justice. Whether or not a jurisdiction makes illegal the irresponsible exercise of government power, it needs to impose obligations and sanctions on Ministers, agencies and agency executives in relation to the documentation and investigability of the rationale underlying their decisions and actions.
Risk assessment and risk management (RA/RM) are well-established processes (e.g. NIST 2012, ENISA 2016, IEC 31010:2019). Their purpose is to ensure that difficulties that might beset an organisation are anticipated, and variously deterred, prevented, detected and mitigated. One of the key responsibilities of governing boards and senior executives is to ensure the effective use of these techniques. However, the Ombudsman found serious shortfalls in fulfilling those obligations. See Appendix 1-6-5.
Despite a risk management plan having been written, there is little evidence of safeguards and mitigation measures having been designed-in or even retro-fitted. Hence any Risk Assessment element of OCI that may have been performed appears to have been no more than a token effort or box-ticking exercise. The Senate Committee concluded that the "procedural fairness flaws" alone were so serious that its Recommendation 1 was that the OCI scheme be suspended (SCA 2017, p. 108). DHS and the Government ignored that call (along with all of the other Recommendations) and continued the scheme for a further 2-1/2 years, prolonging the agony for hundreds of thousands of benefits recipients and thousands of staff-members.
For public sector organisations, however, RA/RM may not be enough, because RA/RM is fundamentally concerned with the interests of the organisation itself. The interests of other stakeholders are relevant in RA/RM only to the extent that those other stakeholders have sufficient power that they represent a threat to fulfilment of the organisation's purposes and hence need to be managed as a risk (Mitchell et al. 1997, Achterkamp & Vos 2008). Particularly for government agencies that recognise as an objective the social and/or economic welfare of disadvantaged people, RA/RM may therefore be a too single-minded technique.
Various approaches can be adopted to reflecting the interests of users and usees, including technology assessment, and environmental, privacy and social impact assessment. However, these are commonly resisted by organisations. An alternative approach that offers promise is expansion of the scope of RA/RM, such that the organisation's understanding is complemented by insights from the perspectives of all stakeholders with material interests in a project's outcomes, not only if they have power, but also if they lack power but their interests nonetheless have legitimacy. An outline of such an alternative approach, Multi-Stakeholder Risk Assessment (MSRA), is in Clarke (2022).
A parallel, and somewhat complementary, path has been traced to that of socio-technical design. Participative analysis and design techniques (Hirschheim 1985), in recent years rediscovered and renamed 'co-design' and 'co-creation', seek better results for system sponsors and other stakeholders alike, through direct involvement of affected parties in the process of developing new systems and adapting existing systems (Sanders & Stappers 2008, 2014). If this is too avant garde for conservative government agencies to contemplate, a slimmed-down approach can be used to gain insights into the perspectives of users and usees. This involves providing sufficient information to advocacy organisations to enable them to engage with the agency on the aspects of greatest concern to the individuals on whose behalf the advocates speak.
In the case of DHS and OCI, it appears that the benefits of transparency and engagement were forgotten, or even suppressed. Insights were needed in relation to the interests and the capabilities of the relevant categories of social welfare recipients. To some extent this required direct access to samples of individuals in each segment, both in focus groups and in laboratory-testing and beta-testing of UI/UX designs. Broader questions are likely to be challenging for individual benefits recipients to understand and address, and some proportion of them may not be able to appreciate the need for a balance between service and control objectives. Such questions are likely to be better addressed through engagement with advocacy organisations. Even if the agency lacked a reference group of such organisations, a suitable pool was very simple to develop. During the Senate Committee's first hearings (SCA 2017), for example, 62 such organisations provided submissions. See Appendix 1-6-6.
The failure of DHS executives in this regard appears likely to have played a major part in both its original blunders in conception and design, and its 2-3 years of denial of what was by then blindingly obvious to the rest of the country. A key lesson from this case study is that even minimalist attention to stakeholders' views can save a great deal of trauma for individuals, and large sums of taxpayer funds.
The authoritative sources of information used in developing this paper contain accusations against named individuals. This case study provides express, detailed information, in order to support the argument that the negative impacts of the Robodebt scheme were exacerbated by a substantial absence of accountability. The authors contend that it is essential to include the names of the individuals whose behaviour was primarily responsible for the harm caused. The whole point of a case study is to provide a graphic sense of the reality that it depicts, and avoid the flattening out of serious subject-matter in bland, fleshless prose.
The Tasmanian Council of Social Service voiced the widespread concern about the absence of leadership, and this was echoed by the Federal Court judgment, 4 years after the event. See Appendix 1-6-7. At the organisational level, DHS suffered no retribution for its breach of the Legal Service Direction. The judge noted the benefits of a class action. See Appendix 1-6-8. However, that required funding, and it was very expensive, and the 'damages' that were secured were far from enough to either recompense those who had been wronged or to represent a material disincentive.
At the political level, a social welfare scandal in The Netherlands involved only 26,000 individuals, 6 percent of the number in the OCI debacle. Yet, when the Report was tabled in January 2021, the principle of governmental accountability was respected, and the Prime Minister and the entire Cabinet of The Netherlands resigned (Erdbrink 2021). In Australia, on the other hand, the notion of Ministerial responsibility has all but disappeared. Evidence presented to the Royal Commission in late 2022 suggests that the proposal originated in the policy agency, the Department of Social Security, or with its Minister at the time. Internal legal advice was ignored, and no external legal advice was acquired, enabling funding to be approved in the 2015 Budget for implementation by the operational agency (Bannister 2022, Morton 2022). The Minister for Social Security at that time, Scott Morrison, appears to have avoided receiving legal advice, or to have been shielded from it, enabling him to instigate a scheme that caused substantial harm to welfare-recipients, and reputational and financial disaster for the agency and the Government. However, his career was clearly not hampered. He moved to the more senior position of Treasurer at the end of 2015, and was subsequently Prime Minister 2018-22.
Meanwhile, during the period 2015-22, the position of Minister for Human Services was recognised as carrying with it considerable negative publicity, and five Government front-benchers were shuttled into and out of the position a total of six times during the seven years. The Ministers were Marisa Payne (during the project's conception and approval), Stuart Robert (with IT industry background, but also with a track-record of parliamentary misbehaviour, for 4 months in the preparation period, and again for 1 year 10 months during the period of denial and collapse), Alan Tudge (also with a record of misbehaviour, for 1 year 10 months, during the early operational period), Michael Keenan (1 year 5 months, during the late operational period, who retired from parliament, handing the chalice back to Stuart Robert), and Linda Reynolds (for the last 14 months of the dying Coalition Government). Each had ample opportunity to investigate, appreciate the problematic nature of the project, and take action to stop it. None did. Before the Royal Commission, all of the Ministers presented excuses for misrepresenting and obfuscating the legality, the effectiveness, and the harm done by, the scheme. Minister Robert, for example, declared under oath that false public statements were necessary because "I have no choice as a Cabinet Minister but to defend government policy" (RRC 2023b, pp.P-4219, 22). All were culpable; yet none have suffered any personal consequences for their behaviour in relation to Robodebt, and it appears unlikely that any of their offensive behaviour constitutes an actionable offence.
As regards the culpability of the senior executives of DHS, the Royal Commission produced considerable insight. As early as 18 December 2014, a senior executive in the policy agency DSS, Andrew Whitecross, sent to relevant DHS senior officers a summary of legal advice that (RRC 2022c, p. 1351, emphasis added):
"A smoothing [of] employment income over an extended period does not accord with social security legislation, which specifies that employment income is to be assessed fortnightly, and it follows the debt amount calculated would not be supported by law"
The receipt by DHS of that advice was expressly denied by the relevant senior executives of DHS, Withnell and Campbell. No evidence has ever emerged to clarify whether that advice was passed to DSS's own Minister, Scott Morrison.
The email from Whitecross also stated that (p. 1352, emphasis added):
"it is unclear whether the proposed algorithm could accurately identify those who did not correctly declare income during the period when they received income support, and there would be particular difficulties where a person has been in and out of employment and subsequently [sic: consequently] on and off income support payments ... Debts calculated in this manner would be likely to be overturned by a tribunal"
The two most critical features underlying the proposed scheme's failure - its illegality and its dysfunctionality - were therefore clearly communicated to DHS during the conception stage, but the senior executives chose to ignore these serious deficiencies.
Chief Legal Counsel for DHS throughout the relevant period, 2015-19, Annette Musolino, absolved herself and her Legal Services Division of any responsibility, despite doing nothing to provide what she acknowledged was necessary advice to agencies, successive senior executives and successive Ministers (Musolino 2023, p. 28). The senior DHS executives, Deputy Secretaries Malisa Golightly, who had direct responsibility for the scheme, and Jonathan Hutson, in a 2017 email-chain disclosed to the Royal Commission, could be seen actively avoiding the acquisition of legal advice on the legality or otherwise of the scheme. They also sustained the calculatedly false public position that "we haven't changed the way we do the data matching or the way we assess or calculate the debts, ie, we are doing it the same way we have done it for many years" (RRC 2022b, pp. 1190-1194). Hutson appears to have retired in July 2018. It is unclear from the public record when Golightly left her role, but she passed away in late December 2021. It appears that Musolino was promoted to one of those vacancies in 2019. As late as early 2023, Musolino remained one of DHS's most senior executives.
Jason McNamara, DHS General Manager, Debt and Integrity Projects, was pressed at length by the Royal Commission on the extent to which he authored segments of text included in the Ombudsman's Report (RRC 2022a). This arose because, in a job application for the position of Deputy Secretary within DHS, McNamara had declared "... I shaped the Ombudsman report" (p. 1083), and that the Ombudsman's Report was then used to counter negative press, describing it as an "external reference that was very useful" (p. 1078). He continued in his senior executive role at DHS 2017-21.
The overall responsibility for the relevant activities, for the entire period 2011-21, was Chief Executive Kathryn Campbell. She had also sustained a part-time military career, and was promoted in 2018 to command the entire Army Reserve, while remaining at DHS. In January 2019, Campbell was awarded one of the nation's highest honours, the Order of Australia (AO), almost exactly 2 years after the disastrous nature of Robodebt had become apparent to everyone outside DHS - although still 10 months before the Government accepted it was a failure and closed it down. In mid-2021, she was promoted to one of the most highly-ranked roles in the public service, Secretary of Foreign Affairs and Trade, rejoining her Minister during 2013-15, Marise Payne. She was Secretary there for 12 months, until shortly after the Labor Government won office in May 2022, when she was appointed to a highly-paid Defence Department sinecure. Immediately after the publication of the Royal Commission's Report in July 2023, she was suspended without pay, and shortly afterwards resigned from the public service.
It remains to be seen whether an as-yet unpublished volume of the Royal Commission's Report will result in any material action to sheet home responsibility for the unlawful behaviour of one of the largest government agencies in the southern hemisphere. Golightly having died in 2021, the possibilities exist that the Secretary, Campbell, and two other senior executives, Withnell and Musolino, may be charged with offences or, in the case of Chief Counsel Musolino, be deregistered as a lawyer. In the interim, however, all of them have prospered, as have other executives whose behaviour was castigated but which appears unlikely to be found to be criminal.
The most parsimonious explanation of how the debacle arose appears to be that the scheme was hatched by the Minister for Social Security during 2014-15, Scott Morrison, despite advice against it by senior executives in his own policy agency, DSS. It was pursued by the Minister for Human Services during 2013-15, Marise Payne, and senior executives in the operational agency DHS, from late 2014 onwards. Morrison, next as Treasurer 2015-18 and then as Prime Minister 2018-22, urged and then supported the project's development and implementation. The rapid succession of Ministers for Human Services, and the senior executives of DHS, did the Cabinet's bidding 2015-19, maintaining the public lie that it was a legal and functional approach to raising debts, until court action eventually forced its withdrawal.
Evidence to the Royal Commission in December 2022 by, in all, eight Ministers, was obfuscatory, particularly that by Morrison and Payne, and all sought to lay the blame on DSS and DHS executives. Evidence by DHS chief executive Kathryn Campbell in December 2022 and February 2023 was similarly obfuscatory. When forced to give evidence to a Royal Commission, eight Ministers, the five most senior executives and several other executives, pointed crosswise at one another, laying the blame anywhere but themselves.
What is sometimes termed 'algorithmic fallout' afflicts people who are subjected to automated processes imposed by ill-advised transformative digitalisation projects. Noble (2018) argues that 'algorithms of oppression' give rise to societal burden (undue feelings of anxiety and distress), distrust in the effectiveness and operationalisation of AI-based systems (such as automated surveillance-based welfare systems), attacks on personal character (causing personal shame, hurt and anguish), and suicidal ideation and suicide. The contrasting impacts on careers in The Netherlands and Australian cases raises the question as to whether other countries' Ministers of State, and government agencies and their senior executives, will suffer reputational harm from their association with mis-conceived, harmful and failed IS projects, or enjoy a protected existence like those in Australia, where both incompetence and cavalier disregard for the impacts on users and usees are being tolerated and even celebrated.
This case study has drawn on authoritative sources to analyse a large-scale exercise in transformative IT applied to the administration of social welfare in Australia, which operated 2014-19, was dismantled in 2019-22, and was subjected to multiple formal reviews 2017-23. The final Report from the Royal Commission confirmed and provided some further articulation of the analysis reported here.
The contribution of this case study is enhanced understanding of the challenges involved in digitalisation in the public sector. It provides deep and multi-layered analysis of the factors that resulted in a large-scale, intentionally transformative project failing to achieve its objectives and causing enormous collateral damage. This depth was achieved by providing a narrative description, followed by examinations of 18 themes that emerged from the substantial and authoritative documents on which the case study depended. Briefly, a massive deficiency at the heart of the technical design escaped into live operation, and did serious harm for over 3 years, because wilful blindness and deafness caused a long series of process safeguards to fail. The case study is a moving picture from real-life that represents refutation of the comfortable assumptions that standards, 'good practice', legal compliance and training are all that are needed to avoid IS failures and harm to parties affected by systems.
One of the largest government agencies, in a large first-world country, with long and deep experience in the application of IT to social welfare administration, was so beguiled by the promise of transformative digitalisation that it abandoned its responsibilities, and reverted to mindless, technologically-determined design, causing massive dislocation and harm to the most disadvantaged members of society. The agency then refused for 3 years to accept that the project was disastrous in conception, design, execution and effect, prolonging the agony. At no stage did the decision-makers exhibit any respect for, or empathy with, the people on whom they were inflicting significant and unjustifiable confusion, effort, cost and anxiety. The summation of the Royal Commission was that "Robodebt was a crude and cruel mechanism, neither fair nor legal, and it made many people feel like criminals. In essence, people were traumatised on the off-chance they might owe money. It was a costly failure of public administration, in both human and economic terms" (RCR 2023, p.xxix).
None of the six design aspects, three process aspects or nine broader issues involve startling new revelations. For 14 of them, well-known principles exist, which were ignored. Even in respect of the three that have some 'bleeding-edge' features (ss.5.1(f) Automation, 6.5 Explanation and 6.7 Risk Assessment in relation to Automation), some sources of guidance are available. The deficiencies in Ministerial and senior executive behaviour catalogued in this case study were numerous. The motivations may have been political, in the sense of the imposition of an ideological view during a particular party's window of opportunity in government. The case study's purpose has been to deliver understanding about the means whereby the development and implementation of the scheme overrode nominal safeguards. It is a reminder to public sector agencies and IS practitioners everywhere of the reasons why socio-technical (rather than merely technical) design, risk assessment and risk management, transparency and engagement, and professionalism, are all so essential to agency activities. Effective guardrails must be in place and operative, to prevent decisions made in comfortable offices in capital cities, remote from the people affected by those decisions, going beyond the impersonal and approaching the inhumane.
More generally, the increasing power of IT has not been matched by advances in the quality of data and the quality of inferencing. Instead, both quality and rationality have been seriously reduced, even where humanly-understandable, genuinely algorithmic data analytics are used. The much-touted, but opaque and a-rational magic of empirical techniques such as AI/ML and generative AI, threaten to further lower the quality of data and inferencing, and to undermine replicability, auditability, correctability, and hence accountability.
This case study has recounted how an Australian agency irresponsibly ran the Online Compliance Intervention project, without direct accountability by the relevant Ministers, or by agencies, or by the agency's senior executives, and without any material retribution once their gross misbehaviour was revealed. Where government IS projects are permitted to be conducted in such a manner, public confidence in both agencies and IT is likely to be seriously harmed, and public scepticism, sullen non-compliance and active opposition to initiatives are likely to become engrained.
The Government has zero tolerance for rorting of our welfare system. We will put a strong welfare cop on the beat focusing on deterrence, detection, investigation and prosecution to track down suspected welfare fraud and non-compliance. From 1 July 2015, the Government will increase DHS's capability to detect, investigate and deter suspected welfare fraud and non-compliance. ... It will achieve net savings of around $1.5 billion. (MRR, 2015)
The key concern with the OCI process, is the outsourcing of the income checking process to individuals. With this comes an inherent reversal of the burden of proof - the department claims an income discrepancy and requires an individual to seek the information required to prove the discrepancy does not exist. If the individual fails, they will owe a debt of potentially many thousands of dollars to the department. The two fundamental resources a person needs to undertake this process is a method of communicating, and once that communication channel is opened, the receipt of information that is both comprehensive and comprehendible [sic]. The department is clearly failing to provide those two necessary tools to allow people to challenge the income discrepancy, and is reaping the benefit through debt payments. (SCA 2017, p. 69)
One thing ... that stands out from the objections is the financial hardship, anxiety and distress, including suicidal ideation and in some cases suicide, that people or their loved ones say was suffered as a result of the Robodebt system, and that many say they felt shame and hurt at being wrongly branded "welfare cheats". Some of the objections were heart-wrenching and one could not help but be touched by them. (Pry 2021, p. 7)
More than 2030 people died after receiving a [OCI] debt notice ... Of those, 429 - roughly one-fifth - were aged under 35. The figures cover a period from July 2016 to October 2018 ... there were 3139 deaths of people aged between 15 and 35 in 2016 overall. (Medhora 2019)
The committee recommends all data-matching guidelines and protocols be adhered to, including the Data-matching Program (Assistance and Tax) Act 1990, regardless of whether the department is using tax file numbers. This will require the department to halt the Online Compliance Intervention process while steps are taken to ensure compliance ... . (SCA 2017, p. 109)
`Averaging' refers to the practice of treating income as if it was earned at a consistent rate over a total period of employment rather than applying the precise amounts against the fortnights in which the income was actually earned. DHS currently applies averaging where a person accepts the PAYG data or does not enter data for all fortnights. (Omb 2017, p. 7, fn13)
[This is a] labour market in which people are trying to work and comply with their Centrelink requirements, ... a market in which people get bits and pieces of work; work irregular hours and often spend periods of time across a financial year out of the workforce. This leads to it being way more complicated and extremely onerous to comply with a ... system that assumes that people either have or do not have a job across a financial year. (SCA 2017, p. 35)
In the course of the proceeding the Commonwealth admitted that it did not have a proper legal basis to raise, demand or recover asserted debts which were based on income averaging from ATO data. The evidence shows that the Commonwealth unlawfully asserted such debts, totalling at least $1.763 billion against approximately 433,000 Australians. Then, including through private debt collection agencies, the Commonwealth pursued people to repay these wrongly asserted debts, and recovered approximately $751 million from about 381,000 of them. (Pry 2021, p. 1)
[I]t is fundamental that before the state asserts that its citizens have a legal obligation to pay a debt to it, and before it recovers those debts, the debts have a proper basis in law. ... Having regard to that, and the profound asymmetry in resources, capacity and information that existed between them and the Commonwealth, it is self-evident that before the Commonwealth raised, demanded and recovered asserted social security debts, it ought to have ensured that it had a proper legal basis to do so. The proceeding revealed that the Commonwealth completely failed in fulfilling that obligation. Its failure was particularly acute given that many people who faced demands for repayment of unlawfully asserted debts could ill afford to repay those amounts. (Pry 2021, pp. 3-4)
[T]he process of checking the ATO lump sum income records against the department's fortnightly income records, a time-consuming process previously undertaken by departmental personnel, has [as part of OCI] been outsourced to the individual income payment support recipients. (SCA 2017, p. 2)
"Recipients are [now] directed to an online portal to check the information and provide supporting evidence of their fortnightly income, dating back to 2010 for some people" (SCA 2017, p. 17)
The committee is concerned about the shift in the onus from the department to the individual recipient to verify whether or not a purported debt exists. The committee is particularly concerned that individuals do not have access to the same resources and coercive powers as the department to access historical employment income information. (SCA 2017, p. 19)
The Committee identified psychological, financial and time impacts; the change in the time when checking records held by the department is done; and the absence of a basis in law (SCA 2017, pp. 79-84).
The committee accepts that challenging these purported debts has taken considerable effort on behalf of those individuals. The committee notes that no other party is entitled in law to assert that a debt exists and require the other party to disprove it (SCA 2017, p. 84)
[W]here a purported debt is raised against an individual, the committee considers that the department should be forthcoming with the calculations that demonstrate how the debt was arrived at. The committee considers that these factors contributed to the increased number of requests for assistance received by community legal centres. Funding for these legal centres remains a substantial issue. (SCA 2017, p. 89)
"The committee considers that it is important that calculation of debts is based on complete and accurate information, and that the fluctuations in recipient's income, particularly if they are employed on a casual or part-time basis, should be closely reviewed before issuing a debt notice" (SCA 2017, p. 36).
The project management team failed to ensure that key external stakeholders were effectively consulted during key planning stages. It also failed to effectively communicate with stakeholders after the full rollout of the OCI in September 2016, resulting in confusion and inaccuracy in public statements made by key non-government organisation (NGO) stakeholders, journalists and individuals (Omb 2017, p. 23)
[A] key lesson for government agencies and policy makers when proposing to rollout large scale measures which require people to engage in a new way with new digital channels, is for agencies to engage with stakeholders ... . (Omb 2017, p. 24)
... lack of procedural fairness is evident in every stage [from] the drafting of the policy [through] the testing phase ... in the failure to carry out a risk assessment before the process started. In sending letters without checking addresses and taking a lack of response as a refusal to engage. In the averaging of income data, which invents a fortnightly income-earned sum for the purposes of then charging people with a debt knowing full well it is going to be wrong. In the millions of calls that went unanswered ... In the lack of information released to individuals which they required in order to challenge a debt ... in the institution of a debt recovery program reaching back six years, despite online departmental advice that welfare recipients need only retain records for six months. A lack of procedural fairness is evident in all these stages. The system was so flawed that it was set up to fail. This lack of procedural fairness disempowered people, causing emotional trauma, stress and shame (SCA 2017, p. 107)
I sat with my daughter--and I have worked for the public sector for many years--and attempted to go through the questions which were in this form with her. Some of the questions were nonsensical. I had no idea what was being asked. You cannot progress unless you answer the question, so people are making a guess. They are putting in whatever information in order to get the form completed. They do not understand. (SCA 2017, p. 41)
That figure of my debt of $3,154.11 was remarkably precise, but if there was any kind of detailed computation behind it I have never seen it. I have even pulled an FOI on my case and I cannot make head or tail of how that figure was arrived at. They came up with this figure, but they provided no accounting for it and they provided no explanation, initially, as to how it arose. They just said, 'Here's your debt; pay it or prove you don't owe it'. (SCA 2017, p. 71)
As the provider of social security in Australia, the department holds a position of power in its recipients' minds, and the power imbalance this creates cannot be underestimated when considering individuals' reactions to the OCI system. (SCA 2017, p. 36)
There is a significant power imbalance between income payment recipients and the department, and communication therefore does not take place on a level playing field ... In discussing the impacts of this power imbalance, where individuals tend to assume 'the department is right', organisations pointed to a number of adverse outcomes for individuals, such as people paying purported debt notices without question and people accepting pre-filled income data that averages their income without checking it for accuracy, leading to incorrect debt calculations. (SCA 2017, p. 41)
[The Committee has] concerns that the OCI system has created a climate of fear where recipients of letters feel they cannot challenge the information provided by the department or risk losing the financial safety net which the department provides. (SCA 2017, p. 34)
"DHS' project planning did not ensure all relevant external stakeholders were consulted ... " (Omb 2017, p. 3)
DHS' planning ... fell short [in relation to] testing, ... rollout ... [and] communication to staff and stakeholders ... The risk management plan recognised the need for a strong stakeholder strategy to mitigate the risk that insufficient communication with appropriate key stakeholders may result in failure to realise expected program savings. (Omb 2017, p. 23, fn.48)
The 62 advocacy organisations that submitted to the Senate Committee hearings in 2017 can be categorised as follows (SCA 2017, pp.123-129):
Where is good government, good decision-making and leadership when a system is failing? Where is the leadership that is bold enough to say: 'We got this wrong. We will pull it back. We will rework it. We will review it. We will talk to the stakeholders who know best to try and get it right.' Where is good government in understanding and taking seriously its duty of care to its citizens to protect the most vulnerable and not cause vulnerability or harm its own citizens? (SCA 2017, p. 107)
The proceeding has exposed a shameful chapter in the administration of the Commonwealth social security system and a massive failure of public administration. It should have been obvious to the senior public servants and to the responsible Minister(s) at different points who designed and were charged with overseeing the Robodebt system that many social security recipients do not earn a stable or constant income, and any employment they obtain may be casual, part-time, sessional, or intermittent and may not continue throughout the year. (Pry 2021, p. 2)
For those perpetual critics of the ... class action regime, the present case is one more example where the regime has provided real, practical access to justice. It has enabled approximately 394,000 people, many of whom are marginalised or vulnerable, to recover compensation from the Commonwealth in relation to conduct which it belatedly admitted was unlawful (Pry 2021, p. 8)
Omb (2017) 'Centrelink's automated debt raising and recovery system: A Report about the Department of Human Services' Online Compliance Intervention System for Debt Raising and Recovery' Australian Ombudsman, April 2017, at https://www.ombudsman.gov.au/__data/assets/pdf_file/0022/43528/Report-Centrelinks-automated-debt-raising-and-recovery-system-April-2017.pdf
SCA (2017) 'Design, scope, cost-benefit analysis, contracts awarded and implementation associated with the Better Management of the Social Welfare System initiative' The Senate Community Affairs References Committee, Australian Parliament House, June 2017, at https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Community_Affairs/SocialWelfareSystem/Report
GR (2017) 'Government Response to the Senate Committee Report', Australian Government, September 2017, at https://www.aph.gov.au/Parliamentary_Business/Committees/Senate/Community_Affairs/SocialWelfareSystem/Additional_Documents?docType=Government%20Response
Pry (2021) Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634 (11 June 2021) - the eventual judgment (see Prelims below), at http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2021/634.html?context=1;query=Prygodicz;mask_path=
SCA (2022) 'Accountability and justice: Why we need a Royal Commission into Robodebt' Senate Community Affairs References Committee, May 2022, at https://parlinfo.aph.gov.au/parlInfo/download/committees/reportsen/024846/toc_pdf/AccountabilityandjusticeWhyweneedaRoyalCommissionintoRobodebt.pdf;fileType=application%2Fpdf
RRC (2022a) 'Transcript of Proceedings for Monday, 5 December 2022' Royal Commission into the Robodebt Scheme, December 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-13-5-december-2022.pdf
RRC (2022b) 'Transcript of Proceedings for Monday, 6 December 2022' Royal Commission into the Robodebt Scheme, December 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-6-december.pdf
RRC (2022c) 'Transcript of Proceedings for Monday, 7 December 2022' Royal Commission into the Robodebt Scheme, December 2022, at https://robodebt.royalcommission.gov.au/system/files/2022-12/transcript-hearing-day-16-8-december-2022.pdf
RRC (2023a) 'Transcript of Proceedings for Monday, 20 February 2023' Royal Commission into the Robodebt Scheme, February 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/transcript-hearing-day-32-20-february-2023.pdf
RRC (2023b) 'Transcript of Proceedings for Monday, 2 March 2023' Royal Commission into the Robodebt Scheme, March 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-03/transcript-hearing-day-40-2-march.pdf
RCR (2023) 'Report of the Royal Commission into the Robodebt Scheme', 7 July 2023, at https://robodebt.royalcommission.gov.au/publications/report
DMA (1990) Data-Matching Program (Assistance and Tax) Act 1990, at http://classic.austlii.edu.au/au/legis/cth/consol_act/dpata1990349/index.html
DMP (1994) 'Guidelines for the Conduct of the Data-Matching Program' Delegated Legislated under the Data Matching Program (Assistance and Tax) Act 1990, October 1994, nominally enforceable, but applicable only if the matching process uses the Tax File Number (TFN), at https://www.legislation.gov.au/Details/F2009B00268
OAIC (2014) 'Guidelines on data matching in Australian Government administration' Office of the Australian Information Commissioner, rev. 18 June 2014, at https://www.oaic.gov.au/privacy/guidance-and-advice/guidelines-on-data-matching-in-australian-government-administration
Centrelink (2016) 'Data Matching Protocol', undated, but apparently of mid-2016, suppressed by Centrelink for a year, and only disclosed incidentally by another agency as an attachment to a letter of 16 May 2017 by the Privacy Commissioner responding to a Senator's questions, at http://www.aph.gov.au/DocumentStore.ashx?id=4f473c96-f345-4314-ad5d-a2c2eecfba47
OAIC (2017) Vacuous statements by the Privacy Commissioner, January-March 2017, at https://www.oaic.gov.au/media-and-speeches/statements/centrelink-debt-recovery-sstem [broken link on 6 Sep 2022]
DHS (2017) 'If you owe us money, you'll need to pay us back' Department of Human Services, 2016-17, https://www.servicesaustralia.gov.au/owing-money-for-centrelink-debt [but continually changing]
Amato (2019) Amato v The Commonwealth of Australia (Federal Court VID611/2019), at https://www.comcourts.gov.au/file/Federal/P/VID611/2019/3859485/event/30114114/document/1513665
Thodey D., et al. (2019) 'Independent Review of the Australian Public Service' Department of the Prime Minister and Cabinet, December 2019, at https://www.pmc.gov.au/sites/default/files/resource/download/independent-review-aps.pdf
Prygodicz v Commonwealth of Australia [2020] FCA 1454 (17 September 2020) http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2020/1454.html?context=1;query=Prygodicz;mask_path
Commonwealth of Australia v Prygodicz [2020] FCA 1516 (14 October 2020) http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2020/1516.html?context=1;query=Prygodicz;mask_path
Pry (2021) Prygodicz v Commonwealth of Australia (No 2) [2021] FCA 634 (11 June 2021) - the eventual judgment, at http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2021/634.html?context=1;query=Prygodicz;mask_path
SA (2021) 'Individuals' Services Australia, 24 June 2021, at https://www.servicesaustralia.gov.au/individuals/centrelink
DHS (2022) 'Explanation or formal review of a decision (Form SS351)' Department of Human Services, March 2022, at https://web.archive.org/web/20220707163123mp_/https://www.servicesaustralia.gov.au/sites/default/files/ss351-2103en-f.pdf
Prygodicz v Commonwealth of Australia (No 3) [2022] FCA 826 (23 March 2022), at http://www.austlii.edu.au/cgi-bin/viewdoc/au/cases/cth/FCA/2022/826.html?context=1;query=Prygodicz;mask_path
APS (2022) 'APS Code of Conduct' Australian Public Service Commission, 10 May 2022, at https://www.apsc.gov.au/working-aps/integrity/integrity-resources/code-of-conduct
Royal Commission into the Robodebt Scheme:
SA (2022) 'A guide to Australian Government payments' Services Australia, 2022, at https://www.servicesaustralia.gov.au/sites/default/files/co029-2207.pdf
Musolino A. (2023) 'Exhibit 3-4208 - AMU.9999.0001.0003_R - 20221031 NTG-0012 - Response (Annette Musolino)(46258465.1)' Royal Commission into Robodebt, 30 January 2023, at https://robodebt.royalcommission.gov.au/system/files/exhibit/Exhibit%203-4208%20-%20AMU.9999.0001.0003_R%20-%2020221031%20NTG-0012%20-%20Response%20%28Annette%20Musolino%29%2846258465.1%29.pdf
Podger A. (2023) 'Report to the Royal Commission into the Robodebt Scheme' Royal Commission into the Robodebt Scheme, February 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-02/andrew-podger-report-to-robodebt-royal-commission.pdf
Deloitte (2023a) 'The use of data and automation in the Robodebt scheme: A report to the Royal Commission into the Robodebt Scheme' Royal Commission into the Robodebt Scheme, March 2023, at https://robodebt.royalcommission.gov.au/system/files/2023-03/Use-of-data-and-automation-in-the-Robodebt-Scheme.pdf
Deloitte (2023b) 'Robodebt Process Maps' Royal Commission into the Robodebt Scheme, March 2023, at https://robodebt.royalcommission.gov.au/publications/deloitte-robodebt-process-maps
The Primary Case Sources, comprising Investigation Reports and Other Relevant Official Documents, are listed above, separately from the References.
Abbas, R. & Michael, K. (2022) 'Socio-Technical Theory: A Review' In S. Papagiannidis (Ed) 'TheoryHub Book', at: https://open.ncl.ac.uk/theories/9/socio-technical-theory/
Abbas R., Michael K., Sargent J.P. & Scornavacca E. (2021) 'Anticipating Techno-Economic Fallout: Purpose-Driven Socio-Technical Innovation' IEEE Transactions on Technology and Society 2,3 (September 2021) 111-113, at https://ieeexplore.ieee.org/iel7/8566059/9500078/09500093.pdf
Abbas R., Michael K., Michael M.G., Perakslis C. and Pitt J. (2022) 'Machine Learning, Convergence Digitalization, and the Concentration of Power: Enslavement by Design Using Techno-Biological Behavior' IEEE Transactions on Technology and Society 3, 2 (June 2022) 76-88, doi: 10.1109/TTS.2022.3179756
Abbas R., Pitt J., & Michael K. (2021) 'Socio-technical design for public interest technology' IEEE Transactions on Technology and Society 2,2 (2021) 55-61
Achterkamp M.C. & Vos J.F.J. (2008) 'Investigating the Use of the Stakeholder Notion in Project Management Literature: A Meta-Analysis' International Journal of Project Management 26 (2008) 749-757
AHRC (2020) 'Using artificial intelligence to make decisions: Addressing the problem of algorithmic bias Technical Paper' Human Rights Commission, Australia, 2020, at https://tech.humanrights.gov.au/sites/default/files/2021-05/AHRC_RightsTech_2020_algorithmic_bias.pdf
Anderson S. & Belot H. (2017) 'Centrelink's debt recovery system working, Human Services Minister Alan Tudge says' ABC News, 11 Jan 2017, at https://www.abc.net.au/news/2017-01-11/centrelinks-debt-recovery-system-to-remain-government-says/8174644
APF (2017) 'The Imminent Threat of Automated Government' Australian Privacy Foundation, 18 January 2017, at https://privacy.org.au/2017/01/18/the-imminent-threat-of-automated-government/
APSC (2022) 'APS Code of Conduct' Australian Public Service Commission, 10 May 2022, at https://www.apsc.gov.au/working-aps/integrity/integrity-resources/code-of-conduct
Avison D.E. & Wood-Harper A.T. (1990) 'Multiview: An Exploration in Information Systems Development' Blackwell, 1990
Bajkowski J. (2022) 'Services Australia handed $5.6m more work to robodebt collector Panthera following ACCC prosecution' The Mandarin, 3 November 2022, at https://www.themandarin.com.au/204378-services-australia-handed-5-6m-more-work-to-robodebt-collector-panthera-following-accc-prosecution/
Bajkowski J. (2023) 'Jane Halton links `chilling' effect of FOI to robodebt's 'frank and fearless advice' epic fail' The Mandarin, 11 July 2023, at https://www.themandarin.com.au/224898-jane-halton-links-foi-to-robodebt-failure-of-frank-fearless-advice/
Banks D. & London A.J. (2017) 'Algorithmic Bias in Autonomous Systems' Proc. 26th Int'l Joint Conf. on Artificial Intelligence (IJCAI 2017), at https://www.cmu.edu/dietrich/philosophy/docs/london/IJCAI17-AlgorithmicBias-Distrib.pdf
Bannister M. (2022) 'Commissioner appalled by robodebt evidence' AAP/The Canberra Times, 2 November 2022, at https://www.canberratimes.com.au/story/7965606/commissioner-appalled-by-robodebt-evidence/
Barbaschow A. (2017) 'Privacy Foundation calls for human intervention in Centrelink debt collection' zdNet, 2 April 2017, at https://www.zdnet.com/article/privacy-foundation-calls-for-human-involvement-in-centrelink-automated-debt-collection-process/
Baumer E.P.S. (2015) 'Usees' Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15), April 2015
Belot H. (2017a) 'Centrelink debt recovery: Government knew of potential problems with automated program' ABC News, 12 Jan 2017, at https://www.abc.net.au/news/2017-01-12/government-knew-of-potential-problems-with-centrelink-system/8177988
Belot H. (2017b) 'Centrelink's controversial data matching program to target pensioners and disabled, Labor calls for suspension' ABC News, 17 Jan 2017, at https://www.abc.net.au/news/2017-01-17/labor-calls-for-suspension-of-centrelink-debt-recovery-program/8187934
Belot H. & McGhee A. (2017) 'Centrelink debt recovery program to undergo changes following public criticism' ABC News, 16 Jan 2017, at https://www.abc.net.au/news/2017-01-16/government-to-change-centrelink-debt-recovery-program/8185366
Benbasat I., Goldstein D. & Mead M. (1987) 'The Case Study Research Strategy in Studies of Information Systems' MIS Qtly 11,3 (September 1987) 369-386
Berleur J. & Drumm J. (Eds.) (1991) 'Information Technology Assessment' Proc. 4th IFIP-TC9 International Conference on Human Choice and Computers, Dublin, July 8-12, 1990, Elsevier Science Publishers (North-Holland), 1991
Bower C.M. & Christensen C.M. (1995) 'Disruptive Technologies: Catching the Wave' Harvard Business Review (January-February 1995) 43-53
Braithwaite V. (2020) 'Beyond the bubble that is Robodebt: How governments that lose integrity threaten democracy' Aust J Soc Issues 55,3 (September 2020) 242-259, at https://static1.squarespace.com/static/5c05f8595cfd7901fc57139d/t/5f8a808b459ade460acd28dc/1602912397885/Beyond+the+bubble+that+is+Robodebt+How+governments+that+lose+integrity+threaten+democracy.pdf
Brennen S. & Kreiss D. (2016) 'Digitalization and Digitization' International Encyclopedia of Communication Theory and Philosophy, October 2016, PrePrint at http://culturedigitally.org/2014/09/digitalization-and-digitization/
Brookes J. (2022) '197,000 notices wiped from `shameful' robodebt scheme' Innovation Australia, 12 October 2022, at https://www.innovationaus.com/197000-notices-wiped-from-shameful-robodebt-scheme/
Burgess K. (2019) 'Robodebt Centrelink compliance program cost $600m to recoup $785m so far' The Canberra Times, October 28 2019, at https://www.canberratimes.com.au/story/6407984/four-of-five-robodebt-reviews-hit-with-a-debt-as-cost-revealed/
Carney T. (2019) 'Robo-debt illegality: The seven veils of failed guarantees of the rule of law?' Alternative Law Journal 44,1 (2019) 4-10, at https://journals.sagepub.com/doi/pdf/10.1177/1037969X18815913
Checkland P. (1981) 'Systems Thinking, Systems Practice' Wiley, 1981
Cherns A. (1976) 'The principles of sociotechnical design' Human Relations 29,8 (1976) 783-792
Cherns A. (1987) 'Principles of sociotechnical design revisited. Human Relations, 40,3 (1987) 153-161
Christensen C.M., Raynor M. & McDonald R. (2015) 'What Is Disruptive Innovation?' Harvard Business Review (December 2015) 2-11
Citron D.K. (2008) 'Technological Due Process' Washington University Law Review 85 (2008 1249, at https://core.ac.uk/download/pdf/229123628.pdf
Clarke R. (1987) 'Just Another Piece of Plastic for your Wallet: The 'Australia Card' Scheme' Computers & Society 18,1 (January 1988), Addendum in Computers & Society 18,3 (July 1988), PrePrint at http://www.rogerclarke.com/DV/OzCard.html
Clarke R. (1992a) 'The Resistible Rise of the National Personal Data System' Software Law Journal 5,1 (January 1992), PrePrint at http://www.rogerclarke.com/DV/SLJ.html
Clarke R. (1992b) 'Extra-Organisational Systems: A Challenge to the Software Engineering Paradigm' Proc. IFIP World Congress, Madrid, September 1992, PrePrint at http://www.rogerclarke.com/SOS/PaperExtraOrgSys.html
Clarke R. (1994a) 'The Digital Persona and its Application to Data Surveillance' The Information Society 10,2 (June 1994), PrePrint at http://www.rogerclarke.com/DV/DigPersona.html
Clarke R. (1994b) 'Dataveillance by Governments: The Technique of Computer Matching' Information Technology & People 7,2 (December 1994) 46-85, PrePrint at http://www.rogerclarke.com/DV/MatchIntro.html
Clarke R. (2014) 'Promise Unfulfilled: The Digital Persona Concept, Two Decades Later' Information Technology & People 27, 2 (Jun 2014) 182 - 207, PrePrint at http://www.rogerclarke.com/ID/DP12.html
Clarke R. (2017) 'Guidelines for the Responsible Application of Data Analytics' Computer Law & Security Review 34, 3 (May-Jun 2018) 467- 476, PrePrint at http://www.rogerclarke.com/EC/GDA.html
Clarke R. (2019a) 'Why the World Wants Controls over Artificial Intelligence' Computer Law & Security Review 35, 4 (2019) 423-433, PrePrint at http://www.rogerclarke.com/EC/AII.html
Clarke R. (2019b) 'Principles and Business Processes for Responsible AI' Computer Law & Security Review 35, 4 (2019) 410-422, PrePrint at http://www.rogerclarke.com/EC/AIP.html
Clarke R. (2022) 'Evaluating the Impact of Digital Interventions into Social Systems: How to Balance Stakeholder Interests' Proc. Int'l Digital Security Forum (IDSF22), Vienna, 1 June 2022, PrePrint at http://rogerclarke.com/DV/MSRA-VIE.html
Clarke R. & Davison R.M. (2020) 'Through Whose Eyes? The Critical Concept of Researcher Perspective' J. Assoc. Infor. Syst. 21, 2 (March-April 2020) 483-501, PrePrint at http://rogerclarke.com/SOS/RP.html
CPI (2022) 'Delay and Decay: Australia's Freedom of Information Crisis' Briefing Paper, The Centre for Public Integrity, August 2022, at https://publicintegrity.org.au/wp-content/uploads/2022/09/FOI-Delay-and-Decay-Final.pdf
Cukier K. & Mayer-Schönberger V. (2013) 'The Rise of Big Data' Foreign Affairs (May/June 2014) 35
Emery F. (1959) 'Characteristics of socio-technical systems' In 'The Social Engagement of Social Science: A Tavistock Anthology' Volume 2. pp. 157-186), University of Pennsylvania Press
ENISA (2016) 'Risk Management:Implementation principles and Inventories for Risk Management/Risk Assessment methods and tools' European Union Agency for Network and Information Security, June 2016, at https://www.enisa.europa.eu/publications/risk-management-principles-and-inventories-for-risk-management-risk-assessment-methods-and-tools
Erdbrink T. (2021) 'Government in Netherlands Resigns After Benefit Scandal', The New York Times, 15 Jan 2021, at https://www.nytimes.com/2021/01/15/world/europe/dutch-government-resignation-rutte-netherlands.html
Farrell P. (2019) 'Government halting key part of robodebt scheme, will freeze debts for some welfare recipients' ABC News, 19 November 2019, at https://www.abc.net.au/news/2019-11-19/robodebt-scheme-human-services-department-halts-existing-debts/11717188
Fischer-Huebner S. & Lindskog H. (2001) 'Teaching Privacy-Enhancing Technologies' Proceedings of the IFIP WG 11.8 2nd World Conference on Information Security Education, Perth, Australia
Graycar A. & Masters A. B. (2022) 'Bureaucratic bastardry: robodebt/debt recovery, AI and the stigmatisation of citizens by machines and systems' International Journal of Public Policy 16,5-6 (2022) 333-344
Hammer M. (1990) 'Reengineering Work: Don't Automate, Obliterate' Harvard Business Review (July-August 1990) 104-112
Henriques-Gomes L. (2020) 'All Centrelink debts raised using income averaging unlawful, Christian Porter concedes' The Guardian, 31 May 2020, at https://www.theguardian.com/australia-news/2020/may/31/all-centrelink-debts-raised-using-income-averaging-unlawful-christian-porter-concedes
Hevner A.R., March S.T. & Park S. (2004) 'Design Science in Information Systems Research' MIS Quarterly 28,1 (2004) 75-105
Hirschheim R.A. (1985) 'User Experience with and Assessment of Participative Systems Design' MIS Quarterly (December 1985), at https://www.academia.edu/download/51009081/viewcontent.pdf
Hogan-Doran D. (2017) 'Computer says "no": automation, algorithms and artificial intelligence in Government decision-making' The Judicial Review 13 (2017) 1-39, at https://www.academia.edu/38540521/Computer_says_no_automation_algorithms_and_artificial_intelligence_in_Government_decision_making?email_work_card=view-paper
IEC 31010:2019 'Risk management - Risk assessment techniques' International Standards Organisation, 2019
Jobin A., Ienca M. & Vayena E. (2019) 'The global landscape of AI ethics guidelines' Nature Machine Intelligence 1 (September 2019) 389-399, at https://doi.org/10.1038/s42256-019-0088-2
Karp P. (2019) 'Government admits robodebt was unlawful as it settles legal challenge' The Guardian, 27 November 2019, at https://www.theguardian.com/australia-news/2019/nov/27/government-admits-robodebt-was-unlawful-as-it-settles-legal-challenge
Knaus C. (2016) 'Centrelink officer says only a fraction of debts in welfare crackdown are genuine' The Guardian, 23 December 2016, at https://www.theguardian.com/australia-news/2016/dec/23/centrelink-officer-says-only-a-fraction-of-debts-in-welfare-crackdown-are-genuine
Knaus C. (2017) ''Reasonably clear' Alan Tudge's office broke law in Centrelink case, says Labor legal advice' The Guardian, 3 Apr 2017, at https://www.theguardian.com/australia-news/2017/apr/03/reasonably-clear-alan-tudges-office-broke-law-in-centrelink-case-says-labor-legal-advice
Knaus C. & Bassano J. (2019) 'How a flawed freedom-of-information regime keeps Australians in the dark' The Guardian, 2 January 2019, at https://www.theguardian.com/australia-news/2019/jan/02/how-a-flawed-freedom-of-information-regime-keeps-australians-in-the-dark
Land F. & Hirschheim R. (1983) 'Participative systems design: Rationale, tools and techniques' Journal of Applied Systems Analysis, 1983
Landis-Hanley J. (2023) 'An APS code of conduct 'injustice'' The Canberra Times, 29 July 2023, at https://www.canberratimes.com.au/story/8286894/complete-lack-of-justice-campbell-resignation-prompts-calls-for-aps-reform/
Lee A.S. (1989) 'A Scientific Methodology for MIS Case Studies' MIS Qtly 13,1 (March 1989) 33-50
Liu K., Sun L. & Bennett K. (2002) 'Co-Design of Business and IT Systems' Introduction by Guest Editors, Information Systems Frontiers 4,3 (2002) 251-256
LSD (2017) 'Legal Services Directions 2017', 2 July 2018, at https://www.legislation.gov.au/Details/F2018C00409
Lycett, M. (2014), Datafication: Making Sense of (Big) Data in a Complex World, European Journal of Information Systems, 22, 4 (December 2014) 381-386, at http://v-scheiner.brunel.ac.uk/bitstream/2438/8110/2/Fulltext.pdf
Lyytinen K. & Rose G.M. (2003) 'The Disruptive Nature of Information Technology Innovations: The Case of Internet Computing in Systems Development Organizations' MIS Quarterly 27, 4 (December 2003) 557-595
Martin B. (2013) 'Whistleblowing: a practical guide' Irene Publishing, Sweden, 2013
Medhora S. (2019) 'Over 2000 people died after receiving Centrelink robo-debt notice, figures reveal' ABC TripleJ, 18 Feb 2019, at https://www.abc.net.au/triplej/programs/hack/2030-people-have-died-after-receiving-centrelink-robodebt-notice/10821272
Meske C., Abedin B., Klier M. & Rabhi F. (2022) 'Explainable and responsible artificial intelligence' Electronic Markets 32,4 (2022) 2103-2106, at https://link.springer.com/content/pdf/10.1007/s12525-022-00607-2?utm_source=toc&utm_medium=email&utm_campaign=toc_12525_32_4&utm_content=etoc_springer_20230315
Michael, K. (2003) 'The technological trajectory of the automatic identification industry: the application of the systems of innovation (SI) framework for the characterization and prediction of the auto-ID industry' PhD thesis, School of Information Technology and Computer Science, University of Wollongong, 2003, at http://ro.uow.edu/theses/309. See in particular 'Section 4.3 The Evolution of the Citizen ID Number'
Michael K., Abbas R. & Pitt J. (2021) 'Maintaining Control over AI' Issues in Science and Technology: Forum XXXVII, 3 (Spring 2021)
Michael K. & Michael M.G. (2006) 'Historical lessons on ID technology and the consequences of an unchecked trajectory' Prometheus 24,4 (2006) 365-377
Mitchell R.K., Agle B.R. & Wood D.J. (1997) 'Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts' Academy of Management Review 22, 4, 853-886
Morton R. (2022) 'Robo-debt: Liberals knew it was illegal before it started' The Saturday Paper, 5 November 2022, at https://www.thesaturdaypaper.com.au/news/politics/2022/11/05/robo-debt-liberals-knew-it-was-illegal-before-it-started#mtr
MRR (20125) 'Welfare Integrity, Fairness and Sustainability for all Australians' Minister for Social Security, Scott Morrison, Budget Media Release, 12 May 2015, at https://formerministers.dss.gov.au/15850/welfare-integrity-fairness-and-sustainability-for-all-australians/
Mumford E. (2000) 'A socio-technical approach to systems design' Requirements Engineering 5,2 (2000) 125-133
Mumford E. (2006) 'The story of socio-technical design: reflections on its successes, failures and potential' Info Systems J 16 (2006) 317-342, at https://executiveinsight.typepad.com/files/the-story-of-socio-technical-design.pdf
Newell S. & Marabelli M. (2015) 'Strategic Opportunities (and Challenges) of Algorithmic Decision-Making: A Call for Action on the Long-Term Societal Effects of 'Datification'' The Journal of Strategic Information Systems 24, 1 (2015) 3-14, at http://marcomarabelli.com/Newell-Marabelli-JSIS-2015.pdf
NIST (2012) 'Guide for Conducting Risk Assessments' National Institute of Standards and Technology, Special Publication SP 800-30 Rev. 1, September 2012, at http://csrc.nist.gov/publications/nistpubs/800-30-rev1/sp800_30_r1.pdf
Noble S.U. (2018) 'Algorithms of Oppression: How Search Engines Reinforce Racism' New York University Press, 2018
Piller G., Schubert P., Koch M. & Moeslein K. (2004) 'From Mass Customization to Collaborative Customer CoDesign' Proc. ECIS 2004, 118
Redden J. (2018) 'The Harm That Data Do' Scientific American, 1 November 2018, https://www.scientificamerican.com/article/the-harm-that-data-do/
Rinta-Kahila T., Someh I., Gillespie N., Indulska M. & Gregor S. (2022) 'Algorithmic decision-making and system destructiveness: A case of automatic debt recovery' European Journal of Information Systems 31,3 (2022) 313-338, at https://www.tandfonline.com/doi/pdf/10.1080/0960085X.2021.1960905?needAccess=true
Sanders E.B.-N. & Stappers P.J. (2008) 'Co-creation and the new landscapes of design' CoDesign 4,1 (2008) 5-18, at https://www.tandfonline.com/doi/pdf/10.1080/15710880701875068
Sanders E.B.-N. & Stappers P.J. (2014) 'Probes, toolkits and prototypes: three approaches to making in codesigning' CoDesign 10,1 (2014) 5-14, at https://www.researchgate.net/publication/262851486_Probes_toolkits_and_prototypes_Three_approaches_to_making_in_codesigning
Sarah M. (2016) 'Welfare debt squad hunts for $4bn' The Australian, 5 December 2016, at http://www.theaustralian.com.au/national-affairs/welfare-debt-squad-hunts-for-4bn-in-overpayments/news-story/e19c5b0d4a39aa07364a41269fdc11c9
Trist E.L. (1981) 'The evolution of socio-technical systems' Vol. 2, Toronto: Ontario Quality of Working Life Centre, 1981
TT (2016) 'Guilty until proven innocent ... Centrelink gone rogue ...' Tasmanian Times, 5 December 2016, at https://tasmaniantimes.com/2016/12/guilty-until-proven-innocent-centrelink-gone-rogue/
Venkatraman N. (1994) 'IT-Enabled Business Transformation: From Automation to Business Scope Redefinition' Sloan Management Review 35,2 (Winter 1994) 73-87
Von Bertalanffy L. (1950) 'The theory of open systems in physics and biology' Science, 111,2872 (1950) 23-29
Wachter S. & Mittelstadt B. (2019) 'A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI' Forthcoming, Colum. Bus. L. Rev. (2019), at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3248829
Wahlstrom K. & Quirchmayr G. (2008) 'A Privacy-Enhancing Architecture for Databases' Journal of Research and Practice in Information Technology 40, 3 (August 2008) 151-162, at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.454.3815&rep=rep1&type=pdf
Walsham G. (1995) 'Interpretive Case Studies in IS Research: Nature and Method' European Journal of Information Systems 4, 2 (May 1995) 74-81
Whiteford P. (2021) 'Debt by design: The anatomy of a social policy fiasco - Or was it something worse? Aust. J of Public Admin 80,2 (June 2021) 340-360, at https://onlinelibrary.wiley.com/doi/epdf/10.1111/1467-8500.12479
Whiteford P. (2022) 'For the poorest 20% of Australian households, social security payments provide more than 70% of their income' John Menadue's Public Policy Journal, 20 May 2022, at https://johnmenadue.com/social-security/
Wong K. (2022) 'Robodebt scheme told to `cease and desist' AAP, 4 November 2022, at https://www.aap.com.au/news/debt-collectors-to-front-royal-commission/
Wynn D. & Williams C.K. (2012) 'Principles for Conducting Critical Realist Case Study Research in Information Systems' MIS Quarterly 36, 3 (September 2012) 787-810, at https://ecommons.udayton.edu/cgi/viewcontent.cgi?article=1076&context=mis_fac_pub
Yin R.K. (2017) 'Case Study Research and Applications: Design and Methods' 6th edition, Sage, 2017
Zeng Y., Lu E. & Huangfu C. (2019) 'Linking Artificial Intelligence Principles' Proc. AAAI Workshop on Artificial Intelligence Safety (AAAI-Safe AI 2019), 27 January 2019, at https://arxiv.org/abs/1812.04814
Zwass V. (2010) 'Co-creation: Toward a taxonomy and an integrated research perspective' International Journal of Electronic Commerce 15,1 (2010) 11-48
Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professor associated with the Allens Hub for Technology, Law and Innovation in UNSW Law, and a Visiting Professor in the Research School of Computer Science at the Australian National University. He is also a longstanding Board-member of the Australian Privacy Foundation.
Katina Michael is a Professor at Arizona State University, a Senior Global Futures Scientist in the Global Futures Laboratory and has a joint appointment in the School for the Future of Innovation in Society and School of Computing and Augmented Intelligence. She is the director of the Society Policy Engineering Collective (SPEC) and the Founding Editor-in-Chief of the IEEE Transactions on Technology and Society. She is also a longstanding Board-member of the Australian Privacy Foundation.
Roba Abbas is a Senior Lecturer and Academic Program Director in the School of Business, at the University of Wollongong in Australia. Her work is focused on methodology approaches to complex socio-technical systems design. She is a Co-Editor of the IEEE Transactions on Technology and Society.
| Personalia |
Photographs Presentations Videos |
Access Statistics |
 |
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax. From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 65 million in early 2021. Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer |
Xamax Consultancy Pty Ltd ACN: 002 360 456 78 Sidaway St, Chapman ACT 2611 AUSTRALIA Tel: +61 2 6288 6916 |
Created: 9 September 2022 - Last Amended: 5 August 2023 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/DV/RDCS.html
Mail to Webmaster - © Xamax Consultancy Pty Ltd, 1995-2022 - Privacy Policy