Roger Clarke's 'Eras of Dataveillance'

The practice of data surveillance has developed within a context. That context comprises factors at the levels of society and organisation, compounded by technology. These notes argue that the tendencies inherent within information technology have changed significantly during the last couple of decades, and are still changing. These changes have a significant effect on the practice of dataveillance.

I depict the first era as being one of justifiable centralisation. Until about 1980, Grosch's Law prevailed. This asserted that the processing power of computer hardware grew exponentially with the price, i.e. that bigger was more efficient. In respect of the original, 'mainframe' architecture computers, that was demonstrably true.

Based on a different component technology (integrated circuits or ICs), the 'mini-computer' architecture emerged during the 1960s and 1970s, and matured into the 'mid-range' machines of the 1980s and 1990s. Initially, these too obeyed Grosch's Law. As communications matured, the dominant topology used was the 'star-network', which involves a processor at the centre, and more or less 'dumb' terminals at the peripheries. Virtually all processing is performed by the master processor.

The combination of 'bigger is better' computer technology and star networks was eminently compatible with centralist approaches to data management, hierarchical organisational structures, authoritarian notions of organisational leadership. Data was collected (as close as possible to its source, in order to reduce costs and ensure quality), and passed to the centre for storage and processing.

Large-processor / star network technology fuelled assertions about the technological imperative, and the centralist and authoritarian tendencies inherent in computers. Orwell seemed to be vindicated: the corporation and the State had a weapon of suppression available to it.

2000-: The Era of 'Dispersed Power'

In the near future, which I depict as the third era of dataveillance, the technological context is profoundly different. Commencing in the early 1970s, Very Large Scale Integrated circuit (VLSI) component technology spawned yet another machine architecture, which were originally referred to as 'micro-computers'. One of the effects of this revolution was the repeal of Grosch's Law: many micro-computers were, initially in principle, and increasingly in practice, capable of delivering greater processing power for less price than either mainframe or mini-computer architectures.

Micro-computer impact began in a small way, embedded in other equipment, in hobby-sets and then in the Apple II. In 1981 the IBM PC was launched, and this was followed by more usable alternatives, in particular the Macintosh in 1984. These low-end, inexpensive machines delivered computing for the masses. No longer could companies' Information Systems Managers regulate the supply of computing power to the company's employees. PCs appeared in locations throughout organisations, and then spread like wildfire. Many of these machines were used primarily for document preparation and spreadsheet modelling, but in time a considerable amount of data processing which the centralised IS departments had not yet been able to support was performed locally on PCs.

The extent of the PC's impact can be further dramatised. There is a line of argument which proposes that the collapse of Communism generally, and of the U.S.S.R. in particular, was due to the PC. In order to maintain competitiveness with the West, it was essential that the Soviets exploit microcomputer technology. Moreover, it was necessary that the energies of the populace be harnessed, by distributing processing power. But every PC was a samizdat press (i.e. it provided the means for the disenchanted to express and disseminate their views). The lies which had been sustained for 70 years could not withstand the white light of microcomputer technology.

From the mid-1980s onwards, much more powerful machines than mere populist PCs became available, and were commonly designated as 'scientific workstations'. These have had an increasing effect on PC design, such that PCs of the mid-1990s are much better conceived as workstations for white-collar workers than as mere playthings.

Communications technology has also undergone a significant revolution. Star networks have given way to alternative topologies which better support multiple processors. Since the late 1980s, the trend in local area networks has been towards 'client-server' architectures. By this is meant that workstations are the clients which are to be served by background machines which offered shared data storage and processing resources: the old order of master-and-slave has been inverted.

Meanwhile, in wide area networks, star topology has proven to be too lacking in resilience: since all processing is dependent on a single node, any failure in that node undermines the entire network. Alternative, more robust topologies are preferred, with various combinations of closely-coupled and loosely-coupled processors, and peer-to-peer communications among computers, rather than master-slave relationships.

In the mid-1990s, these are all established technologies. Current developments are in full connectivity among the world's myriad machines; connectionless linkages between clients and servers and directly between clients; the capture, storage, manipulation and output of many different types of media; and the integration of these varied I/O technologies into virtual reality applications.

The implications of contemporary information technology for data surveillance are profound. Storage is being dispersed. Within corporations, some success is being achieved in attempts to sustain the sense of centralisation, in the form of a 'logical' or 'virtual' corporate database. It is proving much more difficult to develop open, multi-corporate logical databases, i.e. to virtually integrate the data collections of multiple organisations.

In the third era of dataveillance, the dominant mode of operation would appear to be cooperation among peers, rather than master-slave relationships (or markets rather than hierarchies, to use a piece of academic jargon). The centralist, authoritarian tendencies which appeared to be inherent in computers is giving way to dispersed power. In the new context, there may be more danger of anarchy and ungovernability than of repression by a powerful State.

1980-2000: The Uneasy Era of 'Unjustified Centralisation'

That leaves the question as to how to typify the present, second era of dataveillance, roughly 1980-2000. I depict it as the uneasy interim era of unjustified centralisation. Corporations and governments are clinging to the old, trusted and altogether simpler centralist notions. ("After all, there are few who are competent and who should be entrusted with important responsibilities, and they are us").

This creates tensions. People who are envisioning the future are inevitably impatient with those who cling to past patterns; and conservatives by definition do not want to lose the stability which has served (at least them) in the past.

The coming years will see a struggle between the social and organisational factors on the one hand, and the possibilities created by contemporary information technology on the other. If the assertion of a technological imperative is right, the practice of data surveillance in the future of dispersed and relatively autonomous peers will present considerable challenges. It remains to be seen whether powerful social and political forces will be capable of overcoming these technologically induced tendencies, and sustaining the power relationships of the past.

Personalia Photographs
Presentations
Videos Access
Statistics

The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.

From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 65 million in early 2021.

Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer

Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916

Created: 13 October 1995 - Last Amended: 13 October 1995 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/DV/NotesDVEras.html
Mail to Webmaster - © Xamax Consultancy Pty Ltd, 1995-2022 - Privacy Policy