Roger Clarke's Web-Site

© Xamax Consultancy Pty Ltd,  1995-2024
Photo of Roger Clarke

Roger Clarke's 'Research Methods for eCommerce'

Appropriate Research Methods for Electronic Commerce

Roger Clarke

Principal, Xamax Consultancy Pty Ltd, Canberra

Visiting Fellow, Department of Computer Science, Australian National University

Version of 19 April 2000

© Xamax Consultancy Pty Ltd, 1989, 1994, 1996, 1999, 2000

This document is at http://www.rogerclarke.com/EC/ResMeth.html


Abstract

The research domain of electronic commerce is particularly challenging, because of the lack of established definitions, and the high volatility of the phenomena. The urgent need for quality information presents a classic case of the need for instrumentalist research, which pursues outcomes of relevance to practitioners of the discipline, subject to the constraint of achieving sufficient rigour.


Contents


1. Introduction

Specialist journals in electronic commerce, in particular the International Journal of Electronic Commerce, and Electronic Markets, carry very few articles that directly address the question of which research methods are appropriate to this domain. This might be because textbooks are available that address the question of research methodology, and that these suffice (e.g. Emory 1985, Kervin 1992, Cooper & Schindler 1998).

Alternatively, the absence of discussion may be a sign of maturity in the information systems discipline, because there are already ample references in the I.S. literature, which are studied by doctoral candidates during their preparation for entry to the research profession. See, in particular, Ives et al. (1980), Nolan & Wetherbe (1980), Sprague (1980), Hamilton & Ives (1982), McFarlan (1984), Benbasat (1984), Mumford et al. (1985), Jenkins (1985), Galliers (1985), Culnan (1986, 1987), Culnan & Swanson (1987), Galliers & Land (1987), Alavi et al. (1989), Banville & Landry (1989), Galliers (1990, 1992a, 1992b), Orlikowski & Baroudi (1991), Nissen et al. (1991), Alavi & Carlson (1992), Galliers (1993), Benbasat & Weber (1996), Lee et al. (1997), and Mingers & Stowell (1997).

This paper is written because the author believes that any such sentiments would be exaggerated. Electronic commerce is a domain of study within various disciplines, importantly including the discipline of information systems. Many domains that are examined by I.S. specialists present serious challenges; but electronic commerce presents even more significant obstacles to the academic seeking to undertake and present high-quality research. Among these are:

The purpose of this paper is to provide a perspective on the methods that should be applied to electronic commerce research. It should aid both the assessment of articles, and the preparation of research designs.

There is a great deal of scope for terminological ambiguity in this topic. This paper uses the following terms in the following ways:

The paper commences by reviewing several different schools of thought in relation to research approaches. The challenges confronting information systems research are summarised, so that electronic commerce research can be situated within it. Important considerations in empirical research are identified, and assessment criteria suggested for a research design. Some specific instances of electronic commerce research are considered, and conclusions are drawn.


2. The Dimensions of Research Approaches

The discipline of Information Systems in general, and the research domain of electronic commerce in particular, are both crossroards at which differing approaches to research intersect.

In Clarke (1992), it was argued that information systems overlaps with both the computer science and the business clusters of disciplines; for example, software engineering and database management and some aspects of application software development are properly studied within computer science, and systems analysis and organisational behaviour within business-related disciplines.

A further distinction can be drawn, based on the motivation underlying the research. The primary distinction is between:

A further dimension of research is associated with the form of the theory on which research is based. The theory may be merely descriptive of aspects of the domain, or it may be explanatory of behaviour, or (most ambitiously), it may have predictive power. It was argued in Clarke (1997) that information systems research also has a policy dimension, which demands that some work extend even further, into prescriptive or normative mode.

Beyond these distinctions lies another fundamental difference in the manner in which research can be approached. The following section examines three separate traditions, the first two of which derive from the business disciplines, and the third from computer science and engineering.


3. Research Traditions

This section provides a brief overview of each of the three major research traditions that are evident in information systems in general, and electronic commerce in particular. They are:

The first two are business discipline approaches, and there is a fundamental tension between them. This was much debated during the 1980s and 1990s. See, in particular, Hirschheim 1985, Boland 1985, Klein & Lyytinen 1985, Turner 1990, Lee 1991, Orlikowski & Baroudi 1991, Rouse & Dick 1994, Robey 1996. By the mid-1990s, King & Appleglate (1997) argued that the debate was no longer generating useful information.

The appellations used in this paper, 'scientific' and 'interpretivist', naturally over-simplify the nature of both of them, and the distinction between them. The alternative labels 'quantitative' and 'qualitative' are often used in the literature. Those terms are properly used to describe the nature of the data that is gathered in empirical research (e.g. Van Maanen 1979, Marshall & Rosman 1984, Miles & Hubermann 1984, Cash & Lawrence 1989, Patton 1990, Myers 1997a, Lee et al. 1997). Their use to distinguish schools of thought in research is, on the other hand, quite deficent and misleading. Their use in that sense is accordingly avoided in this paper.


3.1 The Conventional Scientific Research Tradition

This section presents a simplified description of the notion of scientific method, closely akin to what Kuhn (1962, 1970) referred to as 'normal science'. It is intended as a recapitulation of key elements of a graduate unit in philosophy of science or research methods, not as a substitute for such studies.

Conventional scientific approaches to research were developed over several centuries in the context of what are now referred to as 'the physical sciences'. It has been co-opted by 'the social sciences', which can be loosely depicted as being those whose domains of study include agents that exercise free will, or at least appear to do so. Some information systems research is close to the realm of the physical sciences; but most of it sits squarely within the social sciences.

Conventional science adopts the assumption that there is a real world, comprising objects and processes. This real world cannot be directly understood by humans, nor 'captured' into human artefacts. However it can be observed. On the basis of observation of the real world, humans form theories as to how it came to be the way it is, and how and why the processes take place. In an applied discipline such as Information Systems, it is common to depend on theories borrowed from 'reference disciplines'.

Theories should ideally be expressed in deductive form, such that a set of axioms or postulates, operated on by conventional deductive logic, lead to inferences. A scientific theory, á la Popper (1959, 1968), is one which is capable of generating inferences which are, at least in principle, refutable by reference to the real world. Religious and ideological theories generally result in inferences such as "through grace, a believer is saved" (Christian liturgy) and "only a virtuous ruler can survive" (Machiavelli), which cannot be subjected to such testing. Such theories are therefore a-scientific (a term which is descriptive, and should not be interpreted as being pejorative).

In order to establish whether the theory has the capability to describe, explain, and especially to predict, the behaviour of the real world, it is necessary to construct tests of the inferences arising from a theory. Inferences are generally expressed at a conceptual level, however; for example, "experienced auditors are better judges of the extent to which substantive tests of controls should be undertaken". In order to actually perform tests, it is necessary to express them in 'operational' terms, in the form generally referred to as hypotheses, e.g. "the proportion of auditors who select the same extent of substantive testing as recommended by a panel of experts will be highly correlated with their number of years of auditing experience".

The outcomes of hypothesis testing provide feedback to the theory. If the hypotheses are not confirmed, then evidence exists that the theory may be wrong (in Kuhnian terms, this is an 'anomaly'). If the results can be reproduced by others, then at least some aspects of the theory should be regarded as refuted. If the data support the hypotheses, then there is justification for increased or sustained confidence in the theory's effectiveness, or at least continued use of the theory. As findings accumulate which support a wide range of hypotheses generated from the theory, it tends to be regarded as authoritative within some range of applicability.

Research that stops at that point is 'pure'; but it is also incestuous, because there is a risk of the ideas rattling around the abstract realm of theory, perhaps tested from time to time against the real world, but without useful implications for that world. In an instrumentalist discipline, which is how most people would regard information systems in general and electronic commerce research in particular, the outcomes of research must be demonstrated to be relevant and useful to professionals and managers as well.

Research that is based on observations of the real world is referred to as 'empirical research'. Empiricism is relevant at two points in the conventional cycle:

It is important to appreciate the role of theory. It is perfectly feasible to create hypotheses directly from a small collection of observations, without any statement of underlying principles and logical derivation of inferences. The weakness of this approach is that the outcomes of the research can merely refute, or provide conditional support for, those specific hypotheses; there is no accumulation of knowledge. On the other hand, where the hypotheses are derived from a body of theory, the results arising from the research accumulate, and can be used again by the researcher and others to flesh out knowledge.

Information Systems is an applied discipline, however, and lacks a solid body of existing theory. In practice, therefore, it is uncommon in IS to have a theory readily available. It is most common to have a partial theory that requires embellishment, and/or to co-opt a theory from a 'reference discipline' such as organisational behaviour, management accounting or computer science.

In the various reference disciplines, many theories are available. Many of them, however, were developed for purposes related at best tenuously with IS. Some which appear relevant may well prove not to be appropriate, e.g. because they are specific to a particular culture (often the U.S.A. or some part thereof), or the unit of analysis is individuals in a loose-knit community rather than individuals acting within an organisational context (e.g. Rogers' diffusion of innovations theory), or the domain to which they are applied is significantly different from that for which the theory was developed (e.g. Hofstede's theory of cultural variation).

Where no appropriate theory can be found as a basis for research, it may be necessary to undertake 'exploratory research'. This involves open-ended study, largely unguided by theory and intended to provide a new body of empirical knowledge from which theories might be able to be postulated.

The following diagram depicts that process of conventional scientific method that was described above. The button below the diagram provides an annotated animation of the flow. [Note that the animation appears in a new window, which can be closed after it has been run].

Exhibit 1: Conventional Scientific Method

In summary, conventional science consists of extracting new hypotheses from an existing theory, testing them, and adding the results to the pool of knowledge. It presupposes the existence of:


3.2 The Interpretivist Approach to Research

Conventional science is based on 'rational positivist' thought. This includes the presumptions that there is a 'real world', that data can be gathered by observing it, and that those data are factual, truthful and unambiguous. The 'post-positivist', 'interpretivist' philosophy, on the other hand, asserts that these assumptions are unwarranted, that 'facts' and 'truth' are a chimera, that 'objective' observation is impossible, and that the act of observation-and-interpretation is dependent on the perspective adopted by the observer.

Interpretivists criticise even the physical scientists for the narrowness of their assumptions. Their criticisms strike home particularly strongly in the social sciences, where the objects of study are influenced by so many factors, and are extremely difficult to isolate and control in experimental laboratory settings. See, for example, Boland (1985), Hirschheim & Klein (1994), and Walsham (1993, 1995a).

The interpretivist approach confronts the difficulties presented by the nature of the research domain, and in particular:

This leads to a requirements that multiple interpretations of the same phenomena must be allowed for, and that no truth is attainable.

A further approach, philosophically distinct but usefully dealt with here (Myers 1997a), is the 'critical theory' school of thought. This considers that a researcher is trapped within the status quo, and that research involves the surfacing of the tensions and working towards their relaxation.


3.3 The Engineering Approach to Research

Within the information systems segment of the computer science and engineering discipline, the research that is most directly relevant to electronic commerce is of an engineering rather than a scientific orientation, and is essentially concerned with technology, including artefacts, techniques and combinations of both of them.

Information systems research undertaken within this tradition tends to be applied or problem-solving in its orientation. It is of two broad types:


4. Research Techniques

Research techniques can be distinguished firstly on the basis of whether or not they are empirical, that is to say that they involve observation of the 'real world'.

Among the empirical techniques, 'scientific' techniques are distinguishable from 'interpretivist' techniques on the basis that science claims to be able to achieve a high degree of 'objectivity', in some sense such as the replicability of experiments, whereas interpretivists question whether objectivity is attainable, or even meaningful.

This section categorises research techniques into the following groups:


4.1 Non-Empirical Techniques

The following techniques are detached from real-world data. This is not to say that they are necessarily totally remote or irrelevant, but rather that they are once-removed, depending on synthetic data, or on conceptual thinking about abstractions. The primary techniques are:


4.2 Scientific Research Techniques

The following are common techniques that can be applied by information systems researchers to the electronic commerce research domain, within the scientific tradition:

A number of additional techniques may be applied within either a scientific or an interpretivist context. They are listed below.


4.3 Interpretivist Research Techniques

The following are techniques which are unequivocally interpretivist in their style:

Additional techniques, which may be applied within either a scientific or an interpretivist context, are listed below.


4.4 Research Techniques at the Scientific/Interpretivist Boundary

Several research techniques can be applied within either a scientific or an interpretivist context. In each case. there are differences in the detailed application of the technique, because work undertaken within the scientific tradition requires careful attention to the rigour with which the instrument is designed and validated, and the data is captured and analysed; whereas, in the interpretivist tradition, the focus is on openness to alternative perspectives and on the identification of ambiguities in the data and the setting.

The techniques include:


4.5 Engineering Research Techniques

Information systems research conducted within the computer science and engineering context uses two categories of research technique:


5. The Quality of IS Research

Underlying all discussions about research is the question of what quality means. This in turn depends on the objectives that are ascribed to it.

If the primary purpose of research is to increase the pool of abstract knowledge, then the criteria of quality are inherently internal, and rigour is critical. If, on the other hand, the primary purpose of research is to contribute to the practice of information systems (instrumentalism), then the criteria are external, and relevance is much more important, and rigour of process is a contraint. The relevance-versus-rigour debate is reawakened from time to time. See Keen (1980), Benbasat & Zmud (1999), and Lee (1999).

Whether the pure research or the instrumentalist perspective is adopted, many observers feel that the quality of information systems research leaves much to be desired. Until the mid-1980s, conceptual (i.e. non-empirical) papers dominated the literature, even in the most highly-regarded journals. Despite this, there has been, and remains, a shortage of theories, and even an absence of a 'cumulative tradition' which would lead to the establishment of theories (Keen 1980). Where pre-theoretical 'research frameworks' have been established, they have not been used to any great extent. Conceptual, theoretical contributions continue to be common, and there continues to be a high incidence of articles excusing themselves from the harsher tests of rigour by depicting themselves as 'exploratory research'.

There has been, and remains, a shortage of validated constructs (i.e. conceptually-expressed notions for which operational definitions have been established and tested in a variety of settings), and of validated survey instruments. See Straub (1989). Some maturation has been noticeable recently, however, with the establishment of a database of instruments (Newstead et al. 1998). In addition, particularly prior to the last decade, the quality of statistical understanding and analysis demonstrated by researchers undertaking research in the scientific tradition was relatively low (Baroudi & Orlokowski 1989, Pervan & Klass 1992, Grover et al. 1993). A great deal of case study research has been undertaken in such a manner that it could provide nothing more than a free-standing instance of interest, possibly useful in teaching, but of no research significance Yin (1984), Benbasat et al. (1987) and Lee (1989).

Empirical research has been dominated by descriptive approaches. Most models have limited explanatory power, and even less predictive value, let alone clear prescriptive implications.

There has been inadequate application of the principle of 'triangulation', i.e. the gathering of data from multiple sources, and the comparison and cross-analysis of that data. Of particular importance has been the inadequate exploitation of the complementariness between quantitative and qualitative data, and a marked unpreparedness of those working in the scientific tradition to accept qualitative data as having a place in research. See, for example, Jick (1979) and Kaplan & Duchon (1988). The attitude of researchers working in the scientific tradition sometimes goes to the extreme of denying that the interpretivist approach is empirical; and some interpretivists are dismissive of the meaningfulness of research conducted in the scientific tradition.

A further serious concern has been that, even though the study of time-variant phenomena necessitates the use of longitudinal rather than cross-sectional studies, longitudinal studies have remained uncommon (Vitalari 1985, Vitalari & Venkatesh 1991). This reflects, in part, deficiencies in the funding of research, the short time available to doctoral candidates to conduct the empirical component of their thesis work, and the career-structures available to researchers.


6. Application to Electronic Commerce

The preceding sections have provided a scan of information systems research, designed to establish a firm foundation for a discussion of research in the electronic commerce domain. The following sub-sections consider quality factors, identify challenges, and examines some specific categories of research.


6.1 Definition and Scope

The research domain of electronic commerce is capable of a variety of definitions. In particular, it could be as broad as any form of business undertaken with the assistance of electronic tools, or only those that involve trading. There are many phases through which trading passes; and many sub-domains depending on the particular categories of goods and services, on the nature of the trading parties, and on the social and technological networking used.

For discussions of the term's definition and scope, see Clarke (1993 and 1997) and Zwass (1996).


6.2 Quality Considerations

Each instance of information systems research in the domain of electronic commerce requires:

In order to satisfy contemporary expectations, research in electronic commerce needs to exhibit the following characteristics:

The following, more specific considerations are suggested as being particularly important:

An application of these quality factors is to be found in the selection criteria for the Outstanding Paper Award at the annual Bled Electronic Commerce Conference.


6.3 Challenges

On the basis of the preceding discussion, specific difficulties confronting research into electronic commerce can be identified:

Research in any new domain is highly risk-prone. Research in electronic commerce is especially challenging.


6.4 Instances

It is useful to consider some common instances of research in electronic commerce, and to reflect on the extent to which they are subject to the difficulties identified above.

These quick reviews of particular kinds of EC research further evidence the extent and depth of the challenges confronting the researcher.


7. Conclusions

Research in the domain of electronic commerce is directly confronted by the 'relevance versus rigour' debate. It is crucial that researchers:

This author doubts the wisdom of pure research in a domain as directly business-related as electronic commerce, and suggests that the instrumentalist motivation is far more appropriate. This implies that researchers need to achieve a degree of relevance and external validity appropriate to the needs of the information systems discipline, and relevance and applicability to the needs of the information systems profession and its clientele, subject to a sufficient degree of rigour , internal quality and external validity.


Bibliography

Alavi M. & Carlson P. (1992) 'A Review of MIS Research and Disciplinary Development' J. Mngt Inf. Syst. 8,4 (Spring 1992) 45-62

Alavi M., Carlson P. & Brooke G. (1989) 'The Ecology of MIS Research: A Twenty Year Status Review' Proc. 10th Int'l Conf. Info. Sys., 1989, 363-375

Banville C. & Landry M. (1989) 'Can the Field of MIS Be Disciplined?' Commun. ACM 32,1 (January 1989) 48-60

Barki H., Rivard S. & Talbot J. (1988) 'An Information Systems Keyword Classification Scheme' MIS Qtly 12, 2 (1988) 298-322

Barki H., Rivard S. & Talbot J. (1993) 'A Keyword Classification Scheme for IS Research Literature: An Update' MIS Quarterly 17,2 (June 1993) 209-226

Barbour R.S. & Kitzinger J. (1998) 'Developing Focus Group Research : Politics, Theory and Practice' Sage, 1998

Baroudi J.J. & Orlikowski W.J. (1989) 'The Problem of Statistical Power in MIS Research' MIS Qtly 13,1 (March 1989) 86-106

Baskerville, R. and A. T. Wood-Harper (1998) 'Diversity in information systems action research methods' European Journal of Information Systems 7, 2 (1998) 90-107

Benbasat I. (1984) 'An Analysis of Research Methodologies' in McFarlan (1984), 47-88

Benbasat I. (Ed.) (1990a) 'The Information Systems Research Challenge: Experimental Research Methods' Vol. 2, Harvard Bus. School, 1990

Benbasat I. (1990b) 'Laboratory Experiments in Information Systems Studies with a Focus on Individuals: A Critical Appraisal' in Benbasat (1990)

Benbasat I. & Lim, L.H. (1993) 'The Effects of Group, Task, Context and Technology Variables on theUsefulness of Group Support Systems: A Meta-Analysis of Experimental Studies' Small Group Research (November 1993), pp. 430-462

Benbasat I., Goldstein D. & Mead M. (1987) 'The Case Study Research Strategy in Studies of Information Systems' MIS Qtly 11,3 (September 1987) 369-386

Benbasat I. & Weber R. (1996) 'Rethinking Diversity in Information Systems Research' Info. Sys. Research 7, 4 (December 1996) 389-399

Benbasat I. & Zmud R.W. (1999) 'Empirical Research in Information Systems: The Practice of Relevance' MIS Qtly 23, 1 (March 1999) 3-16

Boland R. (1985) 'Phenomenology: A Preferred Approach to Research in Information Systems' in Mumford et al. (1985)

Cash, J.I. & Lawrence, P.R. (Eds.) (1989) 'The Information Systems Research Challenge: Qualitative Research Methods' Volume 1, Harvard Business School, 1989

Cavaye A.L.M. (1996) 'Case Study Research: A Multi-faceted Research Approach for IS' Information Systems J. (1996) 227-242

Cheon, M., Grover, V. & Sabherwal, R. (1993) 'The evolution of empirical research in IS: A study in IS maturity' Information and Management, 24, 5, (February 1993) 107-119

Clark P.A. (1972) 'Action Research and Organization Change' Harper & Row, 1972

Clarke R. (1992) 'Fundamentals of 'Information Systems'', September 1992, at http://www.rogerclarke.com/SOS/ISFundas.html

Clarke R. (1993) 'EDI Is But One Element of Electronic Commerce' Proc. 6th International EDI Conference, Bled, Slovenia, June 1993, at http://www.rogerclarke.com/EC/Bled93.html

Clarke R. (1997) 'Electronic Commerce Definitions', January 1997, at http://www.rogerclarke.com/EC/ECDefns.html

Clarke R. (1997) 'Data Surveillance: Theory, Practice & Policy', July 1997, at http://www.rogerclarke.com/DV/ICIS97.html

Clarke R. (1999) 'Focus Groups', March 1999, at http://www.xamax.com.au/Res/FocusGrps.html

Cook, T. D.& Campbell, D. T. (1979) 'Quasi-experimentation: Design and Analysis Issues for Field Setting', Rand-McNally, 1979

Cooper D.R. & Schindler P.S. (1998) 'Business Research Methods' Irwin, 6th Edition, 1998

Cooper H.M. (1998) 'Synthesizing Research, A Guide for Literature Reviews' Sage Publications, March 1998

Culnan M.J. (1986) 'The Intellectual Development of Management Information Systems, 1972-1982: A Co-Citation Analysis' Mngt Sci. 32 (1986) 156-172

Culnan M.J. (1987) 'Mapping the Intellectual Structure of Management Information Systems, 1980-1985: A Co-Citation Analysis' MIS Qtly 11 (1987) 341-353

Culnan M.J. & Swanson E.B. (1986) 'Research in Management Information Systems, 1980-1984: Points of Work and Reference' MIS Qtly 10 (1986) 289-302

Curran . & Blackburn R. (2000) 'Researching the Small Enterprise', Sage, 2000

Delbecq A.L., Van de Ven A.H. & Gustafson D.H. (1975) 'Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes' Scott Foresman, 1975

DeSanctis G. (1990) 'Small Group Research in Information Systems: Theory and Method' in Benbasat (1990), 53-82

Dingwall R. & Miller G. (1997) 'Context and Method in Qualitative Research' Sage, 1997

Ein-Dor P. & Segev E. (1993) 'A Classification of Information Systems: Analysis and Interpretation' Information Systems Research, 4, 2 (June 1993) 166-204

Eisenhardt K. (1989) 'Building theories from case study research' Academy of Management Review, 14, 4 (1989) 532-550

Emory W.C. (1985) 'Business Research Methods' Irwin, 3rd Ed.

Farhoomand A. F. (1987) 'Scientific Progress of Management Information Systems' Database, 18 (1987) 48-546

Frey J.H. (1989) 'Survey Research by Telephone', Sage, 1989

Gable G.G. (1994) 'Integrating Case Study and Survey Research Methods: An Example in Information Systems' Eur. J. Info. Syst. 3,2 (1994) 112-126

Galliers R.D. (1985) 'In Search of a Paradigm for Information Systems Research' in Mumford et al. (1985), 281-297

Galliers R.D. (1990) 'Choosing Appropriate Information Systems Research Approaches: A revised Taxonomy' in Nissen et al. (1990), pp. 155-173

Galliers, R. D. (Ed.) (1992a) 'Information Systems Research: Issues, Methods and Practical Guidelines', Blackwell, 1992

Galliers, R.D. (1992b) 'Choosing Information Systems Research Approaches', in Galliers (1992), 144-162

Galliers R.D. (1993) 'Research Issues in Information Systems', Journal of Info. Technology 8, 2 (1993) 92-98

Galliers R.D. & Currie W. (Eds.) (1999) 'Rethinking Management Information Systems : An Interdisciplinary Perspective' Oxford University Press, 1999

Galliers R.D. & Land F.F. (1987) 'Choosing Appropriate Information Systems Research Methodologies' Commun. ACM 21,12 (December 1987) 900-902

Geertz C. (1973) 'The Interpretation of Cultures' Basic Books, New York, 1973

Glaser B.G. & Strauss A.L. (1967) 'The Discovery of Grounded Theory: Strategies for Qualitative Research' Aldine, 1967

Grover V. (1997) 'A Tutorial on Survey Research: From Constructs to Theory', at http://dmsweb.badm.sc.edu/grover/survey/MIS-SUVY.html

Grover V., Lee C. & Cheon, M. (1992) 'Research in MIS: Points of Work and Reference' Database (Spring 1992) 21-29

Grover V., Lee C.C. & Durand D. (1993) 'Analyzing Methodological Rigor of MIS Survey Research from 1980-1989' Info. & Mngt 24 (1993) 305-317

Gummesson E. (1991, 1999) 'Qualitative Methods in Management Research', Sage, 1st Ed. 1991, 2nd Ed. 1999

Hamilton S. & Ives B. (1982) 'MIS Research Strategies' Information & Management 5 (December 1982) 339-347

Hart C. (1998) 'Doing a Literature Review : Releasing the Social Science Research Imagination' Sage, 1998

ImaginationHarvey L. & Myers M.D. (1995) 'Scholarship and Practice: The Contribution of Ethnographic Research Methods to Bridging the Gap' Information Technology & People 8, 3 (1995) 13-27

Hersen M. & Barlow D.H. (1976) 'Single-Case Experimental Designs: Strategies for Studying Behavior' Pergamon Press, New York, 1976

Hirschheim, R.A. (1985) 'Information Systems Epistemology: An Historical Perspective', in Galliers (1992), pp. 28-60

Hirschheim R.A. & Klein H. (1989) 'Four Paradigms of Information Systems Development' Commun. ACM 32 (1989) 1199-1216

Hirschheim R.A. and Klein H. (1994) 'Realizing Emancipatory Principles in Information Systems Development: The Case for ETHICS' MIS Qtly 18, 1 (March 1994) 83-109

Hofstede G. (1980) `Culture's Consequences: International Differences in Work-Related Values', SAGE Publications, 1980

Hwang M. (1996) "The Use of Meta-Analysis in MIS Research: Promises and Problems," The Data Base for Advances in Information Systems, 27, 3 (Summer, 1996) 35-48

Ives B., Hamilton S. & Davis G.B. (1980) 'A Framework for Research in Computer-Based Management Information Systems' Management Science 26,9 (September 198ß0) 910-934

Jarvenpaa S.L. (1988) 'The Importance of Laboratory Experimentation in Information Systems Research' Commun. ACM 31, 12 (December 1988) 1502-1504

Jarvenpaa S.L., Dickson G.W. & DeSanctis G. (1984) 'Methodological Issues in Experimental IS Research' Mngt Sci 30, 5 (May 1984) 586-603

Jarvenpaa S.L., Dickson G.W. & DeSanctis G. (1985) 'Methodological Issues in Experimental IS Research: Experiences and Recommendations' MIS Qtly 9,2 (June 1985) 141-156

Jenkins A.M. (1985) 'Research Methodologies and MIS Research' in Mumford E., Hirschheim R., Fitzgerald G. & Wood-Harper A.T. (Eds.) 'Research Methods in Information Systems' North-Holland, 1985 103-117

Jick T.D. (1979) 'Mixing Qualitative and Quantitative Methods: Triangulation in Action' Admin. Sc. Qtly 24, 4 (1979) 602-611

Kaplan B. & Duchon D. (1988) 'Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study' MIS Qtly 12 (1988) 571-588

Keen P. (1980) 'MIS Research: Reference Disciplines and a Cumulative Tradition' McLean E. (Ed.), Proc. 1st Int'l Conf. Info. Sys. 1980, 9-18

Kervin J. (1992) 'Methods for Business Research' Harper Collins

King J. & Applegate,L. (1997) 'Crisis in the Case Study Crisis: Marginal Diminishing Returns to Scale in the Quantitative-Qualitative Research Debate'. in Lee et al. (1997), at http://www.hbs.edu/applegate/cases/research/

Klein H.K. & Lyytinen K. (1985) 'The Poverty of Scientism in Information Systems' in Mumford et al. (1985), pp. 131-161

Kling R. (1980) 'Social Analyses of Computing: Theoretical Perspectives in Recent Empirical Research' Computing Surveys 12 61-110

Klein H.M. & Myers M.D. (1999) 'A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems' MIS Qtly 23, 1 (March 1999) 67-93, at http://www.auckland.ac.nz/msis/isworld/MMyers/Klein-Myers.html

Kling R. & Scacchi W. (1982) 'The Web of Computing: Computer Technology as Social Organisation' in Yovits M. (Ed.) 'Advances in Computers (21)' Academic Press, 1982 1-90

Kraemer K.L. (Ed.) (1991) 'The Information Systems Research Challenge: Survey Research Methods' Vol. 3, Harvard Business School, Boston, MA, 1991

Kraemer K.L. & Dutton W.H. (1991) 'Survey Research in the Study of Management Information Systems' in Kraemer (1991), 3-57

Krueger R.A. & Casey M.A. (1994, 2000) 'Focus Groups : A Practical Guide for Applied Research', Sage, 1st Ed. 1994, 3rd Ed. 2000

Kuhn T. (1962, 1970) 'The Structure of Scientific Revolutions' Uni. of Chicago Press, 1st Edition 1962, 2nd Edition 1970

Kumar R. (1999) 'Research Methodology : A Step-by-Step Guide for Beginners', Sage, 1999

Kvale S. (1996) 'InterViews : An Introduction to Qualitative Research InterviewingSage, 1996

Levin M. & Greenwood D.J. (1998) 'Introduction to Action Research : Social Research for Social Change', Sage, 1998

Landry, M. & Banville, C. (1992) 'A Disciplined Methodological Pluralism for MIS' Research, Accounting, Management and Information Technology, 2, 2 (1992) 77-97

Lee A.S. (1989) 'A Scientific Methodology for MIS Case Studies' MIS Qtly 13,1 (March 1989) 33-50

Lee A.S. (1991) 'Integrating Positivist and Interpretive Approaches to Organizational Research' Organization Science 2, 4 (1991) 342-365

Lee A.S. (1999) 'Rigor and Relevance in MIS Research: Beyond the Approach of Positivism Alone' MIS Qtly 23, 1 (March 1999) 29-33

Lee A.S., Liebenau J. & DeGross J.I. (Eds.) (1997) 'Information Systems and Qualitative Research' Chapman & Hall, 1997

Lee B., Barua,A. & Whinston A.B. (1997) 'Discovery and Representation of Causal Relationships in MIS Research: A Methodological Framework' MIS Qtly, 21, 1 (March 1997) 109-136

Leung K. & van de Vijver F. (1997) 'Methods and Data Analysis for Cross-Cultural Research', Sage, 1997

Lyytinen K. (1987) 'Different Perspectives on Information Systems: Problems and their Solutions', ACM Computing Surveys, 19, 1 (1987) 5-44

McFarlan F.W. (Ed.) 'The Information Systems Research Challenge' Harv. Bus. Sch. Press, 1984

Mansell G. (1991) 'Action Research in Information Systems Development' Journal of Information Systems, 1, 1 (1991) 29-40

Marshall C. & Rossman G.B. (1989) 'Designing Qualitative Research' SAGE, Newbury Park CA, 1989

Miles, M. B. & Huberman, A. M. (1984) ' Qualitative Data Analysis: A Sourcebook of New Methods', Sage, 1984

Miles, M. B. & Huberman, A. M. (1994) ' Qualitative Data Analysis: An Expanded Sourcebook', Sage, 1994

Mingers J. & Stowell F. (Eds.) (1997) 'IS Research: An Emerging Discipline' McGraw-Hill, London, 1997

Moustakas C. (1994) 'Phenomenological Research Methods' Sage, 1994

Mumford E., Hirschheim R., Fitzgerald G. & Wood-Harper T. (Eds.) (1985) 'Research Methods in Information Systems' North-Holland, 1985

Myers M.D. (1997a) 'Qualitative Research in Information Systems' MIS Qtly 21, 2 (June 1997) 241-242, at http://www.auckland.ac.nz/msis/isworld/

Myers M.D. (1997b) 'References on Case Study Research', MISQ Discovery on May 1997, at http://www.auckland.ac.nz/msis/isworld/case.htm

Myers M.D. (1999) 'Investigating Information Systems with Ethnographic Research', Communications of the AIS 2 (23), 1-20

Negandhi A.R. (1983) 'Cross-Cultural Management Research: Trend and Future Directions' J. Int. Bus. Stud. 14 17-28

Newsted P., Huff S., & Munro M. (1998) 'Survey Instruments in IS' MISQ Discovery, December 1998, at http://misq.org/discovery/surveys98/surveys.html, living version at http://www.acs.ucalgary.ca/~newsted/tutor.htm

Nissen H.-E., Klein H.K. & Hirschheim R.A. (Eds.) (1991) 'Information Systems Research: Contemporary Approaches and Emergent Traditions' North-Holland, 1991

Nolan R. & Wetherbe J. (1980) 'Toward a Comprehensive Framework for MIS Research' MIS Qtly 4, 2 (June 1980) 1-29

Orlikowski W.J. & Baroudi J.J. (1991) 'Studying Information Technology in Organizations: Research Approaches and Assumptions' Info. Sys. Research 2, 1 (March 1991) 1-28

Patton,M.Q. (1990) 'Qualitative Evaluation and Research Methods' Second Edition, Sage, 1990

Pervan G.P. & Klass D.J. (1992) 'The Use and Misuse of Statistical Methods in Information Systems Research', in Galliers (1992), pp. 208-229

Pinsonneault A. & Kraemer K.L. (1993) 'Survey Research Methodology in Management Information Systems: An Assessment' J. of MIS, 10, 2 (Fall 1993) 75-105

Popper K.R. (1959) 'The Logic of Scientific Discovery', Basic Books, 1959

Popper K.R. (1968) 'Conjecture and Refutations' Harper & Row, 1968

Remenyi D., Money A., Williams B. & Swartz E. (1998) 'Doing Research in Business and Management : An Introduction to Process and Method, Sage, 1998

Robey, D. (1996) 'Research Commentary: Diversity in Information Systems Research: Threat, Promise and Responsibility', Information Systems Research, 7, 4, 400-408

Rosenthal R. (1991) 'Meta-Analytic Procedures for Social Research', Sage, 1991

Rouse A. & Dick M. (1994) 'The Use of NUDIST, a Computerised Analytical Tool, to Support Qualitative Information Systems Research' Information Technology & People 7,3 (September 1994)

Sanday, P. R. (1979) 'The Ethnographic Paradigm(s)' Administrative Science Quarterly, 24 (1979) 527-538

Sprague R. (1980) 'A Framework for Research on Decision Support Systems' MIS Quarterly 4, 5 (1980) 1-26

Stewart, D. W. & Shamdasani, P. N. (1990) 'Focus Groups: Theory and practice', Sage, 1990

Stone E. (1978) 'Research Methods in Organisational Behavior' Scott Foresman, Glenview IL, 1978

Straub D.W. (1989) 'Validating Instruments in MIS Research' MIS Qtly 13,2 (June 1989) 146-166

Straub D.W., Soon A,, & Evaristo R. (1994). "Normative Standards for IS Research," Data Base, 25, 1 (February 1994)

Strauss, A. & Corbin, J. (1990, 1998) 'Basics of Qualitative Research: Grounded Theory Procedures and Techniques', Sage, 1st Ed. 1990, 2nd Ed. 1998

Stringer E.T. (1996, 1999) 'Action Research : A Handbook for Practitioners' Sage, 1st Ed. 1996, 2nd Ed. 1999

Susman G.L. & Evered R.D. (1978) 'An Assessment of the Scientific Merits of Action Research' Admin. Sci. Qtly 23,4 (December 1978) 582-603

Turner J.A. (1990) 'Relevance Versus Rigor in Information Systems Research: An Issue of Quality' in Nissen et al. 1990, pp. 597-599

van de Ven A.H. & Huber G.P. (1995) 'Longitudinal Field Research Methods : Studying Processes of Organizational Change', Sage, 1995

Van Maanen J. (Ed.) (1979) 'Qualitative Methodology' Sage, Beverly Hills

Vitalari N.P. (1985) 'The Need for Longitudinal Designs in the Study of Computing Environments' in Mumford et a. (1985), pp. 243-265

Vitalari, N.P. & Venkatesh A. (1991) 'Longitudinal Surveys in Information Systems Research: An Examination of Issues, Methods, and Applications' in Kraemer (1991), pp. 115-140

Walsham G. (1993) 'Interpreting Information Systems in Organizations', Wiley, Chichester, 1993

Walsham G. (1995a) 'The Emergence of Interpretivism in IS Research', Information Systems Research, 6, 4 (1995) 376-394

Walsham G. (1995b) 'Interpretive Case Studies in IS Research: Nature and Method' European J. of Info. Sys. 4, 2 (1995) 74-81

Watson R., Ho T. & Raman K. (1994) 'Culture: a fourth dimension of group support systems' Commun. ACM 37, 10 (October 1994) 45-55

Weber R. (1987) 'Toward a Theory of Artifacts: A Paradigmatic Base for Information Systems Research' Information Systems (Spring 1987) 3-19

Weber R. (Ed.) (1998) 'Archive of Research Information Solicited via ISWorld', at http://www.commerce.uq.edu.au/isworld/research/

Weber R. & Wand Y. (1995) 'On the Deep Structure of Information Systems' Information Systems Journal 6, 3 (July 1995) 203-223

Westfall R. (1999) 'An IS Research Relevance Manifesto' Communications of the AIS, Vol. 2 (September 1999), Article 14

Wrigley C.D. (1991) 'Research on EDI: Present and Future' Proc. 4th Int'l EDI Conf., Bled Jugoslavia 10-12 June 1991 353-367

Yin R. (1984, 1994) 'Case Study Research : Design and Methods' Sage, 1st Ed. 1984, 2nd Ed. 1994

Zmud, R.W. (1998) 'Conducting and Publishing Practice-Driven Research', in Larsen,T.J., Levine,L. & DeGross, J.I. (eds) Conference Proceedings of IFIP WG8.2 and WG8.6 joint working conference on Information Systems: Current Issues and Future Changes. Helsinki, Finland Dec 10-13, at http://www.bi.no/dep2/infomgt/wg82-86/proceedings/zmud.pdf

Zwass V. (1996) 'Electronic Commerce: Structures and Issues' Int'l J. Electronic Commerce 1,1 (Fall 1996) 3-23, at http://www.mhhe.com/business/mis/zwass/ecpaper.html


Acknowledgements

This paper is a further development on presentations in graduate seminars at the Australian National University between 1989 and 1995, at the Bled Electronic Commerce Conference and at the Johannes-Kepler-Universität Linz in June 1994, at Monash University's Faculty of Business & Electronic Commerce in Churchill, Victoria in April 1996, at the School of Accounting and Information Systems at the University of South Australia, Adelaide, in September 1999, and at the Australian National University's Department of Computer Science in February 2000.



xamaxsmall.gif missing
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax.

From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 65 million in early 2021.

Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer
Xamax Consultancy Pty Ltd
ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 6916

Created: 11 October 1995 - Last Amended: 19 April 2000 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/EC/ResMeth.html
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2022   -    Privacy Policy