Summary and Analysis of
User-design Research by Carr-Chellman & Savoy (2004)
Faculty of Education and Social Work,
The purpose of this paper is to examine a research stream of user-design by summarizing the main points in Carr-Chellman and
Traditionally, end users of a instructional system is consulted through needs assessment conducted in which the instructional designers find the problem and begins to create a solution to it. Seemingly, the end users play an important role in instructional design circle. However, Carr-Chellman and Savoy (2004) point out that while traditional instructional designers analyzes, creates, and negotiates, and the leaders initiate, approve and decide, users are left to accept or reject the innovation.
User design, in contrast, empowers the users to engage authentically in the decision-making process that is design. It is an approach to design that attempts to actively involve the end users in the design process to help ensure that the product designed meets their needs and is usable. This means releasing power from the administrators to the end users. In this case, on the other hand, rather than only being outside researchers ,consultants, or even facilitators, instructional designers must be users as well, which increase their stake in the effectiveness of the design process and final product.
Though user design has empowering potential, many users still need convincing. This is because users may not desire to be involved in the design process where their responsibility and pressure may be increased. So user-design researchers should try their best to make the users aware that their participation will make a difference, the result will be implemented, and the process of design will be an enjoyable experience (Ehn, 1993).
In terms of objective obstacles, a lack of common standardized language of user-design concepts has been hindering research. Also, the varying knowledge levels the end users possess, and the difficulties in generalizing user groups, impose a dilemma for effective user-design.
Currently in literature, user-design empirical research on instructional systems is almost nonexistent. There are fields that are similar to user-design, and research conducted in those particular fields may offer some hindsight regarding the efficiency and applicability of user-design. One of the authors of “User-design Research”, Carr-Chellman, with Cuya & Breman, used the new approach to conduct a pioneer study of home nursing agents (1998), in which they found that user design was very time-consuming and resource hungry. However, Carr-Chellman and
Based on these problems identified, Carr-Chellman and
The discipline of user-design originates from two models, the Scandinavian user design and the stakeholder participation model.
Scandinavian researchers have a long history of user-centered, user-design, and emancipatory design literatures. This empowerment is possible due to the strong position labour unions hold in Scandinavian countries. However, the relationship between Scandinavian user-design research and the application of user design to instructional design is still quite limited. Although Scandinavian user designs’ primary emphasis is on human-computer interface, many of the ideas are useful for instructional design.
Stakeholder participation has been widely researched. Most studies agree that including stakeholders in the decision making process is beneficial to the end result as long as the administrators actually take the stakeholders input into consideration. Understanding the ways in which leaders can enable stakeholders to take a decision-making role in the design of their own systems of human learning is the next step toward effective implementation of technology and educational practices. However, most of the studies about stakeholder participation lack significant methodological attention and rigor, because they use strategies and techniques that mix and match in somewhat less intentional ways and rarely display the rigor associated with excellent qualitative or quantitative inquiry.
One method suitable in user-design research is the PAR model (Participatory Action Research) that focuses on involving the test subject in the research by providing them with all accessible information. It is collaborative research where the “community” is in control (Stoecker, 1999). Participatory action researchers do not define problems ahead of time but determine, in collaboration with the participants, the problems throughout the research process.
User-oriented Design Research
Though PAR is expected to suit user-design research, it still has little specific research conducted on either process or product (Carr-Chellman, Cuya, & Breman, 1998). So basing future user-design research on similar user-oriented design research will be beneficial. Even though research in user-design is rare, there is a relatively large amount of research conducted in user-oriented design fields. According to Carr-Chellman and
These three designs are evaluated focusing on users’ true partnership in the design process. Empirical research in the three areas for the most part showed positive findings for involvement of users in the creation of their own systems part. However, these studies were overly reliant on certain forms of research methods, such as case studies and surveys, and were not holistic in terms of understanding complete disciplines.
For research method, learner-centered design usually uses multiple research methods to process the research, while user-centered design is criticized for being superficial and merely using case studies and surveys as research methods, but most of emancipatory design researches are case based.
As for research emphasis, learner-centered design focuses on learning outcome; user-centered design focuses on appropriate tools to be produced; emancipatory design focuses on the benefit of management for organisations. So the levels of their research subjects are therefore different: individuals, tools or organisations.
As for research hypothesis, although the three designs acknowledge the importance of users’ contribution for developing and improving education, products and management, their internal structures differ. Learner-centered design hypothesizes that the more the learning needs are clear to instructional designers, the better the learning out will be. User-centered design researchers consider that the more the users’ contexts are clear to product designers, the better the product using experience will be. Emancipatory design researchers hypothesize that the change of individuals will lead to the change of organization that they belong to.
As for users’ powers and role, both leaner-centered and user-centered designs treat users as those who may adopt the results and will not really empower users. Although users are interviewed, observed, surveyed and consulted, the final decisions are still made by designers and administrators. Emancipatory design, however, asks users to find ways to create systems for themselves that serve themselves primarily, which leads to users’ power and indigenous knowledge become more powerful and respected than those of the expert designer.
As for the scopes of involvement, learner-centered and user-centered designs limit users’ experience within a specific area such as a classroom or a specific behaviour such as using software. At the same time, the researchers also try to avoid other factors that may influence the experiment in the scopes. However, emancipatory design researchers consider individual users’ immediate daily life as part of the objects to be empowered. This means emancipatory design is to change entire structures that dominate and oppress.
As for data collection and analysis, learner-centered design mainly collects and analyzes information about learners’ needs, learning outcomes from different approaches, etc. User-centered design needs users’ report of experience of using a specific tools and suggestions for improvement. However, emancipatory design seldom collects data for research as most of them are case based, where analysis is processed qualitatively.
As for difficulty to success, learner-centered design researchers always report increased student motivation, fewer disciplinary problems in classrooms, higher student performance and improved student-teacher relations, which seems that the successful rates are very high, though there are factors that could have distorted these results out of the research context. In contrast, the results of user-centered and emancipatory design research are not so promising because they rely on either designers’ skills and experience or individuals’ success.
This paper has summarized the basic definition of user design, the problems existed, the three models for user design, and three similar user-oriented design models. The three user-oriented design models are elaborated by comparing their methodologies from different aspects, such as evaluation focus, research method, research emphasis, research hypothesis, users’ power and role, scopes of involvement, data collection and analysis, and difficulty to success.
Carr-Chellman, A., Cuya, C., & Breman, J. (1998). User-design: A case application in the health care training. Educational Technology Research and Development, 46(4), 97-114.
Carr-Chellman, A. & Savoy, M. (2004). User-design research. In D. H. Jonassen (Ed.), Handbook of Rsearch on Educational Communication and Technology (2ed., pp. 701-716).
Ehn, P. (1993). Scandinavian design: On participation and skill. In D. Schuler & A. Namioka (Eds.), Participatory design: Principles and practices.
Stoecker, R. (1999). Are academics irrelevant? Roles for scholars in participatory research. American Behavioral Scientist, 42(5) 840-854.