Ernst & Young’s Practice of Designing
Global E-Learning Program:
a Review from the ISD Perspectives
Faculty of Education and Social Work,
Ernst & Young is a global leader in assurance, tax, transaction and advisory services, with about 130,000 staff helping clients retain confidence of investors, manage risk, strengthen controls and achieve potential in more than 130 countries in the world (Ernst & Young, n.d.-a, n.d.-b). To standardise or customise their services and make the staff members rely on explicit and tacit knowledge to solve problems, they have used the codification strategy since they frequently reuse their knowledge to achieve long-term advantage and economies of scale (Smith, 2004). To support this, they needed a flexible learning system to provide a global curriculum that all the staff from different offices in the world can participate in (Werner, 2002). On the other hand, using a blend of Web-based and classroom instruction, Ernst & Young reduced training costs by 35 percent while improving consistency and scalability (Bih, 2007). This paper, focusing on Ernst & Young’s Audit Methodology Learning Program, looks deep into the design process of their global program from the ISD perspectives, and provides suggestions for improvement of the e-learning solution.
The Ernst & Young Assurance & Advisory Business Services (AABS) practice consists of financial statement audit, core assurance service, and six specialty assurance and advisory services. To support its AABS strategy, Ernst & Young created a global audit methodology that is organised in three layers: (1) overview of the methodology, (2) detailed guidance for applying the procedures and (3) examples and leading practices. The staff’s learning about this audit methodology would be critical to its successful deployment and application. So Ernst & Young used six months to design and develop the first 300 hours of the core curriculum of the global e-learning program to initiate and support such learning, which included six main stages (Werner, 2002):
(1) Global Learning Committee Construction. The Committee Members were the learning leaders of Ernst & Young’s main geographic areas, who were responsible for defining the learning strategy and the development process, and approving all finished learning modules.
(2) Content Creation. Ernst & Young used a modular approach to create the initial content which was divided into web-based and instructor-led learning modules based on the global audit methodology activities. Ernst & Young also assigned countries to develop the content for the modules related to a particular activity which would be bundled into logical groups later. All the modules were rated beginner, intermediate, advanced and expert.
(3) Streamlined Development and Pilot Process. Once the relevant documents of the learning content had been ready, they used a streamlined development process to accelerate the actual learning module development and the build phase of the program. The Committee allocated the modules to different development teams and provided them with guidance including the detailed development process map, initial design documents, expanded design documents, leader guides, business English guide and roles description for team members (e.g. subject matter specialists, local project managers, etc.). A showcase was leveraged to test the content and gather feedback, in which approximately 80 hours of learning were delivered to the learning leaders and senior managers. The program manager and methodology team analyzed issues identified during showcase testing, and critical issues and suggestions for resolution were sent to the development teams.
(4) Central Communication Point Creation. A central communication point was created to allow every developing team member from multiple countries to access the learning modules under development and make comments. Thus everyone could see what everyone else was developing.
(5) Peer Review. When a learning module was developed in one country, it was systematically reviewed by a subject matter specialist from another country. Countries were asked to submit learning material related to all methodology activities, regardless of the activities their countries were assigned to develop.
(6) Classroom-based Case Study Exercises Development. A separated case study team was created with a member from every country, which was responsible for creating all the information for a fictitious business. The case study was used in many of the classroom modules to create exercises to reinforce learning.
The six stages are more or less overlapped as there were different teams fulfilling responsibilities within each stage. However, the overlapped parts basically only appeared in the latter half of the whole project as the design process is actually a top-down approach (see Figure 1). The lower the levels of the teams were, the more overlapped the phases of their work were.
Figure 1: The Organisation Structure of Ernst & Young’s AABS Global Learning Development Project (Werner, 2002)
The method used for the instructional design of Ernst & Young’s e-learning program is a systems approach. This approach has been described in literature as instructional systems development (ISD) which provides a practical, step-by-step system for evaluating student needs, developing the program content and determining the effectiveness of the instructional design (Hannum & Hansen, 1989). The widely used models for ISD are ADDIE Model which includes analysis, design, development, implementation and evaluation (Branson, 1975), and the Dick and Carey Model which describes nine phases of an iterative process that starts by identifying instructional goals and ends with summative evaluation (Dick & Carey, 1978). Though systems approaches for instructional design have been criticized by some as being too rigid, too cumbersome, too linear, too inflexible, too constraining, and even too time-consuming to implement (Clark, 2004; Kruse, 2006), Ernst & Young still had to develop their e-learning program in this way because they had become a complex system which has so many locations through out the world (O’Leary, 1998; Seng, Zannes, & Pace, 2002). Such context is much different from a single teacher or lecturer designing instruction for the classes. This section of the present paper adopts the latest edition of Dick & Carey Model (see Figure 2) to describe and discuss the design methodologies of Ernst & Young’s e-learning program development project.
Figure 2: The Dick & Carey Model (6th Edition) (Dick, Carey, & Carey, 2004)
The Dick and Carey model does not emphasize the first phase—analysis of ADDIE as Dick et al. (2004) believe that, before instruction is created, it is necessary to determine the need for that instruction in terms of what problem within the organization will be solved through the use of new skills, or what opportunity can be seized because of new skills in the organization. This step is critically important to the success of the design process. Ernst & Young’ e-learning program design decision was consistent with this assertion. In the very beginning, rather than analyzing the skills to be developed, the learners’ entry behaviour or the learning context, Ernst & Young clearly defined their instructional goal which was to promote their global audit methodologies that their staff was required to use consistently while providing clients with assurance and advisory business services. They valued and emphasised this goal to a considerable extent. To ensure the whole design process continually on the right track, they used a high-level committee to support and control the project, which has reinforced the importance of the project to all participants and made the project strongly goal-oriented.
However, Dick et al.’s (2004) not putting analysis phase as the first phase does not indicate that the analysis can be ignored. Rather, they recognise the constructivist points of view that learning is always a unique product "constructed" as each individual learner combines new information with existing knowledge and experiences, so the Dick and Carey Model’s second phase includes analysing the performance of the instructional goal as well as the learners’ entry behaviours, prior knowledge, learning and application context, etc., but this was much difficult for Ernst & Young to implement and actually they have not done much in this phase because the target learners and their contexts varied from places to place.
To make up the lack of analysis of learners and their contexts, Ernst & Young wrote the learning objectives according to the learners of different levels from beginners to experts and the learning content followed was designed based on these assumptions. The other compensation reported by Werner (2002) was to tie the learning content “to the learners’ previous experiences through open-ended questions (in the Web-based modules) and opportunities to share experiences through storytelling (in the instructor-led modules)” (p. 69). Anyhow, Ernst & Young tried their best to minimize the need for localization of the modules.
Without developing any assessment instrument, Ernst & Young directly proceeded to the fifth and six phases to develop instructional strategies (e.g. modular approach, blended approach, etc.) and materials (e.g. web pages, case study exercises, etc.). Once the pilot modules were completed, the formative evaluation was conducted by project managers and methodology team. Also, each completed module was sent to subject matter specialist from a different country for systematic review. However, the objective of this kind of reviews “was to use as much existing material developed by the different countries as possible” (Werner, 2002, p. 72). Therefore, the “real formative evaluation” only existed in the pilot process. And the feedback from the pilot process formed the basis for the revision of the instructional strategy and material. According to Dick et al. (2004), the data from a formative evaluation should not be simply used to revise the instruction itself, but should be used to reexamine the validity of the instructional analysis and the assumptions about the entry behaviors and characteristics of learners. However, because there were no actual learners participating in the formative evaluation, Ernst & Young could not identify the difficulties experienced by the learners in achieving the objectives and relate these difficulties to specific deficiencies in the instruction. Eventually, there were no data for the revision of the analysis phase. In addition, they even had no chance to validate the assumption for the four-level learners’ competencies.
To sum up, Ernst & Young’s e-learning design process basically went through most phases of the Dick & Carey Model but some components and links of the model were “missing” as labeled in Figure 3.
Figure 3: Ernst & Young’s E-learning Design Process Model
Critical Analysis and Problem Solving
The main decisions within Ernst & Young’s e-learning program design process were basically made by high-level staff (e.g. learning leaders, project managers, subject matter specialists, etc.) without much consulting the target learners. The learners were considered in the design and develop phases, but not involved in the actual design process. As learners are the important factors in the analysis, assessment and evaluation phases (Dick et al., 2004) and Ernst & Young’s design process had no learners involved, these three phases tended to be missing and this would lead the design process to be a vicious circle because of lack of effective revision of instruction supported by the data from learners. Such situation would also lead to learner readiness issues, such as lack of time, low interest in subject matter, low motivation for learning, poor self-study skills, poor time management skills, disrupting life interruptions, lack of necessary e-skills, psychological resistance to losing face to face learning perks, etc. (Phillips, 2002; Romiszowski, 2004). Furthermore, because of such situation, the leaders in the organisation would have not enough understanding on the learners’ needs in terms of the learning context, which would lead to poor internal marketing of courses and events, lack of clear reward structure, failure to provide quality learning environment, failure to provide quality learning equipment, failure to provide managerial feedback and support of learning, failure to provide time on-the-job to train, corporate-wide lack of dedication to a learning culture, blanket mandate of e-learning as the new-new thing (ibid).
Ernst & Young’s current instructional design approach is exactly what Carr-Chellman & Savoy (2004) described about traditional instructional design—“the designer analyzes, creates, and negotiates, and the leaders initiate, approve and decide”, and “the users are left to accept or reject the innovation” (p. 702). If users are truly empowered to participating in designing products or processes, the products or processes will tend to create a significantly different adoption process. Therefore, a user-design approach, which indicates that actions such as initiation, approval, rejection, design, and decision making are negotiated among the users , designers and leaders, would be beneficial for Ernst & Young to avoid the disadvantages of the current design process and the potential risks on the organizational level (ibid).
Since user-design is very time-consuming and resource hungry (Carr-Chellman, Cuyar, & Breman, 1998), the suggestion about user-design approach may not be adopted if its disadvantages significantly constrain the e-learning solution to meet the business goal of Ernst & Young, and the possibility of this supposition tends to be large because saving learning and administrative time, training budgets and other resources is still one of the main reasons why corporations use e-learning (Oakes, 2003; Tai, 2008). An eclectic approach would be revising the traditional ISD models so that it can adapt to the context of e-learning that has special needs and constrains (e.g. lower cost, learners’ locations vary, etc.). Carliner (2002) revised the Dick and Carey Model and then proposed a four-phase model to describe the ISD process for e-learning (see Figure 4).
In the Carliner Model, there are no standalone instructional analysis (distinguishing from the analysis of goals and needs of organisations), assessment and evaluation phases and basically learners are not involved in the design process. Some of the components of each phase of the Carliner Model indicate the differences between designing e-learning and designing traditional classroom courses, and these components generally have existed in Ernst & Young’s e-learning design process. For example, in the Definition Phase, Carliner (2002) suggests collecting demographic data about the learners and preparing prose descriptions of them because within an organisation, learners with different backgrounds and of different levels have different appetites for content and use the content in different ways. Similarly, Ernst & Young prepared their e-learning content for four-different-level learners. We can notice that both the Carliner Model and Ernst & Young’s practice tend to construct the learning content that covers most of the learners and the term “learner” that they used actually refers to the abstract, assumptive or even imaginary concept of “learner”, rather than the learners known as actual people. Such situation is much different from traditional classroom where actual learners are known and reachable, and the learning content will be only prepared directly for them.
Figure 4: ISD Process for e-learning (Carliner, 2002)
In the Design Phase, Carliner (2002) emphases using of style guides and templates to ensure the consistency of similar content (screens, text, number, language, formats of headings, etc.). And in Development Phase, effective communication, technical and editorial reviews, and running tests are also required in the context of designing e-learning. Ernst & Young’s design process included the provision of guidance for streamlined development and the review process. Such “additional” work should be done within the ISD for e-learning, which is not emphasised in traditional ISD because the most of the information from the course will be delivered through teachers’ speech which does not need the style guides, templates, review process, etc.
Assessment and evaluation are the two main missing phases for Ernst & Young’s design process as well as the Carliner Model. However, Carliner (2002) still mentioned assessment and evaluation in the Design Phase but disappointingly he only used limited words to address the importance of them and did not clarify whether the actual learners should be involved or not and whether the assessment and evaluation should be implemented in the Definition Phase to obtain the data for the definition of goals. So regarding to the assessment and evaluation phases in ISD, we probably can adopt some thoughts from Borich’s (1979) systematic evaluation model (see Figure 5) to the e-learning design context.
Figure 5: (a) Traditional view and (b) nontraditional view of the evaluator during planning, development, and evaluation (Borich, 1979)
Borich (1979) noticed that, traditionally, planners, designers, developers and evaluators each often begin their work in different phases in the instructional design process and then proposed a new approach involving evaluators to keep contributing through out the whole process. Here the evaluator, rather than entering the scenario late in the development process, plays an integral role in program planning and development alongside planners and developers. This model and Carr-Chellman & Savoy’s (2004) User-Design approach may meet a compromise and be integrated. While one representation of User-Design approach is design team members acting as users (Carr-Chellman & Savoy, 2004), this kind of “users” can keep contributing to the evaluation process within Borich’s (1979) framework. So if Ernst & Young used this systematic evaluation approach in the design process (e.g. using the special “users” to conduct a formative evaluation while senior leaders were reviewing the pilot modules), this would help make up the disadvantages caused by lacking actual learners involved.
To sum up, lack of actual learners involved in Ernst & Young’s e-learning program design process would lead to various problems and a user-design approach is recommended to the e-learning project. However, a user-design approach requires too much time and resources to support, so a revised ISD model was introduced specifically for e-learning design process. Finally, the evaluation issue was resolved by integrating a user-design approach and a systematic evaluation approach so that the drawbacks of lacking actual learner’s involvement would be overcome.
Reflection and Conclusion
Although there are places that could be further improved, Ernst & Young’s practice of global e-learning program development has given us a very good case to help us better understand Instructional Systems Development’s (ISD) application in the real world. ISD has been used for more than three decades. It experienced the change of contexts—from pure face to face instructional design to e-learning program development. In the middle of this change, there has been a need for us to study how bridge the gap between using it in traditional way and in nontraditional way. If new issues emerge, we need to identify opportunities to use other exist approach to resolve them, or to create new approaches to update ISD’s usage in the new context.
In this paper, we have summarised Ernst & Young’s e-learning design process, used a system approach (e.g. the Dick and Carey Model), and critically analyse the issues identified in this paper and finally provided the conceptual solutions for the issues.
Bih, J. (2007). When it comes to e-learning. ACM Ubiquity, 8(43), 1-11.
Borich, G. D. (1979). A system approach to the evaluation of training. In J. H. F. O’Neil (Ed.), Procedures for instructional systems development (pp. 205-231). New York: Academic Press.
Branson, R. K. (1975). Interservice procedures for instructional systems development: Executive summary and model. Tallahassee, FL: Center for Educational Technology, Florida State University.
Carliner, S. (2002). Designing e-learning. Alexandria, VA: ASTD.
Carr-Chellman, A., Cuyar, C., & Breman, J. (1998). User-design: A case application in health care training. Educational Technology Research and Development, 46(4), 97-114.
Carr-Chellman, A., & Savoy, M. (2004). User-design research. In D. H. Jonassen (Ed.), Handbook of Research on Educational Communication and Technology (2nd ed., pp. 701-716). Mahwah, NJ: Lawrence Erlbaum.
Clark, D. (2004). The Dick and Carey Model – 1978. Retrieved June 15, 2008, from http://www.nwlink.com/~Donclark/history_isd/carey.html
Dick, W., & Carey, L. (1978). The systematic design of instruction. Glenview, IL: Scott, Foresman.
Dick, W., Carey, L., & Carey, J. O. (2004). The systematic design of instruction (6th ed.). Boston & London: Pearson/Allyn & Bacon.
Ernst, & Young. (n.d.-a). Creating the right climate for your business success. Retrieved June 15, 2008, from http://www.ey.com/global/content.nsf/International/Services
Ernst, & Young. (n.d.-b). How we make a difference. Retrieved June 15, 2008, from http://www.ey.com/global/content.nsf/International/About_EY
Hannum, W. H., & Hansen, C. D. (1989). Instructional systems development in large organizations. Englewood Cliffs, NJ: Educational Technology Publications.
Kruse, K. (2006). Introduction to Instructional Design and the ADDIE Model. Retrieved June 15, 2008, from http://www.e-learningguru.com/articles/art2_1.htm
O’Leary, D. E. (1998). Using AI in knowledge management: knowledge bases and ontologies. IEEE Intelligent Systems, 13(3), 34-39.
Oakes, K. (2003). E-learning. Training & Development, 57(7), 17-20.
Phillips, V. (2002). Why does corporate e-learning fail? Virtual University Gazette Retrieved June 15, 2008, from http://www.geteducated.com/vug/june02/vug0602c.htm
Romiszowski, A. J. (2004). How’s the e-learning baby? Factors leading to success or failure of an educational technology innovation. Educational Technology, 44(1), 5-27.
Seng, C. V., Zannes, E., & Pace, R. W. (2002). The contributions of knowledge management to workplace learning. Journal of Workplace Learning, 14(4), 138 – 147.
Smith, A. D. (2004). Knowledge management strategies: a multi-case study. Journal of knowledge management, 8(3), 6-16.
Tai, L. (2008). Corporate e-learning: an inside view of IBM’s solutions. New York: Oxford University Press.
Werner, T. (2002). Best practices for e-learning: top entries in the best practices category Sunnyvale, CA Brandon-Hall.