I will be presenting a self-paced online CPD program I recently developed in the MEDFAC E-Learning Showcase 2011. In my ten minute presentation, I will talk about:

• The background of the CCRE Anxiety Education Project
• The design methods, including how to work with a large SME team of 23 members to design peer-reviewed course content under the same framework
• The development process, using Articulate Studio, Text Aloud, Moodle, Adobe CS5, etc.
• The main features of the e-learning program including screenshots and a quick live pre-view of the actual program
• The future directions, including how to improve the process of CPD program development on a continuing basis

If you are interested in medical-related e-learning projects, you are welcome to attend this free event at New Law 026, University of Sydney, Camperdown Campus, at 12:30 on the 23rd of November 2011. This free event will be showcasing e-learning (broadly defined to include LMSs, multimedia, use of social media etc.) within Sydney Medical School – not just innovations but also work in progress, planned etc. The topics will include:

• Introducing LMS skills: a staged approach
• Using Adobe Connect for research consultations and troubleshooting
• Compass: Sydney Medical Program LMS
• Integrating e-portfolios, Blackboard and Compass
• E-learning modules for physiotherapists and junior clinicians for international (resource-poor) and local settings
• Internationalising your online course
• Using the i-pad to create dynamic lecture materials
• E-learning in Opthalmology using Ocean Browser
• Online OSCARS in Dentistry
• A self-paced program in psychiatry using Moodle and Articulate
• Using social media and webinars in student recruitment
• Computer‐based support for psychiatry
• Teaching/Learning Evidence Based Practice WITHOUT – PowerPoint, Quarbon, ‘Clickers’ and Blackboard
• E-learning strategic projects
• Mobile learning
• Researching e-learning in the Faculty

The full program can be downloaded here.

为学龄前儿童而设的网上教学？

——灵格风中国的家长参与方案

 何文超

发表于悉尼大学CoCo研究中心举办的Learning Technology Research Symposium 2008上的海报

海报PDF下载

家长参与的周末学前儿童外语（英语） 面授课程

网上学习解决方案： 灵格风中国案例研究

A dissertation submitted in partial fulfilment of the requirements for the degree of

Master of Learning Science and Technology

by

Wenchao He

wenchao.he@sydney.edu.au

Centre for Research on Computer-Supported Learning and Cognition

Faculty of Education and Social Work

The University of Sydney , Australia

July 2008

学位论文全文

论文研究计划

参与者信息综述

家长参与者同意函

父母或监护人同意函

教师参与者信息综述

教师参与者同意函

Abstract

This case study was driven by the needs of Guangzhou Kids Centre, Linguaphone Group China Branch to explore the feasibility of developing an e-learning solution to promote parental involvement in order to enhance the learning outcomes of the preschooler-level students attending weekend face-to-face English classes. A pilot e-learning project was initiated in two classes (Level 1 and 2) to help the parents to organise weekly family-based English learning activities under the instruction of the teachers. During a five-week period, the parents were given access to an online learning management system where there were weekly instruction packages including the teachers’ summaries of what was taught in class, family-based English learning activity guide with relevant materials, and a feedback forum. The teachers checked and replied the parents’ feedback during the week to provide further assistance. To have a better understanding of parental involvement in the weekend English class and to avoid influences of any potential technical issues related to the e-learning project, an alternative method was taken at the same time in another two classes (Level 1 and 2), where the parents received the same instruction packages in print. The parents completed some questionnaires before and after the project, and were interviewed by the researcher. The four teachers were also interviewed at the end of the project. The results show that most of the parents of Level 2 students actively participated in the project while the participation by the parents of Level 1 students was limited. The E-Learning Group’s and Print Group’s parents participated in the program using different strategies. The teachers’ and parents’ observations seem to suggest that the students’ progress was associated with the degree of activeness of the parents’ participation. Suggestions for the subsequent development of the e-learning solution are made based on the analysis and discussion of the results.

Acknowledgements

Firstly, I am grateful to my supervisor Dr. Chun Hu for her support and help throughout my study. Her valuable suggestions and comments on my work gave me a strong direction to move further. Discussing with her usually triggered much new idea conducive to the subsequent research procedure. She cared not only my study but also my life in Australia, a foreign country for me. This has led to my confidence and enjoyableness during the study.

I thank Linguaphone China and Ask Idea Educational Studio for providing this study with funding, relevant materials and continual support.

I thank Mr. Yaowei Situ and Mr. Tim Ng for their promoting my proposed parental involvement program to be implemented in Guangzhou Kids Centre, Linguaphone China. Without their support, I could not conduct the case study with Linguaphone.

I thank the four teachers’ participation in this study. They have worked with me collaboratively within the parental involvement program. Without their support and effort, I could not implement the project and conduct the study internationally.

Finally, and certainly not least, my greatest thanks are due to my fiancée for cheerful encouragement and unfailing support during the study. And I thank my parents who are living in China but always encourage me to overcome difficulties via synchronous chat on the Internet.

Licensing

The author of this dissertation, Mr. Wenchao He, owns the copyright of this dissertation which is protected by Copyright Act 1968 and other applicable law in Australia . Without the copyright holder’s written permission, any use of this dissertation other than as authorised under this license is prohibited.

On 1 July 2008, this dissertation will be submitted to Faculty of Education and Social Work, University of Sydney for examination. The Faculty and University of Sydney Library and its agents will be permanently licensed to hold the hard and electronic copies of this dissertation which can be made available to the staff and the students of University of Sydney . Once this dissertation has been submitted, Linguaphone Group and Ask Idea Educational Studio will be respectively granted a permanent, irrevocable, free, world wide, non-exclusive license (including a right of sublicense) to use, reproduce, adapt and exploit the intellectual property rights in this dissertation for any commercial or noncommercial purpose.

From 1 July 2008 to 31 December 2008, the full text of this dissertation should be kept partially confidential and should not be copied, distributed, displayed or reproduced out of the scopes of University of Sydney, Linguaphone Group and Ask Idea Educational Studio. The author and any other staff and student in the three organisations who has received this dissertation should only use the dissertation for their work, study, or research within the three organisations and should not forward the full text of this dissertation to any other party or upload the full text of this dissertation to the Internet to make it available to the public.

From 1 January 2009, this dissertation will be made available to the public and become licensed unde Australian Creative Commons Attribution-Noncommercial-No Derivative Work 2.5 License , which is:

You are free:

to Share — to copy, distribute and transmit the work

Under the following conditions:

Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

Noncommercial. You may not use this work for commercial purposes.

No Derivative Works. You may not alter, transform, or build upon this work.

Any of the above conditions can be waived if you get permission from the copyright holder.

Nothing in this license impairs or restricts the author’s moral rights.

Any request for any use of this dissertation other than as authorised under the above licensing statement can be forwarded to the copyright holder, Wenchao He, who can be contacted at wenchao.he@sydney.edu.au.

List of Contents

1    Introduction. 1

1.1 Problem Statement. 1

1.2 Possible solution. 2

1.3 Purposes of the Study and Research Questions. 2

2    Literature Review. 4

2.1 Parental Involvement. 4

2.1.1 Parental Involvement’s Effectiveness. 4

2.1.2 Parent-school Partnership. 5

2.1.3 Communicative Issue of Parental Involvement Programs in Kindergartens. 6

2.1.4 Parental Involvement Possibility. 7

2.2 E-learning. 8

2.2.1 E-learning’s Representations. 8

2.2.2 Networked Learning. 9

2.2.3 E-learning in a Family Context 10

2.2.4 Using Technology to Promote Family-School Connection. 12

2.3 Second/Foreign Language Learning. 13

2.3.1 Family-based Linguistic Input and Interaction. 13

2.3.2 The Transitional Shift of CALL. 15

3    Methodology. 18

3.1 Research Method Selection. 18

3.2 Participants. 20

3.3 Instruments. 21

3.3.1 Questionnaires. 21

3.3.2 Telephone Interview. 25

3.4 Procedure. 27

3.4.1 Parental Involvement Program. 27

3.4.2 Data Collection. 28

3.4.3 Data Analysis. 28

4    Results. 30

4.1 Overview. 30

4.1.1 Overview of the Students and the Parents. 30

4.1.2 Response Rates. 33

4.1.3 Participation Rate. 34

4.2 Parental Involvement in Children’s English Learning. 35

4.2.1 Parental Modeling in Children’s English Learning. 35

4.2.2 Parental Involvement Preferences. 39

4.2.3 Summary. 43

4.3 E-learning Solution Supporting Parental Involvement. 43

4.3.1 Parent Involved Networked Learning Model 43

4.3.2 Promoting Connections. 44

4.3.3 Summary. 49

4.4 Enhancing the Outcomes of Children’s English Learning. 49

4.4.1 Enhancing English Input and Interaction by CALL. 50

4.4.2 Let the Students Know about their Parents’ Participation. 52

4.4.3 Multiple Language Representations. 53

4.4.4 Parent-Teacher Face-to-Face Communication. 54

4.4.5 Customised Instruction in Class. 54

4.4.6 Parents’ Learning English for Themselves. 55

4.4.7 Summary. 56

5    Discussion and Conclusion. 57

5.1 Summary of the findings. 57

5.2 Implications and Suggestions. 58

5.2.1 Multiple Solutions. 58

5.2.2 Instruction on Parental Involvement 58

5.2.3 Promoting Connections. 59

5.2.4 Promoting English Input and Interaction. 59

5.3 Limitations of the study. 60

5.4 Future research. 61

6     References. 62

Evaluation Proposal for an Online

IELTS Writing Tutoring System

Wenchao He

Faculty of Education, University of Wollongong, Australia

1    Introduction

This plan describes the background, purposes, limitations, stakeholders, decisions, questions, methods, participants, instruments, procedures, logistics, and timeline for the evaluation of Chao Yue IELTS Online Tutoring System, which is an e-learning program for IELTS candidates who have purchased IELTS preparation books on www.cyielts.com.au. The methodology, procedures, and instrumentation included in this plan are based on several meetings between members of the development team and the evaluation team, as well as a review of several draft design documents. The primary objective of this evaluation is to provide timely and accurate information to support decisions regarding the enhancement and marketing of the e-learning program. The evaluation project will involve students and teachers who are to go through the standard instructional process and complete tests and/or questionnaire and/or interview. The system log will also be used for data analysis. The evaluation project will start on 1 June 2009 and finish by the end of the month. The estimated cost will be $27,200. This plan is to be reviewed, revised and approved in the first week of June.
 

2    Background

Chao Yue IELTS Online Tutoring System (http://e-learning.cyielts.com.au) is a Moodle-based learning management system providing IELTS candidates with tutoring service for their IELTS preparation. IELTS stands for International English Language Testing System. People from non-English speaking countries should take IELTS test and use the test result to apply for courses and/or jobs offered in English speaking countries (e.g., Australia). IELTS result is also a requirement for applying for a permanent visa and/or immigration to English speaking countries. Chao Yue IELTS sells IELTS preparation books online and provides the customers with tutoring services. The operation is that the students firstly purchase IELTS preparation books online, then receive and use the books for self-study. Once they have any difficulties in studying the materials, they can ask the online tutors for help on the e-learning system. When they complete an IELTS writing task, they are to submit the essays to the e-learning system and the tutors will provide them with feedback and a web-based customised small course with various learning activities. The tutoring service for the first essay that the student submits is free of charge. After the first round, the student can decide to pay for extra writing tutoring service. So basically, Chao Yue IELTS’s e-learning system offers two kinds of services – general questions answering and IELTS writing tutoring. This evaluation will focus on the IELTS writing tutoring part of the system. A detailed tutoring process is illustrated in Figure 1.

Besides the one-on-one tutoring mode in the IELTS Writing Tutoring Section, once the tutors identify the students’ common problems in IELTS writing, they will organise some instructional materials and make them available to the students. The tutors also moderate an online learning community where students discuss about their learning and help each other.
 

•    Figure 1    Chao Yue IELTS Writing Tutoring Process

 
3    Purposes

The overall purpose of this evaluation is to provide the directors and tutors at Chao Yue IELTS with the timely, accurate information required to support decisions regarding the enhancement and marketing of the IELTS Writing Tutoring program. This evaluation is primarily an effectiveness evaluation in the sense that the information collected will be used as the basis for determining whether the program accomplishes its objectives within the immediate or short-term context of its implementation. As a result of this evaluation and the decisions and actions stemming from it, the e-learning program should be ready for being further promoted in the IELTS preparation education market, or should be enhanced before being further promoted.
Specific sub-purposes of this evaluation are:

*    to collect information for promoting the e-learning program,
*    to investigate the reasons why the students subscribe to or not subscribe to extra tutoring service after they receive the first round of service for free,
*    to measure to what extent the program helps the students improve their English writing skills for IELTS, and
*    to decide if the program needs to be modified for students’ better learning outcome.

 
4    Stakeholders

The clients for this evaluation are the managers of Chao Yue IELTS. Important primary stakeholders include the tutors and marketing personnel involved in the development effort and the students of the e-learning system. The technical staff who is in charge of maintaining the e-learning system and the customer service staff may also be viewed as secondary stakeholders in this evaluation, but dissemination of the evaluation plans and results will be controlled by the clients.

 
5    Decisions

To empower this evaluation to provide timely and accurate information to support decision making, we have identified a set of anticipated decisions that will be made about the e-learning program. Most of these decisions must be made regardless of the quantity and quality of information available to the decision makers, but the evaluation will ensure that the decisions are informed by the best possible information. The following decisions are anticipated:

*    Advertising and marketing decisions will be made about the e-learning program by the managers of Chao Yue IELTS.
*    Modifications will be made in the e-learning program to improve its effectiveness and appeal.
*    Strategies will be made for persuading students subscribe to extra IELTS Writing tutoring service after the free-trial.
 

6    Questions

The following questions will be addressed during this effectiveness evaluation:

*    How are the knowledge and skills of the students changed by the program?
*    What are the students’ reactions to free trial of the program?
*    What are the factors in students’ decision to subscribe or not to subscribe to the program after free trial?
*    What are the fee-paying students’ reactions to the program?
*    What corrections must be made to the program?
*    What enhancements can be made to the program?
 

7    Methods

7.1    Data Collection Matrix

No single evaluation design can encompass the six major questions specified for the evaluation of the IELTS Writing Tutoring e-learning program. Therefore, a mixed methods evaluation design and multiple data collection strategies will be utilised to collect the information required to address these questions. The data collection strategies include questionnaire, interview, automatic data collection and tests. Table 1 specifies the data collection methods and their corresponding evaluation questions.

•    Table 1    Data Collection Matrix

7.2    Learner Questionnaire

7.2.1    Questionnaire Design

Questionnaires are one of the most frequently used methods of collecting data for effectiveness evaluation (Reeves and Hedberg 2008). The Learner Questionnaire for this evaluation will be addressing all the six research questions. The current enrolled and past non-paying and fee-paying students will be invited to complete the questionnaire. Although the questionnaire may be anonymous, the two groups of students will be labelled so that we can differentiate their responses. The objectives of the questionnaire are as followed:

*    to clarify what kinds of benefit in terms of knowledge and skills, if any, the students have got from the program,
*    to let the students recall and specify their reactions to the free trial,
*    to know about the subscribers and non-subscribers’ decision making process in terms of subscription,
*    to let the fee-paying students recall and specify their reactions to the program, and
*    to obtain the data about the weakness and strengths of the program from the students’ perspective.

Generally, the questionnaire will predefine a series of variables represented as a list of 5-point Likert scaling questions. The questionnaire will be web-based so it can be completed on the Internet. The respondents will be given an option to address if they are willing to be contacted for a follow up e-mail or telephone interview.

7.2.2    Rationale of the Questionnaire

A questionnaire is an effective tool to gathered responses in a standardised and objective way. It does not require too much time and effort for evaluators to prepare and analyse, and for students to complete (see Table 2). So this method is relatively quick to collect information. In this evaluation context, a web-based questionnaire is easy to develop in Moodle because this is an existing function built in the system.

•    Table 2    Requirement of Questionnaire (Harvey 1998)

However, it is believed that questionnaire’s response rate could be low, specially the questionnaire can not be delivered and responded to in face-to-face class time (e.g., in an e-learning context). This is because students may not be willing to answer the questions. They might not wish to reveal the information or they might think that they will not benefit from responding. Furthermore, the students may answer superficially especially if the questionnaire takes a long time to complete (Harvey 1998). Thus, we should simplify the questionnaire and most of the questions designed are of 5-point Likert scaling type. Where possible, every respondent should have an opportunity to enter a draw to prizes.

7.3    Teacher Interview

7.3.1    Interview Design

In the teacher interview session, the teachers who are offering IELTS writing tutoring service at Chao Yue IELTS will be asked semi-structured questions regarding the students’ progress and the quality of the program. The objectives of the interview are:

*    to obtain the information about the teachers’ impression and observation of the students’ progress, and
*    to let the teachers address their concerns about the design of the program including the instructional process detailed in Figure 1, the current configuration of the Moodle system, and the instructional materials they are using for developing customised small courses for the students.
The semi-structured interview will start with broad questions and then the teachers are free to add extra information to help achieve the objectives of the interview.

7.3.2    Rationale of the Interview

Generally, there are three types of interview design: (1) standardised interview, where questions are asked in the same way across a sample population by different interviewers, (2) semi-structured interview, where the interaction is kept focused on the same ground while allowing individual experience to emerge, and (3) conversational interview, where the evaluator responds quickly to individual differences and situational changes. According to Harvey (1998), all the three types of interview require much time and effort for preparation, conducting the interview and analysis afterwards (see Table 3). However, as the sample for the interview is not large – less than 5, this part of the evaluation would not be extremely time-consuming. Amongst the three types of interviews, the semi-structured one is adopted because the anticipated information from the interview is based on the teachers’ tacit impression, if we choose standardised interview, some important information may be missed; if we choose conversational interview, the teachers may provide too much information which is irrelevant to the predefined evaluation questions.

•    Table 3    Requirement of Interview (Harvey 1998)

7.4    Automated Data Collection

7.4.1    Automated Data Collection Design

In the Moodle system, users’ every single activity will be logged. The e-learning program includes a number of discrete modules, with each module containing several different types of instructional options, e.g., tutorials, quizzes, practice exercises, and mastery tests. The students’ paths through the modules, their choices among the instructional options, and their quiz and test scores, will all be recorded. However, not all the logged data will be used for the evaluation. Our focus is on the following aspects:

*    the frequency of usage,
*    the time spent on each activity and/or resource,
*    exit page (on which page the users are likely to exit the system), and
*    the comparison between expected activity paths and actual activity paths.

The primary objective of collecting the information about the above aspects is to presume the users’ attitude towards the program (e.g., high frequency and moderate time of usage may imply positive attitude while unexpected and sudden exit could be a sign of dislikes). Another objective is to identify any technical problems within the program which may be indicated by unexpected exit and/or unexpected activity paths.

7.4.2    Rationale of Automated Data Collection

Automated data collection is a step-by-step recording of user interaction with an e-learning program. The level of detail recorded is determined by a purpose-built program and can include mouse clicks, menu calls, and all operations on objects. Modelled on a “record” facility, log data is a useful way of tracking user preferences and navigational choices. This approach does not require any attention from the students and administration because once the logging function is turned on, everything is done automatically. However, the preparation time could be long because not all the data is useful for evaluation purpose (see Table 4). Evaluators need to determine what data needs to be logged, how it should be recorded and summarised and what level of detail of activities should be logged (Harvey 1998).

•    Table 4    Requirement of Automated Data Collection (Harvey 1998)

In the current evaluation context, the logged data is a useful resource for presuming users’ reaction and help refine the questionnaire and interview protocol. The reason that the logged data should be linked to the questionnaire and the interview protocol is that the data does not answer “why” questions and further evaluation is necessary to find explanations for occurring phenomena (Harvey 1998). We acknowledge that this approach has such nature so the logged data will not form a strong evidence for the evaluation result.

7.5    Pre- and Post-Testing

7.5.1    Pre- and Post-Testing Design

As the e-learning program being evaluated is an exam preparation program, to determine its effectiveness, a pre- and post-testing approach should be applied and the result can demonstrate if students are making progress or not. The tests are to solely answer the first evaluation question – “How are the knowledge and skills of the students changed by the program?” The subject of the tests will be the fee-paying students who have paid for extra five rounds of IELTS writing tutoring process (see Figure 1). The objective of the tests is to measure to what extent the five rounds of IELTS writing tutoring service has helped the students raise their IELTS writing scores.

7.5.2    Rationale of the Testing

Tests are widely accepted as an effectiveness evaluation strategy, but we should be wary of tests unless you have evidence of their reliability and validity (Reeves and Hedberg 2008). Pre- and post-testing’s result could be easily affected by external factors which influence student learning. It would be very wrong to assume that we can discount a variety of external factors which will affect student performance when considering the reasons for variation between pre and post test responses (Harvey 1998). The reason why this approach requires long time on preparation (see Table 5) is not only about the test questions preparation but also about organising the students (e.g., trying every effort to exclude external factors, recruiting at least 60 students in order to allow us to make reliable statistical conclusions from the data, etc.).

•    Table 5    Requirement of Pre- and Post-Testing (Harvey 1998)

Despite that, we still need to include this approach in the evaluation because this is the only method that can objectively and quantitatively demonstrate the effectiveness of the e-learning program. One of the evaluation purposes is to collect information for promoting the e-learning program. The result of the tests will become an important resource to response to this predefined purpose.

7.6    Procedure of the Evaluation

The evaluation process can be divided into the following phrases:

*    The evaluators recruit students and teachers for the evaluation. The students should include non-paying students and fee-paying students.
*    The administrator turns on automatic logging function in the Moodle system.
*    The evaluators conduct pre-test for fee-paying students who have paid for five rounds of IELTS writing tutoring process.
*    The students go through their study process.
*    The evaluators analyse the logged data of the students’ activities in the learning system.
*    Based on the analysis of the logged data, the evaluators revise the questionnaire and the interview protocol.
*    When the non-paying students finish their first round free trial, they are given the questionnaire to complete.
*    When the fee-paying students finish their five rounds of study process, they are to take the post test and complete the questionnaire.
*    The evaluators conduct the interview with the teachers.
*    The evaluators analyse all the data.
*    The evaluators conduct follow-up interview with students and teachers, where necessary.
*    The evaluators compose the evaluation report.

 
8    Participants

The participants in this effectiveness evaluation will be:

*    2 evaluators
*    60 fee-paying students
*    60 non-paying students
*    4 teachers

 
9    Sample Instruments

9.1    Student Questionnaire

Appendix A includes the Student Questionnaire that the students participating in this evaluation will complete after they finish the free trial (for non-paying students) or after they finish the five rounds of tutoring process (for fee-paying students).

9.2    Teacher Interview Protocol

Appendix B includes the protocol for teacher interview that will complete with the four teachers who are currently offer the tutoring service in the e-learning program.

9.3    Automatic Data Collection

Appendix C includes a list of all the data to be collected by the computer while users are studying within the e-learning program. This data includes both navigation paths and response choices.

9.4    Pre-Test and Post-Test

Appendix D includes a sample of pre-test. All the recruited students should take the pre-test before they enrol in the e-learning program. The pre-test will take one hour to complete. It includes two writing tasks which are standard IELTS Writing tasks.

Appendix E includes a sample of post-test. It is also a standard IELTS Writing test. The difficulty of the pre- and post-tests are the same.
 

10    Data Analysis

10.1    Student Questionnaire

As for the Likert Scale questions, the evaluators are to check the answers for values out with expected ranges and inconsistencies. The results will be presented in bar charts, histograms, and/or scatterplots.
As for the open ended questions, the evaluators are to check the answers to see if they have common responses that can be coded up in some way. The coding technique will be used to transfer the qualitative data into quantitative data.

10.2    Teacher Interview

The evaluators are to check the interview transcripts to see if different teachers have common responses that can be coded up. As for their differences, the evaluators are to summarise them and present the result in a concise format.

10.3    Automatically Collected Data

The automatically collected data will be analysed and will form the basis for the evaluators to presume the users’ thoughts as they were going through the tutoring process. The assumption will be reflected on the revision of the questionnaire and the interview protocol.

10.4    Pre-Test and Post-Test

The test scores will be input into an SPSS database and will be calculated out whether there is significant difference between the pre-test and post-test.
 

11    Deliverables

Below is an outline of how the evaluation findings will be reported:

11.1    Progress of Students
11.1.1    Self-Rated Progress
11.1.2    Pre- and Post-Tests
11.1.3    Teachers’ Observation
11.2    Free Trial
11.2.1    Students’ Reaction to the Free Trial
11.2.2    Factors that Influence Students’ Decision to Subscribe to Extra Tutoring Service after Free Trial
11.2.3    Students’ Performance Described by Teachers
11.3    Paid Tutoring Service
11.3.1    Students’ Reaction to the Program
11.2.2    Students’ Performance Described by Teachers
11.4    Corrections to Be Made
11.4.1    Instructional Process
11.4.2    Instructional Materials
11.5    Enhancements to Be Made
11.5.1    Instructional Process
11.5.2     Instructional Materials
 

12    Limitations

12.1    Individual Learners’ Learning Experiences Vary

Although the trained teachers are to provide the service in a standard process using the same instructional strategy, individual learners’ learning experience will not be the same. This is because the writing tasks each student does are not the same, either as the teachers’ comments and the customised small courses. This may influence the reliability of the evaluation, especially for the student questionnaire part.

12.2    Difficulty in Controlling the Pre- and Post-Tests

The pre- and post-tests require the students complete the writing tasks online individually in one hour. The students are supposed not to look up words in dictionary or have any reference materials. However, this is not controllable because the evaluators can not monitor the testing process directly (e.g., seeing the students doing the tasks without any form of assistance). If any students do not follow the instruction, the reliability of the tests could be doubtful.
 

13    Logistics

Two evaluators will coordinate the implementation of this evaluation plan, including scheduling, data collection, and data transmission, with the course manager for this project. All data will be processed, analysed, interpreted, and reported by the two evaluators. All reports will be provided to managers and the teachers at Chao Yue IELTS. Further dissemination of the evaluation findings will be determined by the managers. Additional details about the logistics are found in the timeline section of this plan. The due dates for various deliverables are also specified in the timeline.

 
14    Timeline

 
15    Budget

 
16    References

Harvey, J 1998, Evaluation Cookbook, Learning Technology Dissemination Initiative, Edinburgh.
Reeves, T C & Hedberg, J G 2008, Evaluating e-Learning: a user-friendly guide (in press).

 
Appendix A    Student Questionnaire

Dear Student,

We are currently running an evaluation project for our e-learning program. We need your feedback to help us improve the IELTS writing tutoring service. Your responses are anonymous. Once you have finished the questionnaire, you will be entered into a draw to win one of the five $100 Coles Gift Cards. It will approximately take you 10 minutes to complete the questionnaire. Thank you for your time!

1. Please circle your responses to the following statements using the scale below.

1 = Strongly Disagree    2 = Disagree        3 = Neither Agree or Disagree
4 = Agree            5 = Strongly Agree

a. The program helped you identify your weakness in IELTS writing. 1 2 3 4 5

b. The program helped you identify your strengths in IELTS writing. 1 2 3 4 5

c. The tutor’s comment helped you understand how IELTS writing is scored. 1 2 3 4 5

d. The customised small course helps you overcome your weakness in IELTS writing. 1 2 3 4 5

e. Your IELTS writing skills have been improved at the completion of the program. 1 2 3 4 5

d. You satisfied with the free trial. 1 2 3 4 5

e. You decided whether to subscribe to extra tutoring service based on money for value. 1 2 3 4 5

f. You decided whether to subscribe to extra tutoring service based on your experience in the free trial. 1 2 3 4 5

g. You decided whether to subscribe to extra tutoring service based on your availability. 1 2 3 4 5

For Fee-Paying Students:

h. You think your decision to subscribe to extra tutoring service was correct. 1 2 3 4 5

i. You satisfied with the quality of the extra tutoring service. 1 2 3 4 5

j. You actively involved in the learning activities in the paid tutoring process. 1 2 3 4 5

2. What is the best aspect of the e-learning program?

3. How would you improve the e-learning program?

4. Please write any other comments about this e-learning program.

 
Appendix B    Teacher Interview Protocol

*    From your impression and observation, what progress did the students make during the studying this e-learning program?
*    What do you think of the current instructional process?
*    What do you think of the current configuration of the Moodle system?
*    Do you find it difficult or easy to organise relevant instructional materials for developing customised small course for the students?

 
Appendix C    Automated Data Collection

Appendix D    Pre-Test

Task 1

You should spend about 20 minutes on this task. The table below gives information about changes in modes of travel in England between 1985 and 2000. Summarise the information by selecting and reporting the main features, and make comparisons where relevant. Write at least 150 words.

Average distance in miles travelled per person per year, by mode of travel

Task 2

You should spend about 40 minutes on this task. Write about the following topic: Successful sports professionals can earn a great deal more money than people in other important professions. Some people think this is fully justified while others think it is unfair. Discuss both these views and give your own opinion. Give reasons for your answer and include any relevant examples from your own knowledge or experience. Write at least 250 words.

 
Appendix E    Post-Test

Task 1

You should spend about 20 minutes on this task. The graph and table below give information about water use world wide and water consumption in two different countries. Summarise the information by selecting and reporting the main features, and make comparisons where relevant. Write at least 150 words.

Global water use by sector

Water consumption in Brazil and Congo in 2000

Task 2

You should spend about 40 minutes on this task. Write about the following topic: Today, the high sales of popular consumer goods reflect the power of advertising and not the real needs of the society in which they are sold. To what extent do you agree or disagree? Give reasons for your answer and include any relevant examples from your own knowledge or experience. Write at least 250 words.