Dr. Yonnie Chyung
IPT Department, ET330
College of Engineering
Boise State University
1910 University Dr.
Boise, ID 83725

Improve the Motivational Appeal of Online Instruction for Adult Learners:
What’s in it for Me?

Yonnie Chyung, Ed.D
Instructional & Performance Technology
Boise State University
1910 University Dr.
Boise, Idaho 83725-2070

Note: This article was a preliminary report of her evaluation study, presented at the American Educational Research Association (AERA) conference, New Orleans, LA in 2000. Her complete 4-year evaluation report, Systemic and Systematic Approaches to Reducing Attrition Rates in Online Higher Education, has been published in the American Journal of Distance Education (AJDE) in 2001 (Volume 15; Issue 3, p. 36-49). Viewers are strongly encouraged to refer to the article published in the AJDE.

Abstract

Distance education, due to its time and geographic flexibility, has appealed to working adult learners who work full-time and seek continuous education as part-time students. A problem in adult distance education, however, has been a high turnover in enrollment. The department of Instructional and Performance Technology (IPT) at Boise State University offers a distance education option in its Master's degree program. All courses are delivered on line via the Internet. Prior to 1997, the IPT online program faced a high dropout rate and looked for effective interventions to reduce the dropout rate. In this paper, the author describes theory-based and model-based intervention design procedures that guided a redesign of IPT online instructional system to enhance its motivational appeal to adult learners and help them stay with their continuous education. It was revealed that improvement of the motivational appeal of online instruction for adult learners had significantly positive effects on three levels of the instructional system: (a) learners' perceptions toward the online learning environment, (b) learning outcomes, and (c) increased retention rate. Adult online learners were highly motivated to learn, and they achieved a significantly high level of learning outcomes. In addition, the overall dropout rate has significantly decreased from the previous record. From this paper, readers will learn systemic and systematic methods of designing motivating online instruction for adult learners and evaluating the effects of the interventions on various levels of an instructional system.

The U.S. Department of Education defines adult education as the teaching of adults via any educational activities, except full-time enrollment in postsecondary credential programs (National Center for Education Statistics November, 1995). What motivates full-time employed adults to be involved in continuous education? The reasons for continuous learning vary. Houle (1971) conducted a qualitative study in which he identified three types of adult learners: 1. goal-oriented participants, 2. learning-oriented participants, and 3. activity-oriented participants (cited in McCreary, 1990). What kinds of adult education programs are available for them? Verduin and Clark (1991) categorize three main types of adult education programs: 1. Adult basic education (ABE) programs (to acquire ABE), 2. Leisure and enrichment education programs (to increase enrichment in adult life), and 3. Career education programs (to prepare or upgrade their job-related knowledge and skills). The National Household Education Survey (NHES) classifies six types of adult education programs: 1. Work-related courses, 2. Personal development courses, 3. Credential programs, 4. ABE/GED programs, 5. Apprenticeship programs, and 6. ESL programs. According to Kim and Creighton’s report (Spring, 2000) based on the data obtained from the NHES:

• During the 12-month period prior to the NHES 1995 survey, about 76 million adults (about 40 percent of adults) participated in one or more adult education activities. In 1999, the number had increased to 90 million adults (46 percent of adults).
• Among six types of adult education programs, work-related courses (23%) and personal development courses (23%) are the two types of educational activities in which adults participated most as part-time learners. Other four types of programs are credential programs (9%), ABE/GED programs (2%), apprenticeship programs (2%), and ESL programs (1%). The sum of the percentages (60%) is greater than the overall participation percentage (40%) in adult education because some of them reported that they participated in more than one type of adult education program.
• The overall participation rate of college graduates was more than three times the rate of those who did not have a high school diploma. Only two out of ten adults who do not have a high school diploma participated in adult education programs. On the other hand, more than six out of ten adults who have a bachelor’s or higher degree participated in adult education programs.

Based on the literature review, it is reasonable to conclude that adult learners tend to seek higher education in order to improve professional, career-related knowledge and skills to be used at work and accomplish professional and personal goals. This observation seems to support Houle’s identification of the two types of adult learners as learning-oriented and goal-oriented learners.

Distance education is defined as "any formal approach to learning in which a majority of the instruction occurs while educator and learner are at a distance from one another" (Verduin & Clark, 1991, p. 8). Various technologies are used as the delivery media in distance education. The National Center for Education Statistics (NCES) describes distance education as "education or training courses delivered to remote (off-campus) locations via audio, video, or computer technologies including both synchronous and asynchronous instruction."

Especially, asynchronously-delivered distance education due to its time and geographic flexibility has appealed to working adult learners who work full-time and hope to participate in continuous education as part-time students. The demand for adult distance education has substantially increased in recent years. According to the National Center for Education Statistics, an estimated number of adult learners formally enrolled in distance educational courses at 2-year and 4-year higher education institutions during the 1994-95 academic year was 753,640. The number of enrollments had increased up to 1,632, 350 enrollments by 1997-98.

The number of enrollments in adult distance education programs increased over the last several years. However, a problem in adult distance education has been a high turnover in enrollment. Although distance education institutions are usually reluctant to publicly announce their drop-out rates, retention of distance education learners is known to be usually lower than retention of on-campus learners (Kember, 1995; Verduin & Clark, 1991). Kember (1995) reports that within a course, attrition is much higher at the beginning of the course than towards the end of the course. The steep attrition slope is also usual for an online degree program where more students drop out of the online degree program after the first couple of online courses than after they have taken a number of online courses (Fenner, 1998).

If adult learners find distance education as a convenient method to reach their goals of learning, why would they drop out of the distance education course or degree program before completing them? Reasons for dropouts are found to be various. It has been reported that adult learners tend to drop out of distance education courses or programs:

• When they perceive that their interests and course structure do not match (Bartles, 1982 cited in McCreary 1990; Fenner, 1998).
• When they are not confident enough in learning in distance learning environments (Chacon-Duque, 1987).
• When they have learned what they wanted and they lost motivation to continue to learn (Holmberg, 1989 cited in Verduin & Clark, 1991).

Again, the above reasons for dropouts seem to support Houle’s identification of the two types of adult learners as being learning-oriented and goal-oriented learners.

Based on the above literature review, it seems to be safe to make an argument that there are at least two important factors that influence adult learners to get motivated or de-motivated to learn via distance education options. The two factors are:

1. Adult learners are goal-oriented; therefore, distance education programs should provide learning environments where they can accomplish their goals. When they do not or cannot accomplish their goals through the distance education, they very likely quit learning.
2. Adult learners are learning-oriented; therefore, distance education programs should provide learning environments where they can acquire interesting and relevant knowledge and skills. When they do not perceive that they are acquiring the knowledge and skills that they wanted, they very likely quit learning.

The author of this article is a strong believer of the above two hypotheses. However there is not enough research done in the adult distance learners and there is little research that supports such hypotheses.

The author has carried out a long-term evaluation study to make a judgment of the effectiveness of the interventions designed based on the above assumptions. The interventions were designed in order to reduce the high dropout rate in an adult online education program and evaluated was the effectiveness of the interventions in terms of the changed dropout rate. Her long-term case study is described in the following section.

The department of Instructional & Performance Technology (IPT) at Boise State University offers a master’s degree program via both the on-campus option and the online option. The IPT online program uses asynchronous computer-mediated communication software (currently Lotus Notes) to deliver online instruction via the Internet. IPT online students from foreign countries such as Canada, Germany, Japan, Philippines, Scotland and Thailand as well as from various locations in the United States are attending IPT online classes.

A dropout problem was found in the IPT online program as well. Between the fall semester of 1989 and the fall semester of 1996, 44% of online students dropped out of the IPT online program by their third online course (Fenner, 1998).

In order to find out the causes of dropouts, the IPT Associate Program Developer conducted exit interviews with the students who dropped out of the program between 1989 and 1996. She also interviewed the students who continued the IPT online program. From the interviews, it was concluded that IPT online students’ satisfaction levels during the first or second online courses were one of the major factors that determined their decision to continue or not to continue the IPT online program. Forty-two percent of the students who dropped out of the IPT online program expressed ‘dissatisfaction with the online learning environment' as the reason for their dropping. Specific reasons for dissatisfaction toward the IPT online program included:

• discrepancies between their professional or personal interests and the curriculum or the course structure
• low confidence levels in learning via the Internet (without face-to-face human contact)
• incompetence in using the online communication software as an effective learning tool
• feelings of being overwhelmed by advanced knowledge and overloaded information on line

[1] Application of the ARCS Model. 

The cause analysis revealed several motivational factors in the online instruction system that caused the dropouts: i.e.,

• The IPT online learners perceived that online learning environment was not attractive to them.
• The IPT online learners perceived that what they learned from the online instruction was not relevant to their interests or goals.
• The IPT online adult learners had low confidence levels while learning in an online classroom.
• Overall, the IPT online learners had low satisfaction levels toward the online learning environment.

These four motivational factors – interests (or attention), relevance, confidence, and satisfaction - are discussed in John Keller's ARCS model (Keller, 1987). Keller explains that the four factors influence the degree of learners' motivation to learn. Learners lose their motivation to learn and quit learning, especially when they do not perceive instruction as interesting or relevant to their goal. They also lose motivation to learn when they are not confident in learning processes. They more likely quit learning when they are not satisfied with the instructional processes or the overall learning environment. Keller argues that in order to help learners become more motivated to learn, instructional designers should consider improving the motivational appeal of the instruction, focusing on the four factors: i.e.,

1. How interesting to learners are the presentation and contents of instruction?
2. How relevant to learners is the instruction (the contents as well as the methods)?
3. How confident are learners?
4. How satisfied are learners?

The author concluded that it was reasonable to restructure the online instructional system focusing on the four motivational factors in order to help online learners develop positive perceptions about the online instruction and hopefully to reduce the number of dropouts.

[2] Application of the Organizational Elements Model (OEM). 

Analyzing the overall organizational end results from a systemic perspective also helped the author approach the problem with effective means. The OEM provides a systemic framework of designing and implementing effective means to achieve desirable end results. In the Organizational Element Model (Kaufman & Thiagarajan, 1987), there are five elements that interact with each other: Inputs and processes are means, and products, outputs, and outcomes are three types of end results in the chain of events. Products and outputs are the end results occurring within the organization; outcomes are end results that are external to the organization (see Table 1).

Table 1. The Organizational Elements Model

MEANS	1. inputs (raw materials)	Organizational efforts	Internal to organization
	2. processes (how-to-do-its)
ENDS	3. products (learner/instructor accomplishments)	Organizational results
	4. outputs (organizational accomplishments)
	5. outcomes (effects in and for society)	Societal impact	External to organization

Improving the motivational appeal of online instructional system is a means to the end results, which include improving DE learners’ perceptions about the online instruction and reducing the dropout rate.

The ARCS model provides a theoretical foundation of selecting effective means; i.e., instructional inputs and processes. Implementing effective instructional inputs and processes will more likely result in online learners’ positive reaction toward the online instruction, which will more likely help online learners achieve successful learning outcomes. Learners who are satisfied with the instructional inputs and processes and experience successful learning outcomes will more likely continue to be motivated to learn. That is, it will eventually contribute to reducing the dropout rate in the IPT online program.

It was determined from the front-end analysis that a solution to the dropout problem in the IPT online program was to improve the motivational appeal of the online instruction and help online learners feel satisfied with the online instruction and their performance in online classes. Starting with the Spring semester, 1997, the IPT online program started implementing new instructional methods in the first online course to make it more interesting and relevant to its adult online learners and helping them increase confidence and satisfaction levels toward their first experience of IPT online instruction. The ARCS model was used to guide the instructor to modify her instruction systematically. Table 2 below provides a summary of interventions implemented based on frequently exhibited characteristics of the first-time online learners. The third column of the table indicates the main targeted factor(s) to improve the motivational appeal of the online instruction (A: Attention, R: Relevance, C: Confidence, and S: Satisfaction).

Table 2. Designing Interventions based on the ARCS Model

Potential learner characteristics	Interventions	Targeted Motivational Factors
		A	R	C	S
Learners are new to the online learning environment and they do not know how to behave in an online learning environment.	Keep the size of each online class small (less than 17 students).			*	*
	Provide examples of exemplary online behaviors and examples of undesirable online behaviors up front.		*	*
	Monitor individual learners’ online behaviors and provide frequent feedback to reinforce good online behaviors and/or modify undesirable online behaviors.	*		*
	Provide a private online discussion area for each learner and the instructor so that the learner can easily contact the instructor and ask for advice privately.	*		*	*
Learners are not knowledgeable of using the Internet-based media technology as a learning tool.	Provide learners with a technical training program and master the basic technical skills prior to the first online class.			*	*
	Provide ongoing technical support in a timely fashion.			*	*
Learners are new to the field (i.e., new to the learning subject matter).	Provide structured and spiral instruction: easy to difficult, simple to complex, concrete to abstract.			*	*
	Provide alternative assignments and have learners to choose a method to accomplish a learning goal.	*	*
Learners are different in terms of the entry knowledge levels.	Conduct an entry knowledge assessment and use the data to adjust the level of difficulties in instruction for the learners. Provide assignments that are not too easy, not too hard, but challenging enough for their entry knowledge levels.		*	*
Learners have different personal, professional interests and goals.	Obtain individual learners’ demographic background information before and/or as soon as the semester starts. Provide various examples during instruction that are relevant to the learners’ interests and background.	*	*		*
Learners are learning-oriented and goal-oriented.	Provide clearly stated weekly goals and objectives and explain why it is important to achieve the goals.	*	*
	Clearly state how learners will be evaluated.	*	*
	Provide specific guidance on how to successfully accomplish the goals.			*	*
	Provide concrete and constructive feedback on how they are doing in a timely manner.			*	*
Learners have different preferences in the types of learning.	Provide various learning activities and multimedia, and allow learners to select their preferred activities.	*	*
Learners are highly achievement-oriented.	Provide frequent feedback to learners on a regular basis (e.g., weekly feedback) and let them know how well they are doing. Assign a special assignment (e.g., assign as a team leader)		*	*
Learners miss personal contact with the instructor.	Make personal contact to each learner using a personal online discussion area, email and/or telephone.	*			*
Learners miss social interactions with classmates	Assign paired or group activities. Encourage collaborative work. Provide a virtual hallway area, a virtual congratulation bulletin board and a virtual student union area.	*			*
Learners have different learning styles (e.g., visual, auditory, introvert, extrovert, social, independent, etc.)	Use multimedia materials in instruction. Use various instructional methods: e.g., assign group activities, assign independent research projects, let students become weekly discussion leaders or weekly discussion wrappers, use personal discussion areas, etc.	*	*

Kirkpatrick's evaluation model of evaluating training programs (Kirkpatrick, 1996) was applied to the evaluations of different stages of results.

• The level 1 evaluation was to measure learners' perception toward the instructional inputs and processes: i.e., the motivational appeal of online instruction based on the ARCS model's four factors - attention, relevance, confidence, and satisfaction.
• The level 2 evaluation was to measure the organizational product: i.e., the learning outcomes in terms of the difference between pretest scores and posttest scores. A level 1 evaluation and a level 2 evaluation were conducted in each semester.
• The level 3 evaluation was to measure the organizational output: i.e., the change in the retention rate since the IPT department started implementing interventions.

A summary of the systemic intervention and evaluation design methodologies is illustrated in Figure 1.

[1] Learners’ Perception toward Online Instruction. 

At the end of each semester, a course evaluation questionnaire was administered to measure learners' perceptions toward the online instruction. Up to Fall semester, 1998, the response scale used was A: Outstanding, B: Very Good, C: OK, D: Improvement Needed, and E: Unsatisfactory. As coding learners’ responses, the number one was assigned to a response to 'A’, 2 to 'B’, 3 to 'C’, 4 to 'D’, and 5 to 'E’. Since the Spring semester of 1999, a new course evaluation questionnaire has been used, and attention, relevance, and confidence level measurements were not included in the course evaluation questionnaire. The response scale used in the new questionnaire was Likert scales. The number one was assigned to a response to ‘Strongly Agree’, 2 to ‘Agree’, 3 to ‘Neutral’, 4 to ‘Disagree’, and 5 to ‘Strongly Disagree’. The average scores revealed that the online adult learners found the online instruction to be interesting and relevant to their interests and goals. They were confident in learning in the online learning environment and satisfied with the quality of the course (see Table 3).

Table 3. Level 1 Evaluation: Attention, Relevance, Confidence, and Satisfaction Levels

Semester	Attention	Relevance	Confidence	Satisfaction
Spring, 1997	M = 1.41	M = 1.92	M = 1.83	M = 1.92
Summer, 1997	M = 1.17	M = 1.63	M = 1.75	M = 1.50
Fall, 1997	M = 1.25	M = 1.47	M = 1.56	M = 1.31
Spring, 1998	M = 1.40	M = 1.63	M = 1.65	M = 1.50
Fall, 1998	M =1.50	M = 1.16	M = 1.40	M = 1.30
Spring, 1999	N/A	N/A	N/A	M = 1.78
Fall, 1999	N/A	N/A	N/A	M = 1.20
Spring, 2000	N/A	N/A	N/A	M = 1.12

[2] Learning Outcomes. 

In each semester, an entry knowledge assessment and two tests (a mid-term test and a final test) were administered. The entry knowledge assessment was used as a pretest. A combined data of the two tests was used as a posttest data. At the end of each semester, a t-test was conducted to reveal if there was a significant difference between the pretest and the posttest. Each t-test revealed that the average posttest score was significantly higher than the average pretest score (see Table 4).

Table 4. Average Pretest and Posttest Scores and t-test Results

Semester	Means (Learning Outcomes)	t-test results
Spring, 1997	M (pre) = 61.5%, M (post) = 92.1%	t (11) = -20.61, p < .01
Summer, 1997	M (pre) = 64.0%, M (post) = 87.0%	t (16) = -7.64, p < .01
Fall, 1997	M (pre) = 63.5%, M (post) = 87.0%	t (15) = -9.25, p < .01
Spring, 1998	M (pre) = 61.5%, M (post) = 91.3%	t (14) = -16.08, p < .01
Fall, 1998	M (pre) = 65.3%, M (post) = 88.3%	t (19) = -5.09, p < .01
Spring, 1999	M (pre) = 60.3%, M (post) = 91.1%	t (14) = -20.10, p < .01
Fall, 1999	M (pre) = 55.6%, M (post) = 95.7%	t (15) = -21.52, p < .01
Spring, 2000 (a combination of 2 classes)	M (pre) = 56.2%, M (post) = 95.3%	t (28) = -15.32, p < .01

[3] Retention Rate. 

Between the Fall semester of 1989 and the Fall semester of 1996, the average retention rate by the third course was 56%; that is, 44% of online students dropped out of the program by their third course. At the end of 1997, only three semesters after the interventions were implemented, the retention rate increased to 78% (Fenner, 1998). Of those who dropped out of the program, three students dropped out due to hardware and/or software incompatibility problems. Six students cited that they decided not to continue the program because their professional goals and the degree program did not match. Other reasons cited were health problems and time constraints. By the end of 1999-2000 academic year, the retention rate increased up to 85% (Fenner, 2000). The graph below shows a substantial amount of increased credits that the IPT online program offered during the past 3 years since 1997 when the IPT online program started implementing the interventions.

This evaluation case study shows how the ARCS model, the OEM, and Kirkpatrick's evaluation model guided throughout the processes of improving the motivational appeal of the first online course for adult learners and solving the dropout problem in the IPT online program at Boise State University. The impact of the interventions was significantly positive for the whole organizational system, in terms of the benefits to the educational organization itself as well as the online learners’ accomplishment of the learning goals. Through the application of these models in practice, the IPT online program markedly increased the student retention rate during the 3 year period. Due to the organizational efforts of enhancing motivational appeal of online instruction through systemic and systematic methods, the learners perceived the online instruction as more interesting and relevant to their professional and personal goals. They were confident during learning processes and satisfied with the online learning environment and their own accomplishment. Their perceptions that the online instruction was motivating and they accomplished significantly positive learning outcomes seemed to influence their decision to continue to learn via the IPT online program. As a result, the IPT online program has significantly increased the retention rate over the last 3 years. Increasing the retention rate is more cost-effective than having to recruit new students due to a high dropout rate in terms of increasing enrollments and contributing to the organizational growth. The systemic and systematic implementation of interventions has produced a win-win situation.

References

Chacon-Duque, F. J. (1987). A multivariate model for evaluating distance higher education. College Park: Pennsylvania State University Press.

Fenner, J. A. (1998). Enrollment analysis. Unpublished manuscript at IPT distance program at Boise State University.

Fenner, J. A. (2000). Enrollment analysis. Unpublished manuscript at IPT distance program at Boise State University.

Kaufman, R., & Thiagarajan, S. (1987). Identifying and specifying requirements for instruction. In R. Gagne (Ed.), Instructional technology: Foundations (pp. 113-139). Hillsdale, NJ: Lawrence Erlbaum.

Keller, J. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2-10.

Kember, D. (1995). Open learning courses for adults: A model of student progress. Englewood Cliffs, NJ: Educational Technology.

Kim, K., & Creighton, S. (Spring, 2000). Lifelong learning: Participation in adult education in the United States: 1998-99. Education Statistics Quarterly, 2 (1). Retrieved June 13, 2000 from the World Wide Web: http://nces.ed.gov/pubs2000/qrtlyspring/6life/q6-1.html

Kirkpatrick, D. (1996). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler.

McCreary, E. K. (1990). Three behavioral models for computer-mediated communication. In L. M. Harasim (Ed.), Online education: Perspectives on a new environment (pp. 117-130). New York: Praeger.

National Center for Education Statistics (November, 1995). Forty percent of adults participate in adult education activities: 1994-95. Retrieved June 12, 2000 from the World Wide Web: http://nces.ed.gov/pubs/95823.html

National Center for Education Statistics (December, 1999). Distance education at postsecondary education institutions: 1997-98. Retrieved June 12, 2000 from the World Wide Web: http://nces.ed.gov/pubs2000/2000013.pdf

National Center for Education Statistics (n.d.). NCES fast facts: Distance learning. Retrieved June 12, 2000 from the World Wide Web: http://nces.ed.gov/fastfacts/display.asp?id=80

Verduin, J., & Clark, T. (1991). Distance Education: The foundations of effective practice. San Francisco, CA: Jossey-Bass.