Evaluation of E-Learning Courses

The following matrix was designed by Badrul Khan, an expert on e-learning and educational technology, to evaluate online courses.

If the question does not apply to the course being evaluated, print Not Applicable (N/A).

Pedagogical Dimension

· Q1. Does the course provide clear instructional goals (or are clear learning outcomes specified?)

· Q2. If appropriate, are objectives for each section of the course clearly specified?

· Q3. What is the instructor’s role?

· More facilitator than didactic

· More didactic than facilitative

· A combination of both

· Q4. If the course design allows the instructor to serve as facilitator, how/ where does facilitation occur?

· e-mail

· mailing list

· online discussion forum

· audio conference

· video conference

· Q5. Does the course provide clear directions of what learners should do at every stage of the course?

· Q6. Does the course provide a sense of continuity for the learners (i.e., each unit of the lesson builds on the previous unit where appropriate)?

· Q7. Does this course utilize a matrix of media to create a rich environment for active learning?

· Q8. Does the course exploit the flexibility of the hypertext/hypermedia environment of the Web?

Technological Dimension

· Q9. Does the course have personnel who can assist learners to set up for starting the course? (Or does the course have orientation programs that provide technical training prior to a course?)

· Q10. Is the cost of required hardware, software, and Internet service a deterrent to taking this course?

· Q11. Do students receive any guidance on how to set up hardware equipment for desktop video conferencing (if needed for the course)?

· Q12. Are the hardware requirements for the course clearly stated?

· Q13. Are the software requirements for the course clearly stated?

· Q14. Do students receive any guidance on how to do the following:

· send e-mail attachments

· install required software,

· scan a picture

· print within Web page frames

· create online presentation using presentation software

· other

Interface Design Dimension

· Q15. Do Web pages look good (i.e., effectively designed) in a variety of Web browsers and devices-in text-based browsers, all recent versions of Internet Explorer and Netscape, and so on?

· Q16. Does the course use a standard font type so that text appears the same in different computer platforms and browsers?

· Q17. Does the course follow “one idea per paragraph” rule?

· Q18. Is the text chunked and presented in a way that enables scanning and comprehension?

· Q19. Does the course provide structural aids or a site map to guide the learner’s navigation?

· Q20. To avoid bandwidth bottlenecks, does the course ask students to download large audio, video, and graphic files to their hard drives before the instructional events?

· Q21.
How quickly can users find answers to the most frequently asked questions on the course site?

· Q22. Does the course use easy to understand terminology?

Evaluation Dimension

· Q23. How is the learner’s assessment of various parts of the course administered? Check all that apply:

· multiple choice

· true/false

· fill-in-the blanks

· essay questions

· papers

· projects

· assignments

· proctored tests

· portfolio development

· case studies

· lab report

· journal

· other

· Q24. Does the assessment provide students with the opportunity to demonstrate what they have learned in the course?

· Q25. Does the course have a system to accept students’ online evaluations of the following?

· Content

· Instructor

· Learning environment

· Learning resources

· Course design

· Technical support

· Other

· Q26. Does the course provide an instant feedback button on most screens/pages to receive learners’ feedback for improvement of the course?

Management Dimension

· Q27. What content exists and what content must be created? Of existing content, what content requires “reprint” permission?

· Q28. Does the course acquire permission to use copyrighted information and materials?

· Q29. Does the course provide students with designated and secure (e.g., password protected) online spaces to store their notes and resources?

· Q30. Does the course have a system of keeping track of student submissions, and online quizzes?

· Q31. Does the course notify students about any changes in due dates or other course relevant matters (e.g., server down) via the following means?

· mail

· e-mail

· announcement page

· alert boxes

· running footer added to a page

· phone call

· Q32. Does the course provide back-up materials or alternative activities for students (i.e., what students will do) if any of the following is either not operating properly or unavoidable during a scheduled lesson period?

· Access to the courseware

· Discussion forum

· chat room

· e-mail and mailing list

· books

· online resources

· library materials

· study guide

· instructor

· tutor

· technical support

Resource Dimension

· Q33. Does the course provide guidance on how to organize it for online learning?

· Q34. Does the course provide information or ideas about how many hours (approximately) per week students are expected to spend on course assignments?

· Q35. Does the course provide toll-free telephone numbers for online support services?

· Q36. Does the course provide clear guidelines to the learners on what support can and cannot be expected from a help line?

· Q37. Does the course provide examples of previous students’ work on the Web? If yes, select all that apply and circle whether searchable and browsable:

· projects (searchable/browsable)

· papers (searchable/browsable)

· text dialogue from discussion forums (searchable/browsable)

· text dialogue from online conferencing exchanges (searchable/browsable)

· Q38. Does the institution’s library have library resources online? If yes, do students have access to its databases via the Internet or other networks?

· Q39. Does the host institution’s library have a system of getting books and other materials for students via interlibrary loan?

· Q40. Does the library fax or email any documents to students?

Ethical Dimension

· Q41. To improve cross-cultural verbal communication and avoid misunderstanding, does the course try to reduce or avoid the use of jargon, idioms, ambiguous or cute humor, and acronyms?

· Q42. To improve visual communication, is the course sensitive to the use of navigational icons or images? For example, a pointing hand icon to indicate direction would violate a cultural taboo in certain African cultures by representing a dismembered body part (Reeves & Reeves, 1997).

· Q43. Is the course offered to a geographically diverse population? If yes, is the course sensitive about students from different time zones (e.g., synchronous communications are scheduled at reasonable times for all time zones represented)?

· Q44. Is the course designed to have tolerance for learners who adapt to individualized distributed learning environment more slowly than others?

· Q45. Does the course allow students to remain anonymous during online discussions?

· Q46. Does the course provide any guidance to learners on how to behave and post messages in online discussions so that their postings do not hurt others’ feelings?

· Q47. If a student fails to follow the etiquette of the course, how does the instructor work with students to promote compliance?

· Q48. Does the course provide privacy guidelines on online postings?

· Q49. Does the course provide policies regarding fraudulent activities in course-related testing, assignments, and projects?

· Q50. Does the course get students’ permission to post any of the following on the Web?

· Students’ photographs

· Students’ projects

Institutional Dimension

· Q51. Do faculty receive training on how to moderate and/or maintain a listserv?

· Q52. Does technical and other staff receive training on how to communicate with remote learners in difficult situations?

· Q53. Does the course limit the number of students per faculty member?

· Q54. Does the course provide academic quality such as one would expect in a traditional course?

· Q55. Does the site provide any information regarding whether the course is transferable to other accredited institutions?

· Q56. Is the course offered at a lower fee than on-campus courses? It is not uncommon for students to expect lower fees for online courses.

· Q57. Would student information submitted online to the registrar’s office be kept secure and confidential to the extent possible?

· Q58. Are instructor/tutor and technical staff available during online orientation?

· Q59. Can students purchase packages of course-related supplemental reading materials online from the campus bookstore?

· Q60. Can students apply for financial aid/loans online?