Tuesday, September 16, 2008

Measurement Metrics for Online Independent Study Courses

In my Web Analytics for Online Instructional Design class, I have been thinking about various metrics that could be analyzed when evaluating online independent study courses. Here are a few that I think would be helpful.
  • The amount of time students spend reading the content pages.
  • The connection between the amount of time spent reading and the scores on the tests.
  • The background of the students (age, ethnicity, demographics, education, etc.) Then it would be nice to find correlations with the students' background and their scores.
  • How often the students refer to the media resources. It would specifically be helpful to know which media resources are the most used: audio, video, etc.
  • For the students that scored high, what did they spend a majority of their online time doing?
  • How often are students referring to the Objectives Page?
  • Are students using the optional mastery pages? If so, how much time on average do they spend doing them? How many of the master pages do they use? What difference has it made on their score?
  • Are the students using the links that go to external resources? If so, which ones seem to be the most popular (or helpful)?
  • How much time are they spending with each lesson?
  • Which distractors are working, and which ones are not?
  • What is the general webpage navigation order that the students follow? Do they read the content pages, or do they skip right to the test?
  • If a test is open-book, how many student skip back and forth from the test to the content pages?
  • Does additional "fluff" (i.e. content or resources) have any affect on the learning for the high school classes? What about for the college level courses?
  • Which of all the resources are being the most used? Do those that use them score better on the tests?
  • Does the amount of time studying the content pages correlate to better test scores? What about the amount of extra resources that are used?
  • Do students spend more time on content pages that are heavily text-oriented or on pages that are heavily graphics-oriented?
  • When students use video/audio clips, do they view/listen to the whole clip?
  • Do students spend more time on pages that look more modern with graphics, colors, videos, audio, etc. rather than those pages that look older?
  • On which pages do students request for help from the teachers/tutors? (This would help us know what content to change or enhance to prevent the problems in the future.)
  • Which webpages seem to catch the most attention of the students?
  • The student's intent for taking the class. For example, does the student truly want to learn about the content and do well in the class, or will the student settle for any grade just so that he can graduate? How could we make the class applicable for both parties?
That's all for now, folks!