Evaluating Online Educational Materials for Use in Instruction

Author: Robert M. Branch, Dohun Kim and Lynne Koenecke
Date: June 1999

Introduction

The Web publishing revolution can be compared to the desktop publishing revolution. With the widespread use of personal computers and desktop publishing software, the construction of printed publications was sometimes put into the hands of novices. Sometimes the editors and layout artists were removed from the equation. Some publications were very good; many were not.

With the Internet, anything can be published on the Web at a low cost and distribution is virtually worldwide. Profuse amounts of information are put on the Internet every day. In many cases there is no editor, reviewer, or any other kind of review mechanism to determine the credibility, quality, accuracy, or timeliness of the material.

This problem magnifies when searchers find incorrect or out-of-date materials that are supposed to be used in instruction. An unsuspecting learner might be exposed to incorrect information retrieved by the instructor. It is imperative that information gathered on the Web be subject to the same strenuous critique as information that previously would have been gathered from books and other publications.

This Digest will help teachers select good resources to use in their instruction by providing a checklist to evaluate online educational materials.

Getting Started: How to Find Good Sites

There are many useful, high quality Web pages. Many sites have been reviewed, authenticated, or sponsored by highly reputable organizations. Some sites are dedicated to gathering valuable educational resources for educators (see suggested sites on next page). Find some reputable organizations in your field of study that act as reviewers for the Web. A wise beginning strategy would be to ask questions of your school's library media specialist.

If you need to search and find good sites by yourself, the first task to master is searching for pages relevant to your subject. One must research and practice searching techniques to narrow search results to pages that are most probably relevant to the search topic. After learning to effectively find topical Web pages, the next, and probably most critical task, is assessing the pages found. How, then, do teachers determine if their findings are jewels or just stones?

A Checklist for Evaluating Online Educational Materials

The following are seven major topic areas to consider when evaluating web-based materials for use in instructional settings.

1. Judge the accuracy of the information and take note of the date modified.

   Sub-questions to ask yourself:

   • Does the site provide evidence that it comes from reputable sources?
   • Does the site contain any obvious biases, errors, or misleading omissions in the document?
   • Does the site contain advertising that might limit the nature of the content?
   • Is the information current and up-to-date?

2. Is the level of information in this site appropriate for the intended audience?

   Sub-questions to ask yourself:

   • Does the site contain information appropriate for the intended learners with respect to their maturity and cognitive abilities?
   • Does the site contain any extraneous and unsuitable vocabulary, language or concepts, bias, or stereotyping?

3. Is the information in this site presented clearly?

   Sub-questions to ask yourself:

   • Is the information arranged in an orderly fashion?
   • Is the information presented clearly?

4. Is the information in this site closely related to purpose, content, activity, and procedures?

   Sub-questions to ask yourself:

   • Is there a clear tie among the purpose, content, and procedures suggested?
   • Does the site contain any activities irrelevant to the topic?
   • Does the site contain any redundant or isolated activities without a relationship to objectives?

5. Is the information in this site complete in scope and ready for use?

   Sub-questions to ask yourself:

   • Does this site contain complete breadth and depth of information related to the topic it claims to cover?
   • Are there any content gaps in concept development?

6. If a website has activities, are the content, presentation method, and learner activity potentially engaging?

   Sub-questions to ask yourself:

   • Are the suggested activities challenging, interesting, and appealing for the intended learners?
   • Does the information in the site emphasize and promote relevant action on the part of the learner?
   • Does the site have the potential for developing confidence and satisfaction as a result of learner effort?

7. If it claims to be comprehensive, is the information in the site well organized?

   Sub-questions to ask yourself:

   • Is the information in the site easy to use and logically sequenced, with each segment of the resource related to other segments?
   • Does the information flow in an orderly manner, use organizing tools (e.g., a table of contents, a map, or headings), and avoid the use of unrelated elements that are potentially ineffective or overpowering?
   • Are references, bibliographies, or other supporting evidence provided?

After you are comfortable recognizing the elements of good sites by using the above seven questions and sub-questions, find and evaluate some sites on your own. With some practice, finding and evaluating Web materials for instruction will become second nature. Keep in mind that instruction might be found in different sized chunks. You might find several parts in different places to construct your own lesson, or you might find good entire lesson plans.

A Good Starting Point: Suggested Web Sites

There are many starting points on the Web that are very helpful when looking for tools to evaluate websites. Listed below are some helpful sites.

Web Resource Evaluation Related Sites

• Bibliography on Evaluating Internet Resources. A comprehensive bibliography of sites that deal with this topic.

  http://refserver.lib.vt.edu/libinst/critTHINK.HTM

• Criteria for evaluating information resources.

  http://www-lib.usc.edu/Info/Sci/pubs/criteval.html

• Critical Thinking and Internet Resources. Includes: WWW CyberGuide Ratings for Content Evaluation, Teaching Critical Evaluation Skills for WWW Resources, Evaluating Quality on the Net, Thinking Critically about WWW Resources.

  http://www.mcrel.org/connect/plus/critical.html

• Evaluating Internet Resources-A Checklist.

  http://library.berkeley.edu:8000/bkmk/select.html

• Evaluating Websites. Bill Trochim provides useful information and tools for evaluating websites.

  http://trochim.human.cornell.edu/WebEval/webeval.htm

• Evaluating Websites for Educational Uses: Bibliography and Checklist.

  http://www.unc.edu/cit/guides/irg-49.html

• Guidelines for Evaluating Internet Information.

  http://info.wlu.ca/~wwwlib/libguides/internet/eval.html

• Internet Detective. An interactive tutorial which provides an introduction to the issues of information quality on the Internet and teaches the skills required to evaluate critically the quality of an Internet resource. Free, but requires registration.

  http://sosig.ac.uk/desire/internet-detective.html

• Internet Source Validation Project. How to Evaluate Web Pages

  http://www.stemnet.nf.ca/Curriculum/Validate/validate.html

• Kathy Schrock's Guide for Educators-Critical Evaluation Information. A series of evaluation tools, one each at the elementary, middle, and secondary school levels, are provided to help students critically evaluate a Web page.

  http://discoveryschool.com/schrockguide/eval.html

• Selection Policy for Resources and Evaluation Criteria Rating System for Web Sites From AASL

  http://www.ala.org/ICONN/curricu2.html

• Web Site Evaluation. A Collection of Research Papers and Surveys. The links on this page provide criteria that can be used to make judgments about educational Web sites in K-12 and higher education contexts.

  http://web.syr.edu/~maeltigi/Research/RIGHT.HTM

• Web Site Evaluation Guidelines from Ed's Oasis.

  http://www.edsoasis.org/Guidelines.html

References

Beck, S. (1997). "Evaluation criteria." The good, the bad and the ugly: or, Why it's a good idea to evaluate Web sources. [Online]. Available: http://lib.nmsu.edu/staff/susabeck/evalcrit.html [1999, April 27].

Brandt, D. S. "Evaluating information on the Internet." [Online]. Available: http://thorplus.lib.purdue.edu/~techman/evaluate.htm [1999, April 27].

Drake, L.; And Others. (1995). Notes from ERIC. Assessment and Evaluation on the Internet. Educational Measurement: Issues and Practice,14(4), 28-30. (EJ 519 184).

"Evaluating Internet resources: Table of contents." St. Norbert College Library. [Online]. http://www.snc.edu/library/eval1.htm [1999, April 27]. Harmnon & Reeves. (1998). Educational WWW sites evaluation instrument. [Online]. Available: http://itech18.coe.uga.edu/edit8350/wwweval.html [1999, April 27].

Livengood, S. P. (1997). An evaluation instrument for Internet web sites. Master's Research Paper, Kent State University. (ED 413 899) Smith, A. G. (1997). Testing the surf: Criteria for evaluating Internet information resources. Public-Access Computer Systems Review, 8(3), 1-14. (EJ 554 170).

Wilkinson, G. L., Bennett, L. & Oliver, K. (1997). Evaluation criteria and indicators of quality of Internet resources. Educational Technology, 37(3). [Online.] http://itech18.coe.uga.edu/faculty/gwilkinson/webeval.html.