Skyline College
Library

 

Evaluation Exercise

 

Evaluating Sources

Once you find books, articles, essays, web pages or other documents on a research topic, it is very important to evaluate those sources of information to try to develop a general understanding of the purpose, value, perspective and quality of any source you use in your research.

 
Evaluating Information from Web Pages

How do you know information from a Web page is true, accurate and of reasonable quality?

Although publishers, editorial boards or reviewers usually decide whether books or magazine or newspaper articles are accurate and of decent enough quality to be published, anyone can create a Web page with no screening at all; so it is especially important to evaluate information from the Internet. The questions below should be asked when evaluating any web page. They can provide some useful guidelines and criteria to help you evaluate the quality and reliability of web pages. To test your evaluation skills, choose one of the Web pages from the Evaluation Exercise and evaluate it carefully asking each of the following questions:

Who is the author or organization that created the page?
Responsible web publications should name the creator of the information in a readily visible place on the page. Is the author an organization? Is an author not given?

  • If an individual has written the page, is biographical information about the author available?
    • What are the author's QUALIFICATIONS or CREDENTIALS ? What is his/her/their background in terms of education, experience, occupation, position, affiliation, publications, etc. and why does or doesn't this make him/her/they an expert?
    • Try doing a web search (e.g. Google or Yahoo) to see if the author's name comes up in other web pages.
    • Can you discern anything about the CREDIBILITY or REPUTATION of the author?
  • If an organization has authored the page, is information available about that organization, including its purpose or history?
    • Look for a link such as "Who we are" on the home page.
    • Is it an impartial group (like a university) or a group established to promote an idea or point of view (like the National Rifle Association)?
    • Try doing a web search (e.g. Google or Yahoo) to see if the institution's name comes up in other web pages.
  • What clues does the URL (Internet address) provide about the page?
    • Is the Web page part of a more extensive website? If so, who is responsible for the site?
    • Is the page part of an individual’s personal Internet account, rather than being part of an organization's Web site? This type of resource should be evaluated most carefully.
    • Identify the domain type within the URL. This is a key factor when evaluating a site since it indicates where the information originates. There are 6 domain types:
      • .edu: an educational institution (can be anything from scholarly research to zany student pages)
      • .gov: a government body (usually dependable)
      • .com: a commercial enterprise (may be trying to sell a product)
      • .net: network
      • .org: a non-profit organization (may or may not be biased)
      • .mil: a military body
  • Read: U.C. Berkeley's "Evaluating Web Pages: Techniques to Apply & Questions to Ask"

Evaluate the content

  • What is the purpose of the information? Is this purpose clearly stated? Does the page fulfill its stated purpose?
    A description of each of the following types of websites is available by clicking on each type:
  • What type of information is presented?
    • scholarly (written by researchers in the field)
    • professional (written by and primarily for those in a specific profession)
    • popular (written for the general public, i.e. those with no special expertise or training)
    • commercial (an advertisement for a service or product)
    • advocacy (opinion pieces)
    • governmental (provided by local, state, or federal government)
    • entertainment
    • some combination of 2 or more of the above types of information may be offered at the same website
    • Is the content intended to be a brief overview of the information or an in-depth analysis?
  • Sources & Documentation:
    • Where did the author gather the information presented? Was it from original research, experiments, observation, interviews, books and documents?
    • If lots of factual information is given, does the author cite his/her sources? Verify that the author used authoritative sources to back up his/her arguments and conclusions.
    • If the information is opinion, is this clearly stated?
  • Bias:
    • Does the site show signs of any political, ideological, cultural, or personal biases?
    • Is the site presenting an extremist or wildly unorthodox view? (Such views may be inherently biased.)
    • Read: "Information and Its Counterfeits: Propaganda, Misinformation and Disinformation" to help you be able to distinguish real information from its three lookalikes, or counterfeits: propaganda, misinformation and disinformation.
  • How current is the information?
    • Is there a date on the page that indicates when the page was created and/or updated?
    • Is there a copyright date?
    • Is the information or the site itself out of date?
    • Is currency not a crucial factor for the topic covered by this website?
  • Who is the audience?
    • Is the page intended for the general public, or is it meant for specific groups, such as scholars, members of an organization, children, etc.
  • Writing style:
    • Is the content free of grammatical, spelling, and typographical errors? These types of errors indicate carelessness and lack of professionalism.
    • Is the information presented in a clear, reasonable, and orderly way? Or is written in a confusing, disorganized or illogical manner?
    • What is the author's overall writing style? Is it appropriate to the topic being presented? Be aware that an author's style (dry, humorous, objective, overly complex, sarcastic, histrionic, tedious, etc) can prejudice your perception of the information.

For further information on evaluating web pages, see "Bibliography on Evaluating Internet Resources" by Nicole Auer, Virginia Tech University Libraries and Evaluating Information on the Internet" by D. Scott Brandt, Purdue University Libraries.

 Evaluating Articles

- Types of Periodicals
- Finding Information about Periodicals
- Finding Information About Authors

"Periodicals" is a general term used to refer to newspapers, magazines and journals (publications that are published "periodically".) When you use periodical articles in your research, you should evaluate the article by asking questions similar to those you asked when evaluating web pages. The first question to ask is: Who is responsible for the article? This is sometimes referred to as evaluating the authority of the article. When asking about the authority of an article, you should look for information about two things:

  • the publication in which the article was published and
  • the author(s) who wrote the article.

Types of Periodicals

The first question to ask about a periodical is: What type of periodical is it? The distinction between magazines and journals is not always very precise, but it is important to understand the general differences between these types of publications. Popular magazines (such as Time, Newsweek or Sports Illustrated) include those found on most newsstands, usually with flashy covers intended to entice a reader and commonly printed on glossy paper with many illustrations and photographs. Magazines are usually written by journalists for a fairly general audience so they are usually easier to read than journals, but the information they provide tends to be less detailed or authoritative. Articles in popular magazines usually report events, the author's opinion or simplified versions of the findings of others. Footnotes and bibliographies are rarely included in popular magazine articles. Browsing through popular magazines can sometimes be a useful method of finding an interesting topic for a research project. Articles in popular magazines usually provide some general overview information on current events, issues and trends.

Scholarly journals tend to be the most common sources of bibliographic research for serious researchers. These journals are usually oriented to students and professionals in a particular discipline and may be more difficult to understand for those who do not have some familiarity with the general field being discussed. Subject coverage tends to be more specialized and articles usually reflect the most recent research and important issues being discussed in a particular academic area. Scholarly journal articles usually review previous literature and report original research and almost always include extensive footnotes and bibliographies. Scholarly journal articles are often preceded by an abstract-- a brief summary of the article. Many scholarly journals are "refereed" or "peer reviewed" ("refereed" or "peer reviewed" are both synonomous). Articles submitted for publication to a refereed or peer reviewed journal are reviewed by an editorial board made up of scholars or authorities in the field to determine whether they are of high enough academic quality to be published. Many periodical databases (including the InfoTrac OneFile database) include an option to limit searches to refereed or peer reviewed journals. If you find an article in a search in a periodical database, you can check to see if the periodical in which the article was published is refereed by limiting the search to refereed journals. If the article is included in the results list after limiting the search to refereed journals, then it is a refereed journal.

Between popular magazines and scholarly journals are subject magazines (such as Scientific American or Smithsonian), professional journals (such as Anthropology Today or American Psychologist) and trade journals (such as Automotive News or PC Week). These types of publications sometimes look similar to popular magazines but they typically report on developments in a particular subject area, profession, trade or industry and are oriented to those who are either in the field or who have a particular interest in the subject matter. They differ from scholarly journals in that their articles do not usually include original research and, although they are often written by subject experts, the writing tends to be less complex and technical.

It is important to understand the use of each type of periodical in the research process. Each type of literature has a function and value in research. Popular articles often make a good point of entry for research. They can provide a basic introduction to a topic, an overview that may suggest possible approaches to the subject, divergent points of view, and perhaps some color. The college-level researcher will go on to collect scholarly articles that offer more focused, in-depth, original, research-based information. Subject or professional magazines may serve both sets of functions and are particularly useful for research in scientific and technical areas in which the scholarly materials may be too specialized for undergraduates. They offer the authority of experts without the complexity of original research accounts. For a more detailed description of how to distinguish between popular magazines and academic journals, click here.

Finding Information about Periodicals

What if you want to know more about a periodical, such as its political perspective and the type of articles it publishes?

Publishers' Websites

Information about a periodical can often be found from the publication's website (if they have a website). From the publication's home page, look for any description or hints about the publication's perspectives or affiliations. Look for links such as "About Us" or "About this publication", or "Publisher's Information".

To find a publication's website, you can often simply use a Web search engine (e.g. Google); type in the full title (with quotation marks around it) and the publication's website should come up near the top of the result list. In some cases, it may be more difficult to find the website for certain periodicals. This may be the case if the publication has a common name, if the publication's website is contained within a broader website--such as that of a publisher or an affiliated organization, or for various other reasons.

You may sometimes have an easier time finding a publication's web address by using PubList (publist.com), a web-based directory of over 150,000 domestic and international print and electronic publications. You can do a title search for a publication on PubList to get basic information about a publication, such as how often the periodical is published, the name and address of the publisher and the publication's web address. To search the PubList directory, the site may request that you complete some basic registration information. You can register for free to get your own user name and password or you may use Skyline Library's registration by entering: Skyline Library as User name and library as password.

Magazines for Libraries

Publication information on a publisher's website is not necessarily objective since it is written about its own publication. Magazines for Libraries, an invaluable reference book which is available at Skyline Library on the Ready Reference shelves behind the reference desk (call number: Z 6941 .M23 2002), provides relatively unbiased descriptions of thousands of periodicals. Magazines for Libraries lists about 6900 of the most important magazines in all fields. This source is especially useful because it includes detailed annotations thoroughly describing each of the publications listed. It is updated every three to four years and is available only in print. In addition to the descriptive annotations, Magazines for Libraries also provides publication information about each periodical that can be useful when evaluating the publication. The information for each title includes:

  • frequency of publication: abbreviations indicate if the periodical is published weekly (w), monthly (m), bi-monthly (bi-m), quarterly (q), etc.
  • publisher: you can check whether the periodical is published by an academic organization, a commercial company or other group
  • whether or not the journal is "refereed": if the periodical is refereed, the word "refereed" precedes "Circ." information
  • a list of periodical databases that index the publication

 

Finding Information About Authors

There are numerous questions you should ask yourself in order to evaluate the author (or authors) of any source. Who is the author? What are the author's credentials--educational background, past writings, occupation, position, titles, experience, political perspective? Is the book or article written on a topic in the author's area of expertise? Is the author qualified (or not) to write on the subject?

In articles, brief information about the author is often included on the first or last page. In books, information about the author may be included at the beginning or end of the book, or on the back or on the inside cover. If the book includes an introduction written by someone other than the author, some details about the author will usually be mentioned.

Have you seen the author's name cited in other sources or bibliographies? Respected authors are cited frequently by other scholars. For this reason, always note those names that appear in many different sources.


Author Information from Books, Periodicals or the Internet

Beyond information provided in the work, itself, background information on authors may be found in a variety of other sources.

If the author is an important figure or a very popular person, entire books may be written about him or her. Books about people can be found in library catalogs, (e.g. PLS online catalog) by searching by subject. (Remember to always search by the last name first.)

It is more common to find magazine, journal or newspaper articles or Internet documents about authors. To find magazine or journal articles with information about an author, use periodical databases (e.g. InfoTrac OneFile). Start by searching by subject (last name first) to see if any entire articles are written about the author. If you don't find any articles specifically about the author, you can do a full-text keyword search to see if the author's name is mentioned in any article or document. For full-text searching, enter the name of the author you are researching first name first. To search for any web pages that mention the author, use a Web search engine (e.g. Google) and enter the name in quotes, first name first.


Author Information from Online Reference Databases

Some additional sources of information for literary authors or especially well-known authors may be found in a couple of specialized online databases accessible through Skyline College Library. Literature Resource Center, includes biographical information on over 100,000 writers. Biography Resource Center provides biographical information on on nearly 275,000 people from throughout history, around the world, and across all disciplines and subject areas. (To access either of these databases from off campus, a PLS library card is required.)
 


Finding Evaluations of Books from Book Review Articles

It is particularly useful to find evaluations of a book from other sources. Knowing what other people say about a book may tell you whether the book is worth reading or not. It also can be very helpful to give you an introduction to the author and the ideas presented.

The most common evaluation of books is the book review. A book review is an article that describes and evaluates a specific book. Book reviews are published in magazines, journals and newspapers and are available on the Internet. Book reviews are indexed in most periodical indexes.

To find book review articles, you can use various types of periodical indexes. General periodical databases, such as the InfoTrac OneFile database, and newspaper databases such as the InfoTrac Newspapers database all include book review articles, usually for books of relatively general interest. The New York Times Book Review provides free online book reviews back to 1996. (Use the "Book Search" search box at the right of the page.) In order to find reviews of books on more specialized topics, it is often necessary to use subject periodical databases that index periodicals in the book's subject area.

To search for book reviews in a periodical database, you can either simply do a keyword search just for the title of the book (if it is unique) or you could do a keyword search for both the title AND the author. If searching title and author does not narrow your search enough, in many databases you could limit your search by ANDing the term "review" with the title.

There are a number of Web sites that specifically provide book reviews written either by professional reviewers or by readers who are interested enough to write and send in their personal reviews. Yahoo has a directory of many of these book review sites at http://www.yahoo.com/Arts/Humanities/Literature/Reviews/. A major limitation of most of these sites, however, is that the books reviewed tend to be primarily recent popular works. Newspapers on the Web can be a good source for reviews of recent books.

Amazon.com, an online bookstore, lists a huge number of books of all types, many of which include brief reviews. It is very important to check closely the source of reviews or descriptions included in online bookstore sites since they tend to include more favorable descriptions (that may seem like reviews) submitted by publishers. Brief reviews submitted by readers are also often found on these sites.

General web search engines (e.g. Google) can also be used to try to find book reviews on the Web. Try doing a search for the title of the book (in quotes) followed by the word review.


Questions for Evaluating and Annotating Any Source

Beyond finding out about the author and considering the basic citation information, it is very important to do more thorough critical thinking about any source you use in your research. For books, scan the Table of Contents and the Index to get a broad overview of the material covered. Note whether bibliographies are included. Read the chapters that specifically address your topic.

A very useful method of organizing your ideas about a source is to write a critical annotation of that source. An annotation is similar to an abstract, which is a brief summary of an article or book. While an abstract usually just provides a summary of what is included in a source, an annotation generally includes a critical evaluation of a source. Click here to see an example of an annotated bibliography.

To try to systematize the process of writing critical annotations, librarians at the McIntyre Library, University of Wisconsin-Eau Claire developed a series of questions to ask about any book, article or other publication that is to be critiqued . The questions (slightly revised) are listed below. After the list of questions, some suggestions are given for how to answer each of the questions in order to evaluate different types of sources and to develop your own annotations.

1. What is the author's purpose for writing the work?

2. Who is the intended audience? Is it intended for the general public, scholars, policy makers, teachers, professionals, etc.? Is this reflected in the author's style of writing or presentation? How so?

3. Does the author have a bias or make assumptions upon which the rationale of the work rests? Is the information covered fact, opinion, or propaganda?

4. What method of obtaining data or conducting research was employed by the author? Is the material primary or secondary in nature? Is the work based personal opinion or experience, interviews, library research, questionnaires, laboratory experiments, case studies, etc.? Does the information appear to be valid and well-researched, or is it questionable and unsupported by evidence?

5. What are the author's conclusions?

6. Does the author satisfactorily justify the conclusions? Why or why not?

7. How does this work compare with similar works? Is it in tune with, or in opposition to, conventional wisdom, established scholarship, professional practice, government policy, etc.? Are there specific writings, schools of thoughts, philosophies, etc. with which this work agrees or disagrees?

8. Are there significant attachments or appendixes, such as charts, maps, bibliographies, photos, documents, tests or questionnaires? If not, should there be?

These questions should be used as general guidelines in the process of analyzing a book, article or other work. It will not always be possible to answer all of these questions about a work. If the subject of the work is relatively new to you, it will be more difficult to answer some of the questions; but the process of trying to answer the questions should help you become more familiar with the subject. As you gain more background on the subject, you will increase your ability to critically evaluate writing on the the subject. Whether or not you can answer all of the questions, however, simply taking the time to think about each question for each book or article that you use in your research can be very helpful in developing your critical thinking skills in the research process.

1. The Author's Purpose

Gaining some information about an author's background can often shed some light on the author's motivation for writing a work. Although sometimes an author may purposely hide his reasons for writing something, usually authors will openly state their purpose. This will generally be presented within the first several paragraphs of an article. A book's preface or introduction, whether written by the author or not, will usually provide significant information not only on the book's purpose, but also on the author's background, point-of-view and how the book relates to similar works.

2. Intended Audience

The intended audience of a work is rarely stated explicitly, but it can usually be inferred from various clues. The purpose of a work may suggest a particular audience. Other evidence may be found in the writing style, use of jargon and footnotes. Use of a specialized vocabulary, such as medical or legal jargon, for example, would imply that the work is intended for medical or legal practitioners. Information about the periodical in which an article is published will provide a good indication of the audience for a specific article. As mentioned above, Magazines for Libraries (Ready Reference section: Z 6941 .K21 1995) is an excellent reference tool for finding basic descriptions of periodicals.

3. Point-of-View

An author's point-of-view or bias may be his political orientation or a literary, artistic, scientific or other academic school of thought. Familiarity with the subject of a work is especially helpful in detecting an author's viewpoint. Background information about the author will often provide some evidence of his political perspective or school of thought. Understanding the author's purpose in writing the work will commonly suggest his bias on the subject. The perspective of an author may not always be detectable in his writing, but if it is possible to determine the point-of-view of a particular work, that information can be very helpful in understanding the work as a whole. Particular biases or perspectives should not be thought of as correct or incorrect, but as an important piece of data to be considered when analyzing the work.

It is not always easy to separate fact from opinion. Facts can usually be verified; opinions, though they may be based on factual information, evolve from the interpretation of facts. Skilled writers can make you think their interpretations are facts. Assumptions should be reasonable. Note errors or omissions.

4. Methodology

The method of obtaining data or conducting research for a work will vary according to the type of work and the discipline. More popularly-oriented articles or books may simply be based on an author's personal opinion, first-hand experiences or on a limited amount of interviewing or library research. More academic works generally involve more extensive research. Depending on the discipline, data-collection techniques may include bibliographic research, laboratory experiments, textual analysis, surveys, empirical observation, etc. Most works use various combinations of techniques and the methods used are commonly specifically stated in the work.

Primary sources are the raw material of the research process. Secondary sources are based on primary sources. For example, if you were researching John F. Kennedy's role in the United States' involvement in Vietnam, Kennedy's own writings would be one of many primary sources available on this topic. Others might include relevant government documents and contemporary newspaper articles. Scholars use this primary material to help generate historical interpretations--a secondary source. Books, encyclopedia articles, and scholarly journal articles about Kennedy's role are considered secondary sources. Choose both primary and secondary sources when you have the opportunity.

5. Conclusions

The conclusions of a particular work are usually relatively easy to find in a work. In articles, in particular, sentences or paragraphs near the end of the piece will often begin, "In conclusion," "In summary," or with some other explicit wording. Books often include a concluding chapter. Although the conclusions of a work are very important to consider, they should be thought of as one element in the overall analysis of a work.

6. Justifications

Determining how well an author justifies his conclusions can be quite difficult for those not very familiar with the subject of a work. For example, those more familiar with the subject area are more likely to recognize significant information that was not included in a work. In considering how well a work's conclusions are justified, it is usually helpful to think about the author's purpose, point-of-view, intended audience and methods of collecting data at the same time as analyzing the conclusions. The general question to ask is: Do the facts in the work fully support the conclusions? Some specific questions to consider about the work are: "Is it internally consistent and does the logic follow? Did the bias determine the method of data collection or the interpretation of the data collected? Was the method of data collection the best one for the original purpose?"

7. Comparison with Similar Works

Although knowledge of the discipline of a work is very helpful in determining how that work relates to the subject matter as a whole, authors often explicitly mention other writers with whom they agree or disagree. In addition, footnotes and bibliographies provide specific references which, in turn, can be researched for comparisons to the original work.

8. Attachments

Attachments, such as charts, maps, survey instruments, appendices or bibliographies, can provide important clarification or enhancement to the basic text of a work. Often, when specific data--such as survey responses or statistical data--is referred to in a work, the actual data may be included or summarized in an appendix. At the same time, the lack of certain supporting attachments may suggest a definite limitation in a work. Such supporting appendices should not be overlooked when evaluating a work.


Annotated Bibliographies

An annotated bibliography is a list of annotations on a specific topic organized in a standardized format.

 

Evaluating Journal Articles: Following Citations

Book review articles provide critical analyses of books, but how can you find evaluations of articles from journals? Individual periodical articles are rarely reviewed exclusively in a single reviewing article unless they are extremely significant or groundbreaking new works. While it is rare to find an article solely dedicated to reviewing another article, scholarly journal articles (as opposed to articles in professional journals or popular magazines) commonly refer to and comment on various different authors' articles. Journal articles written in a specific academic research area regularly cite other articles that cover the same area of research. The citations are often used to document support for a particular point being made in an article or, sometimes, to refute a position taken by the article being cited. The way to find evaluations of a particular article, then, is to find other articles that have cited that article. Once you find articles that have cited the article you want to evaluate, read those articles to see what they say about the cited article.

Following the writing of group of researchers who all write about a particular academic research area and who regularly cite each other in their journal articles can be like listening to a scholarly conversation. Each new article usually addresses previous articles and makes new points which may elaborate on or disagree with the articles it cited. This process of finding citations to new articles to be read in a particular research area by following up on references cited in other articles on the topic is probably the most common process of research used by most academic researchers. A researcher may initially find out about a particularly useful article from discussions with a colleague, from listening to a lecture by an expert in the field or by scanning the latest issue of an academic journal in her subject area. As she reads the article, the researcher makes a list of those sources cited in the article which are of interest to her, and she then locates and reads those items. She in turn follows up on any citations listed in those publications which seem promising to her research.

Back to top

 

 

last revised: 9-22-08
by Eric Brenner, Skyline College, San Bruno, CA
These materials may be used for educational purposes if you inform and credit the author and cite the source as: Skyline College Library. All commercial rights are reserved. To contact the author, or send comments or suggestions, email: Eric Brenner at brenner@smccd.net