THE EFFECT OF ADJUNCT QUESTIONS ON LEARNING IN PREADOLESCENT CHILDREN
ABSTRACT
Two groups of subjects from two seventh-grade science classes at a Miami junior high school, were given a 750-word passage of text to read. One group had to answer 10 adjunct questions, two of which were inserted following each 150-word segment of text; the other group had no adjunct questions added to the text. Both groups received a test on the material from the text immediately following training, and another similar test one day later. The adjunct-question group performed significantly better, on both the immediate and delayed post-tests, than the reading-only group.
BACKGROUND
The purpose of this experiment is to study the effects of questions inserted within a text (called adjunct questions) upon learning in preadolescent children. Research using adjunct questions to examine the effects of question level can be loosely divided into two classes: studies employing post-tests that ask only for factual or verbatim recall of information from the passage, and post-tests containing higher-level questions, asking application and semantic questions from the text. Verbatim questions induce subjects to mechanically memorize the individual words in the text, rather than encode the passage meaningfully, while higher-level questions induce encoding of unified structures of ideas. Questions that require comprehension of the text will produce better learning and retention of information than questions that can be answered purely on the basis of surface features. It is argued that effects of rote memorization and verbatim questions have, at best, minimal interest to educational research, while, presumably, the ultimate educational goal is to foster higher cognitive skills, which may lead people to apply rules and principles learned to new life situations.
Most of the studies reviewed in this paper were conducted primarily with adult subjects. The available data on children is limited. Therefore, this experiment seeks to examine the effects of adjunct questions upon learning in preadolescent children.
The insertion of questions within prose material has been shown to facilitate learning from such material (Rothkopf, 1966). Depending upon question level, position of questions within the text, and question frequency, the effect has been shown to be both direct and indirect, facilitating learning of both factual and inferential information. The facilitative effect is marked with post-questions.
Anderson (1972) has shown that encoding is achieved on at least two levels: phonological, or verbatim, and semantic, or meaningful. He postulated that adjunct question placement within a text may result in differences in encoding: questions placed before a text may direct a student to focus his attention on verbatim, target information sought by those questions, and adjunct questions placed after a text segment may prompt the student to encode the previous information more semantically.
Rothkopf (1965) has found that questions elicit a general class of responses, called "mathemagenic behaviors" (from Greek: mathema, learning, and genesis, birth), behaviors which give birth to learning, which aid in the encoding and retention of information. He tested the effect of question position within a text upon learning from written instructional material. He found that when adjunct questions were placed after a passage, the effect was both specific to the question and general, but if such questions were placed before a passage, the only facilitative effect was question specific. It has been repeatedly shown that test-like events, such as adjunct questions, improve performance on subsequent criterion tests. The present study gives strong evidence to suggest that when subjects study a written text, they not only learn the specific content of the text, but may also acquire some general skills of deduction and elaboration, suggesting further that the very act of answering adjunct questions can be instructive.
Rothkopf (1966) hypothesized that post-questions serve as reinforcing or discriminative stimuli. A post-question may serve as a review and synthesis of material just read (reinforcing stimulus), or it may provide a "hint" as to what sort of information the reader is to look for in subsequent paragraphs (discriminative stimulus). Rothkopf's paradigm consists of interspersing relevant questions in a passage of text, and then testing their facilitative effects upon subsequent criterion tests. Typically, either one or two questions are inserted before (pre-question) or after (post-question) a passage of certain length, the question type varying from low-level rote recall of information to high-level comprehension type questions. Questions and texts appear on separate sheets of paper, and the subject is not allowed to turn back to the text, or to take notes while reading. Following the completion of reading and question answering, a post-test is administered on the amount of direct (questioned) information, as well as upon the amount of indirect (unquestioned, inferred) material retained from the entire text read. Typically, the findings show that the adjunct-questioned group of subjects retains more direct and indirect information than the no-question, reading-only group.
Rothkopf and Bisbicos (1967) studied the effects of using specific categories of questions in structural material to see if they facilitated learning of specific content of the target text. They distributed experimental materials consisting of text and written instructional material, bound in a loose-leaf binder to the students. The students worked at their own pace, but were required to record the time it took for each of them to complete their part of the reading. The criterion test was administered immediately following training. The researchers found that subjects performed better on the criterion tests if given questions after exposure to the relevant text segment than if such questions were presented before inspection of a relevant text segment. Additionally, they found that performance on criterion tests was better if the question types on the tests were similar to the ones represented during training. This experiment demonstrated that test-like events, such as adjunct questions of any type, can facilitate learning, especially if placed following a text segment.
Frase (1967) built upon Rothkopf's studies by varying the adjunct question position, length of the passage read and knowledge of the results of the adjunct questions answered during training. He found that if adjunct questions were placed before a passage (pre-questions), they facilitated factual learning, but had a deleterious effect upon inferential learning, since they tended to limit and focus the students' attention directly on the sought-for information, to the practical exclusion of other information. However, when questions were placed following a passage (post-questions), both text-specific and incidental, inferential learning was improved. Scores on incidental questions tended to improve with longer text segments, indicating that inferential learning was facilitated when the text was presented as a cohesive whole. Knowledge of results of adjunct questions had no significant effect upon post-test performance. His study supports Rothkopf's (1966) findings showing that adjunct questions facilitate learning of both specific and incidental information, especially if placed following a text segment.
In another study, Frase (1971) found that deep, semantic post-questions had a facilitative effect upon the recall of both factual and inferential information, regardless of their position within the text, presumably because inferential questions subsume factual material.
Felker and Dapra (1975) combined Rothkopf's (1966) mathemagenic behaviors theory with Anderson's (1972) distinction between verbatim and comprehension-level questions, to study the effect of adjunct question position upon problem-solving ability, defined as the ability to appropriately apply rules and principles to new problem situations, and use previously learned concepts in some combination to achieve certain goals (Gagne, 1964, 1970). Their findings were consistent with previous work done in this area, that high-level comprehension type adjunct post-questions provided the best method of enhancing mathemagenic behaviors, especially with regard to problem-solving ability.
Rickards and DiVesta (1974) compared the effects of two types of questions: rote verbatim questions and meaningful higher-order questions. Their research has shown that with verbatim questions, material is learned in a somewhat random and disorganized fashion. Such learning is not highly coded, but is merely mechanically memorized directly from the text. The subject is not required to process the material, apply it or even understand it. On the other hand, meaningful learning requires a certain organization of concepts, categorization of facts and coordination of higher-order related ideas. Rickards and DiVesta found that adjunct questions in general, and higher-order questions in particular, especially when placed frequently within a text, produced significantly better recall of both related and incidental facts and ideas, a result at variance with Frase's (1967) findings that scores improved with greater question distance from the text. Interestingly, Rickards and DiVesta found evidence to suggest that even rote questions of abstract ideas yielded better results than rote questions of facts or irrelevant questions, regardless of how frequently placed.
Most studies on the effects of adjunct questions have been conducted with adult subjects. There is little available data gathered on school children, who, because of their developmental age, cognitive skills and relative inexperience with studying techniques, may react differently to inspection control devices, possibly because of an attenuated attention span, or a lack of strongly formed reading behaviors. Swenson and Kulhavy (1974) extended their studies to grade school children. Their findings supported prior research conducted with adults by demonstrating that adjunct questions facilitate learning in children as well. However, contrary to available data with adults, adjunct questions only facilitated the learning of direct, critical items with children. It may be that optimum storage of information for children results from direct cueing devices, and that children may tend to disregard incidental material in text. These studies helped support their observation that encoding of information occurs differently within children as with adults.
Furthermore, Swenson and Kulhavy's (1974) studies failed to show any significant relationship between pacing and placement of adjunct questions within a text, as found by Frase (1967), indicating again that younger subjects may be unable to store as much information as more experienced learners.
Palermo and Molfese (1971) have suggested that the age range between 12 and 14 may be important in terms of language development and cognitive changes. A study conducted by Roggenbau- Engelen (1979) showed that 12-year-old children will retain learned material better if required to process words generatively and semantically, rather than reproductively, as in mechanical memorization. Retention was best when they processed randomly arranged words meaningfully, rather than memorizing nonsense syllables.
Allen (1970) studied the effects of providing advance organizers (prior, highly general concepts under which new learning may be subsumed) to facilitate learning. Allen postulated that higher-order questions, requiring the students to mentally manipulate the data, would enhance the facilitative effects of advance organizers. For his experiment, he chose 212 ninth-grade students. Two texts were prepared: one text in the form of an interesting story, the other in textbook style. Advance organizers were provided to all students. Both memory-level and higher-order adjunct questions were used with the reading material. For the criterion tests, four memory-level and four higher-order questions were developed, covering the same information as the adjunct questions used in learning. The results showed that subjects given memory-level adjunct questions performed better on memory-type tests than those given higher-order adjunct questions, and higher-order adjunct question groups performed better on higher-order tests than memory-level groups. Advance organizers did not prove facilitative to learning.
On the delayed post-tests, adjunct questions facilitated performance only for lower-ability students. Students who also received advance organizers performed substantially better than higher-ability studies who were not given advance organizers. On the post-test designed to measure higher-order learning, advance organizers facilitated higher-order learning only in higher-ability studies, but not that of lower-ability students.
Combining the various studies outlined herein, this experiment attempted to evaluate the facilitative effects of verbatim post-questions upon learning of factual information in preadolescent children. It was expected that adjunct questions would produce better retention of information in the group answering such questions in conjunction with the target text, and would produce higher scores on both the immediate and delayed criterion tests.
Method
Subjects
Subjects were randomly selected students in two seventh-grade, junior high school science classes, each with 24 students, who participated in this experiment as part of a class project. The experiment constituted a supplement to their regular curriculum.
Materials
The material used was a 750-word abstracted text from National Geographic entitled, "Bats Aren't All Bad." Both groups received the same text. The text was typed on white 8-1/2 x 11-inch paper, with approximately 150 words per page. The pages were bound into booklets to be distributed to each student. The students of one class received booklets containing only the reading material; the students in the other class received booklets containing adjunct questions in addition to the excerpted text. The adjunct questions were low-level, factual information questions requiring short responses. There were 2 questions per page, and each question page was inserted after a 150-word passage. There was a total of 10 questions per booklet.
Two post-tests were developed, each containing 10 low-level, factual information questions similar to the adjunct questions. One post-test was administered immediately following the reading session, the other the following day. Both groups were given the same post-tests.
A sample from the text given all subjects is as follows:
"Among orders of mammals, bats rank second only to rodents in numbers and diversity of species. By far the greatest number live in the tropics. They abound in temperate climates, but being nocturnal and usually secretive, rarely attract attention or arouse curiosity." (Novick, 1973, p. 624).
An example of the type of adjunct and post-test questions is as follows:
1. Among mammals, where do bats rank? 2. Where does the greatest number of bats live? 3. Do bats attract attention? 4. What type of climate do bats prefer?
Procedure
At the outset of the experiment, the students in each class were informed that there was to be a slight deviation from their regular curriculum, and that they were going to be given a short text to read in class. Booklets containing only the reading material were distributed to the subject of one group who were instructed to read the material on each page thoroughly, and upon completion, proceed to the next page until the entire booklet was completed. They were not permitted to take notes or turn back to a page once completed. They were told to expect a test at the end of their reading, but were not told the type of test or its length in terms of the number of questions. (Further research was done discussing the "surprise quiz"). They were not told that there would be another test given the following day (so they expected the immediate post-test, but did not expect a test the following day). All test questions were low-level, verbatim factual information tests. The subjects were allowed to work at their own pace.
The subjects in the other group were given the booklets containing both the text and adjunct questions. They, too, were instructed to read the text on each page carefully, and then answer the questions appearing on each page following a page of text. They were permitted to write their answers directly on the pages containing the questions, but were not permitted to either take notes or turn back to a page once it was completed. They were informed that there would be a test following the reading and question answering, but were not told the type or length of the test, nor that there would be another test the following day. They were permitted to work at their own rate. Subjects who finished the task ahead of the others were instructed to raise their hands so that their booklets would be collected. In neither group was performance made part of their final grade.
Immediately upon completion of the tasks, a test was administered. The post-test contained 10 questions of the type used as adjunct questions. Both groups received the same test. Time allotted for completion of the initial post-test was 20 minutes.
The following day, during the scheduled class meeting, the second post-test was administered to both groups. Neither group had been told that another test would be given, and both groups received the same test. Time allotted for completion of the second test was 20 minutes.
A two-way analysis of variance in a within-subjects design was used to measure the effect of post-test timing on either treatment condition. The main effect of adjunct questions was significant, F(1, 23)=50.12, p<0.1. The main effect of test timing was significant, F(1, 23)=26.74, p<0.01. The interaction of adjunct questions and test timing was significant, F(1, 23)=17.38, p<01, indicating that subjects receiving adjunct questions performed significantly better than their reading-only counterparts on both post-tests, presumably because reading-only subjects focused less efficiently on the text material, and therefore, showed a greater degree of "forgetting" over time than the other group. More research is needed, but it seems clear that the act of answering questions engages a part of the learning mechanism and memory system that are responsible for longer-term deeper learning.
Results
The mean scores obtained from both groups, on the immediate and delayed post-tests, are illustrated in Table 1. For the adjunct-question group, the difference in score means, from the immediate to the delayed post-tests, was significant, t(23)=6.97, p<.005, indicating that performance on the delayed post-test deteriorated significantly over performance on the immediate test. For the reading-only group, however, the difference in score means from the immediate to the delayed post-tests was not significant, t(23)=.24, p>.10, indicating that performance on the delayed post-test was not affected by post-test timing, nor was it improved as a function of the practice effect of the immediate test.
The significance levels of the difference in mean scores obtained on each post-test for the two groups were measured. On the immediate post-test, there was a significant difference in mean scores between the adjunct-question group and the reading-only group, t(23)=2.92, p<.005, again indicating that adjunct questions facilitated performance, even when a delayed post-test as unannounced.
Discussion
Research has repeatedly shown that adjunct questions have a facilitative effect upon learning. Perhaps such facilitative effects are due to the practice effect of test-like events, such as adjunct questions, along with the strengthening of information in memory as a result of retrieval. The results obtained in the current experiment with preadolescent children agree with those of previous studies. It has been shown that children do, in fact, benefit from adjunct questions, even though the ones tested here were low-level, verbatim type questions, designed to measure subjects' performance on verbatim information learned from the text.
As observed in this experiment, the group given adjunct questions performed significantly better on both the immediate and delayed post-tests, than the reading-only group. It is interesting to note that, although subjects' performance on the immediate post-test may have been predicted since they anticipated that test, neither group anticipated the second test, yet the adjunct-question group again performed significantly better than the reading-only group. Conversely, even though the reading-only group anticipated the immediate test, they performed no better on it than on an unanticipated delayed test (*presumably because of no adjunct questions to reinforce learned information).
It is the ultimate aim of educators to promote learning at a high level of competence, instilling skills necessary for the application of learning rules and principles to new problem situations. It is useful to note that even though children are not as cognitively developed or as educationally sophisticated as adults, they nevertheless profit from tools designed to bring about effective learning techniques. Most teachers, educators and psychologists would agree that higher-level learning is the ultimate goal of education, yet the majority of teacher-prepared tests (90-95%) require only rote recall of specific information. The practical educator should be more concerned with whether students understand concepts and principles than whether they can recall details.
Perhaps this is yet another indication that teachers and educators need to revise current teaching methods to include adjunct questions as learning tools.
The overriding conclusion to be drawn is that tests serve to reinforce learning, even more importantly than to measure performance, and should not be feared.
REFERENCES
Allen, D.I. (1970). Some effects of advance organizers and level of questions on the learning and retention of written social studies material. Journal of Educational Psychology, 61, 333-339.
Anderson, R.C. (1972). How to construct achievement tests to assess comprehension. Review of Educational Research, 42, 145-170.
Felker, D.B., & Dapra, R.A. (1975). Effects of question type and question placement on problem-solving ability from prose material. Journal of Educational Psychology, 67, 380-384.
Frase, L.T. (1967). Learning from prose material. Length of passage, knowledge of results and position of questions. Journal of Educational Psychology, 58, 266-272.
Frase, L.T. (1971). Effects of incentive variables and type of adjunct question upon text learning. Journal of Educational Psychology, 62, 371-375.
Gagne, R.M. Problem solving, In A.W. Melton (Ed.), Categories of human learning. New York: Academic Press, 1964.
Novick, A. (1973, May). Bats Aren't All Bad. National Geographic, pp. 615-637.
Palermo, D.S., & Molfese, D.L. (1971). Language acquisition from age five onward. Psychology Bulletin, 78, 409-428.
Rickards, J.P., & DiVesta, F.J. (1974). Type and frequency of questions in processing textual material. Journal of Educational Psychology, 66, 354-362.
Roggenbau-Engelen, M. (1979). Memorization learning in school: Does it provide optimal methods? Zeitschrift fur Entwicklungspsychologie und Pedagogische Psychology, 11(1), 59-66.
Rothkopf, E.Z. (1996). Learning from written instructive material: An exploration of the control of inspection behavior by test-like events. American Educational Research Journal, 3, 241-249.
Rothkopf, E.Z. Some theoretical and experimental approaches to problems in written instructions. In J.D. Krumboltz (Ed.), Learning and the educational process. Chicago: Rand McNally, 1965, 193-221.
Rothkopf, E.Z., & Bisbicos, E. (1967). Selective facilitative effects of interspersed questions on learning from written materials. Journal of Educational Psychology, 58, 56-61.
Swenson, I., & Kulhavy, R. (1974). Adjunct questions and the comprehension of prose by children. Journal of Educational Psychology, 66, 212-215.
