Progress 10/01/07 to 09/30/10

Outputs
OUTPUTS: Due to my sabbatic in Year 3, the funds were returned. Thus I report on years 1-2. "Thinking Like A Scientist" trains teachers to inculcate scientific thinking by focusing on real world examples relevant to students' daily lives. Teachers' approaches are changed as a result of learning about and participating in a scaffolded implementation of this program they come to emphasize more often how to think critically and reason effectively, and less often how to memorize facts and figures. Students taught TLAS see, often for the first time, that scientific thinking is something they can use to deal with problems they face in their lives. Their attitudes toward science are changed by this experience; they also learn effective thinking skills, which once learned can be used and maintained. In this project, I was forced to adjust to the economic crisis in New York City public schools by shifting focus to program development and evaluation in the Ithaca City School district. New York City teachers who had participated in Year One found themselves reassigned or out of work. Remaining teachers were heavily overcommitted, and consequently it was a difficult time to work in this venue. Thus, I returned 50% of the allocated funds in Year 2, and used the remainder of the money to work intensively on development and evaluation of a "Thinking Like a Scientist" program for elementary school students. The chief goal was to write, revise, and conduct a detailed longitudinal evaluation of this program with an eye toward eventually making it available broadly to teachers of disadvantaged elementary-aged youth. This evaluation required the development of three versions of an open-ended scientific-critical-thinking test which was administered to the students before, during, and after the program was taught. The newly created "Thinking Like A Scientist" educational outreach program for elementary-age students focuses on teaching basic critical thinking and reasoning skills as applied to science education. I worked with Laurie Rubin, a second-grade teacher in Ithaca with 25 years experience, much of it spent at Beverly J. Martin elementary school, which has a substantial African American/Latino/economically-disadvantaged student enrollment. The PI and Rubin, along with an undergraduate honors student of the PI's (Jessica Zulawski), developed and revised the curriculum, which was then taught by Rubin. The materials focus on the psychology of eating (a topic the young children can relate to), and use this topic to teach critical thinking and reasoning. The second major effort during Year Two involved offering a summer science program in the Department of Human Development as part of the annual 4-H Cornell extension-outreach to high school students in late June-early July. We taught 20 students highlights of the TLAS program, focusing on careers available in science and how to acquire the education necessary to pursue these careers. We offered this 4-H summer program again in 2010. PARTICIPANTS: Nothing significant to report during this reporting period. TARGET AUDIENCES: Nothing significant to report during this reporting period. PROJECT MODIFICATIONS: Nothing significant to report during this reporting period.

Impacts
I evaluated whether critical scientific thinking skills of elementary school children can be enhanced through the "Thinking Like a Scientist" approach. Critical thinking skills were defined as the ability to apply the scientific method successfully in real-world decision-making. The TLAS program includes discussions and activities on how to generalize the scientific method. Participants were a class of 19 public-school 2nd graders (Mage=7.2 years, SD=0.7), 10 girls and 9 boys. The curriculum included lessons linking the scientific method to real world issues for 7-8 year olds (e.g., making healthy decisions about food and exercise); it introduced basic concepts such as "biased source," and "how to create a hypothesis." The evaluation measures contained open-ended questions assessing students' ability to apply domain-general scientific reasoning (e.g., Kyle picks a brand of granola bar because the TV commercial has a famous movie star in it. Is this: Good Thinking, Not-so-good thinking, or Don't Know. Why). I hypothesized that TLAS lessons on critical thinking would increase these particular skills and, consequently, assessment scores in the students. I developed three versions of the evaluation measure (A, B, C) which were used in an experimental design in which the class served as its own control. Version A (pretest) provided baseline data. Version B, administered 2.5 months after Version A, captured any maturational effects due to being taught by this particular teacher, and evaluated acclimation to the testing process and familiarity with the general measure. Version C (post-test) measured gains in domain-general scientific reasoning resulting from the program; it was administered 2.5 months after Version B, during which time (between B and C) the teacher taught the program. Students were tested individually by the researcher who asked students the test questions and wrote down their answers. Each question probed one concept, such a correlation vs. causation, or distinguishing good vs. bad sources of information. The tests were rated by one graduate student unaffiliated with the project plus the research assistant, using a 1 (poor)-to-5 (excellent) scale for each answer (scoring by two independent raters minimized rating bias; Cohen's kappa measure of reliability, k=0.61). The baseline (Version A) per-item mean was 1.86(SD=.39). At second data collection (Version B), scores were comparable (M=1.94, SD=.48). Next, the curriculum was taught. At third data collection (Version C; after curriculum) scores increased substantially (M=3.03, SD=.87). A within-subjects repeated measures ANOVA was significant, F (2,15 )= 41.6, p<.05; from time 1 to 2 the increase in student scores was not significant (before program; desired result); however, differences were significant between time 1 and 3 p= <.05 and time 2 and 3 p= <.05 (after program). The results revealed the TLAS elementary program was associated with a significant increase in quality of student responses. Overall, implications of this initial longitudinal evaluation are promising, suggesting that it is possible to teach critical scientific thinking skills at the elementary level.

Publications

• Ceci, S. J. & Williams, W. M. (2010). Sex differences in math-intensive fields. Current Directions in Psychological Science, 19(5), 275-279. Ceci, S. J., Williams, W. M., & Barnett, S. M. (2009). Womens underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135 (2): 218-261.
• Ceci, S. J., Williams, W. M., & Barnett, S. M. (2011). Sex differences in mathematical and spatial ability. In: Encyclopedia of Giftedness, Creativity, and Talent. New York: Sage.
• Rindermann, H., Ceci, S. J., & Williams, W. M. (2010). Whither cognitive talent Understanding high ability, its development, relevance and furtherance. In S. B. Kaufman (Ed.), Beyond talent or practice The multiple determinants of greatness. Oxford: Oxford University Press.
• Whitecraft, M. A. & Williams, W. M. (2011). Why are there so few women in computer science In: Beautiful Code (second edition), ed. G. Wilson. Cambridge, MA: Riley.

Progress 10/01/08 to 09/30/09

Outputs
OUTPUTS: "Thinking Like A Scientist" trains teachers to inculcate scientific thinking by focusing on real-world examples relevant to students' daily lives. Teachers' approaches are changed as a result of learning about and participating in a scaffolded implementation of this program they come to emphasize more often how to think critically and reason effectively, and less often how to memorize facts and figures. Students taught TLAS see, often for the first time, that scientific thinking is something they can use to deal with problems they face in their lives. Their attitudes toward science are changed by this experience; they also learn effective thinking skills, which once learned can be used and maintained. In this second year of a three-year project, I was forced to adjust to the substantial fallout from the economic crisis in New York City public schools by shifting focus to program development and evaluation in the Ithaca City School district. New York City teachers who had participated in Year One found themselves reassigned or out of work. Remaining teachers were heavily overcommitted, and consequently it was a difficult time to work in this venue. Thus, I returned 50% of the allocated funds and used the remainder of the money to work intensively on development and evaluation of a "Thinking Like a Scientist" program for elementary school students. The chief goal was to write, revise, and conduct a detailed longitudinal evaluation of this program with an eye toward eventually making it available broadly to teachers of disadvantaged elementary-aged youth. This evaluation required the development of three versions of an open-ended scientific-critical-thinking test which was administered to the students before, during, and after the program was taught. The newly created "Thinking Like A Scientist" educational outreach program for elementary-age students focuses on teaching basic critical thinking and reasoning skills as applied to science education. I worked with Laurie Rubin, a second-grade teacher in Ithaca with 25 years experience, much of it spent at Beverly J. Martin elementary school, which has a substantial African American/Latino/economically-disadvantaged student enrollment. The PI and Rubin, along with an undergraduate honors student of the PI's (Jessica Zulawski), developed and revised the curriculum, which was then taught by Rubin. The materials focus on the psychology of eating (a topic the young children can relate to), and use this topic to teach critical thinking and reasoning. The second major effort during Year Two involved offering a summer science program in the Department of Human Development as part of the annual 4-H Cornell extension-outreach to high school students in late June-early July. We taught 20 students highlights of the TLAS program, focusing on careers available in science and how to acquire the education necessary to pursue these careers. We will offer this 4-H summer program again in 2010. PARTICIPANTS: Nothing significant to report during this reporting period. TARGET AUDIENCES: Nothing significant to report during this reporting period. PROJECT MODIFICATIONS: Nothing significant to report during this reporting period.

Impacts
This year, I evaluated whether critical scientific thinking skills of elementary school children can be enhanced through the "Thinking Like a Scientist" approach. Critical thinking skills were defined as the ability to apply the scientific method successfully in real-world decision-making. The TLAS program includes discussions and activities on how to generalize the scientific method. Participants were a class of 19 public-school 2nd graders (Mage=7.2 years, SD=0.7), 10 girls and 9 boys. The curriculum included lessons linking the scientific method to real world issues for 7-8 year olds (e.g., making healthy decisions about food and exercise); it introduced basic concepts such as "biased source," and "how to create a hypothesis." The evaluation measures contained open-ended questions assessing students' ability to apply domain-general scientific reasoning (e.g., Kyle picks a brand of granola bar because the TV commercial has a famous movie star in it. Is this: Good Thinking, Not-so-good thinking, or Don't Know. Why). I hypothesized that TLAS lessons on critical thinking would increase these particular skills and, consequently, assessment scores in the students. I developed three versions of the evaluation measure (A, B, C) which were used in an experimental design in which the class served as its own control. Version A (pretest) provided baseline data. Version B, administered 2.5 months after Version A, captured any maturational effects due to being taught by this particular teacher, and evaluated acclimation to the testing process and familiarity with the general measure. Version C (post-test) measured gains in domain-general scientific reasoning resulting from the program; it was administered 2.5 months after Version B, during which time (between B and C) the teacher taught the program. Students were tested individually by the researcher who asked students the test questions and wrote down their answers. Each question probed one concept, such a correlation vs. causation, or distinguishing good vs. bad sources of information. The tests were rated by one graduate student unaffiliated with the project plus the research assistant, using a 1 (poor)-to-5 (excellent) scale for each answer (scoring by two independent raters minimized rating bias; Cohen's kappa measure of reliability, k=0.61). The baseline (Version A) per-item mean was 1.86(SD=.39). At second data collection (Version B), scores were comparable (M=1.94, SD=.48). Next, the curriculum was taught. At third data collection (Version C; after curriculum) scores increased substantially (M=3.03, SD=.87). A within-subjects repeated measures ANOVA was significant, F (2,15 )= 41.6, p<.05; from time 1 to 2 the increase in student scores was not significant (before program; desired result); however, differences were significant between time 1 and 3 p= <.05 and time 2 and 3 p= <.05 (after program). The results revealed the TLAS elementary program was associated with a significant increase in quality of student responses. Overall, implications of this initial longitudinal evaluation are promising, suggesting that it is possible to teach critical scientific thinking skills at the elementary level.

Publications

• Barnett, S.M., Rindermann, H., Williams, W. M., & Ceci, S.J. (2010). The relevance of intelligence for society: Predictiveness and relevance of IQ for societal outcomes. The Cambridge Handbook of Intelligence.
• Ceci, S. J., & Williams, W. M. (2009). The mathematics of sex: How biology and society conspire to limit talented women and girls. New York: Oxford University Press.
• Ceci, S. J., Williams, W. M., & Barnett, S. M. (2009). Womens underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135 (2): 218-261.
• Ceci, S. J., & Williams, W. M. (2009). Should scientists study race and IQ Yes: The scientific truth must be pursued. Nature, 457 (12 February 2009), 786-789.
• Sternberg, R. J., & Williams, W. M. (2010; released 2009). Educational psychology, second edition. Boston: Merrill.
• Williams, W. M., & Ceci, S. J. (2009). Race: A useful way to glean social information. Nature, 458 (12 March 2009), p. 147. Williams, W. M., Barnett, S. M., & Valla, J. M. (2008). IQ and testing: critiques. In: Encyclopedia of Race and Racism. New York: Macmillan--Thomson Gale.

Progress 10/01/07 to 09/30/08

Outputs
OUTPUTS: This was Year One of a three-year project, and it contained three independent areas of effort. In the first area, the PI was assisted by Steve Hamilton, vice provost for extension, in locating a public high school in New York City with a substantial African American and Latino population. We chose Vanguard High School on Manhattans east side. The PI worked with two science teachers (Scott Livingstone and Ari Sussman) from Vanguard who implemented the TLAS educational materials in their science classes during the spring semester, 2008. Pilot assessments were administered before and after the curriculum was taught. The teachers provided feedback to the PI regarding what worked best and why, as well as what needed to be changed for this population prior to round-two implementation in spring 2009. Next, over the summer of 2008, Scott Livingstone taught a special class of remedial science students who had scored poorly in class during the school year and who thus needed extra work to accrue graduation credits. He used the TLAS program, focusing specifically on the violent videogames, depression, and herbal-supplements for cognitive enhancement modules. The remedial group benefited from the focus of TLAS, which helped to motivate them during the summer sessions. The second area of effort involved initial work on the development of a Thinking Like A Scientist educational outreach program for 2nd-3rd graders. This elementary-age program focuses on teaching basic critical thinking and reasoning skills as applied to science education.The PI worked with Laurie Rubin, a second-grade teacher in Ithaca with 25 years experience, much of it spent at Beverly J. Martin elementary school, which has a substantial African American/Latino/economically-disadvantaged student enrollment. The PI and Rubin, along with an undergraduate honors student of the PIs (Jessica Zulawski), developed a draft curriculum which is being implemented and evaluated in Year Two. The materials focus on the psychology of eating (a topic the young children can relate to), and use this topic to teach critical thinking and reasoning. The third major effort involved offering a summer science program in the Department of Human Development as part of the annual 4-H Cornell extension-outreach to high school students in late June-early July. We taught 18 students highlights of the TLAS program, focusing on careers available in science and how to acquire the education necessary to pursue these careers. We will be offering this 4-H summer program again in 2009. PARTICIPANTS: Nothing significant to report during this reporting period. TARGET AUDIENCES: Nothing significant to report during this reporting period. PROJECT MODIFICATIONS: Nothing significant to report during this reporting period.

Impacts
Thinking Like A Scientist trains teachers to inculcate scientific thinking by focusing on real-world examples relevant to students daily lives. Teachers approaches are changed as a result of learning about and participating in a scaffolded implementation of this program. Students taught TLAS see, often for the first time, that scientific thinking is something they can use to deal with problems they face in daily life. Their attitudes toward science are changed by this experience; they also learn effective thinking skills, which once learned are used and maintained. TLAS also provides high-school students with information about science careers available to them. Increasing the ethnic diversity of professions and careers in scientific areas is a key concern for all educators and indeed, for our entire society, to better enable us to serve the breadth of people in the U.S. population. The ultimate beneficiaries of effective educational programs are developing youth, who are better prepared to succeed and lead fulfilling lives. Teachers also benefit by having programs to offer that are empirically defensible and that do what they promise--in this case, train scientific thinking ability. Not only the youth, but also our society at large benefit when youth of color and disadvantaged youth pursue science education (and complete high school degrees) and ultimately even pursue science careers. Our professions desperately need ethnic diversification so that the next generation of youth have mentors and leaders from their own ethnic and economic backgrounds to show what is possible, despite early disadvantage. Year Ones specific project outcomes included: 1. The PI recruited and trained two New York City high school teachers serving African American and Latino youth in a novel approach for teaching scientific thinking and reasoning about problems in daily life (TLAS). At Vanguard High School in New York City, two classes were taught TLAS during spring semester, 2008, and one large remedial summer class in 2008 was also taught a modified TLAS curriculum. The PI and the teachers administered preliminary evaluation assessments to the students, to collect data on student improvement and curriculum impact. Through these assessments the PI evaluated the appropriateness of the materials for this location prior to use in Year 2. 2. The PI recruited and train one undergraduate and one graduate student research assistant at Cornell University who provided general administrative support for the project, including teaching the summer Cornell TLAS 4-H program. 3. The PI and her undergraduate research assistant, Jessica Zulawski, worked with Ithaca-area teacher Laurie Rubin to develop a first-draft TLAS educational program for 2nd-3rd graders. This program will be taught and evaluated during Year 2. The PI trained Laurie Rubin and Jessica in use of the program, which Jessica assist with in the classroom. 4. The PI and her graduate student and undergraduate student research assistants offered a 3-day class to 18 high-school students on the TLAS program as part of the annual, 4-H summer program for high school students at Cornell in late June-early July.

Publications

• Williams, W. M., & Ceci, S. J. (2007). Striving for perspective in the debate on women in science. In: Why arent more women in science Top researchers debate the evidence. (S. J. Ceci & W. M. Williams, Eds.). Washington, D.C.: American Psychological Association Books.
• Ceci, S.J., & Williams, W. M. (2007). Are we moving closer and closer apart Resolving conflicting views on women in science. In: Why arent more women in science Top researchers debate the evidence. (S. J. Ceci & W. M. Williams, Eds.). Washington, D.C.: American Psychological Association Books.
• Ceci, S.J., Williams, W.M., & Barnett, S.M. (2009, March). A framework explaining the underrepresentation of women in mathematically-intensive fields of science. Psychological Bulletin.
• Ceci, S. J., Williams, W. M., & Barnett, S. M. (2008). Sex Differences in Mathematical and Spatial Ability. In Encyclopedia of Giftedness, Creativity, and Talent. Sage.
• Ceci, S. J., & Williams, W. M. (2008). Do We Limit Women in Science Oxford University Press.