Episode 3: Ensure scientific quality through an appropriately diverse community

This is an example of episodes Gary and Nathan will intersperse strategically for knowledge integration across other episodes, to make connections among topics discussed with various guests, and to highlight important themes across these topics. In this episode, they focus on the theme of community in science to make connections between their interview with Fred Diedrich and future episodes that will address quality in science.

In a prior episode, Dr. Diedrich emphasized the necessity and difficulty of balancing internal and external validity (e.g., rigorousness and relevance) in field-based research. He shared the lessons he learned about balancing these quality criteria over time and multiple investigations in a variety of settings. In the current episode Gary talks with Nathan about the role the broader scientific community has in this strategic balancing act.

Gary discusses how the emergence of scientific quality can be understood in terms of the sociology and philosophy of science as well as in the broader programmatic reflections and strategic plans of the scientists involved in particular lines of investigation. He points out that, while such reflections on science can be quite esoteric, they also have very practical relevance and manifestations to which nonscientists can make substantive contributions. As an example of the latter, he highlights the landmark decision of the U.S. Supreme Court in the case of Daubert v. Merrell Dow Pharmaceuticals (92-102), 509 U.S. 579 (1993) .

Two aspects of peer review are critical in assessing scientific expertise in the law. One is whether the scientific evidence cited by experts in the courtroom has been reviewed favorably by peers in the scientific community. The other is whether the body of work by the expert is recognized and vetted in a scientific community of practice. Gary summarizes the criteria that are important in such broader notions of peer review in a scientific community over time: (1) dialectic by which claims are nonobvious and can be refuted in principle based on validity of assumptions, (2) falsifiability by which it must be possible in principle to acquire evidence that show that claims are false, (3) utility by which the research has a defined real world use such as in engineering, (4) generativity by which particular research stimulates creativity and insight beyond its own claims and evidence, and (5) generalizability enabled by sufficiency of description of both the attendant equivalence and variation.

The episode concludes with some examples that Gary provides from his own experience with scientific communities of practice in which both field research and laboratory research are utilized in balancing internal and external validity with attention to criteria for quality in scientific research. Nathan and Gary preview future episodes in which their guests will discuss such research issues.

Key Terms and Concepts

sociology and philosophy of science
dialectic
falsifiability
utility
generativity
generalizability