Fornaro, C. J., Sterin, K., & Struloeff, K. L. Book Review Current Issues in Comparative Education (CICE), Volume 23, Issue 1, Winter 2021 Book Review Qualitative data collection tools: Design, development, and applications by Felice D. Billups SAGE Publications, 2020, 240 pp. $30.00 (paper) ISBN-10: 1544334826 Christopher J. Fornaro Drexel University, School of Education Kimberly Sterin Drexel University, School of Education Katrina Lynn Struloeff Drexel University, School of Education Where Do We Even Begin? Research is not easy. Learning how to navigate qualitative research can be particularly murky. The three reviewers of Billups (2020) book, Qualitative data collection tools: Design, development, and applications, used the text during an introduction to qualitative research methods and analysis class within their Ph.D. program and strongly recommend it to other PhD students and early scholars. After completing foundational courses and learning about some of the belief systems that drive educational research, such as epistemology and ontology, it was time to tackle a research project. So what now? The three reviewers, all first year PhD students at the time, came to their qualitative course excited to work through pilot studies in their own focus areas. They had all spent time reading research studies, writing about potential research they would conduct, and discussing theoreticals, but this was the opportunity to design and conduct their own research. The reviewers found the following three sections particularly helpful when managing the logistics of their pilot studies to provide the “thick description” (Geertz, 1973) that is vital to qualitative research: (1) Researcher Role, Access, Trustworthiness, and Ethical Concerns, (2) Interview Protocol, and (3) Observation Tools. While these sections are not novel to the Billups text, the authors had read this type of guiding text in similar texts such as Cohen et al. (2007) and Creswell and Guetterman (2018), Billups’s writing is in digestible chunks that provide salient points to support getting out and conduct research. Furthermore, each chapter contains various templates that suggest ways to collect data for a research study. These templates were pivotal in the reviewers’ development of their pilot studies and three salient examples are provided below. Researcher Role, Access, Trustworthiness, and Ethical Concerns (Chapter 3) No matter which epistemological camp qualitative researchers situated themselves within, the vitally and conversation of their role as the researcher must be addressed. © 2021, Fornaro. CC BY 4.0. Current Issues in Comparative Education 23(1), 109-112. https://https://creativecommons.org/licenses/by/4.0/deed.en Fornaro, Sterin & Struloeff Current Issues in Comparative Education 110 Methodologists often state that the researcher is the tool, but actual application of that phrase is one that novice researchers can struggle to embrace. Billups addresses this struggle with a practical conversation of each facet and tools for implementation. First, through the discussion of bias and positionality, Billups outlines how researchers need to provide transparency of who they are, what formed their experiences and beliefs, and how these components bring potential bias to their research practice. Through this, Billups suggests the inputs of building a quality transparent relationship through interviewing, transcribing, and addressing findings with participants. Struloeff found these strategies to be extremely helpful during her communication, interviews, follow-up, and presentation of findings with participants. Interviewing up, that is interviewing participants who hold a higher rank institutionally and socially than the researcher, Struloeff was able to offer transparency and authenticity allowing participants to feel heard, offer additional insights, and ultimately critically engage in their own reflections. Billups created a table that provides the commonalities of rigor between quantitative and qualitative research with a space for the bridge between the two forms of research and the essential questions. This simple representation assisted Struloeff in understanding the terminology, approach, and parallels between the often dichotomous and contentious forms of research. Perhaps more importantly, it provided a starting point, language for discussion, and tactics for how to ensure credibility, dependability, transferability, confirmability, and authenticity. As qualitative research continues to gain recognition and traction, researchers must continue to strive for excellence, especially in the areas addressed in this chapter. Interview Protocols (Chapter 4) Interviews are often the crux of qualitative data collection. Billups first describes the strengths and limitations of an interview, the requisite characteristics needed for a skilled interviewer, and the various formats and types of interviews. What sets the Billups text apart from other books on qualitative research is that she then provides eight interview protocol templates for the researcher to use as models for their particular research study. The first two templates are for unstructured and semi-structured interviews, respectively. Sterin found that having a visual example of what the interview protocol could look like extremely helpful. Billups includes placeholders for logistical information such as the date, time, and place at the top of the template as well as reminders about interview information and procedures to relate to the interviewee. When conducting multiple interviews over long periods of time this information may be helpful for accurate record keeping and consistency of depth of information given to the interviewee about the research study. Next, Billups includes an outline for the sequence of questions that includes bolded sections and a layout for the interview questions. As a novice researcher, Sterin found this guidance to be particularly useful both when designing her interview protocol and when justifying the question sequence in her description of the research design. Billups advises to begin with broad, introductory questions, then move to content questions with potential probing questions, and finally, to conclude with a summary question and an invitation to the interviewee to add anything else they would like to share Qualitative data collection tools: Design, development, and applications Current Issues in Comparative Education 111 on the topic. Sterin’s application of this last open-ended question proved to be notably fruitful when conducting her interviews. The templates conclude with a note to thank the interviewee and reminders about any follow-ups the researcher may want to engage in. After using this template, Sterin can attest to the value of including these final reminders within the interview protocol itself. During the course of an interview, it is easy for the researcher to become so engrossed with the content that they may benefit from having pre-written reminders for logistical next steps. The third template is a note-taking recording sheet designed to capture non-verbal cues and templates 4 through 8 are specific to certain research designs including phenomenological, ethnographic, and narrative. Researchers looking to utilize these designs would benefit from these interview templates. Observation Tools (Chapter 7) Observations seem to be a straightforward process at first. Document what was said, by who, and the group dynamic. However, Billups sets the stage with the researcher needing to consider how they are coming to the observation. She uses Gold’s (1958) categories of a complete participant, observer as quasi-participant, researcher as participant, and complete observer as a consideration for how the researcher is approaching their observation and their level of involvement. The concept of observing an established group as an outsider was challenging for Fornaro to approach. Additionally, due to COVID-19, the observation would take place on a web- based video conferencing platform. This meant that the field of view was contained to what was visible in each participant's camera. Fornaro leaned heavily on two of the three templates provided by Billups that focused on creating a chart for the observation. Billups suggestion to observe “not just what is happening but what is not happening” (Billups, 2020, p. 133) is evident through her inclusion of “group interaction and behaviors, tone/mood” in Template 7.2. By including a similar topic for observation in the pilot studies observational chart, Fornaro was able to take note of how talking time and tone shifted between employees of an organization depending on their experience at the organization. As an outsider to an organization, having the templates from this text allowed Fornaro to stay focused throughout the observation and look for key interactions between participants that may have otherwise been missed. Adding to Your Qualitative Toolbox The sections above are just three of the chapters that contained templates that the authors directly related to their own process. Billups included templates and guidance for conducting discourse analysis, moderating focus groups, document analysis, reflecting on research, and synthesizing data sources in qualitative research. While each section is not exhaustive on these topics, they equip new or early qualitative scholars with the main ideas and tools to design, collect, and analyze quality data for their research projects which was what Billups set out to do with this text. The templates should be utilized as a living document for early scholars to build from and adjusted to their own contexts. Qualitative data collection tools: Design, development, and applications is a text that should be Fornaro, Sterin & Struloeff Current Issues in Comparative Education 112 included in early scholars’ libraries as they look to develop their first qualitative research projects. Christopher Fornaro is a Ph.D. student in Drexel’s School of Education with a focus in STEM education. His current research interests center on supporting instructors in out-of-school STEM programming, the benefits for students of out-of-school STEM programming, and ways to authentically blend together the individual disciplines of STEM. Kimberly Sterin is a doctoral student pursuing a Ph.D. in Education Policy and Leadership from the School of Education at Drexel University. Sterin has spent seven years as an English teacher in K-12 public schools in Montgomery County, Maryland. Sterin earned a Master's degree in Secondary Education from Johns Hopkins University. Katrina Struloeff is a Ph.D. Student in Drexel’s School of Education. She earned a Master’s degree from Carnegie Mellon University Heinz College in public policy. Her research interests focus on leadership and representation for women and minoritized populations, policy design, and principles and practices of effective leaders in education. References Billups, F. D. (2019). Qualitative data collection tools: Design, development, and applications (Vol. 55). SAGE Publications. Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). Routledge. Creswell, J. W. & Guetterman, T.C. (2018). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (6th ed.). Pearson. Geertz, C. (1973). The interpretation of cultures: selected essays. Basic Books. Gold, R. L. (1957). Roles in sociological field observations. Social Forces, 36, 217-223