Sample Size in Qualitative Interview Studies: Guided by Information Power

Qualitative research: standards, challenges, and guidelines

Qualitative research methods could help us to improve our understanding of medicine. Rather than thinking of qualitative and quantitative strategies as incompatible, they should be seen as complementary. Although procedures for textual interpretation differ from those of statistical analysis, because of the different type of data used and questions to be answered, the underlying principles are much the same. In this article I propose relevance, validity, and reflexivity as overall standards for qualitative inquiry.

Evidence appraisal: a scoping review, conceptual framework, and research agenda

Abstract Objective Critical appraisal of clinical evidence promises to help prevent, detect, and address flaws related to study importance, ethics, validity, applicability, and reporting. These research issues are of growing concern. The purpose of this scoping review is to survey the current literature on evidence appraisal to develop a conceptual framework and an informatics research agenda. Methods We conducted an iterative literature search of Medline for discussion or research on the critical appraisal of clinical evidence. After title and abstract review, 121 articles were included in the analysis. We performed qualitative thematic analysis to describe the evidence appraisal architecture and its issues and opportunities. From this analysis, we derived a conceptual framework and an informatics research agenda. Results We identified 68 themes in 10 categories. This analysis revealed that the practice of evidence appraisal is quite common but is rarely subjected to documentation, organization, validation, integration, or uptake. This is related to underdeveloped tools, scant incentives, and insufficient acquisition of appraisal data and transformation of the data into usable knowledge. Discussion The gaps in acquiring appraisal data, transforming the data into actionable information and knowledge, and ensuring its dissemination and adoption can be addressed with proven informatics approaches. Conclusions Evidence appraisal faces several challenges, but implementing an informatics research agenda would likely help realize the potential of evidence appraisal for improving the rigor and value of clinical evidence.

Conducting a Qualitative Document Analysis

Document analysis has been an underused approach to qualitative research. This approach can be valuable for various reasons. When used to analyze pre-existing texts, this method allows researchers to conduct studies they might otherwise not be able to complete. Some researchers may not have the resources or time needed to do field research. Although videoconferencing technology and other types of software can be used to reduce some of the obstacles qualitative researchers sometimes encounter, these tools are associated with various problems. Participants might be unskillful in using technology or may not be able to afford it. Conducting a document analysis can also reduce some of the ethical concerns associated with other qualitative methods. Since document analysis is a valuable research method, one would expect to find a wide variety of literature on this topic. Unfortunately, the literature on documentary research is scant. This paper is designed to close the gap in the literature on conducting a qualitative document analysis by focusing on the advantages and limitations of using documents as a source of data and providing strategies for selecting documents. It also offers reasons for using reflexive thematic analysis and includes a hypothetical example of how a researcher might conduct a document analysis.

Identity, positionality and reflexivity: relevance and application to research paramedics

This article introduces the reader to the concepts of identity, positionality and reflexivity and outlines their relevance to research paramedics. We outline how a researcher’s identity and positionality can influence all aspects of research, including the research question, study design, data collection and data analysis. We discuss that the ‘insider’ position of paramedics conducting research with other paramedics or within their specific clinical setting has considerable benefits to participant access, understanding of data and dissemination, while highlighting the difficulties of role duality and power dynamics. While positionality is concerned with the researcher clearly stating their assumptions relating to the research topic, the research design, context and process, as well as the research participants; reflexivity involves the researcher questioning their assumptions and finding strategies to address these. The researcher must reflect upon the way the research is carried out and explain to the reader how they moved through the research processes to reach certain conclusions, with the aim of producing a trustworthy and honest account of the research. Throughout this article, we provide examples of how these concepts have been considered and applied by a research paramedic while conducting their PhD research studies within a pre-hospital setting, to illustrate how they can be applied practically.

How to Run Surveys: A Guide to Creating Your Own Identifying Variation and Revealing the Invisible

Surveys are an essential approach for eliciting otherwise invisible factors such as perceptions, knowledge and beliefs, attitudes, and reasoning. These factors are critical determinants of social, economic, and political outcomes. Surveys are not merely a research tool. They are also not only a way of collecting data. Instead, they involve creating the process that will generate the data. This allows the researcher to create their own identifying and controlled variation. Thanks to the rise of mobile technologies and platforms, surveys offer valuable opportunities either to study broadly representative samples or to focus on specific groups. This article offers guidance on the complete survey process, from the design of the questions and experiments to the recruitment of respondents and the collection of data to the analysis of survey responses. It covers issues related to the sampling process, selection and attrition, attention and carelessness, survey question design and measurement, response biases, and survey experiments.

Uncertainty limits the use of power analysis.

Why Most Published Research Findings Are False

Summary There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

Document Analysis as a Qualitative Research Method

This article examines the function of documents as a data source in qualitative research and discusses document analysis procedure in the context of actual research experiences. Targeted to research novices, the article takes a nuts‐and‐bolts approach to document analysis. It describes the nature and forms of documents, outlines the advantages and limitations of document analysis, and offers specific examples of the use of documents in the research process. The application of document analysis to a grounded theory study is illustrated.

Argumentation theory | Communication and Mass Media | Research Starters | EBSCO Research

<p>Argumentation theory explores the processes and methods of reasoning and debate used by individuals in both formal and informal contexts. The theory has roots in ancient philosophical discourse, particularly from figures like Aristotle, and has evolved through the contributions of modern philosophers such as Chaïm Perelman and Stephen Toulmin. It highlights how arguments are structured, identifying key components such as claims, grounds (or data), and warrants, which collectively help participants make their case. </p> <p>Additionally, arguments can be categorized into three main types: factual claims, which are verifiable; judgment or value claims, which are subjective; and policy claims, which pertain to proposed courses of action. This framework acknowledges the influence of personal biases, often shaping the reasoning process, and emphasizes the importance of logical support, backing, qualifiers, and rebuttals in strengthening arguments. In academic contexts, the theory suggests that creating valid topics should focus on policy arguments, while also addressing counterarguments to foster a comprehensive debate. Overall, argumentation theory serves as a critical tool for understanding how reasoning and persuasive communication function in various scenarios.</p>

A meta-analysis of the persuasive power of large language models

Large language models (LLMs) are increasingly used for persuasion, such as in political communication and marketing, where they affect how people think, choose, and act. Yet, empirical findings on the effectiveness of LLMs in persuasion compared to humans remain inconsistent. The aim of this study was to systematically review and meta-analytically assess whether LLMs differ from humans in persuasive effectiveness, and under which contextual conditions LLMs are particularly effective. We identified 7 studies with 17,422 participants primarily recruited from English-speaking countries and 12 effect size estimates. Egger’s test indicated potential small-study effects ($$p =.018$$), but the trim-and-fill analysis did not impute any missing studies, suggesting a low risk of publication bias. We then compute the standardized effect sizes based on Hedges’ g. The results show no significant overall difference in persuasive performance between LLMs and humans ($$g = 0.02$$, $$p =.530$$). However, we observe substantial heterogeneity across studies ($$I^2 = 75.97\%$$), suggesting that persuasiveness strongly depends on contextual factors. In separate exploratory moderator analyses, no individual factor (e.g., LLM model, conversation design, or domain) reached statistical significance, which may be due to the limited number of studies. When considered jointly in a combined model, these factors explained a large proportion of the between-study variance ($$R^2 = 81.93\%$$), and residual heterogeneity is low ($$I^2 = 35.51\%$$). Although based on a small number of studies, this suggests that differences in LLM model, conversation design, and domain are important contextual factors in shaping persuasive performance, and that single-factor tests may understate their influence. Our results highlight that LLMs can match human performance in persuasion, but their success depends strongly on how they are implemented and embedded in communication contexts.

Address research questions fast • Gems

Your qualitative study in one place, enhanced by AI.

Designing Information Provision Experiments

Information provision experiments allow researchers to test economic theories and answer policy-relevant questions by varying the information set available to respondents. We survey the emerging literature using information provision experiments in economics and discuss applications in macroeconomics, finance, political economy, public economics, labor economics, and health economics. We also discuss design considerations and provide best-practice recommendations on how to (i) measure beliefs; (ii) design the information intervention; (iii) measure belief updating; (iv) deal with potential confounds, such as experimenter demand effects; and (v) recruit respondents using online panels. We finally discuss typical effect sizes and provide sample size recommendations.

Nothing to Declare

Making big public statements is always fun and people who think themselves to be important love doing it as a way of trying to influence public opinion and/or politics. They are a way for institutions and individuals to organize and try to shine some light onto important issues. We’ve seen many such things in the […]

8  Sample Size Justification – Improving Your Statistical Inferences

This open educational resource contains information to improve statistical inferences, design better experiments, and report scientific research more transparently.

Practicing Information Sensibility: How Gen Z Engages with Online Information

Assessing the trustworthiness of information online is complicated. Literacy-based paradigms are both widely used to help and widely critiqued. We conducted a study with 35 Gen Zers from across the U.S. to understand how they assess information online. We found that they tended to encounter -- rather than search for -- information, and that those encounters were shaped more by social motivations than by truth-seeking queries. For them, information processing is fundamentally a social practice. Gen Zers interpreted online information together, as aspirational members of social groups. Our participants sought information sensibility: a socially-informed awareness of the value of information encountered online. We outline key challenges they faced and practices they used to make sense of information. Our findings suggest that like their information sensibility practices, solutions and strategies to address misinformation should be embedded in social contexts online.