FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence

Introduction

New findings in medicine observed in randomized controlled trials (RCTs) are published in journal articles which describe their design and outcomes. These RCTs "measure the effectiveness of a new intervention or treatment" (Hariton and Locascio, 2018) and are the important basis of evidencebased medicine (Sackett et al., 1996). However, * Equal Contribution understanding such articles requires "specific attention outside of general literacy capacities" (Ad Hoc Committee on Health Literacy, 1999), rendering them effectively inaccessible to most (lay) people. Ideally, healthcare providers would stay current on all medical evidence and share relevant findings with patients, but this is impractical due to the volume and growth of the evidence base (Bastian et al., 2010;Marshall et al., 2021).

LLMs may provide a means for lay readers to access such findings by automatically summarizing and simplifying texts into plain language (August et al., 2023;Shaib et al., 2023). Done successfully, this could allow patients to access the most up-todate literature relevant to their healthcare. In turn, this may promote health literacy broadly by disseminating trustworthy information (Thielmann et al., 2023;Cheng et al., 2022). But given the inherent risks to personal health, the factual correctness of such outputs is paramount in this domain.

While Shaib et al. (2023) showed that GPT-3 infrequently introduced outright errors when simplifying RCT abstracts, inaccuracies are occasionally introduced; this ought to be addressed before wide adoption of such technology. Unfortunately, there is no standard evaluation benchmark for factuality on this important medical evidence text simplification task. Consequently, it is unknown whether and to what degree existing automatic factuality evaluation metrics align with human judgments.

We posit that focusing on critical elements in the RCT structure is key for factual medical evidence communication. This work presents FACT-PICO, an expert-constructed factuality benchmark for the plain language summarization of technical abstracts describing RCTs.foot_0 FACTPICO is a finegrained benchmark focused on key characteristics of trials: Populations (e.g. COVID patients; diabetics), Interventions (e.g. remdesivir), Comparators (e.g. placebo), Outcomes (e.g. 30 day mortality, or pain), as well as Evidence Inference (i.e., whether the intervention yielded a significant difference in the treated group with respect to the outcome; Lehman et al. 2019). PICO is a standard framework to structure clinical questions.

Figure 1 shows an example of FACTPICO annotation. In contrast to standard summaries, plain language summaries have the additional goal of simplifying content for lay readers. This may involve elaboration and explanation of difficult concepts to foster understanding (Srikanth and Li, 2021). Thus, FACTPICO includes a correctness assessment of added content. FACTPICO is also distinct in that it includes expert-written rationales that contextualize the evaluation of these fine-grained characteristics, providing a useful first step in assessing explainable factuality evaluation methods.

FACTPICO includes outputs from a mix of proprietary and open-source LLMs (GPT-4, Llama-2-Chat, and Alpaca). Our findings are somewhat less optimistic than prior work in medical (Shaib et al., 2023) and news summarization (Goyal et al., 2022): Factual errors (occasionally important ones) with respect to key RCT elements are introduced by LLMs. One concerning phenomenon is the extent to which models overgeneralize, resulting in problematic information loss (Trienes et al., 2024).

Using FACTPICO, we evaluated a suite of exist-ing automatic metrics shown to perform well for factuality in summarization (Scialom et al., 2021;Goyal and Durrett, 2021;Fabbri et al., 2022;Zha et al., 2023a;Tang et al., 2023a), as well as newly devised LLM-based evaluations. We find that existing metrics correlate with expert ratings at the system level, but not at the instance level. The best performing metric is an LLM-based approach in which we first identify key RCT elements; this shows that providing models with explicit domain knowledge may help. Analysis of LLM-generated rationales shows that LLMs often provide flawed reasoning when justifying their self-evaluations.

Human Evaluation Framework

We evaluate generated summaries using a set of questions addressing factuality as related to the key PICO aspects of RCTs, as well as information added by LLMs during simplification. We ask evaluators to score model outputs and provide natural language rationales to justify their ratings. Annotation interface details are in Appendix P.

PICO Elements. Population, Intervention, Comparator, Outcome (PICO) elements are the key components of an RCT (Richardson et al., 1995). The trial population concerns characteristics of subjects in the trial, including what condition they have, the number of participants, and their demographics. The intervention is the active treatment being assessed for comparative efficacy; the comparator is the control with respect to which this is being compared. Finally, outcomes are those things that are measured to determine results.

Accurate representation of these essential descriptors in the plain language summaries of an RCT is imperative. More explanations and examples are provided in Appendix Table 7. We ask evaluators to provide a rating between 1 to 4 codifying the factuality with respect to each PICO element, respectively: 4: Mentioned and described accurately; 3: Mentioned but described somewhat inaccurately or vaguely; 2: Mentioned but described with severe inaccuracies and/or is missing critical descriptors; 1: Missing. We describe the rating system in greater detail in Appendix A.

Rationales. Evaluators are also asked to provide natural language rationales justifying their chosen rating. Such rationales may reveal technical justifications for annotations which would be difficult for a layperson to assess (see Figure 2 for examples). More usefully, rationales can express a degree of uncertainty in evaluations. For example, consider the following rationale for an intervention being evaluated as accurately mentioned:

The interventions are clearly described as one group receiving morphine and one triamcinolone, yet their combination with bupivacaine is missing, but doesn't seem very important.

This rationale indicates that the choice to evaluate the intervention as accurate was not a clear-cut decision. By including rationales in FACTPICO, this complex decision process is documented. We compare LLM rationales and expert rationales in Section 6, and envision future work to dive deeply into explainable factuality measures.

Evidence Inference. We additionally evaluate the evidence inference aspect of LLM-generated plain language summaries, i.e., whether the results and findings concerning PICO are reported factually, as PICO covers what outcomes were measured, it does not account for the corresponding results (e.g., if an intervention was found to outperform a comparator). A challenge here is that most trials will report multiple results, any of which may be conveyed (un)factually in the summary. Thus, we collect factuality assessments at the level of individual results, using findings annotated in the source abstracts from the Evidence Inference 2.0 dataset as reference. Evaluators are asked to determine how well each particular inference is reflected in the plain language summary (4: accurate, 3: vague/slightly inaccurate, 2: inaccurate, 1: not mentioned), with a free-text rationale.

Added Information. Unlike traditional summarization where content addition is seen as extrinsic hallucination (Maynez et al., 2020), plain language summarization often requires the model to explain and elaborate complex concepts (Srikanth and Li, 2021). It is important to verify the correctness of content additions. We ask annotators to highlight addition spans, determine whether they are factual, and justify each rating with a free-text rationale.

Annotation

FACTPICO was annotated by two senior students in their fifth year of medical school who are highly proficient in English. They are experienced with

Type κ Type κ Population 0.56 Evidence Inference 0.47 Intervention 0.80 Added Info 0.65 Comparator 0.73 -correctness 0.86 Outcome 0.60 Table 1: Inter-evaluator agreement. For PICO evaluations (left) we show agreement on the subset of 15 undiscussed texts (see Appendix C). For other undiscussed questions (right), we show agreement on all 75 doubly annotated texts.

data annotation for text simplification and summarization tasks. To ensure high annotation quality, we conducted a training phase which involved annotating a set of summaries as pilot (excluded from FACTPICO). Next, we collected two sets of annotations on 75 summaries (25 from each model). For the first 60 of these summaries, we asked annotators to discuss their responses with respect to the PICO elements. These discussions were intended to resolve any conceptual differences related to PICO annotation (Appendix C). The remaining 15 summaries were evaluated individually. Compensation was 12C per hour, which is standard for fixed-hour contracts with the annotators' institutions.

Inter-Evaluator Agreement. Table 1 reports agreement on the 15 held-out (undiscussed) subset for PICO elements, as well as agreement on the full set of 75 texts for the Evidence Inference and Added Information questions (all annotated without discussion). We report Randolph's kappa (Randolph, 2005), a free-marginal version of Fleiss' kappa. 3 With respect to added information, the agreement reported in Table 1 is at the sentence level (for each sentence, whether it is considered as added information). The correctness of added information is reported on the 48 spans both annotators identified as added information. Overall, evaluators showed moderate to high agreement (Artstein and Poesio, 2008). This range of kappa values are expected given the degree of subjectivity inherent to rating the severity of factual errors.

Evaluation and Analysis of Plain Language Summaries

Results of the factuality evaluation of the plain language summaries generated by GPT-4, Llama-2, and Alpaca in FACTPICO are presented in Table 3.

We report average ratings for each PICO category and Evidence Inference, as well as the number and percentage of non-factual added information spans identified by either of the evaluators. Since readability is a key goal of plain language summarization, we also report the change in Flesch-Kincaid Grade Level (Kincaid et al., 1975) between the abstract and the summary (∆FK), as well as the ROUGE-L score (Lin, 2004), which quantifies ngram overlap between the generated plain language summary and the original abstract (i.e., high overlap may indicate low readability).

Factuality vs. Simplicity. We observe a clear trade-off between the simplicity of the generated text and its factuality. Plain language summaries generated by Alpaca are rated as more factual across all dimensions, including most of the automatic metrics (Table 5) discussed in Section 5. However, Alpaca is also the most extractive (i.e., heavily relying on deletion), with a near 0.5 ROUGE-L score and an advanced reading level. By contrast, GPT-4 and Llama-2 both simplify by rephrasing, with similar ROUGE-L and reading levels. Yet the plain language summaries generated by GPT-4 and Llama-2 are less factual, with a significant increase in the number of hallucinations (added non-factual information). Comparatively, Llama-2 produced the least factual summaries.

Can LLMs reliably convey critical RCT elements? The fine-grained framework in FACT-PICO exposes issues with LLM-generated plain language summaries for medical texts. As shown in Table 3, LLMs can explain the Interventions and Outcomes more accurately, while failing to do so for other information, such as the Populations, Comparators, and Evidence Inference.

Overgeneralization and omissions of such critical elements may distort the findings and conclusions of medical research studies in nuanced but important ways. Consider this example where GPT-4 omitted a critical element of RCT in its summary.

Scientists did a study to see if people who know they have a gene that puts them at risk for heart problems would eat healthier. They looked at people's intake of a vitamin called folate, which is good for heart health. The participants in the study were told if they have the risk gene or not. But in the end, there was no difference in how much folate the people with the risk gene ate compared to those without the risk gene. So, knowing if they have the risk gene didn't make people eat healthier.

In the original RCT abstract, participants received "either general healthy eating advice or varying lev-Supplementary Elaboration (factual) Source: We conducted a randomized, placebo-controlled clinical trial to prove efficacy of botulinum toxin for neck pain in chronic whiplash syndrome. Simplification : Scientists did a study to see if a certain type of medicine, called botulinum toxin, could help people who had long-lasting neck pain from whiplash (an injury that happens if your head jerks hard). Rationale: This is an explanation that is not mentioned in the original. Substituting Elaboration (factual) Source: The aim of the study was to compare the effect of co-administered dexamethasone and diclofenac potassium (diclofenac K) with diclofenac K alone on the postoperative pain, swelling and trismus after surgical removal of third molars. Simplification [Llama-2]: The study compared how well two different medicines, dexamethasone and diclofenac K, worked together to help people feel better after having surgery to remove their wisdom teeth. Rationale: This is the correct explanation for 3rd molar. Nonsensical Hallucination (non-factual) Source: Intra-articular morphine was as effective as intra-articular triamcinolone for analgesia in patients with osteoarthritis knee. Intra-articular morphine is possibly a better option than intra-articular steroid as it has lesser side effects. Simplification [GPT-4]: The study concluded that both morphine and triamcinolone are useful, but morphine might be better as it has fewer side effects. So, if granny or grandpa is having pain because of osteoarthritis, an injection with morphine could help them better. Rationale: This is definitely not factual or good advice to give, even though the information within is technically true. Jumping to Conclusions (non-factual) Source: This study aimed to determine if raloxifene prevents the negative effects of diabetes on skeletal fragility in diabetes-prone rats. Simplification [Llama-2]: Raloxifene is a medicine that can help prevent diabetes from causing weak bones. Rationale: The abstract says that this study aims to test IF Raloxifene can help prevent diabetes from causing "weak bones"/"if raloxifene prevents the negative effects of diabetes on skeletal fragility in diabetes-prone rats" Table 3: Human evaluation on a 1-4 scale ↑ (Section 2.2) on the factuality of PICO elements and Evidence Inferences. The average length of the original technical abstracts is 343.5 tokens. ∆FK and Rg-L: Flesch Kincaid Grade Level difference and ROUGE-L between abstract and summary. #N: number of non-factual additions. %N: percentage of summaries in FACTPICO with at least one non-factual addition.

els of personalised nutrition advice". One of the levels of such personalized advice involved testing for a gene "in relation to cardiovascular health and the importance of a sufficient intake of folate". But, the generated summary above fails to mention the comparator, which is the general healthy eating advice; instead, it focuses on the overgeneralized intervention, i.e., personalized advice involving genetic variants (without mentioning the personalized nutrition advice). These omissions not only render the simplification incomplete, but they also make understanding the actual result impossible.

Do LLMs generate accurate elaborations? Table 3 also shows a concerning amount of nonfactual additions within the generated simplifications. Examples of these span-level annotations can be found in Table 2. Most of these non-factual additions may misrepresent the original medical text and consequently mislead lay readers. Even GPT-4 produced many plain language summaries (31.3% of all additions added) with such errors, raising questions as to the trustworthiness of LLMs for tasks in high-stakes domains such as medicine, when used by zero-shot prompting (a most common use-case for lay users).

Factuality Metrics Evaluated

We first evaluate a suite of existing automatic factuality metrics shown to be effective in prior work.

(1) Dependency-Arc Entailment (DAE) (Goyal and Durrett, 2021) decomposes summaries into smaller entailment tasks at the arc-level to assess their factuality. We use a numeric score by taking the minimum of the probability scores assigned to individual arcs. (2) QuestEval (Scialom et al., 2021) uses a QA-based framework to analyze the factual faithfulness of a summary to the original text. This method scores summaries a 0 if there is no common token and a 1 for an exact match.

(3) QAFactEval (Fabbri et al., 2022) is a QA- Table 5: Average of systematic metrics per LLM for plain language summary.

based metric that combines various components from other factuality metrics and assigns scores based on the LERC score (Chen et al., 2020). This score, usually used for evaluating reading comprehension answers, ranges from 1 to 5, where 1 is a completely wrong answer and 5 is a perfect answer. ( 4) AlignScore (Zha et al., 2023b) is an alignment-based method for analyzing factual consistency. The final score assigned is the average of the maximum alignment probabilities between sentences from the summary to context chunks from the original abstract.

LLM Evaluators

Prior work has also shown that LLMs themselves can be good evaluators for factuality in summarization (Luo et al., 2023;Wang et al., 2023;Tang et al., 2023a;Tian et al., 2023). FACTPICO judgments are finer-grained. Therefore, we prompt LLMs with instructions emulating the questions asked of human evaluators in FACTPICO. In addition to ratings, the LLMs also generate rationales for their scores, which we analyze in Section 6.

The implementation details and the prompts used for this task are in Appendix H. Prior work found that LLM evaluations may be biased in that a sys-tem may 'prefer' its own outputs (Liu et al., 2023), but we did not observe this here (Table 5). Full-Text Evaluation. As input, we provide LLMs the full-text of a complex medical abstract and corresponding LLM-generated plain language summary. We instruct the evaluator LLM to find PICO elements in the text and evaluate them according to the provided criteria. In addition to PICO elements, we also evaluate Evidence Inference outputs; here the reference results span (annotated in prior work) is compared against the full text of the plain language summary. We evaluate four LLMs in this way: GPT-4 (OpenAI, 2023), Llama-2-Chat (7B) (Touvron et al., 2023), Alpaca (7B) (Taori et al., 2023), and Mistral (7B-Instruct-v0.1;Jiang et al. 2023).

PICO-R Extraction Pipeline. We also evaluate LLM scorers explicitly informed of the PICO elements and results inferred from evidence (PICO-R). We adopt this two-stage pipeline using GPT-4. We first extract PICO-R from both the original abstract and the plain language summary. 4 For Evidence Inference, this extraction is only necessary from the summary. Next, extracted elements are passed to GPT-4 along with an evaluation prompt.

Factuality Metric Evaluation Results

To conduct a meta evaluation of the system factuality metrics, we compute the Kendall's τ b coefficient, Spearman correlation coefficient, and Pairwise Accuracy coefficient (Deutsch et al., 2023) for automatic vs. human evaluations for each of the PICO and evidence inference aspect. We posit that good metrics must correlate well with the most salient elements in a high-stake domain.

Instance-level Results. We present the results for the above metrics in Table 4. 5 Across most measures, with the exception of Kendall's τ b , the pipeline combining GPT-4 evaluation with PICO extraction has the highest correlations with the FACTPICO ratings. Thus, decomposing the original evaluation task into separate localization then evaluation steps yield better performance, indicating that LLMs benefit from breaking apart a complex tasks into a series of simpler steps completed separately.

FACTPICO ratings correlated more with dedicated factuality models than LLM evaluators with the exception of GPT-4. In fact, the smaller, opensource LLMs had barely any correlation at all with the FACTPICO ratings. These results show that there is a sizeable gap in an LLMs ability to evaluate generated text compared to its generation capabilities.

System-level Results. LLMs in Table 5, with the exception of Alpaca, produce similar results compared to the human average PICO-R ratings in FACTPICO in Table 3. Alpaca rates plain language summaries low, with averages under the minimum in the scale (1), and so has failed to follow the evaluation instructions. Although Alpaca-generated summaries have higher ratings, these aren't meaningful as are extractive.

Visual Analysis. For more visual analysis of the system factuality metrics vs human ratings, Figure 2 shows the top three performing systems vs. averaged human scores. The highest performing system, the PICO-R extraction pipeline, has a the most balanced linear pattern. On the other hand,

GPT-4 Llama-2 Alpaca Mistral Extract P 0.189 0.073 0.153 0.21 0.156 0.201 I 0.173 0.073 0.210 0.11 0.169 0.205 C 0.181 0.092 0.126 0.11 0.173 0.215 O 0.148 0.075 0.111 0.19 0.158 0.176 R 0.061 0.019 0.045 0.60 0.045 0.060 Table 6: BERTScore (rescore baseline) F1 average for PICO and Evidence Inference (R). Note, the results for Alpaca omit rationales. We show the percentage of nonempty rationales next to the BERTScore for Alpaca. We caution comparing the results for Alpaca here against those of other LLMs because instances of empty rationales have been excluded in this evaluation. GPT-4 without PICO-R extraction often rates nonfactual summaries highly. Contrastively, QuestEval, the best performing non-LLM factuality metric, is more cautious and rates high quality summaries lower. We further analyze automatic vs. human rating distributions in Appendix G. These results hint at the potential challenges of factuality assessment brought by the shift in readability.

Preliminary LLM Rationale Analysis

We perform a preliminary analysis on LLM rationales, comparing to expert rationales in FACT-PICO; we leave a thorough human evaluation of LLM rationales for future work. Qualitative analysis of a small sample of rationales show that Llama-2 and Mistral are often able to comprehend the medical text but fail in making correct judgments according to the provided instructions. Most commonly, Mistral focuses on the abstract rather than the evaluated summary, while Llama-2 generates long explanations that eventually arrive at the wrong conclusion. Rationales from GPT-4 and its pipelined counterpart usually did not do this. For the most part, rationales made logical sense. However, some rationales were overly generous in it evaluation, ignoring critical errors. We provide examples of such erroneous rationales in Appendix M.

Table 6 shows the BERTScore (Zhang et al., 2020) between expert vs. LLM rationales across all evaluators experimented in this work. Overall, the rationales are dissimilar. The PICO-R extraction pipeline produced rationales most similar to that of humans, with the exception of rationales for evidence inferences, where GPT-4 rationales are slightly more similar than others. Despite explicit prompting, Alpaca frequently did not produce rationales and stopped generation after outputting a

1.0 1.5 2.0 2.5 3.0 3.5 4.0 Avg. PICO-R 0.2 0.3 0.4 0.5 0.6 0.7 0.8 QuestEval 0 100 0 50 GPT-4 LLAMA-2 ALPACA 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Avg. PICO-R 1.0 1.5 2.0 2.5 3.0 3.5 4.0 GPT-4 0 100 0 100 GPT-4 LLAMA-2 ALPACA 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Avg. PICO-R 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Extract 0 100 0 50 GPT-4 LLAMA-2 ALPACA numerical rating. Despite reporting these preliminary results, we advise caution when adopting reference-based metrics for automatic free-text rationale evaluation, as the results may be misleading. For example, we observed that Alpaca rationales tend to be a somewhat arbitrary variation of the rating descriptions (e.g., The interventions were described accurately), and human evaluators often use the words in these descriptions in their rationales (e.g., The intervention is described correctly as an 8 week program of specific exercises), inflating word overlap scores. By contrast, an example Llama-2 rationale (The comparator in the PICO model is the placebo group...) could present the same idea as a human rationale (ex: Comparator (placebo) is mentioned.), but result in a low BERTScore (ex: -0.182) because rationales from Llama-2 tend to (much) longer than those written by humans. Future work developing metrics for rationale correctness should not solely rely on reference-based metrics. We additionally present a length analysis in Appendix L.

Related Work

Meta-evaluation of factual consistency metrics in summarization (and related tasks) in the "general domain", e.g., Wikipedia and news, has garnered considerable attention (Pagnoni et al., 2021;Honovich et al., 2022;Laban et al., 2022;Tang et al., 2023a;Min et al., 2023). However, in addition to focusing on different domains, these existing benchmarks in summarization include primarily older models. Newer LLMs may yield novel error types (or be more factual overall) (Tang et al., 2023a).

Existing summarization factuality benchmarks also fail to generalize to simplification, in which content addition in the form of elaborations or explanations is often necessary (Srikanth and Li, 2021). Broadly, simplification entails substantial language changes that often lead to the text being more general (Li and Nenkova, 2015). Devaraj et al. ( 2022) evaluated the factuality of automated simplification model outputs at the sentence-level, noticing that content deletion can often lead to factual errors (in contrast to only summarization). Our findings confirm that these errors also exist in plain language summarization; the overgeneralization problem may lead to safety issues in the medical domain. Shaib et al. (2023) and Tang et al. (2023b) evaluated LLM-generated summaries of medical evidence. Notably, Shaib et al. (2023)'s work included an evaluation of plain language summaries. Our work deepens this analysis with a finer-grained evaluation focusing on critical components of RCTs and medical evidence, covering three LLMs. Our findings call for caution against LLM-generated plain language summaries despite the absence of outright inconsistencies. Pal et al. (2023) presents an analysis of "hallucinations" in medical QA tests, focusing on reasoning rather than factual consistency.

The inclusion of human-written natural language rationales in factuality benchmarks is rare, and there is a paucity of work evaluating these. The FELM benchmark (Chen et al., 2023), an opendomain evaluation of LLM-generated long form texts covering factual knowledge, math, and reasoning included human rationales. Work in LLM critiquing has started to incorporate natural language critiques from both humans (Saunders et al., 2022) and LLMs (Cui et al., 2023;Kim et al., 2024). FACTPICO is the first factuality benchmark of the plain language summarization task that includes expert-generated natural language rationales. We introduced FACTPICO, an expert-annotated benchmark in the domain of evidence-based medicine for evaluating the factuality of plain language summarization with respect to clinically important dimensions. Using FACTPICO, we presented an analysis of factual errors along these fine-grained aspects in LLM-generated plain language summaries. We also presented an analysis of methods to evaluate factuality, including both dedicated factuality models and novel LLM-based methods.

Limitations

The process of evaluating texts in FACTPICO was time consuming for human evaluators, requiring close reading of complex, technical language. Consequently, to make the workload manageable, we did not ask evaluators to localize PICO elements (or results regarding these) in texts in plain language summaries. Annotating how these elements are represented overall in the summary would provide more insights in cases where they are vaguely represented and thus required more focus in this evaluation. We automatically localize these elements in the evaluated PICO-R extraction pipeline, but we encourage future work expanding FACTPICO to include human-annotated, span-level annotations.

We primarily evaluated the zero-shot capabilities of LLMs in evaluating the factuality of simplified medical texts. We chose this setting as it best reflects how an end-user-a lay individualwould likely interact with an LLM, as such users are unlikely to provide LLMs with expert-evaluated plain language summaries for few-shot prompting. Future work could explore other LLM evaluation methods methods and use FACTPICO as an evaluation benchmark.

In our meta-evaluation we compared factuality metrics that assess the overall factuality of a text against numerical ratings assessing the fine-grained factuality of key characteristics in these texts. We acknowledge that this is not an equivalent comparison. However, we posit that the aggregation of these assessments should be well-correlated with the overall factuality of RCT texts for them to be useful in this important domain.

predictor of awareness, engagement, and use of a national web-based personal health record: Populationbased survey study. Journal of Medical Internet Research, 24(9):e35772. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. Ultrafeedback: Boosting language models with high-quality feedback. arXiv preprint arXiv:2310.01377. Daniel Deutsch, George Foster, and Markus Freitag. 2023. Ties matter: Meta-evaluating modern metrics with pairwise accuracy and tie calibration. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12914-12929, Singapore. Association for Computational Linguistics. Ashwin Devaraj, William Sheffield, Byron Wallace, and Junyi Jessy Li. 2022. Evaluating factuality in text simplification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7331-7345, Dublin, Ireland. Association for Computational Linguistics. Jay DeYoung, Eric Lehman, Benjamin Nye, Iain Marshall, and Byron C. Wallace. 2020. Evidence inference 2.0: More data, better models. In Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, pages 123-132, Online. Association for Computational Linguistics. Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022. QAFactEval: Improved QAbased factual consistency evaluation for summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2587-2601, Seattle, United States. Association for Computational Linguistics. Tanya Goyal and Greg Durrett. 2021. Annotating and modeling fine-grained factuality in summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1449-1462, Online. Association for Computational Linguistics. Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. News summarization and evaluation in the era of GPT-3. arXiv preprint arXiv:2209.12356. Eduardo Hariton and Joseph J Locascio. 2018. Randomised controlled trials-the gold standard for effectiveness research. BJOG: an international journal of obstetrics and gynaecology, 125(13):1716. David Heineman, Yao Dou, and Wei Xu. 2023. Thresh: A unified, customizable and deployable platform for fine-grained text evaluation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 336-345, Singapore. Association for Computational Linguistics. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3905-3920, Seattle, United States. Association for Computational Linguistics. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. 2024. Prometheus: Inducing finegrained evaluation capability in language models. In The Twelfth International Conference on Learning Representations. J Peter Kincaid, Robert P. Fishburne Jr., Richard L. Rogers, and Brad S. Chissom. 1975. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for navy enlisted personnel. Naval Technical Training Command Millington TN Research Branch. Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-visiting NLIbased models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163-177. Eric Lehman, Jay DeYoung, Regina Barzilay, and Byron C. Wallace. 2019. Inferring which medical treatments work from reports of clinical trials. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3705-3717, Minneapolis, Minnesota. Association for Computational Linguistics. Junyi Jessy Li and Ani Nenkova. 2015. Fast and accurate prediction of sentence specificity. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI'15, page 2281-2287. AAAI Press. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on

A Evaluation Guidelines

Annotators are shown a "source" input (abstract of a technical paper describing an RCT), along with a plain language summary of it, automatically produced by a model, and asked a series of 7 questions for evaluation.

Question I: Added Information For this part of the question, please highlight words, phrases, or sentences in the output that adds or modifies the information from the original document. Afterwards, answer whether this added/modified information is factual or not factual. Also, provide a rationale for why this is added/modified information and why it is factual or not factual.

Questions II-V (one analogous question for each PICO element): For instance, for population, we ask:

□ The population is mentioned, and described accurately □ The population is mentioned, but described somewhat inaccurately or vaguely □ The population is mentioned, but described with severe inaccuracies and/or is missing critical descriptors □ The population is missing in the model summary

Please provide a rationale for why you chose the answer choice in relation to the abstract and the summary. Some examples will actually not quite be randomized trials (e.g., they might be observational studies, or a description of a prospective trial not yet run). In these cases, it may be that there is no meaningful population (or intervention, comparator, and outcome). Here you should select the "N/A" option.

Exhaustive Outcomes (Analyzed in Appendix I): Sometimes, the plain language summary may not mention all the outcome measures described in the abstract. The summary may still be considered factual if the omitted measures are non-critical for the experiment and are not mentioned any further in the abstract. However, please do annotate separately when the plain language summary does exhaustively mention all outcome measures and when it does not.

Question VI: Evidence Inferences You will be presented with a span of text highlighting the inferred result from the experiment presented in the abstract. Based on this span, choose the following based on how this span is presented in the plain language summary:

Please provide a rationale for why you chose the answer choice in relation to the evidence inference span and the summary.

C Improving Agreement Through Discussion

Initially, we had observed low agreement on PICO questions among evaluators on the first 30 doubly annotated documents. Upon deeper analysis of these disagreements, we modified our instructions to be clearer and more detailed. We had asked evaluators to independently re-evaluate the previous annotations as well as doubly annotate 30 more documents. We observed that while agreement had improved significantly overall, evaluators still disagreed substantially on questions regarding population and outcome.

To fix this issue, we facilitated a "soft" discussion between evaluators regarding their annotations. Evaluators were presented with documents in which they had disagreed majorly on any PICO questions from the last 30 documents they evaluated. Then they were asked to come to consensus on how to rate these questions. Afterwards, we asked evaluators to independently reevaluate their previous annotations based on the insights they gained from this discussion. Similarly, they were asked to doubly annotate 15 new documents inde-pendently.

Table 8 shows the resulting agreement through Randolph's kappa after this discussion. For each question type, the first number is the kappa for the entire set of 75 doubly annotated documents. The second number is the kappa for the set of 15 documents that were annotated independently after the discussion. Both sets of number show moderate to high agreement for these questions. Furthermore, the agreement for this undiscussed set being significantly higher than the agreement for all 75 doubly annotated documents indicates that this discussion method was effective at improving agreement.

G System to Avg. PICO-R Visualizations

Figure 4 shows all system evaluations plotted against human evaluations. Figure 3 describes the distribution of averaged FACTPICO ratings alongside those of the automatic evaluation methods. These graphs display Gaussian approximations of the standardized distributions, allowing for easy visual comparison despite differences in scales. We only observe the average of the PICO-R ratings across human and LLM evaluations. Our primary focus in this analysis is observing how closely the distribution of automaticallyderived factuality scores are to the human evaluations in FACTPICO. Upon visual inspection of Figure 3, we see that GPT-4, Mistral, and Align-Score seem to have the most closely aligned distributions to the average human rating distribution in FACTPICO. In order to quantify this, we also calculated the KL divergence between the distributions of the evaluated metrics and the distribution of the averaged human ratings from FACTPICO as they are presented in Figure 3. These results can be found in Table 11. As previously hypothesized, the three distributions closest to the averaged human distribution are from AlignScore, Mistral, and GPT-4, in that order. Another trend that is observable in Figure 3 are the differences in the narrowness of the distributions between some LLM evaluators and dedicated factuality models. Ratings from GPT-4, Mistral, and, to a certain extent, Alpaca typically have narrow distributions, indicating an overall preference towards a single rating. On the other hand, distributions from dedicated factuality models are wider, signaling more variation.

Interestingly, these is a mismatch between wellcorrelated metrics and metrics whose distribution aligns closer to that of the human ratings (Figures 2 and 4). Having a closely aligned distribution does not imply good correlation. Similarly, good correlation, unless it is exceptionally high, does not imply closely aligned distributions. This is the most evident in the case of AlignScore. However, this mismatch does highlight interesting trends. A left or right skew in a score distribution references the "strictness" of the evaluator, with a more leaning skew indicating the evaluators are less strict and vice versa. Through this lens, the human evaluators could be viewed as not as "strict" compared to many of the metrics. A possible explanation of this effect could be that the simplified nature of the text or the accurate elaborations present in the text GPT-4 Llama-2 Alpaca Mistral Extract τ b Pop. 0.323 0.017 0.095 -0.052 0.377 Inter. 0.233 0.053 0.036 0.081 0.222 Comp. 0.506 0.001 0.064 -0.001 0.575 Out. 0.260 0.086 0.090 0.145 0.261 Evd. Inf. 0.561 0.137 -0.045 0.068 0.506 Avg. 0.475 0.055 0.081 0.047 0.474 ρ Pop. 0.363 0.019 0.110 -0.059 0.429 Inter. 0.246 0.058 0.040 0.089 0.246 Comp. 0.558 0.001 0.072 -0.001 0.649 Out. 0.279 0.094 0.100 0.159 0.292 Evd. Inf. 0.667 0.170 -0.051 0.078 0.604 Avg. 0.619 0.080 0.115 0.065 0.633 acceq Pop. 0.469 0.331 0.365 0.287 0.482 Inter. 0.538 0.379 0.365 0.420 0.419 Comp. 0.604 0.339 0.366 0.343 0.632 Out. 0.502 0.381 0.383 0.450 0.423 Evd. Inf. 0.665 0.419 0.342 0.287 0.631 Avg. 0.676 0.482 0.481 0.402 0.686

H.5 Evidence Inference Full-Text Prompt

In FACTPICO, evaluators analyze if individual evidence inference spans from the abstract are accurately represented in the plain language summary. The LLM evaluation is modeled after this as well, comparing evidence inference spans from the abstract to the full text of the summary. The following is the system prompt used for this evaluation. You are given a result inference span from an abstract, and you are given a summary. A result inference span corresponds to an inferred result in an experiment. Find the corresponding result inference in the summary and use it to rate the summary between 1 to 4.

The ratings are as follows:

1 -The result inference is mentioned and described accurately. 2 -The result inference is mentioned but is described vaguely or is slightly inaccurate. 3 -The result inference is critically inaccurate. 4 -The result inference is missing in the model summary.

Please provide only the rating and the rationale for the rating. Provide the rating after stating "Rating:".

The model is then queried as follows:

Result Inference Span: <Span from abstract> Summary: <Summary>

H.6 Evidence Inference Extraction Prompts

We used the following prompt to extract evidence inferences from the plain language summary. These extractions are subsequently compared against evidence inferences from the abstract for the evaluations. The extraction prompt is as follows:

An result inference span corresponds to an inferred result in an experiment.

Identify result inference spans in the following passage. Pull direct quotes from the passage:

Here is the prompt for the evaluation itself:

You are given a result inference span from both an abstract and a summary. A result span corresponds to an inferred result in an experiment. Use the result inference spans from the abstract and the summary to rate the summary between 1 to 4.

The ratings are as follows:

Please provide only the rating and the rationale for the rating. Provide the rating after stating "Rating:".

J Contradictions

#C %C ALPACA 11 8.70 GPT-4 16 10.4 LLAMA-2 36 25.2 Table 12: Total number of contradictions (#C) and percentage of FACTPICO that is a summary with at least one contradiction (%C).

We define Contradictions as pieces of information in the plain language summary that meaningfully disagree with the input abstract. In addition to the characteristics described in the main paper, we also ask evaluators to find spans in both the original abstract and the plain language summary that contradict each other and provide a rationale. The results are presented in Table 12, following similar trends to that found for other dimensions in Table 3.

We observed poor agreement overall for this question, largely due to "fuzzy" nature of this task. We define fuzziness as the difficulty to exhaustive distinguish such characteristics within a text. Table 13 shows a fuzzy, contradiction example which demonstrates how certain exaggerations may not be detected by the reader because it may be considered acceptable to use as part of the simplified language style. Annotators with different levels of discernment will likely suffer from low agreement for these types of annotations due to this phenomenon (Trienes et al., 2024). These collected annotations will also be included in the released data.

Complete Contradiction

Source: Compared to OP group, the post operative diclofenac requirement was significantly less in LP group (mean 107.14 mg) and OP group required mean of (682.35 mg). Simplification [Llama-2]: The study found that laparoscopic pyeloplasty takes longer to do and needs more medicine to help with pain after the surgery, but it also has fewer problems and the person stays in the hospital for less time. Rationale: The LPA group needed LESS pain medication, not more. Misinterpretation Source: The effects of add-on prolonged-release melatonin (PRM)...were investigated in 80 patients... In this randomized, double-blind, parallel-group study, patients were treated for 2 weeks with placebo and then randomized (1:1) to receive 2 mg of PRM or placebo nightly for 24 weeks, followed by 2 weeks placebo. Simplification [Llama-2]: The medicine was given to 80 people with AD, some of whom also had trouble sleeping. Rationale: According to my understanding the 80 people were split 1:1 so that only 40 people were treated with melatonin and 40 received a placebo. Exaggeration (fuzzy) Source: Although, laparoscopic pyeloplasty has the disadvantages of longer operative time and requires significant skill of intracorporeal knotting but it is here to stay and represents an emerging standard of care. Simplification [GPT-4]: So, they think the camera surgery is becoming the best choice even if it's a bit more difficult to do. Rationale: This is not necessarily linked with each other.

Table 13: Examples of contradictions found in plain language summaries within FACTPICO. This Llama-2 rationale assigns a rating of 4 to a summary that it describes as accurate and sufficiently detailed, which is the opposite of the rating system provided in the prompt.

"The population in the abstract and summary is described accurately and with sufficient detail. The authors mention the type of subjects involved in the trial, including their demographics (age, sex, and duration of HIV infection) and any specific shared conditions (abacavir use). The summary provides a brief overview of the study population, including their baseline characteristics and the results of the intervention.The summary accurately reflects the information provided in the abstract and provides a clear and concise summary of the study population. The authors use descriptive language to highlight the key findings related to the study population, such as the slight improvement in endothelial function after maraviroc intensification. The summary does not contain any inaccuracies or vagueness that could affect the reader's understanding of the study population. Therefore, I would rate the summary as a 4, indicating that the population is described accurately and with sufficient detail."

Additionally, the following rationale demonstrates the hallucinations that Llama-2 makes about the rating scale, as it assumes a ranking order of 1-5 with 5 being the most accurate. Not only is 5 out of the range of the rating options given in the prompt, but also this scale is inconsistent with Llama-2's other evaluation scale of using 4 as its highest rating.

"The population in the abstract and summary is described as "drug users" or "people using illicit opiates," which is a broad and vague term that does not provide critical demographic information. The summary does not provide enough detail about the specific characteristics of the population, such as age, gender, ethnicity, or comorbidities. Additionally, the summary does not indicate whether the population was selected randomly or if there were any exclusion criteria.

To rate the summary, I would give it a score of 4 because it does not provide sufficient information about the population involved in the study. A rating of 5 would indicate that the summary provides detailed and accurate information about the population, including critical demographic information and any specific shared conditions." This analysis explains why non-flipped Llama-2 ratings were more correlated with human evaluation. Llama-2 has a tendency to not follow rating instructions and rate more accurate summaries with higher numbers, defeating the purpose of flipping its ratings in post-processing. Furthermore, Table 5 shows how Llama-2 (non-flipped) is capable of rating plain language summaries similarly to human annotators in Table 3. If we flipped Llama-2 ratings, the average ratings for Llama-2 would have given opposite results. Therefore, we report the non-flipped Llama-2 evaluations throughout this paper to most accurately depict Llama-2's intended ratings.

M LLM Rationale Errors

Table 16 shows several examples of erronous rationales generated by various LLMs. These rationales illustrate several patterns of errors exhibited when these systems generate rationales. The examples from Mistral show the tendency to "forget" to evaluate the summary as described in Section 6. Mistral here either completely ignores the input summary or it passes off text from the abstract as belonging to the summary, as exemplified by the very first example. In addition to this issue, we also observe that Mistral generates illogical and contradictory language in its rationales (ex 3). The examples from Llama-2 exemplify the length of its rationales as well as its tendency to use irrelevant ideas in its justifications. For example, it may acknowledge factual information from the summary ("While the summary mentions the age range of the patients (adults) and the type of surgery (elective)"), but draws invalid conclusions ("it does not provide additional demographic information").

Rationale examples from GPT-4 here ignores critical factual errors in the summary. For example, the first GPT-4 example shows that the model acknowledges that the summary left critical characteristics missing ("Although the summary doesn't specifically mention 'adult' patients or the type of hospital"), but claims that "this doesn't seem to be crucial information", downplaying its effect during evaluation.

We also included some examples from Alpaca, showing its rationales tend to be very similar to the the rating descriptions provided through the prompt, and exceptions to this tend be nonsensical and illogical text.

N Full Text of Abstract in Figure 1 Due to space constraints, Figure 1 substituted the full abstract used during evaluation with a link. The full text of this abstract is included below: NicVAX®, a nicotine vaccine (3'AmNic-rEPA), has been clinically evaluated to determine if higher antibody concentrations are associated with higher smoking abstinence rates and if doses and frequency of administration are associated with increased antibody response. This randomized, double-blinded, placebo-controlled multicenter clinical trial (N=301 smokers) tested 200 and 400 µg doses administered 4 or 5 times over 6 months compared to placebo. 3'AmNic-rEPA recipients with the highest serum anti-nicotine antibody response (top 30% by AUC) were significantly more likely to attain 8 weeks continuous abstinence from weeks 19 through 26 than the placebo recipients (24.6% vs. 12.0%, p=0.024, OR=2.69, 95% CI, 1.14-6.37). The 5 injection 400 µg dose regimen had the greatest antibody response and had significantly higher abstinence rates than placebo. This study demonstrates proof-of-concept that 3'AmNic-rEPA elicits antibodies to nicotine and is associated with higher continuous abstinence rates, justifying its further development as a treatment for nicotine dependence. Information), and highlighting text. Figure 6 shows how evaluators can select spans in text to evaluate. Evaluators answer questions by clicking the pencil icon next to each edit. Figure 7 shows how questions are displayed for evaluators to answer.