CER LITERATURE REVIEW | Clinical Evaluation |

Appraising and evaluating the search results

In our previous blog, we focused on performing a successful database search and doing the first screening of search results through titles and abstracts.

In this final, third part of our three-part series, we would like to show you how to do the second level of screening, i.e., full-text analysis, and capture relevant data for the clinical evaluation report.

If you are interested to join our newsletter email list and receiving our template for the Clinical Evaluation Report (CER), please write us directly.

Step One: Find and gather the full text of selected documents

In our previous post, we talked about the first level of screening the results. On that stage we excluded publications that are obviously not relevant on the basis of the contents of the abstract or title. We left the ones that are, at first glance, important for our Literature Review.

In this step, your aim will be to find the full texts of all the references you’ve selected.

According to MEDDEV 2.7/1 Rev. 4 and MDCG 2020 13, the full-text papers and documents should be obtained for the appraisal stage.

Step Two: Evaluating the Relevance

Read and critically appraise the full text of each article you selected. It is crucial to determine if the identified data is relevant for the medical device under evaluation. According to MEDDEV 2.7/1 revision 4, when evaluating the relevance of the full-text articles, it is essential to consider whether the data are intended to:

directly demonstrate adequate clinical performance and clinical safety of the device (pivotal data), or
serve as an indirect supportive role.

The pivotal data must have the quality to demonstrate the clinical performance and the safety of the device under evaluation. Therefore, it must be generated with the device in case or with an equivalent device used for its intended purpose.

For more information on this topic, see the MEDDEV 2.7/1 Appendix A6, Appraisal of clinical data – examples of studies that lack scientific validity for demonstration of adequate clinical performance and/or clinical safety and MEDDEV 2.7/1 Appendix A1, Demonstration of equivalence.

Data that are not pivotal are generally appraised for their contribution for identifying and defining:

– the current knowledge/ state of the art in the corresponding medical field, to define acceptability criteria for the evaluation of the benefit/risk profile and of specific side-effects of the device under evaluation;

– hazards; individual case reports may be used for identification of new and previously unknown hazards that are associated with the device;

– the validity of criteria used for the demonstration of equivalence (if equivalence is claimed) etc.

Questions that help determine whether data are relevant are listed in Section 9.3.2.c of MEDDEV 2.7.1 rev. 4, we provide here in the summarized form:

Are the data relevant to the intended purpose or claims of the device?
Suppose the data are relevant to specific aspects of the intended purpose or claims. Are they applicable to particular device model, user group, medical indication, age, gender, the severity of the condition, or range of time?
To what extent are the generated data representative of the device under evaluation?
What aspects are covered?

If the article is relevant

If you decide that an article is relevant, the next step is to weigh its clinical contribution. Unfortunately, there is no single method for weighting clinical data. You should choose a rating system you believe it is the most adequate for the device under evaluation (for example, OCEBM, SORT, GRADE, or other rating scales.)

The scientific validity of the clinical data set: Level of Evidence of the publication

Level of Evidence		Type of publications
MA / SR		Evidence obtained from meta-analysis or systematic review
I	Ia	Evidence obtained from properly designed randomized and controlled clinical trial
I	Ib	Evidence obtained from pseudo-randomized controlled trials, randomized controlled trials with limitation
II	IIa	Evidence obtained from not randomized prospective studies comparative with current control
	IIb	Evidence obtained from not randomized prospective studies comparative with historical control
	IIc	Evidence obtained from not randomized prospective studies not comparative
III	IIIa	Retrospective trial: consecutive patients extracted from prospective database or registry
III	IIIb	Retrospective trial, not consecutive patients
IV		Evidence obtained from case report
V		Data presented from poster, abstract or conference

If the article is not relevant

You are expected to provide a rationale for each article excluded from the analysis. The reasons might vary:

Different devices, Different indications, Different population
Inappropriate number of patients, duration, no statistical analysis,
The non-comparative study, inappropriate clinical outcome
The preclinical study not on the device under evaluation
Socio-economic assessment, a cost-effectiveness study
Comments on an article, poster, abstract, video not on the device under evaluation
Non-peer-reviewed opinion/journal not on the device under evaluation
Foreign language not generally understood
Out of the scope
Single Case study report without any relevant safety information
The article is duplicated (same author, same study…)
Others

You can add the table with the codes for the exclusion rationale:

Code	Description
NE	Non-Equivalence	Different device, Different indication, Different population
M	Methodology	Inappropriate number of patients, duration, no statistical analysis, Non-comparative study, inappropriate clinical outcome
T	Type of publication	Preclinical study not on the device under evaluation
		Socio-economic assessment, cost-effectiveness study
		Comments on an article, poster, abstract, video not on the device under evaluation
		Non-peer-reviewed opinion/journal not on the device under evaluation
		Single Case study report without any relevant safety information
		Review, guidance or guideline published from more than 2 year
OS	Out of scope	Out of the scope
L	Language	Foreign language not generally understood
D	Duplicate	Duplicate of article (same author, same study…)

Important! The sum of included and excluded articles should sum up to the number of articles you’ve got by the literature search.

If the available data is limited

According to MEDDEV 2.7/1 revision 4, in exceptional situations, when an evaluation is based on limited data, you can describe and justify that in the clinical evaluation report. Please, see additional information and specific considerations in Appendix A8 (Devices for unmet medical needs – aspects to consider).

Step Three: Write the Literature Search Review Report

Finally, you can write the Literature Search Review Report which contains the results of your search that address safety, performance, benefits to the patient, side-effects, adverse reactions, state of the art of the device in question.

A CER literature review report should include Introduction (Purpose), Objectives, Method, Search results, Analysis, and Conclusion.

For Introduction, Objectives and Methods you should use the info pre-defined in your literature review protocol. Provide a summary of the searches that were performed, the number of results, and the number of articles excluded. The analysis of the reviewed literature should contain a comprehensive summary and analysis of the information identified during the literature review process, both favorable and unfavorable, and it should address each of the CER literature review objectives. In Conclusion, you should provide a summary to indicate how the objectives of the literature review have been met.

At the end, you should add Appendices to your Report: Bibliography, Detailed data analysis tables and spreadsheets, Literature search results list and detailed criteria for the article exclusions.

Good Luck!

BioReg Services specializes in systematic literature searches and reviews in compliance with the MDR Regulation (EU) 2017/745. Our team can design a comprehensive literature search protocol with complete documentation of the literature screening and data extraction process. We can obtain clinical evidence for the literature review for your new medical device, or we can update the literature review of a CE-marked or legacy device to create an MDR compliant document. Don’t hesitate to contact me directly: daniel.albahari@bioregservices.com.

BioRegards,

Jelena