systematic literature review master thesis

Structured literature reviews – A guide for students

This is a step-by-step guide aimed at Master's students undertaking a structured literature review as part of their Master's thesis.

There are several different kinds of literature reviews, but any literature review typically includes an extensive literature search. Whenever a systematic approach is used, the literature search features a methodical step-by-step procedure. However, as a Master's student, it might not be possible to fulfill all the criteria of a systematic review when writing a literature review-based thesis; you should rather do a structured literature review, which will include only certain aspects of the systematic review methodology.

In this guide we will go through the different steps of a structured literature review and provide tips on how to make your search strategy more structured and extensive. Additionally, make sure to follow any programme and course specific requirements.

Step 1: Formulate and delimit your research question

• It will be much easier for you to perform a structured information search if you first define and delimit your research question in a clear way.
• One way to define and structure your question is to break it down into different parts .
• PICO and PEO are two different frameworks that can be used for breaking down a research question into different parts.
• You also need to define the most important key concepts of your research question.

The formulation of your research question is partly connected to what kind of literature review you are doing. This article by Maria J. Grant and Andrew Booth usefully compares different kinds of reviews . While a systematic literature review is usually grounded in a clearly delimited and structured question, a scoping review may, for instance, feature a wider problem formulation. The wider a research question is, the larger number of search hits it tends to generate.

To be able to perform a literature review, you need to consider a subject area in which there seems to be a sufficiently large number of original research studies. Therefore, it may be a good idea to test search a database for previous research on the subject while you are trying to formulate and delimit your research question.

One way to structure your research question is to break it down into different parts. A well-delimited question often consists of three to four different parts. PICO and PEO are two examples of frameworks that can help you identify and define your research question.

• PICO ( P opulation, I ntervention, C omparison, O utcome) is primarily used for quantitative research questions.
• PEO ( P opulation, E xposure, O utcome) is primarily used for qualitative research questions.

Structuring your research question in accordance with a framework, such as PICO or PEO, will also help you decide on the inclusion and exclusion criteria of your literature review.

PICO & PEO

After you have delimited your research question, you also need to identify the key concepts that make up your question. Based on these key concepts, you will create " search blocks " that you will use to organise your search terms.

Step 2: Find search terms and create search blocks

• Test searching is a good way to investigate the terminology of a subject area and find search terms.
• Reading key articles can help you gather additional search terms for your final search strategy.
• Find subject headings  for PubMed with the help of the US National Library of Medicine's MeSH database.
• Find& free-text search terms by investigating what words that occur in the title and abstract of relevant articles.
• A good way of achieving a structured final search query is to arrange your search terms into search blocks ; these blocks should arise from the key concepts of your research question.

While working on a literature review-based thesis, you will need to search for articles on several occasions. In the beginning of your project, it is often good to do a couple of unstructured and simple search queries, so-called test searches, in academic databases. This way you are off to a good start, as test searching helps you investigate the terminology of your subject area and find relevant search terms. While the final search strategy is typically reported in full, you don't need to present your test queries in your thesis.

Try to find a couple of key articles, that is, articles that correspond to the type of studies that you are planning to include in your review. Use key articles to gather additional search terms for your final search strategy. Analyse the terminology of your key articles by examining what subject headings (MeSH terms, etc.) that the articles have been tagged with and what words that occur in the titles and abstracts.

Test search - find keywords and narrow down your topic

Test search in pubmed & cinahl.

To retrieve as many relevant studies as possible, you will need to include free-text search terms as well as subject headings in your final search strategy. Free-text search terms are words that occur in the article's title and abstract – words used by the authors themselves. Subject headings are subject-related words that an article is tagged with when the article is added to the database.

• In PubMed , articles are tagged with MeSH terms ( Me dical  S ubject  H eadings). You can look up and browse MeSH terms in the US National Library of Medicine's  MeSH database .
• Databases such as CINAHL , PsycInfo , ERIC , and Sociological Abstracts  have their own subject heading lists; look up subject headings in each database's subject heading list.
• There are also so-called free-text databases, such as Web of Science . These databases lack subject heading lists. Hence, when searching a free-text database, you can only use free-text search terms.

Find subject headings

An effective way to increase the structure of your final search strategy is to arrange your search terms in so-called  search blocks . Create your search blocks based on the key concepts of your research question.

Create search blocks

This search strategy worksheet might help you document and organise your search terms.

Download worksheet

• Worksheet for search terms (Word, 30.54 KB)

Step 3: Search in a structured way

• To get a comprehensive search result, you will need to search for articles in  several different databases .
• Your search strategy should be as uniform as possible in every database, but you may have to  adapt your use of subject headings .
• As you search the databases, combine your search terms and blocks with the help of  AND  and  OR .
• Save time by  documenting your search queries .

When doing a literature review-based thesis it is often wise to use at least two different databases. Many databases overlap, but may also contain unique content. At KI it is common for Master's students to use PubMed and Web of Science when doing a structured literature review as part of their Master's thesis. Depending on your research question, other databases may also be appropriate and useful. Read more about the most frequently used databases at KI .

Your search strategy should be as uniform as possible in every database. However, as mentioned in Step 2, databases may use different subject headings, and some databases only let you use free-text search terms. This means that you need to adapt your use of subject headings depending on the database.

Example: How subject headings may differ between databases 

If you want to search for articles about day surgery in PubMed, you should use the MeSH term Ambulatory Surgical Procedures . However, if you also want to perform your search in a database such as CINAHL, you need to use the corresponding CINAHL Headings term instead: Ambulatory Surgery .

There are many different ways of searching databases. Most databases have one simple, basic Google-like search box and one advanced search form. One advantage of the latter is that combining search terms with AND and OR is usually easier in an advanced search form, especially if you will be using both AND and OR within the same search query. However, you can often combine search terms with AND and OR in a basic search box too, and in that case, you often isolate your different search blocks from each other by enclosing each block in parentheses.

Example: A search query that contains AND, OR, and parentheses

( inflammatory bowel diseases OR ulcerative colitis OR crohn disease) AND (adolescent OR child OR young adult OR teenager) AND (self-management OR self care OR self efficacy )

By choosing the advanced search form you will also be able to exert more control over your search process. The advanced search form lets you specify more closely and decide exactly how you want the database to interpret your search terms; this way you can make your search query more precise.

You should always document your search strategy in order to remember what search terms you have used, how these search terms have been combined, and whether you have applied any limits to your search. The easiest way to do this is to copy and paste your search history from the database into a text document. Also, academic databases often let you create a personal account, so that you may save your searches online.

How to do a structured search in PubMed

Step 4: narrowing or broadening your search.

• Briefly examine your search results to see if you need to narrow or broaden your search query.
• Investigate whether your key articles are present in the search results.
• By using the advanced search form you can improve your search.

Prepare yourself for having to modify and redo your search query several times, before deciding on your final search strategy. After you have combined all your search terms and made your very first database search, you should examine the search results and analyse whether your search query is able to generate the type of search hits that you are looking for.

Analyse your search results

• Are all your key articles present in the search results, or are there some key articles that your search query is unable to retrieve?
• Are you getting too few search hits ? Investigate why. Perhaps you need to remove one of your search blocks, add one or several synonyms within a search block, or search for parts of words by truncating one or several of your free-text search terms, in order to broaden your search ?
• Does your search strategy generate too many non-relevant search hits that have nothing to do with your research question? Investigate why. Perhaps you need to add another search block, remove one of the synonyms from one of your search blocks, or search for phrases by enclosing one or several of your free-text terms in quotation marks, in order to narrow your search ?
• More tips on how to improve your search strategy .

It is important to remember that there is nothing wrong, per se, if your search query generates irrelevant hits. This is quite normal when performing a structured literature search. What's important is that your search strategy is able to retrieve the type of articles that you are looking for, and that you are not overwhelmed by the total number of hits (given the time frame of your thesis project).

We recommend that you use the advanced search form when improving your search strategy. By using the advanced search form, you will for example be able to specify which search fields your search terms must be present in.

Narrowing your search 

Broaden your search, how to specify the field you would like to search in pubmed, step 5: select and review articles.

• After you have completed your search, you will need to go through all your search hits and select which articles to include in your review.
• When selecting articles, read through the titles and abstracts of each article to decide its relevancy .
• Check the quality of each study that you include in your review.
• When checking the quality of articles, it is common to use critical appraisal worksheets or checklists .

Selecting articles

When you have completed and feel satisfied with your search, it is time to go through all the search hits and select which studies to include in your review. All relevant studies, that is, those studies that correspond to your research question and your previously set inclusion criteria, should be included. You decide on the relevancy of a study primarily by reading through the title and abstract. If you feel unsure, go through the whole article. You can describe your selection process with the help of a flow chart, such as the frequently used PRISMA flow diagram .

One of the challenges of systematic literature searches is that the search strategy should be exhaustive, but at the same time the number of search hits also needs to be kept within reasonable boundaries. A search query needs to be broad enough to retrieve all relevant studies, but on the other hand, this also means that a large portion of the search results will be irrelevant. Hence, even though your search strategy may have generated hundreds of hits, it is fine to only include ten to twenty articles in your review in the end.

Saving articles

If you create a personal account in a database it will be easier for you to save any references that you may find there. Another way of saving and organising article references is to use reference management software. There are several different reference management software, for example Endnote Online and Zotero.

Read more about reference management and see software guides.

Reviewing articles

When you have made your selection, you should critically examine the quality of all articles included in your review. The assessment is typically performed with the help of a critical appraisal guide or checklist. The purpose is to assess the reliability of the study results and whether there are any methodological flaws that may have impacted the results. Qualitative research articles are often reviewed with a focus on authenticity, credibility, and validity.

There are many different critical appraisal worksheets and checklists. Some examples are the SBU checklists for assessing the quality of randomized studies, observational studies, and qualitative research. In the course book How to do a systematic review in nursing there is a review guide that can be used for assessing different kinds of studies (both qualitative and quantitative); the original source is Caldwell, Henshaw & Taylor, 2011 .

Review worksheets and checklists contain criteria and questions that may help you identify flaws, errors, or bias. Sometimes different aspects of the study are scored separately. Later, all scores make up a final score that indicates whether the study is of high, medium, or low quality.

Many programmes and courses provide instructions on which checklists to use when reviewing articles, so check your course guidelines.

Step 6: Report your search strategy

• Describe your search strategy in a manner that makes it possible for your readers to replicate the search and get the same results.
• The search strategy is often presented in the form of a table .
• Look at the search history to see what words and limits that you have used when searching a database.

An important aspect of doing a structured literature review is transparency. It has to be easy for your readers to follow what you did when you searched and selected the articles that you have included in your review. In the method section of your literature review you should describe how you searched different databases. This is also where you describe any manual searches that you did. Search strategies are commonly reported in the form of tables. Present one table for each database.

You can examine your search terms and any limits you have applied when searching a database by visiting its search history.

Read more about how to report your search strategy and view examples.

Checklist for search strategies

Here is a checklist to help you review your own or someone else's search strategy.

If you would like us to get back to you, please submit your contact information in the form below along with your feeback.

• Request new password
• Create a new account

Doing a Systematic Review: A Student's Guide

Student resources, chapter 1: carrying out a systematic review as a master's thesis.

What Is The Difference Between A Systematic Review And A Meta-Analysis?

What Is The Difference Between A Narrative Review And A Systematic Review With Narrative Synthesis?

Writing your thesis and conducting a literature review

• Writing your thesis

Your literature review

• Defining a research question
• Choosing where to search
• Search strings
• Limiters and filters
• Developing inclusion/exclusion criteria
• Managing your search results
• Screening, evaluating and recording
• Snowballing and grey literature
• Further information and resources

Most PhD and masters’ theses contain some form of literature review to provide the background for the research.  The literature review is an essential step in the research process. A successful literature review will offer a coherent presentation and analysis of the existing research in your field, demonstrating:

• Your understanding of the subject area
• Gaps in current knowledge (that may in turn influence the direction of your research) 
• Relevant methodologies

There are different approaches and methods to literature reviews, and you may have heard of terms like systematic, structured, scoping or meta-analysis. This is when the literature review becomes the research methodology in its own right, instead of forming part of the research process.

This table shows the differences between a traditional literature review and a structured or systematic literature review.

Structured vs traditional literature reviews

What is a traditional literature review?

A traditional literature review is a critical review of the literature on a particular topic. The aim of this type of literature review is to identify any background research on your topic and to evaluate the quality and relevance of the literature. You will use your literature review to understand what has already been researched, help develop your research questions and the methodology that you should follow to collect and to identify any areas that your research can explore. You want your research to be unique so you will use a literature review to prevent you duplicating any previous research but also identifying any errors or mistakes that you would want to avoid.

A literature review is aimed at Masters (MSc students) and research level.

What is a structured literature review?

A structured literature review involves bringing many research studies together to use them as the data to determine findings (known as secondary research). There is no other form of data collection involved such as creating your own surveys and questionnaires (primary research). This approach allows you to look beyond one dataset and synthesise the findings of many studies to answer your clearly formulated research question.

Sometimes a structured review can be described as being a systematic literature review. A structured review typically does not fulfil all of the criteria for a full systematic review but may take a similar approach by taking a systematic, step by step method to find literature. They tend to follow a set protocol for determining the research studies to be included and every stage is documented.

To help you prepare for your structured literature review please complete this interactive workbook.

For Logistics students only

To help you prepare for your systematic literature review please complete this interactive workbook.

What is a systematic literature review?

A systematic literature review is a specific research methodology to identify, select, evaluate, and synthesise relevant published and unpublished literature to answer a particular research question. The systematic literature review should be transparent and replicable, you should follow a predetermined set of inclusion and exclusion criteria to select studies and help minimise bias. A systematic literature review may be registered, so that others can discover and minimise duplication, and can take several years to complete.

The systematic literature review is aimed at research (PhD students) level.

Useful background reading

Cranfield Libraries have several books offering guidance on how to approach and conduct literature reviews, and structured or systematic literature reviews:

• Reading list for  literature review and study skills
• Reading list of items to support a structured or systematic literature review

Looking at previous structured and systematic literature reviews is an effective way to understand what is required and how they should be structured and written up. Structured literature reviews can be found in the Masters Thesis Archive (MTA) and systematic literature reviews can be found in the Cranfield University institutional Repository, CERES. Check out the Theses  link.

• << Previous: Writing your thesis
• Next: Defining a research question >>
• Last Updated: Oct 24, 2023 3:48 PM
• URL: https://library.cranfield.ac.uk/writing-your-thesis

• Research Guides

MSc In Management Research Guide

• Writing Literature Reviews
• Core Library Research Skills
• Literature Searching
• Cited Reference Searching

Conducting & Writing Literature Reviews

Literature reviews: online guides, writing a literature review section - video tutorial, writing a literature review paper - video tutorial, literature reviews: an overview (video tutorial, 9m 38s).

• Reading & Note Taking
• Find Preprints & Working Papers
• Academic Integrity / Citing Sources
• Explore Research Methods
• Writing Your Thesis / LaTeX Guides
• Find Publishing Opportunities
• Journal Rankings & Impact Factors
• Statistical Software
• Research Data Management
• An important step in the research process for your Master's thesis is the literature review. This is the section of your thesis where you identify and critically discuss previously published literature (e.g. journal articles or other theses) relevant to your research topic.
• There are many guides available on how to conduct and write up literature reviews. Here are a few to get you started:

Systematic Approaches

• Systematic reviews, scoping reviews, and other evidence synthesis Overview of evidence synthesis reviews and relevant strategies, tools and resources.

Start Here:

• Reviewing the Literature (Project Planner via Sage Research Methods database)
• Chapter 7 Literature Review (Designing and managing a research project: a business student's guide) Discusses the literature review as part of the research project.

Other Online Guides:

The following videos from San Jose State University's King Library provide an in depth introduction to writing literature reviews. They distinguish between writing a literature review section (of a paper, or thesis), and writing a literature review paper (a standalone paper which synthesizes the literature on your topic).

This video tutorial, from North Carolina State University Libraries, is targeted specifically at graduate students, and explains how literature reviews fit in to the overall research process.

• << Previous: Cited Reference Searching
• Next: Reading & Note Taking >>
• Last Updated: Nov 2, 2023 9:10 AM
• URL: https://researchguides.library.brocku.ca/MSCM

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Open access
• Published: 12 December 2017

Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe

• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 1 , 2 , 3 &
• Damir Sapunar 3  

Systematic Reviews volume  6 , Article number:  253 ( 2017 ) Cite this article

14k Accesses

21 Citations

78 Altmetric

Metrics details

Systematic reviews (SRs) have been proposed as a type of research methodology that should be acceptable for a graduate research thesis. The aim of this study was to analyse whether PhD theses in European biomedical graduate programs can be partly or entirely based on SRs.

In 2016, we surveyed individuals in charge of European PhD programs from 105 institutions. The survey asked about acceptance of SRs as the partial or entire basis for a PhD thesis, their attitude towards such a model for PhD theses, and their knowledge about SR methodology.

We received responses from 86 individuals running PhD programs in 68 institutions (institutional response rate of 65%). In 47% of the programs, SRs were an acceptable study design for a PhD thesis. However, only 20% of participants expressed a personal opinion that SRs meet the criteria for a PhD thesis. The most common reasons for not accepting SRs as the basis for PhD theses were that SRs are ‘not a result of a PhD candidate’s independent work, but more of a team effort’ and that SRs ‘do not produce enough new knowledge for a dissertation’. The majority of participants were not familiar with basic concepts related to SRs; questions about meta-analyses and the type of plots frequently used in SRs were correctly answered by only one third of the participants.

Conclusions

Raising awareness about the importance of SRs and their methodology could contribute to higher acceptance of SRs as a type of research that forms the basis of a PhD thesis.

Peer Review reports

Systematic reviews (SRs) are a type of secondary research, which refers to the analysis of data that have already been collected through primary research [ 1 ]. Even though SRs are a secondary type of research, a SR needs to start with a clearly defined research question and must follow rigorous research methodology, including definition of the study design a priori, data collection, appraisal of study quality, numerical analyses in the form of meta-analyses and other analyses when relevant and formulation of results and conclusions. Aveyard and Sharp defined SRs as ‘original empirical research’ because they ‘review, evaluate and synthesise all the available primary data, which can be either quantitative or qualitative’ [ 2 ]. Therefore, a SR represents a new research contribution to society and is considered the highest level in the hierarchy of evidence in medicine [ 3 ].

SRs have been proposed as a type of research methodology that should be acceptable as the basis for a graduate research thesis [ 4 , 5 ]. To the best of our knowledge, there are no reports on the acceptance of SRs as the basis for PhD theses. A recent review addressed potential advantages and disadvantages of such a thesis type and presented opposing arguments about the issue [ 5 ]. However, there were no actual data that would indicate how prevalent one opinion is over another with regard to the acceptance of a SR as the primary research methodology for a PhD thesis. The aim of this cross-sectional study was to assess whether a PhD thesis in European biomedical graduate programs can be partly or entirely based on a SR, as well as to explore the attitudes and knowledge of individuals in charge of PhD programs with regard to a thesis of this type.

Participants

The Organization of PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS) includes 105 institutional members from 40 countries and six associate members from Canada, Georgia, Iran, Kyrgyzstan, Kazakhstan and the USA [ 6 ]. The ORPHEUS encompasses a network of higher education institutions committed to developing and disseminating best practice within PhD training programs in biomedicine, health sciences and public health. ORPHEUS approved the use of their mailing list for the purpose of this study. The mailing list had 1049 contacts. The study authors were not given the mailing list due to data protection and privacy. Instead, it was agreed that ORPHEUS officials would send the survey via email to the mailing list. The General Secretary of the ORPHEUS contacted individuals responsible for PhD programs (directors or deputy directors) among the institutional members, via e-mail, on 5th of July 2016. These individuals were sent an invitation to complete an online survey about SRs as the basis for PhD theses. We invited only individuals responsible for PhD programs (e.g., directors, deputy directors, head of graduate school, vice deans for graduate school or similar). We also asked them to communicate with other individuals in charge of their program to make sure that only one person per PhD program filled out the survey. If there were several PhD programs within one institution, we asked for participation of one senior person per program.

The survey was administered via Survey Monkey (Portland, OR, USA). The survey took 5–10 min to complete. One reminder was sent to the targeted participants 1 month after the first mail.

The ethics committee of the University of Split School of Medicine approved this study, which formed part of the Croatian Science Foundation grant no. IP-2014-09-7672 ‘Professionalism in Health Care’.

Questionnaire

The 20-item questionnaire, designed specifically for this study by both authors (LP and DS), was first tested for face validity and clarity among five individuals in charge of PhD programs. The questionnaire was then modified according to their feedback. The questionnaire included questions about their PhD program; whether PhD candidates are required to publish manuscript(s) before thesis defence; the minimum number of required manuscripts for defending a PhD thesis; the authorship requirements for a PhD candidate with regard to published manuscript(s); whether there is a requirement for a PhD candidate to publish manuscript(s) in journals indexed in certain databases or journals of certain quality, and how the quality is defined; the description about other requirements for defending a PhD thesis; whether a SR partly or fully meets requirements for approval of a PhD thesis in their graduate program; what are the rules related to the use of a SR as the basis for a PhD thesis; and the number of PhD theses based on SRs relative to other types of research methods.

Participants were also asked about their opinion with regard to the main reasons that SRs are not recognised in some institutions as the basis for a doctoral dissertation, and their opinion about literature reviews, using a four-item Likert scale, ranging from ‘agree’ to ‘disagree’, including an option for ‘don’t know’. In the last question, the participants’ knowledge about SR methodology was examined using nine statements; participants had to rate each statement as either ‘correct’, ‘incorrect’, ‘unsure’ or ‘I don’t know’. Finally, participants were invited to leave their email address if they wanted to receive survey results. The survey sent to the study participants can be found in an additional file (Additional file  1 ).

Data analysis

Survey responses were entered into a spreadsheet, checked by both authors and analysed using Microsoft Excel (Microsoft Inc., Redmond, WA, USA). Descriptive data are presented as frequencies and percentages. All raw data and analysed data sets used in the manuscript are available from authors on request. A point-biserial correlation (SPSS, IBM, Chicago, IL, USA) was used to measure the strength of the association between results on the knowledge test (continuous variable) and the attitude towards SRs as the basis for dissertations (dichotomous variable; we used the answer to the following question as this measure: ‘Do you agree that a systematic review, in whole or in part, meets the criteria for a publication on which a doctoral dissertation can be based?’).

Study participants

There are 105 institutions included in the ORPHEUS network. We received a response from 86 individuals representing 68 institutions from 37 countries (65% institutional response rate). There were more respondents than institutions because some institutions have several PhD programs and thus several program directors. Those responders were used as a unit of analysis in the analysis of attitudes and knowledge; institutions were the unit of analysis when analysing criteria for theses. Some of the questionnaires ( n  = 15) were only partly completed. In most cases, the missing data were related to knowledge about SR methodology.

Overview of requirements for a dissertation

Based on the information provided by the graduate program directors, in the majority of the included PhD programs, students were required to publish a research manuscript prepared within their PhD thesis prior to their thesis defence (83%; n  = 64). Among 13 programs (17%) that did not have this requirement, five respondents (38%) indicated that in their opinion their school’s rules related to a PhD thesis should be changed such as to specify that each thesis should be based on work that is already published in a journal.

The minimum number of published manuscripts necessary for the PhD thesis defence was prespecified in 94% ( n  = 60) of the programs that required publication of research manuscripts prior to the thesis defence. In most of the programs (37%; n  = 22), the number of required manuscripts was three or more. Two manuscripts were required in 30% ( n  = 18) and one was required in 33% ( n  = 20) of the programs. In four programs, there was no formal policy on this matter, but there was a strong expectation that the student will have contributed substantially to several manuscripts in peer-reviewed journals.

In most cases, the PhD candidates’ contribution to published manuscripts within the PhD thesis was determined through first authorship. A requirement that a PhD candidate should be the first author on a manuscript(s) that constitutes a PhD thesis was reported in 82% ( n  = 64) of the graduate programs.

In 60% ( n  = 52) of the graduate programs, the quality of the journals where a PhD candidate has to publish research manuscripts as a part of a PhD thesis was defined by the database in which these journals are indexed. The most commonly specified databases were Web of Science (41%; n  = 35) and MEDLINE/PubMed (13%; n  = 11), followed by Science Citation Index, Scopus, Current Contents, a combination of several databases or, in two cases, a combination of journals from a list defined by some governing body.

Systematic reviews as a PhD thesis

SRs, in whole or in part, met the criteria for acceptable research methodology for a PhD thesis in 47% ( n  = 40) of programs, whereas 53% ( n  = 46) of programs specifically stated that they did not accept SRs in this context (Fig.  1 a, b). Among the programs that accepted SRs, theses could be exclusively based on a SR in 42% ( n  = 17) of programs, while in the remaining programs, SRs were acceptable as one publication among others in a dissertation.

a European PhD programs that recognise a systematic review as a PhD thesis (green dot) and those that do not (red dot). Half red and half green dots indicate the five universities with institutions that have opposite rules regarding recognition of a systematic review as a PhD thesis. The pie chart presents b the percentage of the programs in which systematic reviews, in whole or in part, meet the criteria for a dissertation and c the opinion of participants about whether systematic reviews should form the basis of a publication within a PhD dissertation

The majority of participants (80%; n  = 69) indicated that SRs did not meet criteria for a publication on which a PhD dissertation should be based (Fig.  1 c). The main arguments for not recognising a SR as the basis for a PhD thesis are listed in Table  1 . The majority of respondents were neutral regarding the idea that scoping reviews or SRs should replace traditional narrative reviews preceding the results of clinical and basic studies in doctoral theses. Most of the respondents agreed that narrative or critical/discursive literature reviews preceding clinical studies planned as part of a dissertation should be replaced with systematic reviews (Table  2 ).

Most of the programs that accepted SRs as a research methodology acceptable for PhD theses had defined rules related to the use of an SR as part of a PhD thesis (Fig.  2 ). The most common rule was that a SR can be one publication among others within a PhD thesis. Some of the respondents indicated that empty (reviews that did not find a single study that should be included after literature search) or updated reviews could also be used for a PhD thesis (Fig.  2 ).

Frequency of different rules that define the use of systematic reviews as a part of a PhD thesis in European biomedical graduate programs

The results of the survey regarding knowledge about SR methodology indicated that the majority of respondents were not familiar with this methodology. Only three out of nine questions were correctly answered by more than 80% of the participants, and questions about meta-analyses and the type of plots frequently used in a SR were correctly answered by only one third of the participants (Table  3 ). The association between participants’ results on the knowledge test and attitudes towards SRs was tested using a point-biserial correlation; this revealed that lack of knowledge was not correlated with negative attitudes towards SRs ( r pb  = 0.011; P  = 0.94).

In this study conducted among individuals in charge of biomedical graduate programs in Europe, we found that 47% of programs accepted SRs as research methodology that can partly or fully fulfil the criteria for a PhD thesis. However, most of the participants had negative attitudes about such a model for a PhD thesis, and most had insufficient knowledge about the basic aspects of SR methodology. These negative attitudes and lack of knowledge likely contribute to low acceptance of SRs as an acceptable study design to include in a PhD thesis.

A limitation of this study was that we relied on participants’ responses and not on assessments of formal rules of PhD programs. Due to a lack of familiarity with SRs, it is possible that the respondents gave incorrect answers. We believe that this might be the case since we received answers from different programs in the same university, where one person claimed that SRs were accepted in their program, and the other person claimed that they were not accepted in the other program. We had five such cases, so it is possible that institutions within the same university have different rules related to accepted research methodology in graduate PhD programs. This study may not be generalisable to different PhD programs worldwide that were not surveyed. The study is also not generalisable to Europe, as there are no universal criteria or expectations for PhD theses in Europe. Even in the same country, there may be different models and expectations for a PhD in different higher education institutions.

A recent study indicated a number of opposing views and disadvantages related to SRs as research methodology for graduate theses, including lack of knowledge and understanding by potential supervisors, which may prevent them from being mentors and assisting students to complete such a study [ 5 ]. This same manuscript emphasised that there may be constraints if the study is conducted in a resource-limited environment without access to electronic databases, that there may be a very high or very low number of relevant studies that can impact the review process, that methods may not be well developed for certain types of research syntheses and that it may be difficult to publish SRs [ 5 ].

Some individuals believe that a SR is not original research. Indeed, it has been suggested that SRs as ‘secondary research’ are different than ‘primary or original research’, implying that they are inferior and lacking in novelty and methodological rigour as compared to studies that are considered primary research. In 1995, Feinstein suggested that such studies are ‘statistical alchemy for the 21st century’ and that a meta-analysis removes or destructs ‘scientific requirements that have been so carefully developed and established during the 19th and 20th centuries’ [ 7 ]. There is little research about this methodological issue. Meerpohl et al. surveyed journal editors and asked whether they consider SRs to be original studies. The majority of the editors indicated that they do think that SRs are original scientific contributions (71%) and almost all journals (93%) published SRs. That study also highlighted that the definition of original research may be a grey area [ 8 ]. They argued that, in an ideal situation, ‘the research community would accept systematic reviews as a research category of its own, which is defined by methodological criteria, as is the case for other types of research’ [ 8 ]. Biondi-Zoccai et al. pointed out that the main criteria to judge a SR should be its novelty and usefulness, and not whether it is original/primary or secondary research [ 9 ].

In our study, 80% of the participants reported negative attitudes, and more than half of the respondents agreed with a statement that SRs are ‘not a result of the candidate’s independent work since systematic reviews tend to be conducted by a team’. This opinion is surprising since other types of research are also conducted within a team, and single authorship is very rare in publications that are published within a PhD thesis. On the contrary, the mean number of authors of research manuscripts is continuously increasing [ 10 ]. At the very least, the authors of manuscripts within a PhD will include the PhD candidate and a mentor, which is a team in and of itself. Therefore, it is unclear why somebody would consider it a problem that a SR is conducted within a team.

The second most commonly chosen argument against such a thesis was that SRs ‘do not produce enough new knowledge for a dissertation’. The volume of a SR largely depends on the number of included studies and the available data for numerical analyses. Therefore, it is unfair to label a SR as a priori lacking in new knowledge. There are SRs with tens or hundreds of included studies, and some of them not only include meta-analyses, but also network meta-analyses, which are highly sophisticated statistical methods. However, limiting SRs within a thesis only to those with meta-analysis would be unfair because sometimes meta-analysis is not justified due to clinical or statistical heterogeneity [ 11 ] and the presence or absence of a meta-analysis is not an indicator of the quality of a SR. Instead, there are relevant checklists for appraising methodological and reporting quality of a SR [ 12 , 13 ].

The third most commonly chosen argument against SRs within PhD theses was ‘lack of adequate training of candidates in methodology of systematic reviews’. This could refer to either insufficient formal training or insufficient mentoring. The graduate program and the mentor need to ensure that a PhD candidate receives sufficient knowledge to complete the proposed thesis topic. Successful mentoring in academic medicine requires not only commitment and interpersonal skills from both the mentor and mentee, but also a facilitating institutional environment [ 14 ]. This finding could be a result of a lack of capacity and knowledge for conducting SRs in the particular institutions where the survey was conducted, and not general opinion related to learning a research method when conducting a PhD study. Formal training in skills related to SRs and research synthesis methods [ 15 , 16 ], as well as establishing research collaborations with researchers experienced in this methodology, could alleviate this concern.

One third of the participants indicated a ‘lack of appreciation of systematic review methodology among faculty members’ as a reason against such a thesis model. This argument, as well as the prevalent negative attitude towards SRs as PhD theses, perhaps can be traced to a lack of knowledge about SR methodology; however, although the level of knowledge was quite low in our study, there was no statistically significant correlation between knowledge and negative attitudes. Of the nine questions about SR research methodology, only three questions were correctly answered by more than half of the participants. This could be a cause for concern because it has been argued that any health research should begin with a SR of the literature [ 17 ]. It has also been argued that the absence of SRs in the context of research training might severely hamper research trainees and may negatively impact the research conducted [ 18 ]. Thus, it has been recommended that SRs should be included ‘whenever appropriate, as a mandatory part of any PhD program or candidature’ [ 18 ].

It has recently been suggested that the overwhelming majority of investment in research represents an ‘avoidable waste’ [ 19 ]. Research that is not necessary harms both the public and patients, because funds are not invested where they are really necessary, and necessary research may not be conducted [ 17 ]. This is valid not only for clinical trials, but also for other types of animal and human experiments [ 20 ]. SRs can help improve the design of new experiments by relying on current evidence in the field and by helping to clarify which questions still need to be addressed. SRs can be instrumental in improving methodological quality of new experiments, providing evidence-based recommendations for research models, reducing avoidable waste, and enabling evidence-based translational research [ 20 ].

Four respondents from three institutions indicated that empty SRs are accepted as a PhD thesis. While it makes sense to include such a SR as a part of the thesis to indicate lack of evidence in a certain field, it is highly unlikely that an entire thesis can be based on an empty SR, without a single included study.

There are many advantages of a SR as a graduate thesis [ 4 , 5 ], especially as a research methodology suitable for low-resource settings. A PhD candidate can prepare a Cochrane SR as a part of the PhD thesis, yielding a high-impact publication [ 4 ]. Non-Cochrane SRs can also be published in high-impact journals. A PhD candidate involved in producing a SR within a PhD thesis goes through the same research process as those conducting primary research, from setting up a hypothesis and a research question, to development of a protocol, data collection, data analysis and appraisal, and formulation of conclusions. Graduate programs can set limits, such as the prevention of empty reviews and the recognition of updated reviews as valid for a PhD thesis, and engage experienced researchers as advisors and within thesis evaluation committees, to ensure that a candidate will conduct a high-quality SR [ 4 ]. Conducting a SR should not be mandatory, but candidates and mentors willing to produce such research within a graduate program should be allowed to do so.

Further studies in this field could provide better insight into attitudes related to SRs as graduate theses and explore interventions that can be used to change negative attitudes and improve knowledge of SRs among decision-makers in graduate education.

Raising awareness about the importance of SRs in biomedicine, the basic aspects of SR methodology and the status of SRs as original secondary research could contribute to greater acceptance of SRs as potential PhD theses. Our results can be used to create strategies that will enhance acceptance of SRs among graduate education program directors.

Gopalakrishnan S, Ganeshkumar P. Systematic reviews and meta-analysis: understanding the best evidence in primary healthcare. J. Fam. Med Prim Care. 2013;2(1):9–14.

Article   CAS   Google Scholar  

Aveyard H, Sharp P. A beginner’s guide to evidence-based practice in health and social care. Glasgow: McGraw Open Press University; 2011.

Google Scholar  

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Puljak L, Sambunjak D. Cochrane systematic review as a PhD thesis: an alternative with numerous advantages. Biochemia Medica. 2010;20(3):319–2.

ten Ham-Baloyi W, Jordan P. Systematic review as a research method in post-graduate nursing education. Health SA Gesondheid. 2016;21:120–8.

Article   Google Scholar  

Organisation for PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS). Available at: http://www.orpheus-med.org/ .

Feinstein AR. Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol. 1995;48(1):71–9.

Meerpohl JJ, Herrle F, Reinders S, Antes G, von Elm E. Scientific value of systematic reviews: survey of editors of core clinical journals. PLoS One. 2012;7(5):e35732.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Biondi-Zoccai G, Lotrionte M, Landoni G, Modena MG. The rough guide to systematic reviews and meta-analyses. HSR proc intensive care cardiovascular anesth. 2011;3(3):161–73.

CAS   Google Scholar  

Baethge C. Publish together or perish: the increasing number of authors per article in academic journals is the consequence of a changing scientific culture. Some researchers define authorship quite loosely. Dtsch. Arztebl. Int. 2008;105(20):380–3.

PubMed   PubMed Central   Google Scholar  

Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statist Med. 2002;21:1539–58.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

Article   PubMed   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.

Sambunjak D, Straus SE, Marusic A. A systematic review of qualitative research on the meaning and characteristics of mentoring in academic medicine. J Gen Intern Med. 2010;25(1):72–8.

Article   PubMed   Google Scholar  

Balajic K, Barac-Latas V, Drenjancevic I, Ostojic M, Fabijanic D, Puljak L. Influence of a vertical subject on research in biomedicine and activities of the Cochrane collaboration branch on medical students’ knowledge and attitudes toward evidence-based medicine. Croat Med J. 2012;53(4):367–73.

Marusic A, Sambunjak D, Jeroncic A, Malicki M, Marusic M. No health research without education for research—experience from an integrated course in undergraduate medical curriculum. Med Teach. 2013;35(7):609.

Mahtani KR. All health researchers should begin their training by preparing at least one systematic review. J R Soc Med. 2016;109(7):264–8.

Olsson C, Ringner A, Borglin G. Including systematic reviews in PhD programmes and candidatures in nursing - ‘Hobson’s choice’? Nurse Educ Pract. 2014;14(2):102–5.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

de Vries RB, Wever KE, Avey MT, Stephens ML, Sena ES, Leenaars M. The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies. ILAR J. 2014;55(3):427–37.

Download references

Acknowledgements

The authors thank the ORPHEUS secretariat for administering the survey and the study participants for taking time to participate in the survey. We are grateful to Prof. Ana Marušić for the critical reading of the manuscript.

This research was funded by the Croatian Science Foundation, grant no. IP-2014-09-7672 ‘Professionalism in Health Care’. The funder had no role in the design of this study or its execution and data interpretation.

Availability of data and materials

The datasets used and/or analysed for the current study are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Cochrane Croatia, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia

Livia Puljak

Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Planinska 13, 10000, Zagreb, Croatia

Laboratory for Pain Research, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia

Livia Puljak & Damir Sapunar

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors participated in the study design, data collection and analysis and writing of the manuscript, and both read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

The Ethics Committee of the University of Split School of Medicine approved the study. All respondents consented to participate in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Online survey used in the study. Full online survey that was sent to the study participants. (PDF 293 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Cite this article.

Puljak, L., Sapunar, D. Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe. Syst Rev 6 , 253 (2017). https://doi.org/10.1186/s13643-017-0653-x

Download citation

Received : 29 August 2017

Accepted : 30 November 2017

Published : 12 December 2017

DOI : https://doi.org/10.1186/s13643-017-0653-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• PhD program
• Biomedicine
• Study design

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Why and how to conduct a systematic literature review

Systematic literature reviews may seem daunting at first, but they offer substantial benefits, especially in enhancing the theoretical framework for your research! Learn more about systematic literature reviews and their benefits. And have a look at a simple step-by-step guide that breaks down the daunting task of conducting a systematic literature review into simple, actionable steps.

What is a systematic literature review?

The difference between a systematic and a regular literature review, the difference between a systematic literature review and a meta-analysis, benefits of conducting a systematic literature review, google scholar to conduct a systematic literature review, web of science to conduct a systematic literature review, scopus to conduct a systematic literature review, step-by-step guide for conducting a systematic literature review.

Systematic literature reviews are a means to rigorously review existing literature on a specific topic. They collect and analyze existing literature in a systematic and replicable way.

Following the definition of a topic of interest and concrete research question, a systematic literature review starts by defining several keywords. Then, academic citation databases are used to retrieve all articles that include these keywords.

Furthermore, inclusion and exclusion criteria for sorting through the existing literature are set up. Think of a time frame, the type of publication (articles, books etc.), a geographic focus, and a disciplinary background. You name it.

Systematic literature reviews are a means to rigorously review existing literature on a specific topic. They collect and analyse existing literature in a systematic and replicable way.

While systematic literature reviews require a lot of work, they can convincingly draw conclusions on the state of the art of existing knowledge, uncover research gaps, and support the creation of new theoretical and conceptual frameworks.

In a systematic literature review, inclusion and exclusion criteria for sorting through the existing literature are set up. Think of a time frame, the type of publication (articles, books etc.), a geographic focus, and a disciplinary background.

Therefore, systematic literature reviews are crystal clear about the process of collecting and analyzing literature, which makes them replicable.

Before starting to consider whether a systematic literature review is right for you, it is important to be aware of the main differences between systematic and regular literature reviews.

The main differences between a systematic and a regular literature review are the process of collecting and analyzing literature, the accuracy of claims, the scope and replicability.

In a regular literature review, authors select articles or other publications ad hoc, to support the arguments of their work. Authors make claims about the state of the art of academic knowledge on a specific topic, but do not necessarily provide systematic evidence to support their claims.

Therefore, the level of accuracy differs in systematic and regular literature reviews.

Furthermore, authors doing regular literature reviews generally do not explain how they conducted their review, and how they selected relevant literature. Therefore, regular literature reviews are usually not replicable, whereas systematic ones are.

Additionally, the scope of a review differs immensely between systematic and regular literature reviews.

Systematic literature reviews tend to methodically analyze hundreds of articles, whereas regular literature reviews are much more limited in scope and highly selective in their choice of publications that are included.

Systematic literature reviews and meta-analyses are frequently confused or used interchangeably, but it’s important to understand that they are distinct methodologies.

While a meta-analysis focuses on synthesizing data and drawing broad conclusions from multiple studies, a systematic literature review is geared towards answering a well-defined research question with a comprehensive and methodical approach.

A meta-analysis involves the comprehensive analysis of results from multiple studies on a particular topic, aiming to identify patterns and draw generalized conclusions. These analyses often rely on statistical methods to combine data from various sources.

On the other hand, systematic literature reviews can take either a qualitative or quantitative approach. They are designed to address a specific and often novel research question, going beyond a mere summary of existing studies.

Conducting a ‘regular’ literature review is perfectly valid, but there are compelling reasons to consider a systematic literature review for certain research endeavors.

One primary advantage of a systematic review is its ability to address the overwhelming volume of relevant literature that can leave researchers, particularly master’s and PhD students, feeling daunted and unsure of when to stop searching for more material.

By conducting a systematic review, all pertinent literature can be analyzed within specific parameters, alleviating concerns about potentially missing critical information.

Moreover, a systematic literature review offers the advantage of identifying and showcasing research gaps, bolstering the academic significance of one’s work. This comprehensive approach to reviewing existing literature contributes to a stronger foundation for the research and its potential impact.

Additionally, by being transparent and explicit about the methodology employed in collecting and analyzing the literature, researchers enhance the credibility and reliability of their statements and arguments. This explicitness reinforces the trustworthiness of the research findings, which is crucial in the academic world.

Academic citation databases suitable for systematic literature reviews

To conduct systematic literature reviews, it is recommended to utilize a citation database that provides built-in features to refine search queries effectively.

Google Scholar is a widely used search engine among scholars, but its sidebar for limiting search queries is relatively basic. Despite this limitation, the advantage of using Google Scholar is that it is freely accessible to all users, including those without institutional affiliations.

Web of Science has very elaborate search options. The search engine allows you to choose from various databases and indexes, such as the Science Citation Index or the Social Science Citation Index. Additionally, it enables you to search for keywords within topics or titles, as well as author names, affiliations, publication years, and more.

Web of Science provides a wide range of Booleans (depicted in the image below on the right) that you can utilize to precisely specify your search criteria. Booleans are a system of logic employed in programming and computer sciences, featuring operators like “AND,” “OR,” and “NOT.” By using these operators, you can fine-tune your search queries and filter results to precisely match what you are looking for.

After obtaining search results on Web of Science, you have the option to further refine and organize them based on various criteria, such as research areas, disciplines, languages, regions, document types, and more.

Although it may require some time and practice to become proficient in developing precise search queries on Web of Science, the database offers tremendous opportunities for conducting systematic literature reviews. Its extensive filtering and organizing features empower researchers to gather comprehensive and relevant literature on specific topics.

Scopus the citation database by publisher Elsevier, boasts access to over 36,000 journals. Its layout and search options closely resemble those of Web of Science, making it a highly recommended platform to explore for research purposes. If you are familiar with Web of Science, you will find Scopus user-friendly and worthwhile to explore for comprehensive academic content.

Citation databases can be utilized in combination for more comprehensive research. It is recommended to begin with popular ones like Web of Science and Scopus, as they provide extensive coverage. However, using additional databases can serve as a valuable double-check to ensure no relevant entries are overlooked. Employing multiple databases enhances the likelihood of capturing a comprehensive range of scholarly materials for your research.

While each systematic literature review is unique, there are commonly followed steps that serve as a foundation for the process:

• Clearly define a focused and narrow research question.
• Identify relevant keywords to lead you to articles that can potentially answer your research question.
• Choose a suitable citation database and establish inclusion and exclusion criteria, considering factors like timeframe, discipline, journal, research area, and geographical context.
• Download all pertinent results, which may amount to hundreds of articles.
• Systematically screen all titles and abstracts, manually excluding irrelevant articles based on your criteria.
• Create your own database, such as using Excel, to store the remaining entries and read them more thoroughly.
• Categorize the entries and/or their content in a way that provides meaningful insights to answer your research question.
• Develop visual representations like graphs and tables to present your results.
• Write a concise and engaging presentation of your findings, summarizing the key outcomes of the systematic review.

Following these steps will help you conduct a thorough and well-organized systematic literature review.

Get new content delivered directly to your inbox!

Subscribe and receive Master Academia's monthly roundup.

Getting the most out of thesis supervision meetings

Separating your self-worth from your phd work, related articles.

The 11 best AI tools for academic writing

Journal vs conference papers: Key differences & advice

How to address data privacy and confidentiality concerns of AI in research

10 powerful methodology courses for PhD students [online]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

A faster, more affordable way to improve your paper

Scribbr’s new AI Proofreader checks your document and corrects spelling, grammar, and punctuation mistakes with near-human accuracy and the efficiency of AI!

Proofread my paper

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved November 18, 2023, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Thesis Helpers

Find the best tips and advice to improve your writing. Or, have a top expert write your paper.

How To Write Literature Review For Thesis? Read On To Find Out!

Table of Contents

• 1. What is a literature review?
• 2. Thesis literature review example
• 3. Importance of the thesis literature review
• 4. Literature review structure
• 4.1. Step 1: Look for the Relevant Scholarly Resources
• 4.2. Step 2: Evaluate the Resources
• 4.3. Step 3: Identify Gaps in Current Resources
• 4.4. Step 4: Develop the Outline of the Master Thesis Literature Review

Types Of Literature Review

• 4.5. Write Your Literature Review
• 4.6. Step 7: Write Your Bibliography

What Is A Literature Review

A thesis literature review is a complete analysis of scholarly sources on a selected topic of study. It is crafted to give an overview of the current knowledge, to help the researcher know the methods, theories, and gaps that exist in research.

Thesis Literature Review Example

Why is Literature Review for Thesis Important?

When you are working on your graduate thesis, one of the core components needed to make it complete is a literature review. Here is a demonstration of the main benefits of carrying a literature review for your thesis.

• Allows you to show how familiar you are with the topic of study.
• Offers you an opportunity to develop a comprehensive methodology.
• Demonstrate how your research will address the existing gap in your topic of study.
• Make your contribution to your area of the study felt.

Doing a literature review requires you to collect and analyze scholarly resources that are related to your topic. When conducting a literature review, the process can be broken down into five key stages.

Literature Review Structure

• Look for relevant scholarly resources . This is checking for different resources, such as journals and books, which are related to your study.
• Evaluate the resources. This is careful sorting of the different resources to identify the most relevant ones.
• Identify debates and gaps in these resources . This is further analysis of the scholarly resources to establish the main arguments and possible gaps in research.
• Develop your outline. This is the format of the literature review that tells you what you are supposed to discuss at different points.
• Write the literature review . This is the final step that involves putting down the findings that you found after analyzing different resources.

To help you craft a good literature review for thesis, here are the main steps that you should follow.

Step 1: Look for the Relevant Scholarly Resources

By the time you get to writing the thesis for your literature, you will have worked on chapter one (introduction) that clearly defines the topic. But you can still relook at it before setting off to look for the relevant resources. By defining the problem, you will be able to look at the resources that are closely related to the study questions and problems.

Another method of looking for relevant studies is searching using the keyword. Consider using the main databases for the latest journals, books and articles. Some of these databases include:

• Project Muse .
• Google Scholar .
• Your university library.

After pulling out different resources, check whether it is relevant by going through the abstract. If the resource is relevant, peruse to the last section, the bibliography, for additional resources. When you find a specific resource recurring in the resources, it means it is very relevant.

Step 2: Evaluate the Resources

Once you have gathered an assortment of resources, the chances are that not all of them will be used during the study. So you will need to evaluate them further to determine which ones to use in the study. So here is how to evaluate every resource:

• What problem is addressed in the resource?
• How has the author defined the main concepts?
• What theories and methods are used in the resource?
• What is the conclusion of the resource?
• What is the relationship between the resource and other resources?
• How does the resource contribute to knowledge about the topic?

You should only pick the most relevant resources. Also, it is important to appreciate that if you are in the sciences, the review has to be focused on the latest resources. But if your thesis is in humanities, it might be necessary to check older resources to bring out the historical perspectives. As you read through, keep track of the resources by taking notes, capturing the pages, and citing them properly.

Step 3: Identify Gaps in Current Resources

Before you can organize the arguments in the literature, it is prudent to comprehend how the resources are related. So what should you look for?

• Patterns and trends, especially in theories, methods, and results.
• Debates, major conflicts, and contradictions.
• Gaps on what is missing in the literature.
• Pivotal publications.

Step 4: Develop the Outline of the Master Thesis Literature Review

The outline of your literature provides you with a breakdown of what you should discuss at what different stages. There are a number of strategies that you can use to prepare your literature review.

• Chronological . This approach involves tracing the development based on the topic occurrence over time. It is the simplest strategy.
• Thematic . This strategy involves presenting the review based on different themes.
• Methodological . If the resources you use for the review have varying methods, a methodological presentation can helps you to compare the results as well as conclusions.
• Theoretical . This approach involves exploring the theories, definitions, concepts, and models used in the resources. You might also want to focus on particular theories depending on the topic of study.

Note that you can opt to use one or combine several of them to make your literature review more articulate.

Step 5: Write Your Literature Review

Like other forms of academic writing, your literature review should take this format: introduction, body, and conclusion. Here is what to include in every section:

• Introduction: This should be used to give the focus of the literature review.
• Body: In the body of the literature review, you get into the finer details of the review. Here you should do the following:
• Summarize, analyze, and interpret.
• Evaluate comprehensively.
• Write carefully in properly structured and easy to read paragraphs.

Literature Review Example

To help you craft a great literature review thesis, it is important to also have the entire project in mind. This means that although you are reviewing literature, the methods you will use should be clear the back of your mind. Here is a thesis literature review example paragraph. The paragraph is borrowed from literature review of a thesis on the effects of cyberbullying.

“ Cyberbullying gives the bully a much larger spectrum to choose from when it comes to how exactly they want to intimidate their victims, which may be why it is often easier for them to carry out the act. Of all the different ways to cyberbully Faucher et al. (2014) found the most common platforms for cyberbullying to be social media, text messaging, and email, which were used to bully students about half of the time followed up by blogs forums and chat rooms which were 25 percent. This is no surprise that social media is the most common platform for cyberbullying because it can allow for the bully to remain completely anonymous to your average victim. This allows people who may not fit the mold of your average bully to create a fake account and build their own persona in order to bully others.”

• Conclusion.

Once you have written the body of the literature review, you still need to conclude it. This is a summary of the literature review that captures the main points that you have discussed.

Step 6: Write Your Bibliography

This guide on how to write literature review for thesis cannot be complete without including a bibliography. This is a complete list of all the resources that you have used during the review. It is important to ensure that you follow the method that your supervisor recommends for formatting and referencing. See two reference examples presented below.

Abeele, M., & Cock, R. (2013). Cyberbullying by mobile phone among adolescents: The role of gender and peer group status. Communications: The European Journal of Communication Research, 38(1), 107-118. Doi:10.1515/commun-2013-0006

Arntfield, M. (2015). Toward a Cybervictimology: Cyberbullying, Routine Activities Theory, and the Anti-Sociality of Social Media. Canadian Journal Of Communication, 40(3), 371-388

Make PhD experience your own

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

• MDR CER Writing
• Systematic Literature Review
• Literature Review Templates
• Full Service PMS
• Vigilance Monitoring and Reporting
• IVDR Writing
• Consulting Services
• SLR Software – CiteMed.io
• EU MDR Templates for CER and PMS
• Free Downloads
• Featured In …

Systematic Literature Reviews

The MDR forces manufacturers to set up processes for continuous literature reviews and reevaluation of the available clinical data. In May 2021, the European Medical Device Directive (93/42/EEC) was replaced with the Medical Device Regulation (MDR, 2017/745). Unlike MDD, the MDR considers clinical evaluation to take place during the entire lifecycle of medical devices, meaning clinical data must be updated continuously. Seeing as clinical data is usually sourced from clinical investigations, post-market surveillance, and literature reviews.

  Literature reviews are fundamental to any scientific discipline, social sciences, medicine, or linguistics. Over the past 100 years, scientific documentation and sharing of new research have evolved into a labyrinthine field of publication databases, each housing thousands of articles, journals, research manuscripts, and conference summaries and abstracts. Yet, with all this knowledge at the tip of our fingers, how do we gather and select information for analyzing scientific queries, and how do we interpret the collected literature about our topic?  

What is a Literature review?

Literature reviews are an analytical approach to investigating a specific topic or research question. For the scientist, they help gain insight into current knowledge, relevant theories and methods, and any gaps or missing exploration opportunities in the existing research. For the reader, they should help situate the research within the current body of knowledge and provide context to the exploration of the chosen research subject.   A literature review comprises four steps; investigation of published literature on the chosen topic; summarizing the chosen literature and synthesizing its relevance to the chosen topic; analyzing and critically evaluating the selected literature; and; presenting the literature in a structured way.   Literature reviews are performed for various scientific works, such as theses, dissertations, journal articles, books, conference manuscripts, scientific journalism, and patient brochures and instructions for use. Most importantly, literature reviews are vital in clinical evaluations of drugs and medical devices. The latter is the focus of this article.   In medical devices and clinical evaluations, literature reviews aim to provide the reader with context and background for the chosen treatment or disease or act as a general overview of a topic.   The systematic literature review is a type of literature review that aims to answer a specific clinical question. While literature reviews can take a few weeks to a few months by a single author, conducting a systematic review can take a team of up to five authors a year or more.  

History of the systematic literature reviews and meta-analyses

The systematic literature review asks a specific clinical question about the effectiveness of an intervention or treatment. It seeks to answer it by critically summarizing and analyzing the current evidence for that intervention. It is considered the highest form of evidence-gathering for medical science. The systematic review process can be further elevated by using meta-analysis methods on data in the chosen literature, resulting in a completely objective evaluation of the research findings.

James Lind , a Scottish doctor preoccupied with helping sailors in the royal navy avoid scurvy, was the author of the first-ever systematic literature review, in which he published a paper providing a clear and unbiased review of the existing evidence on scurvy. Although his article  A treatise of the scurvy , published in 1753, was practically ignored, he started science on a path of systematic literature review that is still underway.  

Archie Cochrane

In 1972,  archie cochrane  wrote:.

“ It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials .”

Gene Glass and Mary Lee Smith

Research synthesis and analysis emerged formally in 1975 when Gene Glass and Mary Lee Smith coined the term meta-analysis in their research paper  Meta-analysis of psychotherapy outcome studies 1.    During the rise of evidence-based medicine in the late 1970s and early 1980s, systematic research analysis and synthesis were applied to medicine and health. For example, researchers in Oxford began a program of systematic literature reviews on the effectiveness of health interventions, thus paving the way for evidence-based medicine and committing to the principles of accumulative scientific knowledge.  

Characteristics of a Systematic Literature Review protocols

Systematic reviews are frequently used in evidence-based healthcare, public health interventions, and evidence-based policy and practice. They are designed to provide a comprehensive and complete summary of the current literature on a specific topic and to minimize bias in the analysis and interpretation of the collected research.

  Many systematic literature reviews include meta-analysis, which applies statistical analysis to the available research. In contrast, others focus on the qualitative synthesis of data or mixed-method reviews that have both.  

The systematic literature review should display the following characteristics:

Clearly defined and specific research questions and objectives

• Pre-defined inclusion and exclusion criteria for choosing relevant literature
• Pre-defined and systematic search strategy, including search terms and keywords
• Defined eligibility criteria to be applied to all the Specified used in the review
• A systematic and clear evaluation of the quality of the literature included
• Identification of excluded sources and justifications for exclusion
• Analysis of the gathered data and information
• References to incoherences, errors, and limitations in and of the chosen literature

What does PRISMA stand for?

The  PRISMA  framework (Preferred Rating for Systematic Reviews and Meta-analyses) was developed to ensure a standardized way of conducting a systematic review to ensure transparency and completeness. The framework is now required by more than 170 medical journals worldwide. In addition, it has been extended to support specific review types or aspects of the review process, for example, PRISMA-P for review protocols and PRISMA-ScR for scoping review.

What are the other tools used for Systematic Literature Review?

Similarly, the  ENTREQ  (Enhancing Transparency Reporting the Synthesis of Qualitative Research) guidelines exist for quantitative reviews,  RAMESES  (Realist and Meta-narrative Evidence Syntheses) for meta-narrative and realist reviews, and  eMERGe  (Improving Reporting of Meta-Ethnography) for meta-ethnography reviews.  

How to conduct systematic literature reviews and meta-analyses?

Defining your study objective/research question

The definition of the research question and/or study objectives is the most important part of the systematic review. Without a clearly defined research question and quantifiable endpoints, the rest of the review can lack focus and accuracy, ultimately ending up with conclusions and weak and unfocused literature interpretations. Care should be taken to avoid research questions that are too wide or too narrow.   For medical interventions and treatments, the PICO method is frequently used. The PICO framework helps develop search strategies focused on the patient, population, or problem (P); intervention (I); comparison, control, or comparator (C); and; outcome (O).  

Lietrature Search Protocol

A clearly defined set of criteria must be set up to include or exclude research on the topic, and these criteria are included in the search protocol. The search protocol is your master document that guarantees transparency, repeatability, and audibility for your review.   One of the most critical aspects of the systematic review is that it can be objectively evaluated for the accuracy of methods and search criteria, and the search protocol ensures that. The search protocol should be carefully planned according to the research question and explicitly documented before the review starts. It supports the review team, ensuring consistency and integrity in all the carried searches.   Once the review is complete, you should be able to identify every paper you read and every piece of information you searched, critiqued, and defined. While doing all this, it is important to include everything – from the terms you used to search, strategies incorporated, and limitations you encountered along the way.  

Literature Search Databases

Literature for clinical evaluations can be found in scientific journals, academic dissertations, books, bibliographic databases, and online databases. Most systematic reviews are based on 5-7 online publication sites, such as PubMed , Cochrane , and Embase (see below for more information), where publications are typically journal articles or conference abstracts. Literature can also be found in “gray” sources, such as dissertations, theses, fact sheets, government reports, and pre-prints of articles. Every source should be included, analyzed, synthesized, and acknowledged in a comprehensive literature review. It is also important to review multiple sources to avoid publication bias in the data interpretation. It is important to apply appropriate use of terminology when searching for literature. Unless a commonly accepted group of terminologies are used, a gap remains in data encryption – be it manual or automated. It is easy to find databases and thesauri highlighting common terminologies used in various fields. Furthermore, alternative spellings and similar conceptual roots should also be considered to ensure all relevant publications are included and evaluated. Boolean expressions can widen, focus, and improve search scope in almost all databases and should also be considered.  

Data analysis and synthesis

Once you’ve done your searches, the eligibility criteria from your search protocol come into effect, and the methodological quality of your evaluation criteria is tested. Your data extraction from the literature should be performed precisely as established in the search protocol. In addition, it must be recorded so that it can be replicated faithfully by others in the future.

  After selecting your data and literature, you are ready to start analyzing it. The more data is included in your review literature, the better and more unbiased your results will be. If appropriate and established in your search protocol, you can use meta-analysis (i.e., analyzing data from multiple sources through statistical methods) to analyze your data or other analytical tools, such as qualitative meta-synthesis, which is the synthesis of data from qualitative studies.  

Data presentation

These sections are followed by acknowledgments, references, and tables and figures (the latter may also be included in the general text, depending on the journal).

Cochrane  is the largest, and arguably one of the most important, international organizations in medicine and health today. Cochrane consists of more than 37,000 specialists in healthcare fields who systematically review randomized trials. Cochrane reviews are published in the  Cochrane Library , in the Cochrane  Database of Systematic Reviews section , and are a valuable resource for anyone looking to conduct a systematic literature review.

PubMed is a free publication resource, housing more than 32 million citations and abstracts of biomedical literature. Although PubMed does not provide full-text citations, they link their citations to the full text, often through the publisher’s website. The largest component of Pubmed is  MEDLINE , which consists most significant actions from biomedical publications through the  National Library of Medicine .

Embase  is another comprehensive medical literature database, similar to Pubmed, where you can search for full-text content or abstracts. Embase is focused on supporting pharmacovigilance and regulatory authorities and includes biomedical and pharmacological publications from 1947.

PubMed and Embase are powerful search engines for retrieving biomedical and life science literature. However, both sites require a thorough knowledge of the optimization of their search engines, the use of BOOLEAN operators, and the indexation systems of  Medical Subject Headings, MeSH .

Systematic literature reviews and the EU MDR

In May 2021, the European Medical Device Directive (93/42/EEC) was replaced with the Medical Device Regulation (MDR, 2017/745). Unlike MDD, the MDR considers clinical evaluation to take place during the entire lifecycle of medical devices, meaning clinical data must be updated continuously.   Seeing as clinical data is usually sourced from clinical investigations, post-market surveillance, and literature reviews, the MDR forces manufacturers to set up processes for continuous  literature reviews  and reevaluation of the available clinical data.   Several sections of the MDR propose using literature to support and source clinical data, such as for demonstrating equivalence, post-market clinical follow-up, clinical investigation plans, and investigators’ brochures. As such, systematic literature reviews are an invaluable part of sourcing clinical data for each clinical section in the MDR.  

1) Smith, Mary & Glass, Gene. (1977). Meta-Analysis of Psychotherapy Outcome Studies. The American psychologist. 32. 752-60. 10.1037//0003-066X.32.9.752.

Want more eu mdr and regulatory insights.

We send weekly emails with the latest regulatory developments, templates, and strategies straight to QA/RA Professionals like you. Sign up below to get access today.

" * " indicates required fields

Step 1 of 4

Want Access to Everything EU MDR?

Join over 3,000 Regulatory and Quality Directors, Engineers and Consultants who receive weekly industry Whitepapers, Templates, and EU MDR news right to their inbox

By signing up, you agree to receive email marketing

Start your free trial

Arrange a trial for your organisation and discover why FSTA is the leading database for reliable research on the sciences of food and health.

REQUEST A FREE TRIAL

• Thought for Food Blog

What is the difference between a systematic review and a systematic literature review?

By Carol Hollier on 07-Jan-2020 12:42:03

For those not immersed in systematic reviews, understanding the difference between a systematic review and a systematic literature review can be confusing.  It helps to realise that a “systematic review” is a clearly defined thing, but ambiguity creeps in around the phrase “systematic literature review” because people can and do use it in a variety of ways. 

A systematic review is a research study of research studies.  To qualify as a systematic review, a review needs to adhere to standards of transparency and reproducibility.  It will use explicit methods to identify, select, appraise, and synthesise empirical results from different but similar studies.  The study will be done in stages:  

• In stage one, the question, which must be answerable, is framed
• Stage two is a comprehensive literature search to identify relevant studies
• In stage three the identified literature’s quality is scrutinised and decisions made on whether or not to include each article in the review
• In stage four the evidence is summarised and, if the review includes a meta-analysis, the data extracted; in the final stage, findings are interpreted. [1]

Some reviews also state what degree of confidence can be placed on that answer, using the GRADE scale.  By going through these steps, a systematic review provides a broad evidence base on which to make decisions about medical interventions, regulatory policy, safety, or whatever question is analysed.   By documenting each step explicitly, the review is not only reproducible, but can be updated as more evidence on the question is generated.

Sometimes when people talk about a “systematic literature review”, they are using the phrase interchangeably with “systematic review”.  However, people can also use the phrase systematic literature review to refer to a literature review that is done in a fairly systematic way, but without the full rigor of a systematic review. 

For instance, for a systematic review, reviewers would strive to locate relevant unpublished studies in grey literature and possibly by contacting researchers directly.  Doing this is important for combatting publication bias, which is the tendency for studies with positive results to be published at a higher rate than studies with null results.  It is easy to understand how this well-documented tendency can skew a review’s findings, but someone conducting a systematic literature review in the loose sense of the phrase might, for lack of resource or capacity, forgo that step. 

Another difference might be in who is doing the research for the review. A systematic review is generally conducted by a team including an information professional for searches and a statistician for meta-analysis, along with subject experts.  Team members independently evaluate the studies being considered for inclusion in the review and compare results, adjudicating any differences of opinion.   In contrast, a systematic literature review might be conducted by one person. 

Overall, while a systematic review must comply with set standards, you would expect any review called a systematic literature review to strive to be quite comprehensive.  A systematic literature review would contrast with what is sometimes called a narrative or journalistic literature review, where the reviewer’s search strategy is not made explicit, and evidence may be cherry-picked to support an argument.

FSTA is a key tool for systematic reviews and systematic literature reviews in the sciences of food and health.

The patents indexed help find results of research not otherwise publicly available because it has been done for commercial purposes.

The FSTA thesaurus will surface results that would be missed with keyword searching alone. Since the thesaurus is designed for the sciences of food and health, it is the most comprehensive for the field. 

All indexing and abstracting in FSTA is in English, so you can do your searching in English yet pick up non-English language results, and get those results translated if they meet the criteria for inclusion in a systematic review.

FSTA includes grey literature (conference proceedings) which can be difficult to find, but is important to include in comprehensive searches.

FSTA content has a deep archive. It goes back to 1969 for farm to fork research, and back to the late 1990s for food-related human nutrition literature—systematic reviews (and any literature review) should include not just the latest research but all relevant research on a question. 

You can also use FSTA to find literature reviews.

FSTA allows you to easily search for review articles (both narrative and systematic reviews) by using the subject heading or thesaurus term “REVIEWS" and an appropriate free-text keyword.

On the Web of Science or EBSCO platform, an FSTA search for reviews about cassava would look like this: DE "REVIEWS" AND cassava.

On the Ovid platform using the multi-field search option, the search would look like this: reviews.sh. AND cassava.af.

In 2011 FSTA introduced the descriptor META-ANALYSIS, making it easy to search specifically for systematic reviews that include a meta-analysis published from that year onwards.

On the EBSCO or Web of Science platform, an FSTA search for systematic reviews with meta-analyses about staphylococcus aureus would look like this: DE "META-ANALYSIS" AND staphylococcus aureus.

On the Ovid platform using the multi-field search option, the search would look like this: meta-analysis.sh. AND staphylococcus aureus.af.

Systematic reviews with meta-analyses published before 2011 are included in the REVIEWS controlled vocabulary term in the thesaurus.

An easy way to locate pre-2011 systematic reviews with meta-analyses is to search the subject heading or thesaurus term "REVIEWS" AND meta-analysis as a free-text keyword AND another appropriate free-text keyword.

On the Web of Science or EBSCO platform, the FSTA search would look like this: DE "REVIEWS" AND meta-analysis AND carbohydrate*

On the Ovid platform using the multi-field search option, the search would look like this: reviews .s h. AND meta-analysis.af. AND carbohydrate*.af.  

Related resources:

• Literature Searching Best Practise Guide
• Predatory publishing: Investigating researchers’ knowledge & attitudes
• The IFIS Expert Guide to Journal Publishing

Library image by  Paul Schafer , microscope image by Matthew Waring , via Unsplash.

• FSTA - Food Science & Technology Abstracts
• IFIS Collections
• Resources Hub
• Diversity Statement
• Sustainability Commitment
• Company news
• Frequently Asked Questions
• Privacy Policy
• Terms of Use for IFIS Collections

Ground Floor, 115 Wharfedale Road,  Winnersh Triangle, Wokingham, Berkshire RG41 5RB

Get in touch with IFIS

© International Food Information Service (IFIS Publishing) operating as IFIS – All Rights Reserved     |     Charity Reg. No. 1068176     |     Limited Company No. 3507902     |     Designed by Blend