Annotation refers to the process of writing notes on an article. There are many ways to do this. The most basic technique is to print out the article and build a binder related to your topic. Raul Pacheco-Vega’s excellent blog has a post on his approach to taking physical notes. While you are there, browse around that website: it is full of amazing tips for students conducting a literature review and graduate research projects. Personally though, I prefer to use my laptop to write notes on an article because I can type faster than I write and typing notes allows me search for keywords. For other students, electronic notes work best because they cannot afford to print every article that they will use in their paper and taking your notes electronically is more environmentally friendly. No matter what you use, the point is that you need to write notes when you’re reading. Reading is research!

There are a number of free software tools you can use to help you annotate a journal article. Most PDF readers like Adobe Acrobat have a commenting and highlighting feature. The best approach may be to use a citation manager like Mendeley or its open-source alternative, Zotero. Again, Raul Pacheco-Vega’s guide to using Mendeley is stellar. Using citation managers, you can build a library of articles, save your annotations, and link annotations across PDFs using keywords. They also provide integration with word processing programs to help with citations in a reference list.

Of course, I don’t follow this advice because I have a system that works well for me. I have a PDF open in one computer window and an Excel file in a window next to it with my Thematic Analysis Grid (I explain this further in a later chapter). It’s a bit low-tech, but it does make my notes searchable and helps me find the full study quickly as needed. This way, when I am looking for a concept or quote, I can simply search my notes using the Find feature and get to the information I need. I also break the research into separate category “sheets” within my spread sheet (or color code themes) as I see themes develop.

Annotation and reviewing literature does not have to be a solo project. If are working in a group, you can use the Hypothes.is web browser extension to annotate articles collaboratively. You can also use Google Docs to collaboratively annotate a PDF using the commenting feature and write collaborative notes in a shared document. By sharing your highlights and comments, you can split the work of getting the most out of each article you read and build off one another’s ideas.

The most important point here is to create a system to organize the research that works for you and to stick to it. There’s nothing worse than having a bunch of studies that you can’t remember your reason for downloading, or to have a great quote in your paper that you can’t find the reference for. The next section deals with ways you might organize the research you find during your literature search.

Common annotations

In this section, we present common annotations people make when reading journal articles. These annotations are adapted from Craig Whippo and Raul Pacheco-Vega. If you are annotating on paper, I suggest using different color highlighters for each type of annotation listed below. If you are annotating electronically, you can use the names below as tags to easily find information later. For example, if you are searching for definitions of key concepts, you can either click on the tag for [definitions] in your PDF reader or thumb through a printed copy of article for whatever color or tag you used to indicate definitions of key terms. Most of all, you want to avoid reading through all of your sources again just to find that one thing you know you read somewhere. Time is a graduate student’s most valuable resource, so our goal here is to help you spend your time reading the literature wisely.

Personal reflections

Personal reflections are all about you. What do you think? Are there any areas you are confused about? Any new ideas or reflections come to mind while you’re reading? Treat these annotations as a means of capturing your first reflections about an article. Write down any questions or thoughts that come to mind as you read. If you think the author says something inaccurate or unsubstantiated, write that down. If you don’t understand something, make a note about it and ask your professor. Don’t feel bad! Journal articles are hard to understand sometimes, even for professors. Your goal is to critically read the literature, so write down what you think while reading! The following table contains some questions that might stimulate your thoughts.

Definitions

Note definitions of key terms for your topic. At minimum, you should include a scholarly definition for the concepts represented in your working question. If your working question asks about the the effect of assessment on student achievement, your research proposal will have to explain how you define assessment, as well as how you define achievement. While you may already know what you mean by assessment, the person reading your research proposal does not.

Annotating definitions also helps you engage with the scholarly debate around your topic. Definitions are often contested among scholars. Some definitions of achievement will be more comprehensive, including things such as test scores, but also grades, graduation rates or employment. Other definitions will be less comprehensive, covering only standardized test scores, for example. Assessment can be formative or summative; criterion or norm referenced; or diagnostic, placement or screening. Assessment is also frequently conflated with testing (measuring discrete knowledge or skills) or evaluation (making a judgement based on collected evidence) while it’s formal definition is as a process by which attributes of phenomena (learning) are described. Often, how someone defines something conceptually is highly related to how they measure it in their study. Since you will have to do both of these things, find a definition that feels right to you or create your own, noting the ways in which it is similar or different from those in the literature.

Definitions are also an important way of dealing with jargon. Becoming familiar with a new content area involves learning the jargon experts use. For example, in the last paragraph I used the term criterion referenced, but that may not be a term you’ve heard before. If you were conducting a literature review on assessment, you would want to search for keywords like criterion and norm referenced , or differentiate assessment from testing and evaluation as relevant to your working question. You will also want to know what they mean so you can use them appropriately in designing your study and writing your literature review.

Theoretical perspective

Noting the theoretical perspective of the article can help you interpret the data in the same manner as the author. For example, articles on student learning may stem from Cognitive, Behaviourist, or Constructivist theory, and understanding which theory is being used is important to make sense of empirical results. Articles should be grounded in a theoretical perspective that helps the author conceptualize and understand the data. Some journal articles are entirely theoretical and help you understand the theories or conceptual models related to your topic. It’s a good idea to note what theories authors mention when talking about your topic area. Some articles are better about this than others, and many authors make it a bit challenging to find theory (if mentioned at all). In other articles, it may help to note which theories are missing from the literature. For example, a study’s findings might address issues of oppression and discrimination, but the authors may not use critical theory to make sense of what happened.

Background knowledge

It’s also a good idea to note any relevant information the author relies on for background. When an author cites facts or opinions from others, you are subsequently able to get information from multiple articles simultaneously. For example, if we were looking at this meta-analysis about domestic violence, in the introduction section, the authors provide facts from many other sources. These facts will likely be relevant to your inquiry on domestic violence, as well.

As you are looking at background information, you should also note any subtopics or concepts about which there is controversy or consensus. The author may present one viewpoint and then an opposing viewpoint, something you may do in your literature review as well. Similarly, they may present facts that scholars in the field have come to consensus on and describe the ways in which different sources support these conclusions.

Sources of interest

Note any relevant sources the author cites. If there is any background information you plan to use, note the original source of that information. When you write your literature review, cite the original source of a piece of information you are using, which may not be where you initially read it. Remember that you should read and refer to the primary source. If you are reading Article A and the author cites a fact from Article B, you should track down Article B and use it when you cite the fact in your paper. You do this to make sure Article A interpreted Article B correctly and scan Article B for any other useful facts.

Research question/Purpose

Authors should be clear about the purpose of their article. Charitable authors will give you a sentence that starts with something like this:

• “The purpose of this research project was…”
• “Our research question was…”
• “The research project was designed to test the following hypothesis…”

Unfortunately, not all authors are so clear, and you may have to hunt around for the research question or hypothesis. Generally, in an empirical article, the research question or hypothesis is at the end of the introduction. In non-empirical articles, the author will likely discuss the purpose of the article in the abstract or introduction.

Results

You should highlight any of the key findings of an article. They will be described very briefly in the abstract, and in much more detail in the article itself. In an empirical article, you should look at both the ‘Results’ and ‘Discussion’ sections. For a non-empirical article, the key findings will likely be in the conclusion.

Measures

How do researchers know something when they see it? Found in the ‘Methods’ section of empirical articles, the measures section is where researchers spell out the tools, or measures, they used to gather data. For quantitative studies, you will want to get familiar with the questions researchers typically use to measure key variables. For example, to measure whether children are having difficulty recognizing letter and number orientations researchers may use the Jordan Left-Right Reversal Test. The more frequently used and cited a measure is, the more we know about how well it works (or not). In education you can find out what we know about many standardized commercially available tests through the Mental Measurements Yearbook (including the Jordan Left-Right Reversal Test). Qualitative studies will often provide at least some of the interview or focus group questions they used with research participants. They will also include information about how their inquiry and hypotheses may have evolved over time. Keep in mind however, sometimes important information is cut out of an article during editing. If you need more information, consider reaching out to the author directly. Before you do so, check if the author provided an appendix with the information you need or if the article links to a their data and measures as part open data sharing practices.

Sample

Who exactly were the study participants and how were they recruited? In quantitative studies, you will want to pay attention to the sample size. Generally, the larger the sample, the greater the study’s explanatory power. Additionally, randomly drawn samples are desirable because they leave any variation up to chance. Samples that are conducted out of convenience can be biased and non-representative of the larger population. In qualitative studies, non-random sampling is appropriate but consider this: how well does what we find for this group of people transfer to the people who will be in your study? For qualitative studies and quantitative studies, look for how well the sample is described and whether there are important characteristics missing from the article that you would need to determine the quality of the sample.

Limitations

Honest authors will include these at the end of each article. But you should also note any additional limitations you find with their work as well.

Your annotations

These are just a few suggested annotations, but you can come up with your own. For example, maybe there are annotations you would use for different assignments or for the problem statement in your research proposal. If you have an argument or idea that keeps coming to mind when you read, consider creating an annotation for it so you can remember which part of each article supports your ideas. Whatever works for you. The goal with annotation is to extract as much information from each article while reading, so you don’t have to go back through everything again. It’s useless to read an article and forget most of what you read, or to waste time digging through dozens of studies to find that one critical point you remember reading (but not where you read it). Annotate!