How to Read a Paper

Keshav, 2007

Notes

First Pass

Goal: gives you a general idea about the paper

Time: 5-10 minutes

Steps:

• Title, abstract, and introduction
• Section and subsection headings
• Glance at the mathematical context to determine the underlying theoretical foundations
• Conclusions
• Glance over the references, mentally ticking off the ones you’ve already read

Answers the following questions:

• Category
• Context
• Correctness
• Contributions
• Clarity

Second pass

Goal: grasp the paper’s content, but not its details

Note down terms you didn’t understand, or questions you may want to ask the author.

• Look carefully at the figures, diagrams, and other illustrations in the paper
• Mark relevant unread references for future reading

Time: 1 hour

Third pass

Goal: to virtually re-implement the paper - making the same assumptions and attempting to re-create the work

• Identify and challenge every assumption in every statement.
• Think about how you would present a particular idea.

Time: 3–5 hours

Annotations

Imported at (2022-06-18, 6:57:26 p.m.)

The key idea is that you should read the paper in up to three passes, instead of starting at the beginning and plowing your way to the end.

The first pass gives you a general idea about the paper. The second pass lets you grasp the paper’s content, but not its details. The third pass helps you understand the paper in depth.

The first pass (5-10 minutes):

• get a high-level view of the paper
• decide whether you need any more passes

1. Carefully read the title, abstract, and introduction
2. Read the section and sub-section headings, but ignore everything else
3. Read the conclusions
4. Glance over the references, mentally ticking off the ones you’ve already read

Questions to answer at the end of 1st pass:

1. Category: What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype?
2. Context: Which other papers is it related to? Which theoretical bases were used to analyze the problem?
3. Correctness: Do the assumptions appear to be valid?
4. Contributions: What are the paper’s main contributions?
5. Clarity: Is the paper well written?

In the second pass, read the paper with greater care, but ignore details such as proofs. It helps to jot down the key points, or to make comments in the margins, as you read.

• Look carefully at the figures, diagrams and other illustrations in the paper. Pay special attention to graphs.
• Remember to mark relevant unread references for further reading

The second pass should take up to an hour. After this pass, you should be able to grasp the content of the paper. You should be able to summarize the main thrust of the paper, with supporting evidence, to someone else. This level of detail is appropriate for a paper in which you are interested, but does not lie in your research specialty.

The key to the third pass is to attempt to virtually re-implement the paper: that is, making the same assumptions as the authors, re-create the work. By comparing this re-creation with the actual paper, you can easily identify not only a paper’s innovations, but also its hidden failings and assumptions.

This pass can take about four or five hours for beginners, and about an hour for an experienced reader. At the end of this pass, you should be able to reconstruct the entire structure of the paper from memory, as well as be able to identify its strong and weak points. In particular, you should be able to pinpoint implicit assumptions, missing citations to relevant work, and potential issues with experimental or analytical techniques.