HOW TO WRITE A RESEARCH PAPER

A guide for software engineers & practitioners

Adrienne Porter Felt & Kurt Thomas

A research paper is a self-contained technical document that teaches a busy and skeptical person something new.

A research paper is not a blog post, design doc, or post mortem. There are significant differences in content, structure, and tone.

Publishing is a way to demonstrate a commitment to transparency, a high standard of excellence, and advancing science.

Having a PhD is not a requirement to do or publish research. Many engineers do publishable research. However, getting a PhD teaches you how to navigate the publishing game. Here are a few tips to get you started.

CONTENT

STRUCTURE

SKEPTICISM

REVIEWS

What makes a good research paper?

A paper that is very novel might require less evaluation, and

vice versa. However, most successful papers have some of both.

Industry papers are often weak on novelty.

A description of a system or data set is not a research paper. (That’s documentation.) A research paper also needs to teach the reader something new about the world.

Evaluation requires extra effort.

In industry, evaluation usually only goes as far as necessary to launch it. Published papers go further, ideally to rigorous standards.

• Companies have scale. Talk about scaling challenges or run very large-scale experiments.
• Engineers get to work with realistic data sets and workloads. Talk about real world lessons that academics might not be able to see.
• Companies have deep pockets. Get budget to pay for large & diverse user studies.

Leverage industry advantages

Examples

THESIS

STRUCTURE

SKEPTICISM

REVIEWS

Structure your paper so that it’s easy to skim.

Your reader is a busy professional with a long to-do list. Unlike a blog post, people won’t read from top to bottom. They’ll quickly look for one specific figure or hunt around to get the gist.

1. Follow paper formatting conventions
2. Have a clear, concise thesis
3. State your paper’s contributions
4. Use bulleted lists and labeled paragraphs
5. Put important numbers into figures
6. Use the right tools

6 WAYS TO ADD STRUCTURE

• Abstract
• Introduction
• Background
• [Technical content]
• Related Work
• Conclusion

1. Follow paper formatting conventions

Not every field uses exactly this structure. Check out papers in your field to understand your specific field. This is just meant as a primer.

• Abstract
• Introduction
• Background
• [Technical content]
• Related Work
• Conclusion

1. Follow paper formatting conventions

Introduction structure:

1. What is the broad problem area you are part of?
2. What have others done before you? Why is more needed?
3. What did you do?
4. What were your key findings?
5. Summary of contribution

• Abstract
• Introduction
• Background
• [Technical content]
• Related Work
• Conclusion

1. Follow paper formatting conventions

Look at papers in your field to see what subsections are expected, e.g., Limitations, Ethics, Performance.

• Abstract
• Introduction
• Background
• [Technical content]
• Related Work
• Conclusion

1. Follow paper formatting conventions

In some fields, this is at the beginning near the “Background” section.

• Abstract
• Introduction
• Background
• [Technical content]
• Related Work
• Conclusion

1. Follow paper formatting conventions

Here’s a good place to discuss future work.

You should be able to describe your paper’s central research question in a sentence to another expert. Put that sentence in your abstract and introduction.

2. Have a clear, concise thesis

Example 1

Example 2

The abstract, intro, and conclusion should all clearly say what you did and why it’s interesting.

3. State your paper’s contributions

Example 1

Example 2

4. Use bulleted lists and labeled paragraphs

Labeled text helps people find what they’re looking for when skimming, or skip uninteresting paragraphs.

Example 2

Example 1

Example 2

5. Put important numbers into figures

It’s easier to spot key numbers in a table or graph than to have to read through the text for the numbers.

In the text...

...pulled into a figure

• Each conference has its own template that will help you format your paper. It will provide formatting for sections, subsections, lists, etc.
• The template will be in LaTeX.
• Use BibTeX to organize and link your citations.

6. Use the right tools

Structure is extra important if English isn’t your first language. Structure will help the reader follow your point even if there are grammatical errors or the sentences sound awkward.

THESIS

STRUCTURE

SKEPTICISM

REVIEWS

Convince a skeptic that your work is trustworthy. The world is full of bad research; prove that yours is good.

• If you’re proposing a new system to solve $PROBLEM, convince the reader that it solves $PROBLEM better than other alternatives.
• If you’re evaluating someone else’s idea, convince the reader that your evaluation is thorough and fair.
• If you’re testing a hypothesis, convince the reader that your experimental setup is sound.

Examples of what you have to demonstrate

• Never assume anything, even if it’s “obvious”
• Admit your work’s limitations
• Compare fairly to related work
• Provide all information the reader needs

4 WAYS TO WRITE FOR A SKEPTIC

• Always provide citations for pieces of your argument.
• If you can’t provide a citation, get the evidence yourself.
• Common wisdom can be unreliable or biased.
• For example, consider this statement: � “Vermont is colder than California. Data centers in Vermont therefore� require less air conditioning...”�Wait, hold on. Citation please! Cause now I’m wondering if that’s still true in the summer…

1. Never assume anything, even if it’s obvious

• It’s critical to show you know all the shortcomings and considered how they impact your findings.
• Don’t diminish or hide limitations. It is intellectually dishonest and, once detected, will make your reader question the validity of your work.
• I like to have an explicit section for limitations.
• If you tried to mitigate a limitation, say what you did.

2. Admit your work’s limitations

• Novelty is important in academia, so readers want to know how your work differs from prior work.
• Do a literature search:
• Use Google Scholar
• Look over past proceedings of conferences
• Read other papers’ “Related Works” sections
• If you discover you aren’t the first, see if you can do it better, faster, cheaper, at larger scale...

3. Compare fairly to related work

• Imagine a curious reader wants to double check that you did something correctly. Make sure you’ve given them enough information to do so.
• Put full proofs, survey instruments, etc. in an appendix.
• Open source code and data if possible. This is tricky in industry. If you can’t share, describe in detail. How does the code work? What are the data set’s characteristics?

4. Provide all information the reader needs

THESIS

STRUCTURE

SKEPTICISM

REVIEWS

In order to publish a paper, you submit it for peer review by a panel of experts. They will decide whether to accept or reject your paper.

Learn how to work with the review process. Although it can be frustrating, you can extract value from the review process.

• You submit a paper by a deadline.
• Three months later, you find out whether your paper was accepted or rejected, along with reviews.
• If accepted, you get 1-2 months to make any last changes. Then submit the final “camera-ready.”
• You give a talk about your paper at the conference. The paper is published at the same time.

The publishing process

• Each paper gets 3-5 reviews.
• A review includes a recommendation (reject/accept) and an explanation for the recommendation.
• Some conferences give authors an opportunity to respond to the first set of reviews.
• Papers that exceed a score threshold are discussed in a meeting, where the set of accepted papers is decided.

What does the review process look like?

• Reviewers are experts in the field. Caveat: fields are broad so your paper might be outside of their expertise.
• Reviewing is “blind,” so you won’t know exactly who reviewed your paper. However, you can look at the Call For Papers to see the program committee.
• Reviewers are typically overworked. They have a high review workload and are not paid for this work.

Who are reviewers?

• The basics are novelty, evaluation, and presentation.
• Reviewers will also look for technical correctness.
• Individual reviewers bring their own ideas of which research areas are “hot” vs stale.
• Program committees will consider the fit for the audience.

What do reviewers look for?

1. Feel upset for a few days. Complain to your friends about how your reviews were unfair.
2. Cool off and gain perspective. A paper rejection says nothing about you as a person. Plus, conferences reject good papers; human processes are imperfect.
3. Use the reviews to improve your paper.
4. Resubmit to another conference.

How to handle a rejection

In my experience, all papers are rejected for four reasons. The trick is figuring out which reason(s) apply to your paper.

Reasons for rejection

• Paper is boring.
• Work contains a technical flaw.
• Work is under-developed.
• Paper is hard to read.

Reading between the lines

GOOD LUCK!

CONTACT US

To suggest additional content,�or to let us know that you found this helpful!

felt@chromium.org, kurtthomas@google.com