Paper Writing and Research Strategies

Thomas Breuel

Introduction

Here is a brief reminder of how to write papers and what kinds of papers to write.  Please also see the other materials on our web site on paper structure.

What matters most about a paper is what the reader gets out of it, not how much work you put into it.  That is, after reading your paper, the reader should feel that he knows something new, has a better insight into something, or has a better idea of how to do something.

Each paper you write should stick to one main idea, and that idea should be clear enough that you can formulate it in a single sentence.  If you can't tell someone else what the main contribution of your paper is in a single sentence, you haven't worked it out enough.

Writing papers doesn't need to be all that hard because papers fall into a small number of common categories.  Most of them involve some problem that needs to be solved or some observation that needs to be explained; for the paper to be accepted, it's important that this problem or observation is interesting in the first place.  

Solving problems that aren’t interesting to reviewers won't get you published.  It is therefore important that you keep track of the literature and figure out what topics are likely going to be of interest to reviewers.

In writing your paper, keep in mind that many reviewers are graduate students like yourself.  Think about how you approached your last few paper reviews.  How well did you yourself know the literature?  Did you give the authors the benefit of the doubt when you didn’t understand something?  Did you ask for more experiments?  

Paper Types

Now let's look at common different paper types.  I have given these papers short, catchy names because you should be able to identify clearly what kind of paper you are writing.

"I reviewed the literature about solutions to problem X."  (Literature Review)

A literature review should pick a well-defined research area, something that is neither too large nor too small.  An area might be something in which there are 100-200 research papers total that you need to read.

A literature review is a little bit like an experimental paper.  It usually starts with a question, then looks at data, and then draws its conclusions.  The question it asks is usually of the form “what are the unsolved problems” or “where are there areas for improvement”.  In the conclusions, you should answer these questions.  They form the basis for further research.

In fact, when you start on your thesis, you need to do a good literature review anyway, and you need to incorporate it into your thesis.  If you can get this kind of review published, that's really good.  However, publishable review papers are a lot of work and usually require upwards of a hundred original references to be organized and analyzed. Nevertheless, you should at least aim for a group seminar on your chosen topic.

"I looked at the best ways of solving problem X, benchmarked, and compared them."  (Benchmark Paper)

A benchmarking paper asks the question: “given these standard methods, how do they perform under different conditions?”  It isn’t necessarily for finding “the best” of the methods (usually, there is no uniformly best method), but for identifying tradeoffs between different methods.


A significant component of a benchmarking paper is the datasets.  If there are standard datasets and standard algorithms, and they simply haven’t been benchmarked against each other, then it’s easy to do that.   Otherwise, you may have to create your own benchmark dataset.  This can be a lot of work, but it can also be rewarding: if your dataset is novel and interesting, other people will use it and cite your benchmarks.

This is often a good paper to write at the start of your thesis work: you gain experience with the problem you are trying to solve, you implement algorithms that you need to implement as control experiments later, and you collect and/or create databases you can evaluate on.  Papers like this have the potential of getting highly cited.  They are also conceptually fairly simple and you are almost guaranteed to get a useful result.  However, they are a lot of work to do well.

"I came up with a new/better/interesting way for solving problem X."  (Method Paper)

This is perhaps the most common paper people write; it's usually the kind of paper that takes an engineering view of the world.  You usually need to show experimentally that your method is better than existing methods, at least under some circumstances or on some kinds of data (rarely, a complexity analysis may be sufficient).   Here, "better" may mean one of several things:

Additional useful distinguishing features can be the following, but they are rarely sufficient by themselves:

The required experiments make these kinds of papers similar to benchmarking papers, but there are some differences.  While the goal of a benchmarking paper is to gain insights into existing methods (data sets represent a wide range of conditions, authors are supposed to have no preferences), the goal of the experiments of a "new method" paper is to show that the method is actually better.  But that also means that the reviewers will take a more adversarial view of your paper, meaning, they will try to poke holes into your arguments.  Key questions they will ask are:

Generally, when you write up your paper, you need to think about the list of objections that reviewers might make, and you need to have answers to them.

"I performed observations/measurements on X"  (Research Paper)

This is the more standard scientific research paper; you don’t create something and you don’t invent a new method for doing something, instead you observe and interpret existing systems.  In some way, it is like a benchmark paper.

These papers are, on balance, easier to write than method papers, because it is usually easier to get publishable results out of observations.  There are a lot of opportunities for these kinds of papers in computer vision, pattern recognition, social network analysis, etc.

"I have a new mathematical model/theory for observation X." (Model Paper)

Mathematical modeling or theory is a very different approach from experimental approaches, but the basic rules still apply: your result needs to be novel, interesting, relevant, and useful somehow.  Generally, mathematical models or theories should make predictions.  Some of those predictions, you can compare against existing data, others may suggest new experiments.

"I found a new/different interpretation of observation X." (Response Paper)

These papers are generally responses to the literature, where you read a paper and decide that its conclusions, model, or theory aren't quite right.  These papers tend to be shorter and fairly easy to write (because someone else has already done all the background research).  They may involve experiments illustrating the point you are trying to make.  They are often good papers to write, but they are difficult to plan for because, of course, you don't know when you'll come across a paper you can respond to.

"A new framework/'theory' for X." (Framework Paper)

These papers often don't involve any experiments, theory, or serious review of the literature.  Instead, they provide a view of a field, observations, or data.  If you hit the "Zeitgeist" just right, these kinds of papers can be spectacularly successful and highly cited, in particular if you have already made a name for yourself.  However, most of them end up just languishing at a conference.

Preparation for Ph.D.

Although you may be enrolled in a Ph.D. program, you may first need to learn how to work with the literature and how to conduct experimental research.  At some other universities, this phase is delimited from the actual start of the Ph.D. program by the submission and presentation of a formal Ph.D. proposal.  We don’t have that requirement at our university, but the two phases really still exist.

The previous steps are preparatory for your Ph.D.  You really shouldn’t spend too much time on them because you don’t want to spend too much time on your Ph.D.  At some point, when you have a reasonable handle on how to work with the literature, how to run experiments, and what you want to do, you need to pick a Ph.D. topic.  That’s when your Ph.D. studies usually start.

Ph.D. Research

Actual Ph.D. research, like all research, really means three steps:

  1. familiarizing yourself with the literature
  2. getting familiar with the research methods and tools
  3. conducting actual research

Note that you don’t usually start this process from zero; presumably, you have already done a little bit of work in the area--maybe as part of collaborations or projects--simply to determine that you actually are interested in it.  

Each of these steps should produce some output, in the form of a review, a benchmark, and research contributions.

This division and three step approach are useful, and it’s the way researchers generally approach starting research in a new field.  But it is not formally required and many students end up working on a lot of different projects and ideas and then at some point putting everything together into a thesis.  Each of project implicitly still requires a literature review, benchmarks, and new contributions, but they happen in different orders and perhaps aren’t made explicit in the form of publications.  There is nothing wrong with that if that works for you and if you get a good thesis out of it.  However, if you are limited in the amount of time you have for your Ph.D., if you feel lost or are concerned about completing your Ph.D., then you should consider picking a single topic and focusing on the three steps for research.

Scholarships and Ph.D. Proposals

Some scholarship organizations will require you to submit a Ph.D. proposal in order to get the scholarship.  That is a useful starting point, but it is rarely as detailed as a true Ph.D. proposal and literature review should be (although the more detailed you make it, the less work and uncertainty you will have later).  Furthermore, after getting the scholarship, you may still need to spend some time with seminars, projects, etc. to get more familiar with research methodology.

This means that you shouldn’t assume that your Ph.D. proposal is finished just because you submitted something with the scholarship application.  On the other hand, scholarship agencies are generally fairly flexible about proposals and you can change them later as you learn more about a research area.

Problems You Will Encounter

Of course, in real life, things don't necessarily work as smoothly.  Common problems are:

Other Strategies

Here are some other strategies you need to keep in mind: