Supporting what you say in a paper

Everything you say in a research report must have some kind of evidence to back it up. Different types of disciplines use different kinds of evidence. One of the long time difficulties with computing is that the discipline has so many different influences that computing people get very confused as to what kinds of evidence we use/accept/attend to/work with.

The answer, however, is not to throw the concept of evidence to the winds! All mature disciplines develop what might be called a 'hierarchy of evidence', which categorizes, whether strictly or loosely, types of evidence by how comparatively strong that evidence is considered to be. A fairly common hierarchy of evidence might be:

1. citation (strongest) - citing other research reports published in archival sources
2. closed form proof (we are talking about computationally related fields, after all...)
3. proofs that include monte-carlo/simulation/randomization components
4. proof by analogy
5. proof by argument

So where do case studies, exploratory research, description research, design research, etc., fit in? These techniques are on another dimension. Without them, in a field like computing, there's no way to know whether the proof you just finished is relevant to the real-world problem you were trying to solve. It's tempting to therefore slide them in above analogy, but to do so is to confuse linguistics with rhetoric and narrative.

[see also UnderstandingANewField]