by Steve Draper.
This document consists of notes for psychology department students on how to tackle critical reviews (CRs). Although the points here have been discussed with some colleagues, they do NOT represent an agreed departmental view: just my own view. If in doubt, go with your tutor’s instructions.

It draws on a lot of text from feedback I had written to past students about their critical reviews, exercises developed by Stephany Biello that I have found useful, and some text I had contributed to staff discussions about changes to the critical review requirements.

Although this document focusses primarily on just one kind of writing — CRs — you may find it contains useful pointers for other writing too e.g. maxi projects, exam answers, PhD dissertations. And not just in psychology but in other disciplines: I have certainly found it useful in here to point psychology students to resources about writing meant for students in other disciplines, and the reverse probably applies too. This is because:

Many aspects of good writing apply widely: structure, clarity, etc.
Most important of all is expressing a rational argument supported by reasons and evidence: whether your own evidence (maxi project, a PhD), summarising a topic in an exam answer, or offering additional commentary on others’ work (CRs).
Contents (click to jump to a section)

What this document most still needs
What critical reviews are
Doing a review
Being critical
Types of critical-ity
Sources of criticisms
Practising being critical
Expressing your critical views
Further reading
Achieving first class versus merely ordinary work
Types of CR
A particularly good type of CR
CRs as a learning exercise: The pedagogical rationale
The references: the papers on which the CR is based
Finding the identities of papers
Estimating the probable quality of references
Getting copies of papers
Justifying / reporting on your selection
Producing a plan (early work on a CR)
Essay writing
Planning and expressing the structure
Deciding what your argument is
Deciding on how to group and order your points
Indicating your structure: sections, section titles, and “glue”
Gopen & Swan: Organising each sentence and paragraph so as to be clear
Title
Abstract
Introductions
First person constructions
Citations and indirect citations
Spelling and punctuation
Common spelling problems
Apostrophes
Punctuation
Plain English
Footnotes
Get someone to proof-read your writing
Cover pages, page numbers etc.
Finishing it: Keeping and making copies
Extra points about writing in a CR
Revising a paper or CR
Further / advanced work: where CRs fit into wider perspectives
The rhetorical form of scientific literature
Critical thinking
Argumentation
Further reading
Appendix
Postscript: projects
Collected references
What this document most still needs

This is a “to do” list for improving this document. Help of all kinds welcome: and I’ll update the document.
More worked examples of searches, to help students get into doing their own searching. There is one in the appendix. A set of ten varied examples would be good. However students may still really prefer/need a personal demonstration by a tutor.
What critical reviews are

Critical Reviews (CRs) are essays based on scholarship i.e. on finding and reading the literature on a topic, and adding your own considered arguments and judgements about it. CRs thus involve both reviewing an area, and exercising critical thought and judgement.
It is quite likely that in writing a CR you will become more expert in that particular topic than anyone else in the department, staff or student. In many cases they will be useful to other students, and indeed you would be sensible to ask to read other students’ CRs where they are on topics useful or interesting to you.

It is quite a good idea to write a CR with fellow students as your imagined audience. You may think that writing for staff is quite different from writing for fellow students. Actually, this is much less true than you might imagine. A great drawback of work being set for staff to read is that students can’t help assuming that staff know everything so they needn’t explain much. Not only is this not true (even in a critical review you will probably have things to say the marker does not in fact know), but in any case most marks in the end go for demonstrating understanding. If you write clearly enough so that a fellow student both understands and is interested by your paper, this is a good indication that it will look good to a staff member.

Doing a review

Writing a review may involve:
Summarising each paper. This of course reveals what you think is important about it (e.g. results, conclusions, methods, ..).
Summarising the topic or field: not just individual papers, but the field as a whole e.g. commenting on the coverage of the topic by published work.
Offering value judgements i.e. being critical.
Because of that, use of citations is somewhat different than in other kinds of writing (see below).

Being critical

There is more than one way of being critical. Here are some of them. Which you adopt should have more to do with the topic than with your whim.
Types of critical-ity

Write out explicit definitions of central technical terms. Very often, simply writing out definitions of the key terms (e.g. in the introduction) will lead you directly into making some criticisms e.g. of studies that used different definitions for no good reason, that included subjects who don’t fit the usual definition, etc. And of course it can lead to criticising papers that failed to be explicit about how they defined crucial terms. It can also lead directly into defining how you selected your papers.
Criticise the technique used. Mount a critique of the method of each paper, and distinguish serious from marginal criticisms of the method.
Could or should techniques not used have been applied? Offer constructive suggestions about better methods or approaches. Discuss what you would do to do better research in the area i.e. make constructive suggestions about method. Consider whether techniques from other areas could be imported into this one. If you think one line of work has led to no clear conclusion, what would you recommend? Simply better detailed methods but the same kind of study, or a different kind of study? Or of course you might decide that, in a given area, the issues have been adequately resolved and there is no great need for further work. But giving your own view (and reasons for it) in addition to what published papers say, adds value.
Criticise for failure to address other established approaches to the problem. In some areas, there may be whole approaches with different research methods that should be considered, yet many papers stay within one approach and fail even to mention the other. One standard approach to criticism is to comment on such one-sidedness. One example would be any paper that discusses nature (genetics) without nurture (context and development) or vice versa. Another example would be the literature on prescribing Ritalin for hyperactive children: some see it as a social issue for which drugs are largely irrelevant, others as a neurophysiological issue independent of social issues — both of these could be criticised relative to others that address both aspects. A third example is work on emotions which is largely divided between three schools: physiological (emotions are bodily changes), social (emotions are communicative signals between people), and cognitive (emotions are internal processes for shifting attention to crucial life-goals when these are disturbed); little work really deals with all three aspects.
Criticise with respect to independent criteria for the research i.e. criteria clearly not considered by the researchers. In other words, discuss with respect to wider basic considerations (“reality checks”, or going back to first principles). For example in the case of TV violence and viewing, there are at least two wider issues. One is the effect of TV viewing on all other behaviours. Do today’s soap operas make people more spiteful? Do romantic novels make women reject real men and wait for prince charming to appear? etc. If we had evidence of other kinds of effects of TV, surely that would be relevant to violence. A second is to consider (all) other causes of violence. Thirdly, and related to this, I have the impression that in adults TV viewing goes up with age, but violence goes down. If so, that would seem to show a priori that TV violence reduces real violence — or at least mean that strong age controls are required in any studies. Fourthly, what is the role of stories (not just TV) in general? Are they the source of training i.e. do we use them to practise what we do; or on the contrary are they the place that everything unrealistic is “put”. Without a theory of what stories are for, it seems unlikely we could understand an “effect” of one kind of story. Cracker is a particularly interesting case. Full of violence, yet its main effect seems to have been to have created a huge increase in the number of students wanting to be forensic psychologists, not murderers. (And I mean huge: thousands of percent increase within a year or two.) The point of considering other effects of TV is at least twofold: we might understand something of the mechanisms and conditions for causality by studying clear cases; and it gives a comparison which would allow a much more confident interpretation of null effects in other cases.
Anyway, a really good review not only reviews a field in its own terms, but questions whether that line of work and type of study is in fact sensible for answering the questions of interest.

Sources of criticisms

You can drum up criticisms from several kinds of source, including:
The list of types of criticism in the preceding section
Self-criticisms in the paper being reviewed itself
Criticisms in other papers you read (obviously other published reviews of the topic would be very likely to give you criticisms you could then apply to the papers you review).
Implicit criticism in your selection of papers e.g. if you review papers on two sides of a debate, then even if they never mentioned each other explicitly (in philosophy this is apparently quite common), your selection and juxtaposition of papers is itself a criticism, which you could also bring out explicitly (e.g. “paper 2 is clearly unable to deal with the issues raised in paper 1″).
Practising being critical

Stephany Biello has some warmup exercises, focussing on generating criticisms of published papers:
Design Critique Ex.1 (Highly structured worksheet)
Design Critique Ex.2 (Only a few pointers)
Critique Ex.3 (The topic of hypnosis)
Another kind of exercise is to read and discuss published critical reviews.
Another kind is to critique one’s own (or each others’) first drafts of papers or essays: trying to predict the tutor’s criticisms.
An exercise I use myself and have found really useful, is reciprocal critiquing: each student in a group contributes their last critical review, makes copies for everyone, and for everyone to comment on, and preferably to say what marks they would give for, each CR. This exercise is discussed in more detail here. It has many benefits, but students say it is particularly good at demonstrating to them the great differences in approach, all good, that other students take.
Finally, though the department has never done this, it has been suggested that a good idea would be for staff to develop standard critiques, and present them to students, of a few very salient topics we are all likely to get asked about sometime e.g. genetic claims about behaviour, parapsychology.
Expressing your critical views

Make it clear in the CR which bits of criticism are yours and which come from the literature. If they are your own, you get credit for original analysis. If the criticism comes from the articles, you may well get credit for selecting papers to discuss on the basis of a critical structure you designed of presenting contradictions and debates in the literature, provided you also say this is why you selected these articles (a point I mention elsewhere). This will also mean that you don’t present a personal analysis, but you could make it clear why there was no room or need for one because the literature is already full of criticism without resolution.
“Remember, ‘To copy from one work is plagiary, to copy from two is research.’ (That quotation was researched.) ”
(But I myself copied this without checking from a web page http://www.rdd-phru.cam.ac.uk/east/phweb.html .)

Further reading

Abercrombie,M.L.J. (1960) The anatomy of judgement: An investigation into the processes of perception and reasoning (Free association books: London). This has many things about learning to read critically.
First class versus merely ordinary quality

Besides what is specifically wanted in a critical review, here’s a word on quality in student work in general. What examiners really want, besides evidence you know something, is evidence that you understand it. If they see the exact words you were given on the course, they can see your memory for words is good but wonder if you know what they mean. If they see reasonable paraphrases, but the structure of the ideas and their presentation is merely a close mirror of what you were given, they know you understood the words sentence by sentence, but wonder if you actually grasp their implications and can use the ideas in any way. Thus in an exam, students who reproduce what they know without adapting it to the specific question asked raise this question in examiner’s minds. What is more convincing is if the student applies or restructures the conceptual material because that shows understanding at a deeper level. A completely original conclusion would be very convincing (provided that as soon as it were put forward, its reasonableness could be recognised), but that is rare because the examiners have usually been thinking around the area for a lot longer than the student. However some novel rearrangement of the given materials is within anyone’s reach, given a grasp of the elements and their meaning. After all, most of the sentences you speak have never been spoken by anyone before, and this facility at rearrangement for each new situation shows your mastery of English vocabulary and syntax. At a deeper academic level, reorganising the presentation of a topic you know similarly demonstrates you have thought about it, and understand it.
In the end, these are only the same standards you yourself tacitly use in judging whether someone understands you. If you are chatting in a pub about, say, what a pain elder sisters are, then someone who keeps saying “Yes, uh huh” is agreeable; someone who says “My sister is just the same” is better, and better still if they give details of their own experience; but possibly the most convincing could be someone who disagrees with you, while showing they see why you said what you did e.g. “well, my sister was a pain too when I had my hair streaked, but when my boyfriend dumped me she was really supportive”. This kind of thing shows understanding both of what you said and of your reasons for saying it, but goes further by relating your original idea to a wider range of things. Applying this in an academic essay, using different words, organising material into a different structure or order, adapting your knowledge to the question rather than reproducing your notes, assimilating non-standard reading material, illustrating it with examples that didn’t come from the lectures — these are some of the ways you can do this. The course handbook stresses additional reading; I myself find novel, but apt, examples the most convincing, especially if they come from a student’s own experience, as it shows they have been thinking about the material and what it might mean in situations not directly discussed in the lectures.

What might these general points about quality mean in a critical review?

A review, without critical thinking, would summarise the contents of the selected papers. A competent but low quality piece would summarise each in turn, using only the issues raised by the authors and used to structure their papers. A high quality piece might start by deciding on independent grounds how the papers should be summarised (e.g. there are three different experimental methods used in the field, and papers will be grouped under each and described in relation to them), and then structure the review on those independent grounds, mentioning each paper (perhaps in several places) in those terms. This would show that the student had done independent thinking about the issue, and could adapt the material to their own approach. Such an essay or exam answer might have no specifically critical component, but would demonstrate that the student could invent and impose a structure on the material that wasn’t directly suggested by the original authors.

Being critical is an additional consideration. Some alternative ways of being critical were discussed above, but in general terms, critical thought is about explicitly discussing rival arguments and conclusions together with the reasons for choosing one over another, and their degrees of strength and weakness. In terms of quality, though, there is the same range. A relatively low quality, even though competent, critical review will simply reflect without re-structuring the arguments and criticisms directly given in the literature reviewed. At the other end, a high quality critical review will rearrange into its own structure the raw materials of critical thought: possible conclusions, and evidence. The main point of this CR exercise is not merely to get you to select and repeat criticisms — although that is an important extension compared to most of your other work — but to gain practice in reorganising arguments — the elements of critical thought — into new but valid patterns.

Because of the great range within psychology, the most frequently appropriate forms of critical thought tend to be different in different topics. At the sociology end, you get lots of explicit debate, and a low quality CR will just summarise the argument patterns given in the articles reviewed, while a high quality one must seek a restructuring of these e.g. by bringing in another viewpoint. At the physiological end, criticisms are likely to be, not of grand conclusions, but of empirical methods i.e. essentially criticisms of whether evidence is good enough. Here, a low quality CR may merely review without criticism, or limit itself to summarising self-criticisms in articles. A high quality CR may begin by deciding what standards should apply to the topic, briefly arguing why these are appropriate, and then systematically measure each paper against them. This often results in comments about details omitted by authors, and perhaps to suspicions about why they were omitted.

Types of CR

There are many kinds of CR, and no recipe fits more than one kind. A lot of it depends on the topic, but your choice of approach may also be a considerable factor. For instance:
An exhaustive review of the current literature on one specific topic e.g. the use of Ritalin on “hyperactive” children.
In a field like education, there is a very large but mainly low quality literature, because many papers are basically field reports of uncontrolled interventions, like natural history observations. It may be interesting to do a high level survey of a large number of such papers. (One of the great features of Laurillard’s book Rethinking university teaching is that several of her chapters are in effect giant reviews of whole fields of literature, telling you how to think about those fields without having to read them. For instance, the field of instructional design is summed up in a section starting on p.72, and Intelligent Tutoring Systems in the following section.)
At the other extreme, although unusual, a detailed criticism of a single (key) paper can be interesting. Indeed, journals sometimes feature such critiques.
In a very new research field, criticism may well all be tentative, because no-one (including you) yet knows if the methods are sensible or what the alternatives are.
On the other hand in a well established field, methods may be well advanced, and the best criticism may be applying old standard criticisms to new papers.
A completely different tack would be to attempt a synthesis: your conclusions about how papers which do not discuss each other should be combined to draw a conclusion. This might not qualify as criticism, yet it certainly is a review and involves original thinking.
But for me the ideal review, although too ambitious for most of these CR exercises, invents and presents a novel framework for relating existing papers i.e. it offers a new way to think about and structure a topic. An outstanding example of this was Wenger’s book. Most PhD theses yield one or two papers, and a considerable minority yield a book; but Wenger made a well-regarded book just out of his literature review chapter (Wenger,E. (1987) “Artificial intelligence and tutoring systems”).
A particularly good type of CR

Putting together the last two subsections (on first class quality and types of CR), we arrive at the kind of CR I have most admired from my students. The lowest approach to a CR just summarises (i.e. reviews) the contents of some papers. A more adequate approach also collects and presents criticisms, but is still essentially reacting to the papers one by one and bit by bit. But what I most admire is when the CR as a whole has a clear structure invented by the student, not just directly reflecting the papers being reviewed. This is like the last “type” of CR listed in the previous subsection.
Of course this cannot be planned in advance or done by recipe. But I’ve seen a couple like this in the last few years done by my students (both boys, for those interested in gender differences). One decided to review the literature on personality and binge drinking, and noticed on his first pass through the papers that most studies had only reported on one or two of the main personality dimensions. This became his chief criticism and also offered a structure for his whole review: beginning with a summary of personality theory, and discussing the studies under each of the 5 personality dimensions in turn. Another concerned CBT therapies for certain mental illnesses. This student began with an introduction about the relevant general issues of clinical study methods, including references to papers on what is desirable. He then applied those standards to the particular papers he was reviewing: again imposing a pre-selected set of standards which he had independently justified and laid out in advance.

This kind of approach usually cannot be decided on until after reading the set of papers through for the first time, making notes of points, and eventually coming to a decision about the most important overall issue. By imposing their own standards and structure, these students demonstrated they were not just taking the literature uncritically; and this gave their whole CRs a coherence that the students, not the papers, had come up with.

CRs as a learning exercise: The pedagogical rationale

We, as teachers, should be able to write out a rationale for every activity we organise for students; and if we have one we should make it available. Actually we haven’t got one written out and agreed for CRs, but here is an impromptu one by me (this could be a useful critical discussion to have with your tutor some time).
Why are CRs valuable exercises?

CRs exercise critical thinking, which is useful not only within psychology, but probably in most of the jobs you might get later i.e. it is hopefully a transferable, marketable, intellectual skill. (See this section of the document on the value of our degrees.)
It is a significant skill within science in general and psychology in particular. Thus developing it is both part of learning the discipline of psychology, and is part of research-led teaching (a university aim) since learning to use the literature critically as opposed to accepting it passively as a consumer could be seen as exactly the difference between a research-led and a commodity-processing view of learning.
You can’t write a good paper without constantly thinking about how you might criticise it if you were the reader not the author.
In reading the literature you have to decide how much, and what, credence to give each paper you read.
CRs entail direct practice at accessing, searching, and using the literature: an important skill in its own right.
CRs exercise essay writing. Both a generic skill in itself, and one demanded by our exams. There is little other opportunity for you to get detailed feedback on your essays in levels 3 and 4.
It is student-directed work i.e. the learner gets to choose the topic and decide how to develop it. This is generally (and I believe rightly) thought to be good for motivation by connecting with a student’s own prior interests (instead of only with curricular requirements), and also is good by demanding and allowing a lot of student planning of their own work: an important skill.
Currently this department is good at turning out students with 2.1 degrees, not as good as it should be at turning out Firsts. The difference is a lot to do with originality, critical thinking, and adopting and justifying a personal perspective on the material, as opposed merely to reproducing a neutral summary of what you were given with no “added value” from extra processing. A CR differs from a standard essay in not just permitting but positively requiring such an approach. Thus it is the best activity we currently support for practising the skill of doing not just good but excellent work.
Finally, the document on the value of our degrees suggests you may like to offer prospective employers a copy of your final year maxi project and/or CR as a sample of your work: in fact a sample of your research, thinking, writing, and word processing skills all together.
The references: the papers on which the CR is based

There are several parts to this. WWW links to help with all these are collected on a “launchpad” page.
Finding the identities of papers

Start by using the latest edition of the first year textbook (Bernstein or Gleitman): this has a very large set of up to date references on a wide set of topics. If you can find any mention of something on or near your topic in the book, follow the references they mention. Similarly use the books mentioned in the course handbook for the lectures nearest your topic. Find the shelf in the library with the books closest to your topic and have a browse in them.
Discover the best two (or so) journals for your topic. You probably need to ask the member of staff most expert in your topic: this should only take them a minute. Then if at all possible, go to a library with this journal and spend 10 minutes reading through the title indices (one in each bound volume) for the last five years to see whether and how your topic gets mentioned in titles of papers. You may find some papers directly; you may be able to pick up the words used in your topic for use in online database searches etc. The names of the best journals will also be useful for estimating source quality later.
Word of mouth. Most valuable of all, if you can get it, is word of mouth recommendations not only of particular papers and books, but of key authors and researchers, journals, and laboratories or other sites where some of the key research is done. Finding even one of these will probably lead you to the others. Your tutor will only be able to do this for you if they happen to be expert in your topic. But quite a slender lead can turn out well (see the appendix for an example).
The most important research sites. If you happen to know a site (lab, unit, department) where some of the key research in your topic is done, find it on the web and with luck they will have a web page listing their papers.
(For example you might go: Autism .. Baron-Cohen

Cambridge University
Psychology Dept.
Baron-Cohen
Autism Research Centre
Search the library index! (by title, by subject classification, ..)
Online database searches of journal articles
Try a web search. You never know what you may find, and it’s definitely worth spending a few minutes on it. Find a computer, fire up the web browser, get a search engine (below are two of the various search engines). Type into their query box the title of your review; and/or some keywords. For web searches, use long search strings. (For more on web searching, see Stephany’s introduction.)

http://www.google.co.uk/

http://scholar.google.co.uk/

Early on make a determined effort to find the nearest published review of the topic you are working on (e.g. search for papers with “review” in their title, or of course in journals that have “review” in their title such as Psychological Review). You should always discuss this in your critical review, and decide and state how your review is different. If you find one, it will be an excellent source of references. If you don’t, do you REALLY believe you are the first person ever to do a review on this topic? If you can find another student who has done a review in a related area, get hold of it, read it, refer to it, and again state how yours differs. It’s always easy to make yours different if you find out early on; it is never convincing to say you didn’t know about it if your tutor finds out about it later (were you incompetent or dishonest? not a choice you want to offer).
Estimating the probable quality of references

Particularly when getting a paper will cost you time and/or money, it is worth estimating how likely it is that a paper is valuable for you. The first estimate you can get is either to use an online database to look at the number of times that paper has been cited, or use a citation report to see how often that journal in general is cited. If it has dozens of citations, you definitely want to see it; if not, then you need another reason to be sure it is worth it. Another, but only one, measure of this is the quality of journal or book in which it appears. (Another is how famous the author is: for really great researchers it is interesting to read even their worst or most peculiar stuff — although perhaps not for a critical review.) One important standard is whether a journal (or conference proceedings) is peer-reviewed i.e. the papers are only published if they pass reviewers. Perhaps a better way is to look up the journal quality from the point of view of citation ratings: use the “How important is a journal (in terms of citation)? ” link on the launchpad, where there are some extra help instructions.
On the WWW, there are now some peer-reviewed journals online. Apart from that, perhaps an important pointer is whether the document is written by someone at a university or not (e.g. is publicity for a company). Even then, only occasionally will a web document be worth reviewing itself: but you may well find web documents with useful new references that you then retrieve for yourself. (For an example, see the appendix.)

Getting copies of papers

Glasgow university library
Other libraries you can access e.g.
Strathclyde university library
Glasgow Caledonian library
List of other academic libraries in Glasgow
Other academic libraries in Scotland
Inter-library loans (but only exceptionally: they are slow, and cost the department money).
Justifying / reporting on your selection

Search for the nearest published critical review if any, cite it, and explicitly if briefly discuss how your review is differentiated from it (by being more recent, by a different type of analysis, ….).
Include an explicit justification for your selection of papers. On the one hand you may get credit for having good reasons. On the other hand, this often means giving a little sketch of the area as a whole and where your review topic fits into it, which improves the quality of the CR. This could be in the introduction, or elsewhere. But you should explain why you chose what you chose. For instance, you may well have picked papers because they were in contrast or explicitly criticised each other, thus being “critical” at the strategic level of selecting papers to review. But it’s also OK to say you just picked these three as representative, or because there wasn’t space to cover more; but even better to say that you found 10 and decided after skimming all 10 that these four seemed the most interesting or varied. This explains your CR, fends off possible criticisms of “why didn’t she review X?”, and tells the reader useful information about the amount of papers existing on the topic. In fact, it would be even better to indicate what other related topics are and are not there in the literature: it is surprising what doesn’t get worked on (in fact this is itself often a useful kind of criticism).

Keep a log, and include it as a short appendix to the critical review, that states:

How many, if any, inter-library loans you got
How many papers you looked at (skimmed or read)
How many of these were actually useful to you
Producing a plan (early work on a CR)

As a tutor I like to see early on, in email or on paper, about one page of notes on the plan for a critical review. The detailed content of each of the points below can, and in fact should, change repeatedly as you do the review, but it helps both student and tutor to have your current answers to them at any one time.
A title. Actually I often write out up to five possible titles for any paper I’m writing, and every time I work on it I add or delete some: only takes a minute a time, but titles are important and a way for me to review each time what my paper REALLY is about.
A private note on why you are interested in this / what your private agenda is. You don’t put this in the final CR; but it always helps to be clear about what you yourself hope to get out of any piece of work.
Your criteria for picking the articles you pick to review. It is NOT impressive just to pick any five randomly. You should say why.
What the nearest published (critical) review on the topic is. Search explicitly for them; and find them. (E.g. search for papers with “review” in their title, or of course in journals that have “review” in their title such as Psychological Review.) You can refer to them; but you need a story about how yours is different (newer, different slant, …). But pretending you are the only person ever to review the area is not plausible.
Check all the search methods above, and write a sentence or two on what you found from each and whether it looks useful or not. So make sure, besides searching online databases, you have searched the library catalogue, the WWW, Bernstein textbook, etc.
Start the log of papers looked at (ordered, used).
Your current plan about how to be critical on your topic. E.g. criticise their methods, criticise their narrow-mindedness on theory (e.g. considering only biology not social aspects, or whatever) …
Essay writing

Here are some points that apply to any essay, including critical reviews (and indeed to PhD dissertations, web documents written for students, …). The rest of this major section contains what I feel is most important to tell the kinds of students I deal with (including PhD students).
However, even though you may well just be reading this document to get a quick idea about what “critical reviews” are, this section is about writing in general. Though you probably haven’t realised this yet, writing well is likely to be something you will be striving to do for the rest of your life, even in careers where that isn’t obvious. As a postgrad, I thought of doing research as about having good ideas: but in fact being successful depends on the quality of my writing both in papers and in grant applications. My sister works for a charity, organising support of various kinds for children and families in deep trouble. Now she finds this career is not just about understanding children: writing funding applications (to pay for the staff under her) is a cruical skill, on which ultimately any support the children get depends. Giving impromptu talks to billionaires at charity fund raisers is also important, and also depends upon expressing herself clearly. Or, delving again into my family history, my grandfather wrote a moving document when his first wife died in childbirth, as a testament for his child (who survived) to read in later years. Clearly such personally important actions would be undermined if, when you most need it, your powers of writing are limited. So consider how to develop your skills. One (only one) way is to make time at some point to read through books on writing such as Gowers’ The complete plain words.

(As an alternative to reading this section, you could look at other advice, such as Alice Jenkins’ advice on writing English essays (4 pages and pointers to other similar advice). She points to this more elaborate essay guide for literature students.

A paper on writing scientific papers (9 pages), recommended by Lars Muckli, probably highly relevant to writing your maxi projects, and of some relevance to CRs.
Gopen, G.D. & Swan, J.A. (1990) “The Science of Scientific Writing” American Scientist (Nov-Dec 1990), Volume 78, pp.550-558.

Highly recommended by an ex-student of mine is this guidance on writing for anthropology students (33 pages). Also of possible interest: How to write a philosophy essay (18 pages).)
How to write a philosophy essay (1 page).)

Planning and expressing the structure

There are really four stages here: collecting material for the essay, deciding what your main message and argument really are, deciding what structure to adopt in order to communicate that, and indicating your structure to the reader. The first is the result of the issues and activities discussed above (e.g. selecting, reading, and thinking), the middle two are about “planning” your writing and perhaps writing an essay plan, the last about making sure your plan is clear to the reader.
Deciding what your argument is

Thus you will probably at this point have a collection of points and facts you think of some value: the raw material for the essay; not a plan, but the elements to fit into the plan. Some students will now realise they have too much material to fit within the size limit. They must decide what is important to include, and what to leave out. The way to do that selection is to do what everyone must do anyway: decide what the overall point or conclusion is, what are the points or evidence that most directly and strongly support those conclusions, and so on. When you know what the main point or points are, then you can decide what is most important for supporting those conclusions.
If this is proving difficult for you, one technique I find useful in my own writing is to put away my notes, and try to speak a 60 second version. For example grab a friend, preferably one who doesn’t know about your topic and isn’t all that interested, and tell them what your essay is going to say. You automatically make it brief to keep their attention, and mention only the most important things without the details. When you hear yourself give the summary, that tells you what for you is the most important point; and hence how to organise your essay. E.g. “My CR is about autism, and the main feature of this field is all the different theories that don’t really fit together, yet all of them seem to have some support.” If you hear yourself say that, then you probably want an essay with one section for each of the major different theories, and a concluding discussion pointing out how they conflict. Or you might have found yourself saying “My CR is on autism, and although there are various theories, I’m just concentrating on the claim that it comes from a specific neurophysiological deficit. There’s a few papers on this, and I’m going to focus on how strong the evidence they present really is. There are really two classes of problem here: firstly the evidence for the deficit is scanty and might be questioned at least until more studies are done, and secondly it is hard to see how all the symptoms and consequences can really be the effect of a single deficit when they vary so much from case to case.” Here the essay might take each of a few selected papers in turn, and apply the basic criticisms repeatedly to each.

Deciding on how to group and order your points

Now you have decided what your main message is, and so know how to decide what is important to include, you must decide on the order and structure of the points you make. Writing is inherently a single unbranching sequence, but the logic of any argument is about grouping (e.g. where several points all support one conclusion) and subgrouping; and often any one point could belong (logically) to several groups. Thus in deciding the detailed structure of your essay you are making many decisions, some of them for strong reasons, some for weak reasons. E.g. a conclusion comparing theories has to go after the main sections because it needs to refer back to all of them; but if you have one section for each of three main theories there may be no strong reason for which order those three go in, although obviously all the points about one theory belong together in that one section. Again, you could decide to have a main section for each of several papers you critique, repeating some standard critical issues in each section; or else you might have one section per critical issue, repeatedly discussing each paper with respect to one issue at a time. Both are logical, but you can only adopt one of these schemes.
Your essay plan (whether written or mental) describes both the grouping and the sequence of your points. Your final essay can’t avoid displaying the sequence, but by itself this could just be an enjoyable experience as each sentence slips past and the reader bounces from point to point. However the more you get the structure clear in your head the better organised the essay will be, and the more you convey that structure to the reader, the more likely it is that they will understand how the parts hang together, be able to remember your argument afterwards, and give you credit for clarity of thought.

Indicating your structure: sections, section titles, and “glue”

It is a good idea to divide your essay into sections, each with a clear title (e.g. Introduction, Conclusion, …). Unless you are unusual and read all papers from start to finish without exception, you will already know that these are helpful to you when you read others’ writing; they allow jumping to and fro to find what you want, they tell you what to expect in each section, and so on. They are also quite helpful in writing, as they represent the plan you should be making of the structure of the paper. So do it.
You may sometimes consider not only using sections, but several different levels of headings, which you might or might not actually number, e.g. section 2 Main review; section 2.1 The paper by Burton; section 2.1.2 lack of a valid control group.

The argument against using sections is if you are writing a smoothly flowing story where each little point (a paragraph) leads to the next with no need to tell the reader where they are going, or to resummarise at the end, or to make any links other than to the paragraph before and after the current one. In all other cases, sections help, by allowing a way to represent a hierarchy or grouping, rather than just a sequence with everything at the same level, and unrelated except to its two neighbours. But even if you do have a smooth story, it isn’t hard to divide it into sections, even if these seem not very important.

The more general principle is to make sure the reader knows what every aspect of your plan is and furthermore the reason for it i.e. why, explaining wherever necessary either in the introduction (saying what comes later) or in “glue” sentences at the start of a section or both. Section titles, if clear, may well in effect express what the essay plan is (e.g. if each mentions a different paper, it will be clear you are critiquing each of a set of papers in turn), but you might want to say why you chose this structure (e.g. in the introduction, say “this review discusses each paper in turn since the critical issues are somewhat different for each of these papers”). Tell the reader how and why you selected the papers (e.g. “6 papers were selected on the basis of being recent, published in good journals, and being easily available. The abstracts of a much larger set of related papers were looked at, but do not suggest that different issues are raised by them”). Additionally, explain to the reader what your critical strategy was e.g. “this review is mainly based on an explicit controversy in the literature” or “this review takes a methodological framework common to medical studies in general, as discussed in Pocock (1993), and applies it to the selected papers which, as will be seen, do not measure up well to these standard criteria”, or “the few papers in this area are critiqued partly on general grounds and partly by taking each point any of the authors themselves raise in discussion and applying them systematically to all the papers”.

Gopen & Swan: Organising each sentence and paragraph so as to be clear

This section deals with the intermediate level of writing between details such as spelling (see sections below), and how to organise the overall structure of the whole CR (see sections above). The test is: can others understand your writing easily? If so, then you may not need advice here, and if not, then revising the paragraphs they complain about will probably get you through.
In my experience, our psychology students don’t often have great problems here, so if you are just reading this document to get started on critical reviews, then you probably don’t need to read this section. However if you are interested in advice on writing as a whole, particularly advanced advice, then you may want to check this out.

This section is basically about a paper by Gopen & Swan (1990) (reference at the end of this document including a direct link to a copy), that was recommended to me by Lars Muckli. What is fascinating about it is that they give several examples of writing for scientific journals that don’t have problems with spelling or grammar but left me feeling it must just be too technical for me to understand. They then demonstrate that in fact it was bad writing, but I wasn’t perceptive enough to see that; and they offer an analysis. However I’m not convinced that their paper offers a practical approach I can use effectively to solve these problems: but they do have both a real problem and a theoretical analysis that I don’t have.

Although its examples are all drawn from highly technical bits of science papers, the principles are general, and address both the structure of sentences and the connection between them i.e., roughly speaking, how to organise paragraphs. The paper is particularly good at analysing why some sentences make you confused or uneasy although you can’t put your finger on why; and at explaining what is wrong with the simplistic rules you sometimes see about how short sentences are good, long are bad; active verbs good, passive ones bad. Most often, too, as they show, once you have rewritten the sentences to be easier to understand, then it becomes clear that important bits of information were entirely missing and need to be added. If you want a master class on this level of writing: this is the paper (9 pages long) to study.

Their general theory

Readers (not only authors) affect the meaning and impact and effect of text by interpretation: in particular, by their expectations, which are set partly by context.
Expectation has a significant impact at several scales:
The structure of the paper e.g. into sections
The scientific content: what was the experimental structure? how many subjects? etc.
The topic/predicate (or topic-stress position) structure of sentences (basically, old information, and the new information that is the point of the sentence).
The subject-verb basic structure of a sentence
Authors need to be constantly aware of all these expectations their readers have, and either to satisfy them or else to manage them by clearly signalling how they should be modified.

The paper also demonstrates that the principles apply as much to the structure and layout of (data) tables as to the prose in the main text.

Practical procedures

Given their demonstration that this issue can lead to serious problems in a text, that need to be corrected, how can this be done? Gopen & Swan in effect only discuss one approach, but there are (it seems to me) three quite different ones.
Trial and error. I.e. as a writer, have no explicit theory like theirs about what good writing is, but generate alternative phrasings repeatedly until it looks right and you, and your friends, judge it is now clear when read. This approach has no theory or conscious method for generating drafts, and no explicit theory of how to correct them, just a focus on readers’ reactions.
The detailed pre-planning approach. Plan and re-plan the detailed structure of the essay focussing all the time on its “logical skeleton”: i.e. on the logic of the argument; organise everything around this argument; and use lots of “glue” text to connect the parts and tell the reader what relationship they have to each other. This is what I tend to do, and is essentially using essay planning extensively and down to quite a fine level: of every substantive point that is mentioned, even though there can be more than one of these in a single sentence. This will deal with all their problems, but by planning and not by correction. However the literature on writing makes it clear that, whatever teachers say, creating a plan first is only a natural way to write for about half the people: so perhaps it is this large set of people who might find the Gopen & Swan approach important in practice. This approach focuses on a method of producing good versions, not correcting them.
The editorial approach: the Gopen & Swan way. Analyse and correct drafts, using their explicit theories, and their seven rules or rather principles (given in the next subsection) to critique the current draft. This approach is about how to correct versions. I can’t help feeling that this method is much more appropriate for an editor trying to improve someone else’s text than it is for an author trying to improve their own writing method.
In the end, good writing probably uses all three methods to some extent. A writer who doesn’t expect to read over and revise their work — and have others do this too — is very unlikely to produce optimum results (A). A writer who doesn’t plan when writing may never sort out a clear structure. I say this, because in the more involved Gopen & Swan examples, their rewriting leads to a stage where it becomes clear there is missing material — i.e. that re-planning the content is now essential (B).

Where do good plans come from? I, at least, find that blurting out a version is the only way I have of discovering what I have to say, and what my real message is — whether that “blurting” is a first draft full of the problems Gopen & Swan identify, a rough “plan” that lets me see those problems before I even attempt complete sentences, or a quick summary to a friend when I’m still struggling with finding a good plan. This is really the “bottom up” approach to writing where you (I) write first, and work out the plan and message afterwards. Gopen & Swan (C) seem to be analysing what happens when you have done this, but not gone on to revision.

Their seven principles

Here I reproduce their seven principles, but to understand them you may have to read the paper. The over-arching principle is about managing readers’ expectations through some key aspects of sentence structure. “None of these reader-expectation principles should be considered ‘rules’.”
Follow a grammatical subject as soon as possible with its verb.
Place in the stress position the “new information” you want the reader to emphasize.
Place the person or thing whose “story” a sentence is telling at the beginning of the sentence, in the topic position.
Place appropriate “old information” (material already stated in the discourse) in the topic position for linkage backward and contextualization forward.
Articulate the action of every clause or sentence in its verb.
In general, provide context for your reader before asking that reader to consider anything new.
In general, try to ensure that the relative emphases of the substance coincide with the relative expectations for emphasis raised by the structure.
Title

The title should ideally be 6 words or less. It should tell people what is in the essay. The ideal title should (in order of importance):
Describe accurately what is in the essay
Be 6 words or less long
Be witty and memorable.

This is almost always impossible, but it is what to aim for. Note that a title is used, and should be designed, in order to pick out the item from a particular context. In a book on educational evaluation, my chapter’s title does not need to say “education” or “evaluation”, but in a general education journal it should indeed say “evaluation”. Your CR will/should have “critical review” on the cover page anyway, so it probably does not need “critical” or “review” in the title; whereas if it were published in a journal, it probably would need “review” to distinguish it from the surrounding papers that are reporting new research.

Abstract

Abstracts are virtually always useful on documents (though they may be called “executive summaries” or something else on company reports, university strategy documents, etc.).
The first thing you need to know (or decide) is how many words. The hard part of writing an abstract is to get it to fit the word limit, and this varies wildly in different cases: from 150 words to 2,000 word “extended abstracts” is the range I myself have had demanded of me. If I were imposing a limit, I might specify 250 words for a level 3 CR, and 450 words for the level 4 CR.

An abstract MUST summarise the whole paper, and it must be short. Because of this, it cannot also introduce the topic carefully in all other ways. Its implicit aim is to tell the reader whether they want to read the paper, so you want to pack it with things that will attract suitable readers, but if it leaves many questions of detail unanswered, that is fine: they will be answered in the main paper. The most common error is for the abstract just to be a general introduction for the reader to read first. On the contrary, abstracts may be read without the paper, and the paper may be read without the abstract; they should be written for this as independent standalone items.

So a reasonable approach in constructing an abstract is a statement of the topic or area, a summary of the main points (you could even begin with stringing together the section titles), and a summary of your conclusion. Bear in mind that part of how readers select papers is by type or method: so for an experimental study mentioning the number of subjects may be a good idea in areas where many papers have too few subjects to be worth reading, but not in other areas, where saying that your subjects were selected in the workplace and were not students may be what picks your study out. For a CR, if your title doesn’t have the words “critical review” in it, then putting them in the first sentence of the abstract is probably a good idea e.g. “A critical review centering on four papers on the topic of Ritalin abuse by overworked child minders …”; or “44 papers are briefly surveyed and grouped into three themes to provide an overview of the heavily researched area of …”.

A good tactic I sometimes follow for writing abstracts is to start a paper by scribbling down an abstract before starting on the paper (as a kind of plan for myself); and then returning at the end to change it to reflect the paper as it turned out. But writing it at the end is also fine. You probably can’t write it properly before you have finished the paper.

The most common fault I see in abstracts is having them introduce the topic, but not get on to saying what the paper says and concludes.

A tricky issue is overlap of the abstract with the rest of the paper, especially the introduction. Many readers will have just read the abstract when they start reading the introduction, so if you repeat the wording they will be a bit bored and irritated with you; but some will not have read the abstract so you must repeat all the information elsewhere. I frequently make up an abstract, at least to start with, by pasting together sentences from elsewhere, but try to ensure at least that no two consecutive sentences are the same in the abstract and another section.

Introductions

In my opinion, introductions have more than one function. The general one is to introduce the story “Once upon a time ….”. For an academic piece, as opposed to a novel, this can be expanded as:
State what the topic is (and is not). Readers of all kinds need to be told as soon as possible what this is really about: partly so those who don’t want to know can drop it quickly (remember, you must have scanned hundreds of papers it turned out were not what you wanted); and partly so they don’t expect the wrong thing. The job is to guide readers’ expectations, so they don’t have to fill the vacuum by assumptions and then criticise you for not satisfying the wrong expectations they were allowed to form.
Set the scene e.g. where the topic comes from, one or two classical references from which it sprang.
But, unlike a novel, it is very helpful to academic readers to tell them where the paper is going: give an outline in a sentence or two. Abstracts are supposed to do this, but it should also be in the introduction, whether or not there is an abstract. This really helps the reader to know what to expect, and helps them to suppress questions that occur to them half way through (“why hasn’t she mentioned X …?”) which in fact are going to be answered just as soon as they can be fitted in. Surprise is a virtue in a novel; but on the whole is a bad idea in an academic paper.
In a word: introductions should manage readers’ expectations by anticipating what they are expecting, and warning them about anything that is different from that.

So introductions should both introduce the topic by setting the scene conceptually and historically, and introduce the paper by saying what it covers and where it is going to end up.

In summary, an introduction should cover:

Define/delimit what the topic is and is not.
Say where the topic comes from.
Define/delimit this essay’s scope (e.g. “reports two experiments” or “reviews four papers, and touches on six others” or “deals only with recent drug therapies, not the wider alternatives”).
Say where the essay is going to.
If these seem such different things that they don’t fit smoothly together, do not hesitate to have several subsections within the introduction.
First person constructions

Many people have been trained to avoid first person constructions in scientific or scholarly writing i.e. using “I”, “we” (and “us”, “me” etc.). Thus even though much of their writing is reporting actions they have taken, judgements they have made, and ideas they have had, they do not say this directly.
However this convention is now under active debate in the best scientific (not just psychology) journals, and will probably change. Of course, it will be 40 years before all those trained up until now have died out, and some of those will never change their habits. Thus continuing to follow this practice can save you trouble in the short run, but may increasingly come to look archaic.

However, a mistress of language should be able to express what is necessary regardless of restrictions on syntactic forms. In my (!) view, the underlying issue of substance is to convey whose is each view: what status, what origin, what authority, what grounds for plausibility it has. This is particularly important in a critical review where what is being conveyed comes from a mixture of sources: unquestioned consensus, particular authors being reviewed, and ideas and points originating with the reviewer. And on the petty scale, it is a real question in the mind of staff marking CRs: is this an original criticism, a reproduction of one from the literature, or a new (original) use of an old point e.g. a critique read in one context applied to a new context? These all deserve credit, but different kinds of credit: for original analysis, for intelligent selection of the literature, for appropriate transfer of an idea from one part of the literature to another.

The trouble with the convention of no-first-person in the hands of many mediocre practitioners is that it allows people to pretend that, or anyway to write as if, an unfounded opinion is universally known and accepted; and for unquestioned consensus (the law of gravitation) to be written about in the same way as the author’s report on their own lab observations (“the subjects took a mean time of 3.5 secs for the task”), and also as their personal suggestions or analytic points. These are quite different grounds for considering a proposition, and critical thought requires that both writer and reader know and consider these differences. In the context of a standard paper reporting an experiment, these different statuses are usually clear anyway, so it doesn’t matter much (e.g. the introduction deals with what the paper is going to take for granted, the results section is about what this author claims to have observed, the discussion and also the use of “may” is about the author’s opinion or interpretation as in “this effect may be due to xxx”).

However in a critical review this is often much less clear. Nevertheless even within the no-first-person convention, it is possible to signal clearly about the origin and/or status of each proposition, and in my view you should do so, whether or not you adhere to the old convention. For example “Dawkins (1999) offered the criticism that xxxx. This might also be applied to other work, yyy.” (i.e. this is the reviewer’s new application of Dawkins’ original critical point). “An additional criticism, not apparent in the literature so far, is that zzzz” (i.e. the reviewer’s own personal point). Similarly “Another problem would seem to be zzzz” (which equally makes it pretty clear it is the reviewer’s own point, not supported by other authority).

In summary, you can’t write a CR without opposing the critic’s — i.e. your own — views to those of published authors. You should clearly label whose views are whose. My own preference is to use direct first person language, but if you wish (or are instructed) to avoid the first person, you should find clear indirect ways to convey this essential information.

Citations and indirect citations

The basic idea of citations is to list all papers mentioned in a section at the end called “References” (or possibly “Bibliography”), each with enough detail so that anyone would be able to obtain the paper themselves, and to refer to (“cite”) the papers more briefly within the text e.g. “Smith (1990)”. That is, the reference in the text should allow the reader to pick out without any uncertainty or difficulty the fuller reference in the Bibliography section, and that fuller reference should allow the reader to retrieve the paper from a library. (What is rather seldom done, but perhaps should be more often, is to give a way of finding, once the reader has got hold of the cited item, the place in that book or paper that actually holds the statement, figure, or argument being referred to.)
Many variations exist with different disciplines using quite different conventions. Psychology students should stick to those used in the psychology literature, but many variations are possible even within this style. For instance just occasionally you might give the title in the text if this was directly relevant to the sentence it was in; you might use the author’s name as part of the text e.g. “Smith (1990) argues that …”; or you might tack on references at the end of a paragraph rather than within sentences, to show your backing for the paragraph’s argument as a whole. E.g. “(Smith, 1990; Jones, 1988.)”. You will see examples in nearly every paper you read, and in the course handbook. Full details on the many variations can be found for instance in the APA style manual: Publication Manual of the American Psychological Association, Fourth Edition. (The library has the second edition: [Level 5 Main Lib; Psychology B235 AME].)

However in these CRs, you should attend to a new issue. Before your third year, the style of teaching and of the essays you have written has been to refer to work and theories that you have been told about, but for which you haven’t usually read the original papers yourself, simply as a way of referring to the idea and authority for it (e.g. “Newton’s laws” or “Piaget, 1970″). However in a review, you are not presenting yourself as a student mentioning knowledge your reader already possesses. Instead, you are offering your personal description and evaluation of a few specific papers. If you mention a paper here, the presumption is that you personally have read it.

The common reasons for mentioning a paper are

To show off your reading: but if you haven’t read it yourself, that would be misleading. (What you are supposed to get marks for is wide reading, not longer lists of papers you should have read.)
As an authority to support an assertion you make. But if you haven’t read it, then you don’t know for sure what it really says (let alone if its claims are true). A review is in part about checking such things.
To avoid appearing to imply that an assertion is your own idea.
A scrupulous accuracy in all your statements is the way to deal with all this.

A weak approach to this is to allow yourself to refer to papers you have not yourself read, but to be explicit about it: we might call this indirect citation. You could do this as follows: “Smith(1990) quotes Jones (1989) as saying ‘Blah’ and that all swans are black”; or “According to Jones (1989), as cited in Smith (1990), sheep may be carniverous…”. Then list both the Smith and the Jones references if you have them; if you don’t have a precise citation for the Jones paper, then just leave it out of your reference list and delete the “(1989)” in the examples. Although this is always inferior to checking the references yourself (which is always desirable as so many mistakes and distortions occur in references made in published papers), this makes it clear to the reader what you have and haven’t claimed you have verified, and gives them maximum information for checking themselves if they wish. It can be inevitable if references to unobtainable technical reports are made, and conversely you may do it to make a criticism as in “Smith(1990) quotes Jones (1989) as saying that sheep are carniverous, but reference to the original text shows that Jones’ claim was only that sheep had been observed to eat ham sandwiches when wrapped in grass”.

However the tough standard here is to avoid citing any paper that you haven’t read yourself. Why would you want to mention one? After all, if you haven’t read it, you don’t know for sure what it really says (let alone if its claims are true). You have read Black (say), so just stick to what Black says. Why does Black mention White? just to support a claim they, Black, wanted to make. So you could simply say “Black asserts that cows are carniverous”, or “recent papers on the topic all assume X, citing work from 1970 as their authority, but without offering any other verification”. On the other hand, if Smith is citing Jones in order to dispute their alleged claim, then in a review you surely would want to have checked Jones’ paper because it is a crucial part of Smith’s argument, which you are reviewing, as opposed to being merely a passing support for a point Smith believes in.

An advantage of this tough standard, is that your reference list will contain only the papers you have actually read, preventing readers from being misled by your bibliography (even if they would not be misled by your main text). A disadvantage is that readers cannot use your bibliography directly to go deeper than you, but would have to go to the papers you read to get the references to earlier literature.

In fact there are quite a lot of wrong citations in the published literature, almost certainly from authors failing actually to read what they cite. Simkin & Roychowdhury have estimated from this that “only about 20% of citers read the original” [Simkin,M.V. & Roychowdhury,V.P. (2002, Dec 3). "Read before you cite!" WWW document]. URL http://arXiv.org/abs/cond-mat?0212043 (visited 2002 Dec 15)]. However you should not do this a) because it is bad practice and attracts criticism whenever noticed, and b) because reviews above all are to provide reliable accounts of the literature (at least in other papers the main point is to present new data or theory).

Spelling and punctuation

This is a very common cause of moaning from employers about graduates, and many student essays make me feel the same. It also annoys examiners and gives them an excuse for not trying harder to understand what you mean. (It is also true that the quality of manuscript I, as a reviewer, see submitted by academics to many academic journals is in this respect much the same as students in this department: very variable, with plenty of bad practitioners. And, I am sorry to say, I still have quite basic things to learn myself. Recently a co-author had to point out to me that after 48 years I still hadn’t learned the difference between when to use “practice” and when “practise” — like “advice” and “advise” she taught me.)
Help on spelling, punctuation and apostrophes is also available from someone else here. My own advice is as follows.

If you want to improve, then

Critique each others’ writing at this level. It is MUCH easier to spot others’ errors than one’s own.
Use a dictionary. Don’t rely (only) on computer spelling checkers which are only good at spotting the things you can spot easily by eye. The hard to spot ones are missed by the checkers (the/then, of/or, affect/effect, practice/practise, missing ‘s’ in plurals etc.).
Use Fowler’s “Modern English Usage”, or rather Burchfield’s new (third) edition of it, for things other than simple spelling, such as apostrophes. The new Fowler’s modern English usage edited by R.W. Burchfield. 3rd edition (Oxford: Clarendon Press) (1996) [Level 9 Main Lib, and ULL; English Ref D460 BUR]
The complete plain words by Ernest Gowers (1948), now revised by Bruce Fraser, is also very good on these and other matters. [Level 9 Main Lib; English D421 GOW5]
Common spelling problems

If you use the spelling checker in your word processor and don’t bother to read your writing at all, you will get some complete nonsenses (e.g. “the” for “then”, “of” for “or”).
If you do read your writing but still rely on the checker for words you actually aren’t sure about, then you will get caught especially by words that sound and look similar but actually have different spellings for different meanings or grammatical roles. Common errors currently include: its/it’s (see below), practice/practise, dependent/dependant, affect/effect, principle/principal, illict/elicit.

Apostrophes

The single worst-done issue is that of apostrophes. The main points are in these examples, which are correct usages selected to imply and illustrate the rules:
Fowler’s book, the man’s face, the car’s wheel.
His book, her face, its wheel. No apostrophe in this “its” any more than in “his”.
He’s a sight so he is, she’s a picture so she is, it’s bust so it is.
Apostrophe to mark a missing letter, as also in: he’d better, I’ll go, shan’t I.
Punctuation

There are a lot of wrongly placed commas around. Putting one where you might pause in speaking is a clue, but no more reliable as a rule than, say, speaking whenever someone smiles at you. It’s a correlation all right, but you know how much to trust correlations, don’t you?
Plain English

Plain English means being clear, brief, and not long winded yet harder to understand. Students usually only have minor versions of this disease. http://www.plainenglish.co.uk/guides.htm downloadable short guide
For light relief, here’s a collection of creative words that were just invented on the spot. I do NOT advise you to use them in academic work, yet you could probably argue that they are plain, even though not accepted, English. wordlust

Footnotes

The short advice is: don’t use them.
If you are a psychology student, you probably won’t be tempted to use footnotes, because of course the main rule of writing is to fit in with the community you are writing in and for, and the convention in academic psychology is generally not to use footnotes. However this document is in part general advice on writing, and some disciplines, e.g. History, do use footnotes. In that case, so should you.

However some of us get tempted to use them poorly. The principle to worry about is whether a reader can read comfortably and fluently, or is continually having their reading broken up by having to jump to the bottom of the page, read a bit there, then jump back. A good use of footnotes, where they are the convention, might be to have flowing text in the main text, and footnotes are (only) used to give the citations to the literature or evidence of the main argument. Then readers who aren’t thinking about detailed evidence can ignore the footnotes altogether, but those doing checking can follow the links. A bad use of footnotes however uses them for extra detail. A reader cannot safely avoid them, and all that is achieved is breaking up smooth reading: only someone who had already read the whole text and was re-reading only for a summary could afford to skip the footnotes. This is really a form of the issue discussed earlier, that language and hence text is linear, but arguments are non-linear branching tree structures or networks. Part of the skill of writing is to find a single path through and offer it to the reader. Using sections and subsections can express this non-linear structure, and yet still gives the reader a single order in which to read everything. Footnotes fail to do this. Using footnotes for extra explanation, though logical, is simply failing to address the heart of the authoring task. Either the extra explanation is necessary (if so, put it in the main text), or it isn’t (so leave it out).

(I remember reading a preface in which Oliver Sacks said his editor had finally called him to account on this, pointing out that he now had over a third of his book in footnotes. Deservedly best-selling author though he is for his books on the mind-body borderland, his editor was right.)

Get someone to proof-read your writing

I find it hard to follow this rule, but still I never manage to write anything without at least one or two silly errors in. (In a document this long, even though I will have read it through two or three times, there is probably at least one. In fact in re-reading this in 2000, I found three typos that had survived all previous checking.) Getting someone else to read it through after you’ve done your own best checking is much the best tactic. We all have different blind spots, and the chances are that your errors will be obvious to them (and vice versa). It is also good for catching sentence constructions which, even if legal, are in practice too difficult for others to understand. Good writing should not just be correct, but easy to read: and that can only be judged by someone other than the writer.
Cover pages, page numbers etc.

(See also the requirements in the course handbook.)
The cover page. Before printing off a piece of work, just consider the reader for a moment. Check you have your name on it, and that it is labelled for the type of work (critical review in this case) so someone can classify it even if it got dumped in a pile of incoming “mail” of different kinds. Most likely staff will search for the name, not the title: so having your name in the biggest clearest print is actually helpful to them, not boastful. For you, you may search through a pile of your own work (so your name won’t be useful) and you will be looking for something that means “critical review number 2″ (you will be doing three critical reviews): so that information is useful to you, and perhaps to staff if the paper gets mis-filed somewhere. No harm in having other information on the cover page, especially if it could be useful (matric number, date e.g. “level 3, term 2, 1999″, whatever), but make the most useful information the biggest/clearest; and consider the different kinds of user: secretaries (which student’s CR is this?), tutors (is this a research paper I should read? or something I should mark? and if so which of my groups does it come from?), and yourselves (which of my bits of work was this? e.g. level three, second CR).

Page numbers. It is useful to have page numbers. If your CR is ever going to be unbound, ever going to go through a photocopier, get dropped on the floor, or shuffled in a pile, page numbers are the best safeguard. In fact, as you hand it in, how do even you know you have all the pages and in the right order?

Since you all print single-sided, the binding (e.g. staple) will almost certainly be on the left hand side. That means if the reader is flicking through looking for a page number, they will be looking at the right hand side (the left side will be the last bit to be exposed as the page is opened). Personally, I find it easiest to look at the top (right hand) corner. That is where page numbers are most useful. So page numbers in the centre of the bottom may look nice when the page is open by itself, but are less useful when page numbers are being used in earnest.

Finishing it: Keeping and making copies

The formal requirement is to hand in a single printed copy (see the handbook). However both you and your supervisor may like to have a separate copy for reference in either or both electronic and paper versions. Electronic takes less space, and re-using chunks is easy; paper can be easier to put through a photocopier. Certainly I sometimes like to give students CRs from the past: they are often useful literature reviews that can help start a new maxi or CR. Furthermore, you may like to offer it to employers as a sample of your work: certainly there is nothing like a quick look at a CR or maxi report to tell me if this person can write competently or not, and indeed it is also a demonstration of your word processing skill. And offering it makes it look like you are proud of your work, even if the employer doesn’t really look at it.
So plan how you are going to preserve your work for at least a year or two beyond the life of your university computing account; and ask your supervisor if they want a spare copy. This advice probably applies even more to your maxi project than to your CR.

Extra points about writing in a CR

These have each been mentioned above, but I’ll stress them again. They are particular points about structure and style that apply to CRs rather than other essays or dissertations.
Justify your selection of the papers you chose to review: otherwise it looks random. Say (briefly) why you chose them: the most interesting, representative of the different views, the most recent? And listing the papers you have chosen briefly (probably in your introduction) helps the reader be clear about the scope and structure of your CR.
The introduction should manage the readers’ expectations; in particular tell the reader how you have structured the rest of the CR: by paper, by theme, by critical point, ….
Provide explicit definitions of technical terms and categories wherever this is central (as opposed to merely relevant) either to the criticisms you will discuss, or to how the topic is defined and the papers selected. It’s surprising how simply writing out an exact defintion immediately leads you to making some critical comments.
Be extra-scrupulous about citations: do not cite papers you haven’t personally read (use indirect citations if appropriate, as discussed above). Even if you think it’s OK to cite papers you haven’t read in other writing, in a review the whole essay is about your personal discussion of what others have written.
Tell the reader which criticisms are your own thoughts, and which you are repeating from others (e.g. an author’s self-criticism). Both are interesting, but tell the reader which is which. (See above for the issues of using or avoiding first person language, which comes up in this connection.)
Revising a paper or CR

“Writing isn’t writing, it’s rewriting.” [P.Caputo A rumor of war p.349 (1977/96) (Pimlico: London)]
If you want to improve quality, revise what you write. When you no longer want to change it when you yourself read it through, get someone else to read it and comment on it. This is easily the single biggest thing you can do to improve quality, but you have to allow some time for it. It won’t work to delay work until near the deadline, write it in a hurry, then wonder about readers: you have to plan ahead.

Even if you have a very patient friend, you only get the best out of any reader the first time, when it is new to them. So if you are serious about quality, line up more than one reader, and revise it after each set of comments before using your next reader. If you think one reader is better (more expert) than another for this piece of work, save the best reader till last (get the less good one to find the worst of the spelling mistakes etc. first).

As mentioned earlier, it is probably a good idea to think of fellow students as your audience: this will encourage you to be interesting, and not to assume too much, which will then lead to you writing more clearly and impressing everyone especially staff.

Putting all this together, and doing basic time management reasoning on it, you might have a work plan something like this:

Deadline – 4 weeks: Finish reading articles and making notes both about content and about critical points you might make. Now you can plan a structure.
Deadline – 4 weeks: Tell a friend orally (i.e. aloud, not in writing) what your CR is about and its main points. This will make it clear to you what the most interesting thing(s) you have to say is. Organise the CR around that.
Deadline – 3 weeks: finish a complete draft, and revise it obsessively. Get someone to read it through for typos.
Deadline – 2 weeks to 8 days: give your best draft to your last and best reader.
Deadline – 1 week: get back comments from your last and best reader
Deadline – 2 days: hand it in. You need to plan for 2 days to allow for broken printers, storms that prevent you travelling into Uni to hand it in, etc.
If, when revising, you are told by your readers, or can see for yourself, that your writing just isn’t clear, but you can’t see what’s wrong with it and how to improve it, then you may want to look at the section above on the Gopen & Swan approach to organising each sentence and paragraph so as to be clear.

Further / advanced work: where CRs fit into wider perspectives

Most of you will just regard this web document as an aid to getting through a compulsory exercise. But if you wonder where, or whether, CRs fit into anything wider, then here are two leads.
The rhetorical form of scientific literature

The published literature you examine in CRs is written to persuade other scientific readers, and a considerable part of doing a CR is learning to examine how well a given paper addresses this purpose. Rhetoric is the traditional discipline of persuasion. For a somewhat startling view of the role of the literature — what part it is playing in technoscience — see chapter 1 of: Latour,B. (1987) Science in Action (Open University Press) [Level 5 Main Lib, and ULL; Gen Sci M8 1987-L]. Latour began as an anthropologist, and they of course pride themselves at seeing through the little myths we like to tell about ourselves. I admire him enormously, but don’t expect his views to go down well with most academics.
Critical thinking

As mentioned above, an old, but from your viewpoint valuable, text on critical thinking is: Abercrombie,M.L.J. (1960) The anatomy of judgement: An investigation into the processes of perception and reasoning (Free association books: London). [Level 5 Main Lib; Psychology F570 ABR]
There is also a current topic in educational and psychological circles called “critical thinking”, which views it as a teachable generic mental skill, and would view CRs as an exercise to develop it. If you would like to explore this perspective, a good reference is Kuhn,D. (1991) The skills of argument (Cambridge University Press: Cambridge) [Level 5 Main Lib; Psychology F570 KUH].

Actually I feel rather contemptuous of some of this literature, as it fails to understand the extent to which it is a partial re-invention, without realising this or grasping the wider generalisations available, of the rather better preceding literature initiated by Perry. (A brief description of Perry’s views and some references are included in this web document associated with my APEC lectures). This literature seems to assume, without looking for evidence, that critical thinking is a general mental skill unrelated to context or discipline; that it “should” be acquired by everyone without teaching (like your first language); and that (they look for evidence of this point) shockingly not all students have automatically got it. My own view is that it is a useful transferable skill, but that like nearly everything it comes from practice (not osmosis, infection or spontaneous generation), and that its role in universities depends strongly on the particular discipline.

Argumentation

Others, however, call much the same thing “argumentation”, and are interested in whether there is a general thing that could be called argumentation, how to teach it to students, and whether computer tools could be useful for this. Jean McKendree offers these links to computer argumentation tools/papers:
Archelogos and Log analysis
Belvedere (not available?)
BGuile see also here
Convince Me
Further reading? There are of course other documents on the web offering advice on this subject. A couple I have come across are:
BRINK-BUDGEN, R van den. 2000. Critical thinking for students: learn the skills of critical assessment and effective argument. 3rd edn. Oxford: How to Books. listed here
critical thinking by Thomson Learning.
Appendix

Here is a sketch of one particular search trail, to illustrate how a combination of clues often can work out.
A student said she was perhaps interested in finding a topic in education, and I said I’d recently seen a TV programme on the issue of whether formal teaching vs. play-based activities were more effective in schooling for 4-7 year olds. I couldn’t remember the relevant names, nor had I ever read anything on this.

I went to the BBC web site. http://www.bbc.co.uk/home/today/
Couldn’t find it at first; remembered it had been a Panorama programme, so after a bit I realised Panorama might be under “News” not “Education”. http://news.bbc.co.uk/
Found a pointer to the programme: http://news.bbc.co.uk/hi/english/education/newsid%5F186000/186707.stm and http://news.bbc.co.uk/hi/english/education/newsid%5F185000/185466.stm That gave me the name Kathy Sylva as the one discussing the research.
WWW search on Kathy Sylva gave a page mentioning her http://www.ioe.ac.uk/cdl/cugrants.html
Poking around on the same site (web page for that unit) http://www.ioe.ac.uk/cdl/ found a personal page with her publications http://www.ioe.ac.uk/cdl/kathy.html (Although emailing her may be necessary to find the literature about that issue.)
Postscript: projects

Although all the skills of critical reviews play a part in writing up any piece of work really well, projects also involve additional skills. In particular, they involve: TEARS: time, expertise, access, resources, and support (as Kay Persichitte puts it).
Writing up a research project in psychology builds on writing lab reports, but it also requires constructing a reasoned argument like a CR. The weakest reports just say what was done and have no real conclusion — no real argument.

Collected references

Here I have collected, in case it is convenient for you, copies of all the references I mention above.
Alice Jenkins’ advice on writing English essays (4 pages and pointers to other similar advice).

Essay guide for literature students.

Guidance on writing for anthropology students (33 pages).

How to write a philosophy essay (18 pages).
How to write a philosophy essay (1 page and 10 more pointers).

Abercrombie,M.L.J. (1960) The anatomy of judgement: An investigation into the processes of perception and reasoning (Free association books: London). [Level 5 Main Lib; Psychology F570 ABR]

APA Publication Manual of the American Psychological Association, Fourth Edition. (The library has the second edition: [Level 5 Main Lib; Psychology B235 AME].)

Brink-Budgen, R van den (2000) Critical thinking for students: learn the skills of critical assessment and effective argument 3rd edn. (Oxford: How to Books)

P.Caputo A rumor of war p.349 (1977/96) (Pimlico: London)

Fowler & Fowler The new Fowler’s modern English usage edited by R.W. Burchfield. 3rd edition (Oxford: Clarendon Press) (1996) [Level 9 Main Lib, and ULL; English Ref D460 BUR]

The complete plain words by Ernest Gowers (1948), now revised by Bruce Fraser [Level 9 Main Lib; English D421 GOW5]

Gopen, G.D. & Swan, J.A. (1990) “The Science of Scientific Writing” American Scientist (Nov-Dec 1990), Volume 78, pp.550-558.

Kuhn,D. (1991) The skills of argument (Cambridge University Press: Cambridge)

Latour,B. (1987) Science in Action (Open University Press) [Level 5 Main Lib, and ULL; Gen Sci M8 1987-L].

Laurillard’s Rethinking university teaching

Wenger,E. (1987) Artificial intelligence and tutoring systems

http://www.psy.gla.ac.uk/~steve/resources/crs.html

This is my description of an exercise I feel is good, and have run many times with level 3 tutorial groups here at Glasgow, in Psychology, as have a least a couple of other tutors. It goes particularly well with supporting critical reviews. This document is written as if I were talking to tutors, but there is no reason why students should not organise and do this themselves. See also here

.

There are also some studies of this: see here and a talk: abstract, slides, handout.

It requires making each student come with a piece of old work (e.g. an essay) and have students critique each others’ work. Afterwards (not before!) they say how valuable it was for at least these reasons:

Pedagogic rationale

• Learners see how differently other students write: most are quite unaware of the variety of different “good” ways there are of doing something, and any group usually illustrates this well. Seeing other students’ work is much more convincing that hearing some tutor say this.
• Each learner must acquire the ability to judge quality if only to regulate their own writing production. We should give them direct explicit practice, if not teaching, at this.
• Furthermore, this gives them insight into what it is like to play the tutor’s role of having to judge another’s work.
• Learners experience the usefulness of getting feedback from peers, as an apprenticeship preparing them to make this a standard part of their practice independently of staff: they will achieve better coursework quality if they do what most academics do and get local feedback.
• If they are going to do this, they need to practise giving useful feedback to be any use to others in return for the same service.
• Critical reviews are often interesting and useful to other students for their content alone when they are on exam-related topics. It is worth students asking around and reading others’ reviews as part of extending their coursework. However this benefit will not be apparent to students from this exercise if they have all done a CR on the same topic (to make it easier for the tutor), and when they are stuck in the same group all year (to make it easier for the class administration).

Brief overview

Everyone produces something from their files e.g. their last submitted piece of work; two others get to read and comment on it; the rules are: always say what is best and worst about it (not just one or the other), minimum is a comment on it overall, better is a separate best and worst for each section, better still is best/worst with respect to each of the assessment criteria for that piece of work, a useful extension is to suggest a mark/grade for the work.

One version of instructions for participants is here.
You can download (and edit) a prompt sheet here.

Reading two pieces each is probably as much work as is both useful and not too onerous.

More practical detail

• Everyone selects their piece to be critiqued. Can be either paper or re-printed from their files, but it is best if it doesn’t have a tutor’s comments on it already. Make sure it has the writer’s name on it.
• Get each person to write on the cover what feedback they would most like to get e.g. “was my argument clear?”, “how else could I have been critical?”, …
• Organise the cycle of critiquing e.g.
  • Kathryn -> Richard, Gavin
  • Rhiannon-> Clare, Steve
  • Richard-> Steve, Gavin
  • Steve-> Rhiannon, Clare
  • Clare-> Kathryn, Rhiannon
  • Gavin-> Kathryn, Richard

  (If some of the work was joint, or on similar topics, it may be worth designing this cycle so everyone is reading something new to them; and/or so that people with different interests or expertise are coupled as critics of a given piece.)

• Arrange to meet for half an hour to do the photocopying, and distribution.
• Instructions:
  • Issue a clear instruction about the comments to generate before they do the reading and/or give them a comment sheet pro forma to use. E.g.
    “The absolute minimum is to produce, for the two papers assigned you, two points: the thing you liked best about the paper, and the thing you thought worst (preferably with a suggestion about how to do it better). Better, is to produce best and worst points for each section separately, plus for the paper as a whole. In addition, or as part of that, suggestions about other WAYS the paper could have been critical.”
  • Decide whether students should additionally suggest a mark/grade for the work.
  • Decide whether critics should additionally mark the printout of the essay direction for typos etc, in addition to the higher level overall comments.
• On the feedback day, go round the group, asking for the two oral pieces of feedback for each piece of work in turn. If necessary prompt for both positive and negative from each person. If the tutor is producing comments, s/he should speak after the other critic(s) of the piece.
• Optionally, get the recipient to say what was and wasn’t helpful about the comments i.e. critique the critique by again saying what was best and worst about them.
• End with a few minutes discussing what (if anything) they found useful about the exercise.
• I then say: if you found it useful, then you could do this again with your current assignment by organising this reciprocal critiquing before handing in your final version. If you want to do this, it means finishing a full version one week earlier, and putting this in your diaries now. My last several groups have done this, and carried it out in due course.

Variations, issues

Tutor join in critiquing?

One way to do it is for the tutor to join in completely, and contribute a piece of work for critiquing. This is good for the tutor’s soul, particularly the first time of running the exercise, but the difficulty is finding a piece of writing the students can and will comment on usefully. If the tutor has recently written a handout or notice for students, this might be a good choice.

Another approach for tutors nervous in another way, is for them to try and read and produce comments on all the pieces of work. This soothes a nervous fear that no comments will be produced, but gives the tutor much more work than the others. Without this, the exercise can scale over quite a range of different group sizes.

The lazy and disorganised, but professional-seeming, tutor will take a copy of all the pieces, but only read a few depending on whim and time.

Best and worst

I like asking for comments on both the best and worst aspects as it goes a long way to producing balanced comments, and avoiding both students who think “critiquing” means being negative, and those who feel that politeness means they should only be positive. You can usually tell an overall assessment without it being stated from the relative strength of the best and the worst comment produced. And both of course are useful: negative comments tell you what needs changing, positive comments tell you what to retain and repeat. You could call them “what to keep and what to change” instead of best and worst.

An alternative to this is to ask for stop/start/continue comments:

• Stop i.e. what is bad and should be stopped
• Start i.e. what was omitted and should be added
• Continue i.e. what was good and should be retained

Note that “best/worst” is asking for judgements on what was written, while “stop/start/continue” is asking for specifications for what should change i.e. for constructive comments only. Constructive comments are always more useful to the recipient, but not always possible to give. We can always say whether we are unhappy, almost always where in the writing or what is making us unhappy, usually why it makes us unhappy, but only sometimes what should be done to change it.

More elaborate specifications for writing feedback can be found here. However these are not likely to be appropriate for a simple introductory exercise like the above.

Giving the feedback face to face

The first time this is done, it is best to deliver the feedback face to face in a group, not privately; even though they will probably have written it out on the comment sheets. There are multiple advantages of this:

• It is harder to be rude face to face than in writing
• In any case, the tutor will be there to moderate any such tendency, but also to prevent vacuous politeness: and so to establish the productive tone for this new kind of conversation.
• It is valuable when a more general conversation ensues on some point, including the whole group. This will probably happen to some extent.
• It is valuable for authors to hear comments that other authors receive: it brings up points that they might not have received, and illuminates ones they did by adding a wider perspective.
• It ends up bonding the group further, as joint tasks generally do.

Giving marks/grades

A variation is to require students also to propose a mark or grade for the piece of work they are commenting on. This is mainly of benefit to the critic. Although students find it stressful to mark each others’ work, it is in fact very useful as a step towards understanding and exercising standards.

Giving feedback on spelling etc.

It is worth including feedback on spelling and language. It is what I always do for colleagues, and there’s no other provision for this low level feedback for students. This is best done by marking a copy of the essay, so paper copies may be best for this.

Time management consequences

Could end with punching home the time management message:

• Do you think getting such peer critiques would help you improve your coursework?
• (if yes) so what does that mean about setting yourself deadlines for finishing your next piece of coursework? Yes, finish a week early.
• Get out the course handbook, look up the deadline, work back a week, put it in your diary.
• Also you need to get your pal to commit to commenting in advance, even though they may themselves be panicking over the same deadline.

Investigations have suggested that the most crucial failing in students’ time management is grasping the most basic idea of working back from the deadline to the implied time needed for component actions. I.e. students have to grasp and act on two notions:

• A task (e.g. writing an essay) is NOT an indivisible activity but has component subtasks.
• Each subtask must be assigned a different time, calculating back from the overall deadline.

Further exercises

If you want to follow up this exercise with more, then the next steps might be:

• Aim for better quality feedback: see this document.
• Require the feedback (and marks/grades) to be submitted in writing; and then have the recipient critique the critiques for clarity and utility.

http://www.psy.gla.ac.uk/~steve/resources/tactics/tutcr.html

By Steve Draper,   Department of Psychology,   University of Glasgow.

Students often, and sensibly ask for example CRs (critical reviews). Often if you go to the right PAL session, the facilitators will bring theirs in as examples. If you do the reciprocal peer critiquing exericse in your tutorial group(s), where each student reads and comments on two other CRs, then you will have seen examples.

I have put some of my past students’ CRs on the web because I think they are generally useful, so you can look at them if you like. No marks or comments are available for these.

• Ian McCubbin on Tinto’s theory of student dropout
• M.Nimmo (2005) Placebo: Real, Imagined or Expected? A Critical Experimental Exploration

http://www.psy.gla.ac.uk/~steve/resources/crseg.html

(last amended 16.10.06)

In order to qualify as a ‘critical’ review you must go beyond the simple description of the sources you read. If you are writing a review with reference to specific research questions or objectives then you might consider:

• the breadth of different pieces of work  – how wide a range of the possible subject matter defined by your question(s) or objective(s) do different pieces of work cover?
• the depth of the different pieces of work – how detailed is the analysis of the subject matter in each piece of work?
• the relevance of each piece of work to your specific question(s) or objective(s) – how much of the subject matter you are focusing on do different pieces of work cover? Are subject(s) missed out in different sources? Are the research methods adopted in one source more useful in answering your question(s) than those adopted in another?  – and so on.
• gaps in the form of relevant questions that do not appear to have been tackled by the authors you read.
• contradictions and inconsistencies, both within single pieces of work ,and as a result of making comparisons between the work of different authors

The preceding suggestions may be helpful in obtaining the lower pass-level grades in assessed work, but the award of higher grades demands the application of more sophisticated forms of criticism. These are, in the main, associated with the identification of logical flaws in the literature you are reviewing.  Consequently you will need to display the ability to:

• recognise un-stated and invalid assumptions in arguments.
• distinguish facts from hypotheses.
• distinguish facts from opinions.
• distinguish an argument’s conclusions from the statements that support it.
• recognise what kind of evidence is relevant and essential for the validation of an argument.
• recognise how much evidence is needed to support a conclusion.
• distinguish between relevant and irrelevant statements and evidence.
• identify logical fallacies.

NB if you are unsure what is meant by any of the terms in the last list you would be well advised to invest in a textbook on critical reasoning. There are a variety of useful books available on the market including:

Brown, N. & Keeley, S., (1997), Asking the Right Questions: A Guide to Critical Thinking, Pearson: London  (ISBN: 0131829939)

Cottrell, S., (2005), Critical Thinking Skills, Palgrave: Basingstoke; ISBN: 1-4039-9685-7

Cottrell is particularly good on assessing the quality of arguments and evidence and a wide variety of source materials, while Brown & Keeley is excellent on the analysis of the various parts of arguments.

With primary sources it may also be possible to critically comment upon:

• the suitability of the research design.
• the effectiveness of the data collection process.
• the validity of the sample selection process .
• the appropriateness of the chosen research methodology to the subject being researched.

and so on.

Illustrative Examples

The extracts shown below were taken from a mixture of student assignments and published critical literature reviews. This is not an exhaustive list, merely a sample of the many different ways one can go about critically assessing published work. You should also note that the examples were not selected for the quality of their written English. We are not recommending or endorsing any particular writing style or format by quoting these examples. We would like you to focus on the various methods of constructing criticisms of published work. The examples illustrate the following types of criticism:

• Flawed understanding of a phenomenon
• Challenge to a statement of ‘fact’
• Excessively narrow subject focus
• Poor data collection methods
• Omission of potentially relevant information
• Lack of evidence to support conclusions
• Lack of current data
• Challenge to the validity of the conclusions
• Exaggerated and unsubstantiated claims
• Insufficient detail to permit the assessment of the validity and reliability of the conclusions
• Challenge to the generalisability of the conclusions
• Possible confusion between correlation and causation

Most of the criticisms refer to defects in the structure of the arguments the authors have offered in their published work, in other words, to faults in the authors’ reasoning. If you do not fully understand what is meant by the phrase ‘critical reasoning’ you are advised to remedy this gap in your learning as soon as possible. You can make a start by going to the heading of ‘Critical Thinking’ elsewhere on this website.

Flawed understanding of a phenomenon

Neiss (2002) contends that foreign creditors would have suffered huge and immediate losses had the IMF not intervened. However, these are naive comments from a member of an organisation that promotes the free market approach. Foreign creditors would have known the structure of the financial systems in South East Asian countries and the close links businesses would have with their respective governments. [In other words, they would have been well aware of the risks and did not need protection from the Fund. JR]

Challenge to a statement of ‘fact’

1. In this article The Economist is also responsible for portraying the IMF as an organisation that had no choice other than to implement the policies it implemented. Since the Fund had an enormous range of possible courses of action in response to these events this portrayal is clearly not true.

2. There appears to be an error, or at least a discrepancy when the data on waterway freight growth – 9 million tonnes (DFRA, 2002, p. 29) is compared with the figure of 12 million tonnes cited in AINA (2001).

Excessively narrow subject focus

Only Krippendorf (1999) fails to discuss in detail the economic impact, favouring instead a discussion of some of the other effects of tourism. One might argue that the tight concentration by other authors on the purely economic effects may result in unnecessarily narrow, incomplete and possibly biased conclusions.

Poor data collection methods

1. However, Stanton and Bardoni’s (1972) research can be questioned in that whilst they carried out an anonymous questionnaire to enlisted males under 25 years of age in Vietnam, they left the respondent to define ‘flashback’ for themselves. This personal definition is clearly unsatisfactory and does not enable comparisons between studies, nor does it allow for an examination of the nature of such ‘flashback’ reports. Consequently a major variable was not controlled for or quantified.

2. Horowitz (1969) interviewed a smaller sample (n = 31) of individuals said to be representative members of a drug-taking community, found that 32% (that is 8 individuals) reported having flashbacks. However there was poly-drug use amongst this group, which could have confounded the results.

3. The problem of possible participant bias also plagued another study: the Boris and Mandel (1994) study which used 26 consecutive ADHD children referred to their allergy practice. …. The study has been criticized because Boris is an allergist and the group of participants was small.

Omission of potentially relevant information

When defining culture as an acquired element, the authors omit consideration of an individual’s personality and genetic makeup, although both if these factors interact with the culture in which they are brought up to create their observed behaviour.

Lack of evidence to support conclusions

The main shortcoming of this text is the author’s reliance on personal opinion rather than empirical data. For example, on page 325 there is a criticalattack on the government’s policy towards private eye hospitals, but the author offers no data to support the argument he is making. In the absence of any supporting evidence there is no way of judging the validity or reliability of his conclusions and this seriously undermines the value of the work.

Lack of current data

Recent years have seen the publication of a number of critiques of changes in funding regimes, for example: Anderson (2003); Phillips (2002). However this text was published before the recent wave of reforms took place. Consequently, although much of the book’s contents remain valuable, the conclusions on this particular aspect of the problem are at best partial in nature, and at worst no longer relevant.

Challenge to the validity of the conclusions – alternative conclusions drawn from the same supporting evidence

Research by the EU (European Commission, 2002) has also shown that the most important factor impeding SMEs of all sizes from going digital is the belief that e-commerce is not applicable to their type of products or services and the lack of perceived commercial benefit. However this may be due to the survey’s focus on e-commerce rather than e-business. ICT skills gaps seem also to be more important for small enterprises than for larger and medium ones.

Exaggerated and unsubstantiated claims

When creating this theory of JIT in the late 1980’s Harrison and Tersine (1989) p. 234 made the completely unrealistic claim that it is possible through the pursuit of total quality objectives to achieve production with the creation of no waste.

Insufficient detail to permit the assessment of the validity and reliability of the conclusions

A lot of primary research has been conducted to help support policy recommendations in this subject area, but Haywood (2002) is lacking depth in its explanation of both the data collection process and sample selection criteria. Greater elaboration of the research process was offered in an earlier paper: Greensmith and Haywood (1999), but in the current work, in the absence of adequate detail, it is extremely difficult to assess the validity and reliability of the findings.

Challenge to the generalisability of the conclusions

Recent research by Actinic (2002) shows that, in their sample of UK SMEs, 72% believe their e-commerce site is profitable, and that increased sales remain the main justification for e-commerce adoption, …However, it should be noted that ‘early adopters’ may have done so because they could obtain these benefits and it does not follows that all SMEs will benefit in the same ways.

Possible confusion between correlation and causation

Boniface describes Africa having:

• … a low level of economic development. Most African countries fall into the ‘least developed’ category with only a few having breached the intermediate level of development.

• Boniface, 2002, p. 241

In contrast to this, he describes Latin American countries as being at an intermediate stage of development, although tourism in Brazil, for example:

…accounts for less than one per cent of GDP.

• Boniface, 2002, p. 327

This ties in with the previous discussion in which Lumsdon (2001) and Youell (1999) argued that the extent to which a country benefits from its tourism activity depends on the success of its other industries. Thus Latin America is noticeably more developed than Africa, but it can be argued that this may be a result of the possession of more successful industries, rather than higher levels of tourism activity.

The Truman Show Critique

The Truman Show, another movie about reality television, provides an outlet for the fears that we should have in regards to how far the reality television phenomena will go. The movie also elicits fear concerning corporations and just how much they can own. The fact that Truman himself is “fathered” by a corporation is part of the dichotomy of the movie that contributes to the overall disgust towards everyone involved in deceiving Truman.

The scene when Truman is about to leave his created world is perhaps one of the most profound. The show’s director, who has spent thirty years manipulating Truman’s life calls out from beyond the artificial sky in an attempt to keep Truman from leaving. The character’s ego becomes crystal clear and his god complex emerges when he speaks to Truman about the show he created. However, he attests when talking to the public that it is all reality TV. What really comes into question is the validity of the “reality” created by manipulated events and programmed people.

Truman may have been the only real person in his world, but how accurate is that statement if he is being conditioned the entire time and never has the period of adolescence to grow apart from the conditioning role of parents. The triumph of Truman’s escape is juxtaposed against the feeling of horror that these people can watch him suffer, watch him trapped like a caged animal in a zoo and then turn and laud him for his escape. The Truman Show provides a dark view on reality television and explores fears that emerged when the TV genre became popular.

http://hubpages.com/hub/The-Truman-Show-Critique

What is a critical review?

A critical review is an evaluation of an academic text eg: an article, report, essay or book. You are asked to make judgments, positive or negative, about the text using various criteria. The information and knowledge in the text needs to be evaluated, and the criteria that should be used can vary depending on your discipline. That is, management, sociology, information technology, or literature may use different criteria. All critical reviews, however, involve two main tasks: summary and evaluation.

Process of writing a review

1. Skim read the text – note the main question or questions the text tries to answer and the main answers it gives.
2. Think of evaluation criteria. Talk about the text and criteria with classmates.
3. Read the text again and note the important points in detail such as the subject, question, arguments and/or evidence, and conclusions made, and your evaluation using your criteria.
4. Read related texts, note differences, similarities.
5. Outline critical review, matching points of description with evaluation criteria.
6. Start writing review.

Sources:

Coyle, J. 2000, ‘John P. Kotter on What Leaders Really Do’, Human Resource Planning, vol. 23, no. 1, p.45.

Gibson, JW 1999, ‘John Kotter on What Leaders Really Do’, Organizational Dynamics, vol. 28, no. 2, p.90.

http://www.monash.edu.au/lls/llonline/quickrefs/26-critical-review.xml

The Truman Show (USA, 1998); directed by Peter Weir

Chapter 4 of my book, Faith, Film and Philosophy, is titled “Escaping Into Reality: What We Can Learn from The Truman Show about the Knowledge Enterprise. Here are discussion questions for the film The Truman Show that I’ve used in conjunction with this chapter.

1. What is Christof’s purpose in “designing” a life for Truman? What kind of life does he want for Truman? And what is Christof’s purpose in televising Truman’s life?
2. There’s The Truman Show that is the TV show the movie is about, and there’s the movie called The Truman Show that we see in the theatre or on DVD. We’ll call the TV show TS-1 and the movie TS-2. In TS-2, viewers of TS-1 are depicted in various ways. Presumably, they enjoy watching TS-1. What is it about TS-1 that keeps them watching? Why do they like watching? What does this say about them?
3. Those watching TS-1 seem to have opinions about the quality of life Truman has on “Seahaven.” What are they supposed to think about Truman and his quality of life? How does this compare with Christof’s attitude about Truman’s quality of life? Now think about how we are supposed to regard Truman’s life as we view the film, TS-2. Is there a difference between what we’re supposed to think or feel as we watch the movie and what the TV viewers are supposed to think and feel as they watch the TV show? Describe whatever differences you think of.
4. Truman falls in love with Sylvia, who is kicked off the show (TS-1). Later she calls in to speak with Christof during a rare interview on television. What is her thesis about what Christof is doing? Do you agree with her? Are we supposed to agree with her? Does she make a good argument? Can you think of ways to strengthen her argument?
5. What is this movie about? Do you think the filmmakers are making an argument? If so, what is that argument? What is the thesis and what evidence is presented in support of that thesis?
6. What kinds of freedom does Truman exercise while living in Seahaven? What kinds of freedom is he lacking? How is he presented from exercising these freedoms?
7. Are you more free than Truman? In what ways? Are you sure about this? Can you be sure? [This question was suggested to me by David Hunt, a contributor to Faith, Film and Philosophy.]
8. Some critics see Christof as a god-figure in this film and suggest that the film is actually a critique of the Christian worldview. If that’s true, what do the filmmakers assume about the Christian worldview, and especially about the Christian or biblical conception of divine sovereignty and human freedom? Based on your understanding of what the Bible teaches about such things, how is Truman’s life in Seahaven like, and how is it unlike, human existence in the actual world? Support your answer from the Bible and from evidence in the film.
9. In his chapter in Faith, Film and Philosophy, Geivett claims that this film illustrates how a person may be able to acquire knowledge that is important, even when much of his community is determined to deceive him or her. Is this a plausible claim about the film? How could this claim be challenged?
10. What does this film “say” about the responsibilities people have toward each other when it comes to seeking the truth and tracking the evidence? Can you describe some contemporary attitudes about truth, the objectivity of truth, and the possibility of knowing truth? How are these attitudes reinforced socially?

http://douggeivett.wordpress.com/2008/04/15/the-truman-show-a-discussion-guide/

Part of becoming a successful critical reader is being able to translate the thoughts you had whilst reading, into your writing.  Below are some written examples of the observations a critical reader may make whilst commenting on various issues in text.

NOTE:  The critical analysis component of each example below is highlighted in blue.

Overgeneralisations and assumptions

Researchers often make simplifying assumptions when tackling a complex problem, while the results might provide some insight these answers will also likely have some limitations.

Example:

Methodological limitations

Researchers may simplify the conditions under which an experiment occurs compared to the real world in order to be able to more easily investigate what is going on.

Example:

Objectivity of research

Some research may be bias in its structure.

Example:

Limitations due to sample group

Limitations can arise due to participant numbers.

Example:

Limitations can also arise if there is a limited range of participants

Example:

Limits to applicability

There can be concerns with studies’ applicability, for a number of reasons.

Results not replicated

One such reason could be that the study results have not been replicated in any other study.  If results have not been replicated, it indicates that the results are suggestive, rather than conclusive.

Example:

Long term effects unknown

There would be limits to applicability if long term effects have not been tested.

Example:

Omissions

It is important to look for things that have not been discussed within studies to ascertain whether this would limit the applicability of the results.

Example:

Correlation vs. causation

It is important to be aware that just because one variable is correlated with another, doesn’t necessarily mean that once variable is the cause of another.

Example:

http://www.uq.edu.au/student-services/Examples+of+critical+analysis

What is a critical review?
A critical review is a review of an article that combines a summary and a critical comment.
Why are you asked to write a critical review?
Students in the Australian School of Business are required to write critical reviews in some
of your courses to enable you to demonstrate that you can:
- read to understand the main points in an article
-  analyse the findings or argument of the article
- decide the appropriate criteria by which to evaluate the article
- provide a critical evaluation of the article based on the criteria selected.

The ability to read critically is not only important in academic study, it is also important in
business because critical abilities enable practitioners to keep up to date and adjust to
change; to assess and comment on problems and proposed solutions published in professional literature; and to evaluate and comment on solutions proposed in the workplace.
What steps should you take in summarising an article?
1. Take a quick overview of the article by reading
- the title
- the abstract
- the introduction
?? the subheadings
?? the conclusion
2. Read the article without taking notes in order to gain an overall idea of its aim and main
idea.
3. Read the article again analytically and make notes of main ideas and main topic.
- Highlight important ideas.
- Make brief notes in the margin or on paper.
4. Check your notes to ensure that they include:
- the main aim of the article, e.g. to analyse, explain, evaluate, argue, citicise,
discuss opposing views
- the methodological approach, e.g. empirical research, financial analysis,
textual analysis
- the main findings/conclusions
5. Use your notes to write a summary
6. In your summary ensure that you have paraphrased not plagiarised the authors’ words
and used quotations sparingly.
edu@unsw.edu.au
9385 5584
What is involved in commenting critically on an article?

Commenting critically on an article involves analysis and evaluation. Analysis of the article
involves dissecting the information presented in order to identify the purpose, the main
points, the methodology and the findings or conclusions of the article (This is done in the
initial summarising step).
In addition, analysis for critical comment involves identifying:
1. unstated assumptions
2. steps in the argument that are not logical
3. any additional purposes of the article that are not explicitly stated.
Evaluation of the article involves making judgments about the value (both positive and
negative) of the article against specific criteria.
What criteria can be used for evaluating an article?

The following criteria are useful; however, not all of them will be relevant for evaluating all
articles:
• the timeliness of the article
• the degree to which the article makes an original contribution
• the logic of the view put forward
• the validity of the evidence put forward
• the theoretical framework used
Is the framework valid?
Has the framework been applied appropriately?
• the methodology used
Is the methodology appropriate?
Is the methodological approach explained clearly?
Does the methodological approach have any weaknesses?
Is the study sufficiently comprehensive and thorough?
Is anything important omitted in the research?
• the findings
Are the findings presented and described clearly and fully?
Do the findings seem sound?
Could the data be interpreted in another way?
Do/does the author(s) account for everything in the data or do they ignore
something that might be important?
• the validity of the conclusions
• the thoroughness with which the article treats the topic
• its value compared to that of other articles on the topic
• the appropriateness of the article for the intended audience
• the extent to which it might satisfy the specific needs of a specific user.
edu@unsw.edu.au
9385 5584
- 3 -
What is the structure of a critical review?
Like most other writing you do at university a critical review has an introduction, a body
and a conclusion.
Introduction
In the introduction you should:
• provide a context for the article (background information or shared knowledge)
• give the title of the article and name of author (full name is possible here with
subsequent references to the family name only)
• identify the writer by profession or importance if appropriate
• include some indication as to why the subject is important and thus worth writing
about
• identify the purpose of the article
• give an indication of your overall impression of the article in general terms.
Body
In the body you should:
• summarise and analyse the contents of the article
• make clear by frequent reference to the author(s) of the article that you are
presenting the author(s) views, not yours
• evaluate the article.
The following is a suggested structure:
• an analytical summary of main findings/arguments/conclusions of article
• strengths/usefulness of article
• weaknesses/limitations/problems of the article especially for your purposes
(Or you might put these together so that each paragraph includes all four.)
Conclusion
In the conclusion you should:
• summarise the previous discussion
• make a final judgement on the value of the article
• comment on the future of the issue/topic or implications of the view expressed.
edu@unsw.edu.au
9385 5584
- 4 -
What makes a good critical review?
A good critical review:
• gives correct information about the author, date and article in the introduction
• summarises the purpose and main idea of the article in the introduction
• shows evidence of analytical thinking in the summary section
• evaluates the article against a number of criteria
• provides a final evaluation indicating the balance that is seen to exist between the
strengths and weaknesses of the article
• makes sufficient reference to the author of the article
• makes appropriate use of reporting verbs
• makes appropriate use of summarising vocabulary – words that sum up the ideas in
previous sentences and paragraphs
• makes appropriate use of evaluation vocabulary
• provides clear transitions between paragraphs that are helpful in guiding the reader
through the review
• provides full bibliographical details of the article at the end of the review.

http://wwwdocs.fce.unsw.edu.au/fce/EDU/eduwritingcritreview.pdf