How to approach a research proposal: The Literature Review

This is a follow-up to my earlier post “How to approach a research proposal: Writing Questions.” In that entry, I laid out a dynamic hierarchy of questions—conceptual questions, research questions, and field questions—which define the logical structure of a research proposal. They also begin to establish the scholarly contribution and the intellectual context of a research project. Below, I’ll be drawing on the question types I introduced in my earlier post.

The first few pages of a research proposal will be devoted to introducing and explaining your research questions, and introducing or at least suggesting your conceptual and field questions.

After the research proposal has done this, it must spend some time explaining what those questions are about, and the broader empirical and academic context in which the proposal is asking those questions. These things are done in the literature review.

A literature review is a scoping exercise that draws together the scholarship relevant to your project in order to contextualize your questions, by synthesizing the most important related studies in your field. As a proposal writer, it is your job to help your reader understand why anyone would ask the questions you are asking and to show them that (a) these questions are relevant and (b) answerable in the ways you are proposing to answer them. The literature shows us this in relation to the scholarship that has gone before yours. (Thanks to my colleague Eli Elinoff for this articulation.)

There is a whole process to putting together a literature review, but I’m going to limit what I say below to the basic functions of a literature review section in a research proposal.

What the Literature Review is not

Less experienced students often labor under misconceptions of what a literature review in a proposal is, and teachers often assume that students will know what a literature review is without much explicit guidance. So let’s address some common misconceptions.

A literature review is not a record of the things that you have read in preparing your proposal. Students will sometimes write a literature review as though they are providing proof that they have read a sufficient number of books and articles related to their research topic. This tends to lead to pieces of writing that list and summarize some number of sources, without making any strong or clear claims about any of them. But just as most other pieces of academic writing, a literature review must have an argument about the literature that is novel in some respect. (In the case of a research proposal, the argument of a literature review will be about how certain sources or groups of sources inform and justify the proposal’s questions and research plan.)

A literature review is not exhaustive. Just as a research project cannot be about every single thing of any possible relevance, a literature review in a proposal cannot be about every single source of any possible relevance. It must represent a careful and considered selection of the most relevant ones. In preparing for a literature review, you will want to be as exhaustive as possible, but in the actual written product, you must make choices about what belongs and what can be left out. These selections should be informed by the arguments that the literature review is trying to make, and therefore the questions at the center of the proposal.

A literature review is not a book review. In a book review, the purpose is to assess the merits and shortcomings of a book or other piece of academic writing on a wide range of criteria—for example, quality of the argument or data, clarity of writing, significance of contribution to an important debate or discussion, and so on. The use of books and other sources in a literature review must be both more limited and more sharply focused. Each individual source should only be discussed to the extent required to establish its relevance to the literature review’s argument.

Relatedly, a literature review is not a place to state your personal opinions about a book or article. You should not say whether you did or didn’t like a book. This is the wrong framework to use to explain its value in a literature review, because the point of a literature review is to situate sources in relation to each other and, ultimately, to your own research project. When you express a like/dislike opinion about a book, you show the reader that you are only situating it in relation to yourself.

The Functions of a Literature Review

At times, a literature review may be required to do one or more of the above things. Some departments may require broader “surveys of the literature” as part of a Ph.D. proposal, but in many cases, a more focused text is desirable. In my view, a literature review should perform four main functions. As I’ll explain, some of these functions will be addressed in part in other sections of a proposal, but can also be included in a literature review.

The first function of a literature review is to establish that your project and questions are likely to be successful and lead to interesting insights. It does this by explaining how, given similar conceptual questions and empirical foci, other anthropologists or academics have posed and answered their own research questions. Some of these research questions and projects will be similar to yours. By explaining the significance of the work of these adjacent scholars, the literature review can establish an academic justification for your own research project. This is your “academic” or “scholarly contribution.” It is also the main argument of your literature review.

Through your selection of these adjacent scholars, the literature review also fulfills a second function, which is to communicate the relevant intellectual context of your research to the reader. For any given conceptual question, you can probably imagine many different scholars asking all kinds of research questions and answering them through a great range of projects. Your literature review will show which subset of these scholars have similar questions, hypotheses, and methods to you. This is what an anthropologist wants to know, when they ask which scholars you are “in conversation with.”

The third function of a literature review is to show how your research is “in conversation” with these adjacent scholars. In anthropology, it will be rare you will do exactly the same project as another anthropologist has done in the past—it will differ in terms of method, theory, or field site. This means that you must explain the relationships between your work and the work of others. This is usually done by introducing a set of contrasts and similarities.

For instance, your project may have similar research questions but explore them in a new field site. This is “empirical novelty”, that is, what makes your work new or unique is that it is about a place or situation that nobody has looked at before: “Anthropologists have studied burial practices in places X and Y, but nobody knows what they are in Z.”

Or, your project may have different research questions in a similar field site, that is, it explores a new relationship in a known field site: “Anthropologists 1, 2, and 3 have understood ceremony A in place X to be a rite of passage, but following 4, 5, and 6, I ask how these ceremonies may be related neoliberalization.”

In most cases, you should have a bit of literature speaking to both sides of the novelty/similarity contrast: “Anthropologist 1, 2, and 3 have argued that ceremony Y in place X is a rite of passage that ensures the continuity of gerontocratic society. However, anthropologists 4, 5, and 6 have shown that ceremonies similar to A in places Y and Z are also linked to neoliberalization.”

By doing this, you also provide justification for your own hypothesis. If your research project is about showing that ceremony A is linked to neoliberalization, then being able to show that others have shown this link for similar ceremonies helps justify the importance of your own research, and show that other academics are going to see the value of your work.

The fourth function of a literature review is to bring clarity to important terms. If you’ve stated that 1, 2, and 3 argue that a certain type of ceremony is linked to neoliberalization, then you should explain how they define “neoliberalization,” as well as “ceremony.” What phenomena constitute neoliberalization in their view? What other authors do they draw on to formulate their definition? What alternative definitions are there, and why is theirs more useful?

The literature review may also be where you explain key terms from the field: who and what are the “who” and “what” of your research questions? Some of this may take place in a different section, but in any case, you will need to explain somewhere how you define a particular community, and something about their history as a community, and introduce how other scholars have defined and studied that community. You might also add your own preliminary or past research here. The point is to provide whatever a reader who is informed about anthropology but may be naive about your particular interests what they need to know in order to understand what you want to do.

Scoping

It’s important to note that, throughout this, we’ve been talking about connections between your proposal and the existing literature that are made at the level of research question. In most cases, your literature review should not bring in sources that only speak to your work at the conceptual level. You do not need to introduce sources that show how anthropologists have long been interested in social structure, or the diversity of cultures. They should address the levels of research question. This is provided that the reader is someone reasonably familiar with anthropology. If you are writing a proposal that will be read by a person in a completely different field, or outside of university research, then you may want to include such a statement, but only if you judge that your audience really needs it.

(You may discuss things in your literature review at the level of field questions, but these are usually best saved for the methodology section, which I will address in a future post.)

To say just one thing about the process of composing a literature review: The four functions of a literature review can be used as questions that you ask of the things you read, in light of your conceptual and research questions, and which allow you to rewrite your questions or modify or clarify the scope of your research project.

The functions, restated as broad questions are:

  1. What is the academic value of my project and the work of others in my area of research?
  2. Who are the scholars doing work most relevant to my project, and what do they say?
  3. What makes my research similar or different to their work?
  4. What key terms are most useful and effective for communicating my research and my ideas?

Ending your literature review

If your literature review fulfills these four functions, then it should allow you to arrive at a concluding sentiment in your literature review, which should be something like “Therefore, I will study ceremony Y in place X in order to understand its links with neoliberalization.”

In this way, a good literature review is like an essay within an essay. It shows, based on a careful and deliberate selection of relevant sources, how you position your research in relation to what others have done, and to explain the value of your work within that intellectual context. The rest of your proposal can then move on to the how of your research.

How to approach a research proposal: Writing questions

No matter where I have taught, one job that often challenges graduate students is the writing of a research proposal. When first encountered, it is a peculiar piece of academic writing that most undergraduate students may only have to attempt once or twice, if that. But they become increasingly important in graduate school, when writing thesis proposals and grant applications.

Below is a refined version of an explanation of how I approach writing questions for a research proposal, that I have shared in the past with Master’s students. Later installments will address other aspects of the proposal.

Questions

Questions are key to any type of research proposal. They make clear to the reader what your research is going to be about, and helps them to imagine the range of things you might do in your research. Formulating and refining your questions is also useful for helping you clarify your own thinking. You will draft many questions that are close but not quite right, and the process of writing better questions will help you figure out what you are actually doing.

Usually, students are asked “What is your research question?” and research questions are certainly important in a proposal. But in my view, the research question is just one of three types of questions that any effective research proposal must address.

The three types of questions are: conceptual questions, research questions, and field questions.

These question types are arranged in a logical hierarchy, with conceptual questions being the most abstract and field questions the most concrete.

Conceptual Questions

These are the “big” questions, posed at the most abstract level. Conceptual questions can include the biggest questions, like “What does it mean to be human?” or “What do people value in their lives and relationships?” but in research proposals, they usually connect to the broadest issues of academic concern in anthropology. They might be “What is capitalism?” or “How do new technologies affect human life?” or “How do people remember the past?” or “How does race shape peoples lives?”

Note how at this stage, these questions do not yet ask where or with whom your research will be done. Neither do they ask how the research will be conducted. Those matters are addressed by the other question types. What’s important in a conceptual question is to be able to express the potential significance of your contribution as a scholar who is doing this research, and not primarily of this specific research. Conceptual questions should suggest what you want to say about issues of general concern, which will certainly emerge out of specific research projects, but which rely more on what it is you want to be able to say about the world as a scholar, and what you want to see change, or argue is changeable.

Therefore, conceptual questions will often be about a big category of things in the world, like “capitalism” or “kinship,” that you want to critically understand. If you think capitalism is messed up and destroying the world, then your conceptual questions should probably be about capitalism and the environment. If you think what really matters to most people is religion, then your conceptual question should be about religion.

Note also that even though you may be doing anthropology, these conceptual questions may be shared with researchers in other disciplines. A sociologist, biologist, political scientist, or psychologist might also ask “What is race?” An anthropologist will answer this question differently, but it allows partial connections and dialog to happen interdisciplinarily.

For an anthropological reader, and for you as an anthropological proposal writer, conceptual questions will also indicate more or less what kind of anthropologist you are. For instance, if your conceptual question is “What does it mean to be “healthy?” a reader might place you as a medical anthropologist. But perhaps you’re actually interested in what people consider to be “good” or “virtuous,” above what they desire as “healthy,” in which case you might be an anthropologist of ethics. Or, maybe you are a medical anthropologist of ethics. In any case, by providing the reader with a sense of what your conceptual question is will allow those who might not know anything about your research sites a way to begin to understand how to position your work within their view of the discipline.

For most research proposals, you’ll only have one or two conceptual questions. You may not even explicitly pose them in your proposal, though a reader should be able to reasonably guess at what yours are. But knowing your conceptual questions will help you develop your other questions, and orient you within the discipline.

Research Questions

The abstract things that you ask about in your conceptual questions must have some concrete manifestations in the world. Anthropologists may talk about “kinship” but what really exists is people doing things with each other in more-or-less repetitive ways that anthropologists have decided can be usefully discussed using a concept called “kinship.” Research questions show the reader where specifically you think these concepts can be fruitfully investigated in the world. Thus, research questions are more concrete questions than conceptual questions: they get down to the specific who and what of your research. They are the key questions you will ask in your proposal and answer in your thesis.

So, if your conceptual question is “How do new technologies affect human life?” then your research question might be: “How is being an avatar on Platform X changing the way users navigate interpersonal relationships?” If your conceptual question is “What is capitalism?” then your research question might be “How is traditional textile production in Taiwan being affected by global trade?”

In anthropology, research questions are usually “How?” questions, even if they are not written in that way. This is because, in the majority of cases, your research will be about ethnographically establishing that some unstudied or understudied relationship exists between one thing and another thing. This is the case in most research. In the natural sciences, a study often tries to understand a causal relationship between one measurable thing—the independent variable—and another measurable thing—the dependent variable. (e.g. How is the motion of an electron affected by the presence of a strong magnetic field?) In anthropology, the relationships being explored are not usually causal, but analogical or otherwise co-occuring, but the point that the key aim of a study is to find the relationship between a thing and another thing that contributes to what that thing is holds, and your research question should reflect what you understand to be an important relationship between them.

Where the research questions are not explicitly posed as “How” questions, the relationship between one thing and another thing is usually present in more subtle ways. To take a classic example, Evans-Pritchard may have asked “What are magic and witchcraft among the Azande?” In this case, the dependent variables are “magic” and “witchcraft” and the independent variable is “among the Azande.” Unwieldy as it is, you could restate the question as “How are magic and witchcraft changed by being among the Azande?”

By formulating a research question in this way, you articulate a starting point for empirical study. For instance, the avatar question above shows clearly that you should start by trying to get information about interpersonal relationships from users of Platform X, and also that you should pay special attention to how people talk about or experience changes in those relationships associated with their use of the platform.

When you are writing a research proposal, you may only have a vague sense of what the relevant relationship is, but you hopefully have some idea. This can come from pilot research, personal experience, or a reading of the literature. In other words, you will have a hypothesis about what you will find in your research. Therefore, another function of a research question is to give you the opportunity to state this hypothesis. A hypothesis is a statement that answers the research question, and which can be tested for validity. For example, you may believe that users of Platform X become more isolated in their offline relationships as they invest more of themselves into their avatar. In your actual research, you can then go and test this hypothesis. Are users really becoming more isolated in real life? If so, is it because of their use of the platform?

Your research question must be posed in a way that a clear hypothesis can be given in response. One thing to be careful of here is that the research question should not pre-suppose too much of the hypothesis. You must be able to clearly separate the research question you are asking from the answer you think you will establish. A poor research question would be something like “How do people deal with the isolation in their offline relationships caused by using Platform X?” This question already implies that the major effect of using Platform X is isolation, and moreover that using Platform X causes that isolation. The question is posed in a way that already takes things you should probably find out for yourself as already known. Taken as a starting point for empirical study, it means that you might end up ignoring all the ways that people might become less isolated offline, or if they do appear to be becoming more isolated, all of the other things that could be contributing to that isolation that have little to do with using Platform X.

In practice, writing a clear research question is tricky, but it becomes easier with experience, and also by knowing how other people construct and answer their own research questions. More on this later, when I discuss literature reviews.

Field Questions

Field questions are the most concrete questions. If your research question is “How is being an avatar changing the way people navigate interpersonal relationships?” then good field questions might be: Who are the “people” you are interested in? What is an avatar, and how do people choose or create them? Who do they create or maintain relationships with, and how? Field questions are what you will end up asking directly to people in an interview, or trying to answer through participant observation, or asking on a survey, so that you can gather information that will help you answer the research questions.

Good field questions will help bring clarity, detail, and nuance to key terms in your research question. A key term in the earlier example “How is traditional textile production in Taiwan being affected by global trade?” is “production.” Field questions about “production” will help you be able to explain what it is in the context of traditional textiles in Taiwan: Who is doing the producing? What activities do they perform to produce? What do people say about their productive activities? What are the types of products they produce? Where and when does production happen? What other types of production are going on? And so on…

A Dynamic Hierarchy of Questions

As I’ve been discussing, conceptual questions are the most abstract, field questions are the most concrete, and research questions come in between. By “abstract,” I mean that they are about things that they are about things that are present generally, if not universally, in human life. This means that conceptual questions are relatively devoid of specific details. They are about the concept “the sacred” rather than about Christianity or Islam or Buddhism. By “concrete,” I mean that they are about specific details: this person, instead of “people”; this ceremony happening at this time and place, rather than “ritual.”

Therefore, you must be able to arrange conceptual questions, research questions, and field questions in a hierarchy from abstract to concrete. In this hierarchy of questions, you should be able to see that at each level, questions are about different things, i.e. “ritual” rather than “a baptism.” It should be possible to see the research questions as a specific manifestation of a general conceptual question, and each of the field questions as manifestations of the research questions.

You should also be able to see how the key terms in those questions are related to each other in a hierarchy of abstract to concrete. So if a conceptual question is about “the sacred” then there should probably be research questions about “Christianity” or “Buddhism,” and then field questions about “baptism” or “enlightenment” or whatever else. (Exactly what terms go at what level in the hierarchy will depend completely on how you design your project.)

There is also a hierarchy in the number of possible questions that can be posed at each level. For every conceptual question, there are hundreds of research questions that could be asked. Each research question should lead to hundreds of field questions. In any single research proposal, you will probably only have one conceptual question, maybe one or two research questions, and many, many field questions. Conversely, it should be possible to see how answering enough of the field questions can lead to answering the research questions, thereby helping to answer, at least partially, the conceptual questions.

In the actual practice of research, the questions at each level while cause you to rethink and revise questions at other levels. To give a trivial example, you may find that users of Platform X do not in fact use avatars at all, but are taking care of virtual pets. In response, you may decide to change your research question so that it is no longer about avatars but about virtual pets, or maybe you will change your question to be about a different platform. In either case, answers to your field questions will have significantly changed your research questions. You will then adjust your field questions to suit the new research questions. This is normal, and is a feature of ongoing ethnographic research. You discover things are not quite what you thought, and you adjust.

Notice, however, that the conceptual question (“How do new technologies affect human life?”) does not change. In some cases, the conceptual question may change, but this means that things were really not what you thought. Or, it might mean that you want to do a different kind of project as a different kind of anthropologist. This happens too, but probably less frequently.

The point is that, when you’re actually doing research, the hierarchy of questions will turn out not to be as rigidly top-down as it might appear in the research proposal. Nevertheless, the research proposal should usually be presented in a way that a reader can follow how certain conceptual questions imply the research questions which imply field questions; and be able to see how answering the field questions will answer the research questions, and then the conceptual questions. This is often what a reader of a proposal means when they say “logical organization” or something similar about a research proposal

The rest of the proposal

In terms of research design, having clarity about your questions and the relationships between them is crucial for building up the justification for your research project. Depending on the type of proposal that you are writing, you may spend the first one to three pages introducing and explaining your questions. If you’re successful in this, then the literature review and methodology sections, which usually follow, should be on firm logical footing. (I plan to address these other sections in other posts.)

What is “Labour-Based Grading”?

(This is an explanation of Labour-Based Grading that I provided to a first-year anthropology class I taught in 2020.)

Labour-based grading is not what you are used to, so please read the following very carefully so that you understand how this assignment will work. 

This course is entitled “Social and Cultural Diversity.” We will be learning about the incredible diversity of ways that people around the world live their lives. We will see the different ways that people use symbols to interpret the world around them, or how things like family and gender take different forms depending on when and where you look for them. And, we will be learning about many methods and concepts that will help us interpret what these differences mean, the ways that societies and cultures can be organized, and the vast diversity of lifestyles and worldviews that human beings hold around the world.

One of the key ideas that we will be learning about is ethnocentrism. Briefly, ethnocentrism is a standpoint in which a person takes their own ideas about the world as the standard against which the lives of other people are interpreted and measured. For instance, I am used to thinking that wearing your shoes inside the house is inappropriate. I take my shoes off at the door and get annoyed if I see my kids stomping into the house with their shoes. However, when I lived in the US, many people around me wore their shoes inside their homes. If I told people off for wearing their shoes inside their own homes, then I would be acting ethnocentrically. 

In some ways, all of us are ethnocentric: it is nearly impossible to understand the world without drawing on your own background and experiences. Anthropologists, however, must learn to become aware of our own ethnocentrism and work around it if we want to be able to understand how people in other societies live their lives.  We need to be able to see, to the greatest extent that we can, how “they” experience and interpret the world. We need to try and see what “their” standards for seeing the world are. This is a stance we call cultural relativism—each society or culture needs to be understood according to its own terms. If we fail to do this, then other peoples’ lives will only ever seem ‘strange’ or ‘irrational’ or ‘backward’ to us. 

If individual people are sometimes ethnocentric, then the same is true of large institutions, including governments, companies, and universities. The university, for example, has been built over time into a place that prides itself on having “high standards” and educating people to meet those standards. These standards have developed over decades if not centuries, and they reflect the values of the people who established the universities. These standards shape many things, including who is accepted to university, what they are taught, and who is selected to teach them. 

In many universities around the world, including Vic, these standards have begun to change. For instance, in the past, most of the people who studied and worked at universities came from similar ethnic, racial, and class backgrounds. There were also more men than women. Today, we in the university community have much more diverse backgrounds. We bring new and different experiences and ideas to the university, making it a more interesting and stimulating place to be. 

However, there are some parts of the university that have not changed very much. One of the most mundane, but powerful is the system of grading.

In classes like this one, grades are meant to be an indicator of a number of things. For example, students who receive high grades are meant to: 

  1. Have a functional understanding of key concepts in anthropology. 
  2. Have a knowledge of a range of ethnographic material illustrating social and cultural diversity. 
  3. Have the ability to draw connections between ethnography and theoretical concepts and among ethnographic cases. 
  4. Have skills in critical reading, bibliographic research and citation, and the clear presentation of ideas, in oral and written form. 

The above four points are the “Course learning objectives” for ANTH102. Other courses in the humanities and social sciences will have similar ones. 

Assignments, courses, and the university as a whole are set up so that people who achieve As in courses are deemed to have achieved “excellence” in all of these areas, while people who receive Cs or Ds have not.

One of the main ways that students are assessed is through written essays. The assumption is that students who write clear, organized, and well-structured essays have achieved all four of the course learning objectives above. In other words, we assume that “writing quality” directly reflects the quality of a person’s effort and understanding. This is, on the surface, a very reasonable assumption. It is reasonable to assume, for example, that a person must have a good understanding of anthropological concepts and the ability to “draw connections between ethnography and theoretical concepts and among ethnographic cases” in order to be able to write a clear and persuasive essay. 

But in practice, this assumption does not always hold up. A student’s ability to write an academic paper does not alwaysmean that they have understood what the course is teaching to a sufficient degree. In fact, we probably all recognize a little bit, that successfully passing a writing assignment in university is often made much easier if you’re skilled at producing, for lack of a better term, bullshit. 

In fact, we could say that courses like this one are often set up to push students to produce bullshit. According to one definition, something is bullshit when the person who said or wrote it doesn’t know (or doesn’t care) whether what they are saying or writing is correct or not; they only care to convince you that they are saying it or writing it sincerely. 

According to philosopher Harry Frankfurt, in a book entitled On Bullshit (2005), there are certain conditions that make people more likely to produce bullshit: 

Bullshit is unavoidable whenever circumstances require someone to talk without knowing what [they] are talking about. Thus the production of bullshit is stimulated whenever a person’s obligations or opportunities to speak about some topic exceed his knowledge of the facts that are relevant to that topic. (63) 

One of the things that distinguishes a good bullshitter from a bad one is how well they know how something should look and sound. A person who knows what a good academic paper should look like is much better able to write a sincere-sounding paper than a person who has never read or written an academic paper until this year. While lecturers and tutors believe themselves to be skilled at detecting bullshit, the reality is that we often unconsciously let ourselves believe that somebody who has written a nice-sounding essay knows what they are talking about. 

Conversely, a student who may have an extremely strong understanding of anthropological ideas and has been 100% sincere in their efforts to learn and write in a course may not yet have gained the skills to communicate them in academic writing. When this happens, our unconscious biases can work against the student: we sometimes end up penalizing students who don’t write well, because we assume that they don’t understand what they are writing about. 

So, what does this have to do with ethnocentrism? When lecturers and tutors make these unconscious judgements about the quality of a paper, we are doing so by relying on our own standards of what “appropriate writing” are. These standards are based on, to put things bluntly, White middle and upper class values, just as much of the university is. By “White” here, I don’t mean that only White people hold these standards. I mean that if you are White in a society like New Zealand (or in much of the “Western” world), these standards are more likely to feel “obvious” or “natural” or “familiar.” These standards may have been the ones that the majority of people who came to university fifty years ago had before starting their first year. This is not the case now. 

This means that even before anyone has set foot in a university classroom, some students, because of their backgrounds, will have an easier time getting an A, regardless of whether or not they understand any of the material presented. Other students will struggle to get an A even if they have an excellent, critical understanding of what the courses are trying to teach. The difference between these two groups is not that one group of students is “good” or “smart” and the other “bad” or “stupid.” The difference is that the first group has to deal with a much smaller gap between what they are used to doing and what the university expects, while for the other group there is a bigger gap, especially when it comes to written communication.

Conventional academic standards are still important. Inside university and especially in the “working world” beyond university, there are going to be many situations where being able to write clearly and “correctly” in the ways that some people expect will be an important part of achieving success. 

But in a first-year course like this one, we are all still learning. And it strikes me as incredibly unfair to limit any student’s chances for a good grade based on a mismatch between the ‘traditional values’ of the university and the values they carry from their backgrounds, especially if different students are putting in the same amount of labour into their writing. Shouldn’t students who put in a similar amount of work be eligible to receive similar grades?

This is the idea behind “labour-based grading.” Labour-based grading systems have been developed by scholars and teachers of English writing. These scholars want to help students how to write well, but they also have observed that grading them only on the basis of the writing quality of the final product puts students from non-White, non-middle-class backgrounds at a significant disadvantage. The effect is that some students do not have an equal opportunity to achieve grades across the entire range, up to A+. 

Instead, they propose using the amount of labour that each student spends on their writing to determine their final grades. In short, if you spend the required amount of labour on your writing, you will receive a standard grade on the assignment regardless of the “quality” of the final product.

For these scholars, the quality of the writing their students produce still matters. They spend a great deal of time giving feedback on students’ drafts, because spending time and energy on writing is what makes writing better.

For the Article Review Assignment, we will be using a labour-based grading system. 

At this point, you may have questions. 

One might be, what is labour and how could you even measure it? 

Think of “labour” as what a person does in order to produce something of value. We can all probably agree that the more time you spend practicing something, the better you get at it. For the purposes of this course, labour is measured in time used performing a task or word count of the product of a task. These are imperfect ways of measuring labour, but they are better than the alternative, as I’ll discuss now. 

If labour is what a person does in order to produce something of value, we still need to know what “value” is. What is “value”? For the purposes of this assignment, we need to see that there are two kinds of value produced by the labour you perform. We’ll call them (1) “value to you (the student)” and (2) “value to me (the lecturer, or the university)”. 

“Value to you” is the value that acquiring new knowledge, gaining new skills, and building new relationships has to you as an individual. “Value to you” is very difficult to measure as a number or a letter grade, because different things are important to different people, so the value of a piece of knowledge or a skill will vary from person to person, depending on who they are and what they want out of life. However, as we saw above, we can probably agree that the more time you work on something, the more “value to you” is likely to increase. “Value to you” is embodied in you as a person, your skills and your experiences.

“Value to me” is how I, as the lecturer and as a representative of the university, value your work. This is the kind of value that grades are supposed to measure in ordinary grading systems. Your (the student’s) labour produces an essay or test answers, and I assign them a value based on standards that I and the university have for measuring its value. Ordinarily, “value to me” is based on a judgement of the end product in comparison with some academic “standard.”

(In Week 11, we’ll be discussing the ideas of Karl Marx. “Value to you” is similar to what he called “use value” of a commodity, and “value to me” is similar to what he called a commodity’s “exchange value.”)

When your labour produces something of “value to you,” then you have learned something or gained something from the course. Ideally, the “value to me” faithfully reflects the “value to you”: the value you gain from your labour should be recognized by an appropriate grade from me.

But in reality, the “value to me” (the lecturer and the university) is out-of-sync with students’ individual “values to you.” This is because the traditional “standards” of the university can be very different than the “standards” that students have for themselves. This is a version of the problem of the ethnocentrism of the university I mentioned above.  

You, as a student, see that these standards are out-of-sync, and will react in a very understandable and rational way: you will try to minimize the labour you perform while maximizing the “value to me” (the grade, based on traditional standards) you get from that labour. In addition, you will tend to avoid labour that contributes to “value to you” if it takes away from labour that creates “value to me.” 

In short, traditional grading systems can create incentives for students to work against their own educational growth. 

Labour-based grading tries to address this problem by making grades only depend on the amount of labour that you put into the course (as measured by time used and words written), in order to more closely make “value to you” match “value to me.”  

One thing is extremely important to point out: we, the tutors and lecturer, still care about the quality of your work. We will be working to get you individual feedback on your writing, more than in similar courses and at times when it is more useful to you. But labour-based grading is designed on the principle that better quality comes from more labour. This is why your grades in this assignment will reward the amount of labour you put in.  

The basis of this system is “The Grading Contract,” which defines exactly what tasks must be completed and how much time you must spend on each task.

If you clear all of these tasks on time, you will receive a B on this assignment.

The grading contract also specifies the tasks you can complete to receive a higher grade, and the penalties for missing any of the tasks.

Academic Authority and Institutional Power in Anthropology

Update (2022/2/14): Prompted by a question on Twitter, I went back to my data and re-discovered that in 2020 I had updated my data to include dissertations up to 2017, and had put it through an additional round of cleaning through OpenRefine. This shows that John Comaroff is actually ranked 7th in term of dissertations supervised, rather than 3rd as my original post said. I’ve included an amendment below, but otherwise the post is unchanged.

Recent events have raised perennial questions about academic authority and institutional power in anthropology. There are persuasive allegations about more than one famous anthropologist—most recently, John Comaroff—at a powerful institution behaving very badly.

One aspect that may be very difficult for outsiders to anthropology to appreciate, but which is also felt more than known by people within the discipline, is the extent of the influence that these scholars have over the field. Junior scholars were fearful that their future careers were at risk if they spoke up against their famous supervisors. What were they fearful of?

Prompted by a question by Zoe Todd on Twitter, I’ve gone back to an old project where I mapped the networks of citations and doctoral supervisor-student relationships in cultural anthropology.

While my data cannot show who supervised who too far into the past, it does show who of the recent past can exercise a tremendous amount of influence. The data also shows who all of these scholars’ students were, but I’ve decided not to list the students’ names.

The selection of data presented below shows how central to anthropology some of these scholars are.

(This article in American Anthropologist does similar analyses with different data.)

Doctoral Supervisor-Student Connections

Using metadata from Ph.D. dissertations indexed by ProQuest, I extracted the names of dissertation authors and listed supervisors, and used them to produce a map of supervisory relationships. (Note that Proquest’s data is only good for this information from the 1980s, and focuses mostly on US universities. I don’t think any major European universities are represented. As far as I know, the data I have is complete up to about 2016.)

The entire dataset consists of thousands of names and connections. The next image shows some of the most central nodes in the network. Here, “central” means those names with the greatest number of outgoing connections: supervisors who supervised the greatest number of students. These nodes are the largest.

A portion of a network image showing doctoral supervisor and student relationships in cultural anthropology.

You’ll be able to discern the names Jean Comaroff and John Comaroff. In terms of the number of students they supervised (or at least were listed as supervisors for), they rank 1st and 3rd respectively across all of the indexed cultural anthropology dissertations.

This corresponds to 40 students supervised for John Comaroff and 65 for Jean. For comparison, Stevan Harrell (2nd place) has 41, George Marcus is tied for 3rd at 40, and John Kelly is 5th with 39. The vast majority of PhD theses writers have no students who have produced anthropology dissertations, and most have only 1 or 2, though this data does not catch people who supervise students outside of cultural anthropology.

The numbers shouldn’t be treated as exact, because the data is quite dirty. Nevertheless, it’s clear that John Comaroff is among the most prolific of doctoral supervisors in the US.


(Added: 2022/2/14)

Using slightly newer and cleaner data from 2020 that I rediscovered on my computer, I’ve arrived at slightly different rankings. John Comaroff has 38 supervisions rather than 40, and is ranked 7th between John Kelly and Michael Silverstein, rather than 3rd, as I originally wrote above. I don’t know the exact reasons for the difference, but it’s likely to be due to the removal of duplicate entries when I cleaned the data in 2020, and to changes in the script I used to extract the data, which I believe originally introduced some errors. The graphic below shows the top 30 from my revised data. (Confusingly, the information on institutions I discuss in the next section was created from the revised 2020 data.)

List of the top 30 most frequently listed Ph.D. supervisors, up to 2017.

Institutional Power

From another perspective, we can see how powerful the institutions that famous scholars have belonged to are in the discipline.

This is an image I originally posted here, when I discussed the issue. Here, each dot is a university rather than an individual. The lines show connections between the university where a person received their Ph.D. and the university where they produced a Ph.D. student in anthropology.

The top row of institutions is the “elite” tier, including Harvard and the University of Chicago. They produce many, many students, but importantly, those students produce more students in anthropology, so this small group of universities ends up defining what the field is across generations.

The fan shaped patterns around the “elite” schools are indications of how many of its own students a department hires as faculty. The fan pattern is the largest for Chicago. More comments on this image are in the original post.

Scopus and ISI severely undercount citations in the humanities and social sciences

At Anne-Wil Harzing’s Publish or Perish:

When comparing Google Scholar and ISI citation scores, the Business academic has six times as many citations in Google Scholar than in ISI. However, this difference is dwarfed by the increases for the Political Scientist (15 times), the Cinema Studies Academic (25 times), the Linguist (32 times) and the Educationalist (33 times). It is clear that for academics working in the Social Sciences and Humanities, a focus on citations to ISI listed publications only, severely underestimates their citation impact.

With regard to the differences between Google Scholar and Scopus, the Social Science and Humanities academics again follow a different pattern from most of the Science academics. Whilst for the Science academics differences between Google Scholar and Scopus are slightly larger than between Google Scholar and ISI, for academics in the Social Sciences and Humanities, they are slightly smaller, but still substantial in absolute terms. Differences run from 3.5 times as many citations in Google Scholar for the Business academic to 27 times as many for the Linguist. The Cinema Studies academic shows a 180 times increase in citations, but that is caused entirely by the very incomplete coverage of her publications in Scopus.

People might find her Publish or Perish software useful for locating citations and calculated their own metrics.