Academics working today are laboring in a rapidly changing landscape of scholarly communication.

When acclaimed Internet researcher danah boyd published her recent book, “It’s Complicated,” about the social lives of networked teens with the highly reputable academic house Yale University Press, she also put a free PDF of the book up on her own website.  She wrote this about that decision:

“…I didn’t publicize this when I did so. For those who are curious as to why, I want to explain. And I want you to understand the various issues at play for me as an author and a youth advocate.

I didn’t write this book to make money. I wrote this book to reach as wide of an audience as I possibly could. This desire to get as many people as engaged as possible drove every decision I made throughout this process. One of the things that drew me to Yale was their willingness to let me put a freely downloadable CC-licensed copy of the book online on the day the book came out. I knew that trade presses wouldn’t let a first time author pull that one off. …But what I started to realize is that when people purchase the book, they signal to outside folks that the book is important. This is one of the reasons that I asked people who value this book to buy it. Your purchasing decisions help me signal to the powers that be that this book is important, that the message in the book is valuable.” (emphasis in the original)

It’s an important and worthwhile book, and you should buy it and/or download it, depending on what you can manage. What I so appreciate about what she’s done here is to find a way to thread the very thin needle of open access and a prominent, scholarly book.

Elsewhere in that post, she describes her experience with the machinery of publishing, and it goes like this:

“If you haven’t published a book before, it’s pretty unbelievable to see all of the machinery that goes into getting the book out once the book exists in physical form. News organizations want to promote books that will be influential or spark a conversation, but they are also anxious about having their stories usurped by others. Booksellers make risky decisions about how many copies they think they can sell ahead of time and order accordingly. (And then there’s the world of paying for placement which I simply didn’t do.) Booksellers’ orders – as well as actual presales – are influential in shaping the future of a book, just like first weekend movie sales matter. For example, these sales influence bestseller and recommendation lists. These lists are key to getting broader audiences’ attention (and for getting the attention of certain highly influential journalistic enterprises). And, as an author trying to get a message out, I realized that I needed to engage with this ecosystem and I needed all of these actors to believe in my book.”

Her experience with publishing is quite different from the traditional academic’s experience, but then that might be expected as danah boyd is not a traditional academic.  If you’re not familiar, danah boyd is something of a celebrity among folks who study the Internet, works as a Principle Researcher at Microsoft, and is starting her own research shop called Data & Society. Her work is also on two areas  — the Internet and teenagers — that has wide public appeal.

The reality for most traditional academics is that they produce “Long, complex monographs are expensive to produce yet sell only 150 to 300 copies.”

The news is even worse for academic papers published in traditional journals. A study at Indiana University found that:

“as many as 50% of papers are never read by anyone other than their authors, referees and journal editors.” That same study concluded that “some 90% of papers that have been published in academic journals are never cited.”

This is a certain kind of impact, to be sure, if who you are trying to have an impact on is an elite group of specialists in your field.  But this model of publishing is never going to have much of a wider reach.

As Anthony DiMaggio, writing for CounterPunch, notes about his own field of Political Science, that it is dominated by “over-specialization and obscurity” with scholars who carve out “extremely narrow niches” that have “no practical utility.”  DiMaggio minces no words as he calls out social science academics broadly for a lack of relevance and what he deems as cowardice:

“Lack of relevance to the political world doesn’t make one’s research interesting or worthwhile, but this message falls on deaf ears in insulated places like high ed social science departments.  A main reason for scholars’ contempt for political advocacy is cowardice.  The vast majority of scholars have been socialized their entire lives to believe they must always remain ‘objective,’ and that to take a position on an issue would be heretical.  Most scholars operate according to a pack mentality – fearful of engaging in unconventional behavior.  By producing useful real world research, one is challenging the sacred rules governing ‘objective’ social science that celebrate esoteric research agendas. To step outside that mold would be to endanger one’s prestige, and risk that one will be seen as unprofessional in colleagues’ minds. Such pressures ensure that academics remain part of the problem, not the solution. They fail by design to challenge the political and economic power status quo and injustices that occur around them.”

There’s something to what DiMaggio says here, but I don’t know if it’s cowardice as much as institutional reward structures.  Or, perhaps those are two sides of the same coin.

The legacy model of scholarly communication values writing obscure books and papers for tiny audiences makes sense within a certain kind of reward structure. Within legacy academia, the people that sit on hiring, tenure and promotion committees still place value on at things like ‘impact factor’ of little-read journals and the fading prestige of boutique publishers with minuscule runs.

However, the appearance of digitally fluent, hybrid scholars – like danah boyd – who are more interested in reach and impact on a broader public, point to a new kind of reward structure, one that values influence beyond a small group of specialists.

The real challenge, I think, comes when a researcher that doesn’t have the star-power or following of a danah boyd wants to write about something that’s much less appealing than what teenagers are doing on the Internet.  What kind of broad reach or impact can a relatively unknown scholar writing about a topic that’s unpopular expect to have? This remains an open question in this changing landscape of scholarly communication, but it seems to me that the Internet offers a set of opportunities to reach beyond the conventional audiences for academic research.

Still, even when academics use social media there’s little to indicate they are doing so in order to reach a broad, general audience. Indeed, we know from recent research that even when academics use social media, such as blogging, they mostly don’t do this to engage with a broader public. In a recent study of 100 academics blogs, researchers found that most academics are blogging for professionals peers, rather than for the public in any general sense: 73% of the blogs analyzed were geared toward other academics, while just 38% were designed for general readers.

I can’t help but wonder how different academic research would look if we were guided by danah boyd’s goal: “I wrote this book to reach as wide of an audience as I possibly could.”  

The counter to this, of course, and one that I often hear in talks I give about this work, is something along the lines of: “well, small publishers and journals are providing a valuable service for getting academic work published that wouldn’t ever be interesting to a wide, public audience. This work is often too complex, theoretical, esoteric, important, too politically unpopular for a wide audience, so we must rely on the obscure publishing options to keep doing what we do as academics.”

There is something to this argument.  For example, I write about racism – a thoroughly unpopular topic in the US.  My academic books have done ok, but they will never be as popular as the work that danah boyd does.  It’s also the case that academic presses have published books of mine that probably would not have been picked up by trade presses for a general, public audience. Still, what I also know to be true is that the work I do on racism has gotten a much bigger following from my various social media outlets than it has from the books and articles I’ve published.

The skepticism about “reach” for academic work is built on a misconception that there won’t be an audience for that work. In fact, I think there are multiple audiences, varied publics and a wide citizenry that’s really interested in more substantive contributions about the state of the world than they’re currently getting.  And, I think academics can step up and make a contribution, if we’ll begin to re-think what scholarly communication is now.

As we come to the end of the meta-MOOC #FutureEd conversations prompted by Cathy Davidson and our lunchtime interlocutors here at CUNY, I’m pondering how we measure the impact of such a course, and more broadly, the impact of the work we do as academics.

In a recent post at The Chronicle of Higher Ed, Douglas Howard considers the unfinished feeling of so much teaching in general:

Did Jim, who talked about becoming a therapist, go on to graduate school in psychology? Did Jessica, who argued so passionately in class against the death penalty, make it as a lawyer? We fill in the blanks about them based upon what we know (or think we know), and tell ourselves that their stories ended the way that we hoped.

Teaching, in this regard, is the great open-ended narrative, the romantic fragment, the perpetually unfinished symphony.

The fact is, we almost never know if a course we taught, or a book we wrote, or an article we labored over, has any impact on anyone else.

Mark McGuire has a thoughtful post at HASTAC about the difficulty in rating transformational experiences specifically in the context of a MOOC, such as the one on #FutureEd:

We rate hotels, music, live performances, movies, etc., so that others are able to make an informed decision about how to best invest their time and money. Rating and reviewing MOOCs seems like a sensible thing to do for similar reasons, and it would not be surprising to see such a practice develop. However, unlike a hotel or restaurant franchise, a living, changing, organic learning experience cannot be packaged, replicated, and sold to consumers who are looking for a satisfying (and predictable) product or service. It can’t offer the same experience to the same person twice, and one person’s experience may not be a good indicator of the experience another person will have. A transformational learning experience is like a good pot luck dinner party — you might have had the time of your life, but it can never be repeated.

I like the dinner party metaphor for teaching much better than the “unfinished symphony,” perhaps because I’m much more likely to attempt a dinner party than a symphony.  I do think that there is a kind of alchemy in teaching, and good dinner parties, that makes it easy to assemble the same people, elements and conditions but difficult to replicate magic when it happens.

It’s also difficult to think about how one might measure any of this. Unfortunately, to my way of thinking, the word “measure” has become synonymous with “quantify.”  When we think of measurement exclusively in terms of quantification and counting, we lose a great deal of the story of impact.

I want to suggest an alternative way of thinking about impact that includes qualitative measures. In the chart below, you’ll see a way of conceptualizing “transactional” measures – quantifiable things we can count, alongside “transformational” measure – qualitative measures, that are more difficult to count but represent more lasting change.

(Chart content from Bolder Advocacy, h/t gabriel sayegh.
Chart design by Emily Sherwood)

In this schema, transactional metrics include both traditional (or “legacy”) academic measures of impact such as citations, along with alternative measures (or “altmetrics”), such downloads, or mentions on social media. Quantitative, transactional measures of impact can also include lasting social change, such as changes to public policy.  One of the things I like about this conceptualization is that it illustrates the incremental change that altmetrics represent. In other words, altmetrics are just another way of counting things – downloads and social media mentions, rather than citations – but it’s still just counting things. Counting and quantification can tell us somethings, but it doesn’t tell the whole story.

On the “transformational” side are those things that it’s difficult, perhaps even impossible to measure, but that are so crucial to doing work that has a lasting impact. These include identifying allies, building relationships, establishing collaborations, and co-creating projects. Ultimately, transformational work is about changing lives, changing the broader cultural narrative, and changing society in ways that make it more just and democratic for all. These kinds of transformations demand a different kind of metric, one that relies primarily storytelling.

How might this work in academia? Well, to some extent, it already does. 

To take the example of teaching, you may have gotten the advice – as I did – “save everything” for your tenure file. This advice often goes something like, “everytime a student sends you a thank you card, or writes you an email, or says, ‘this class made all the difference for me’ save that for your tenure file.” That’s part of how we ‘finish the symphony,’ to borrow Doug Howard’s metaphor, we get notes from students, we compile those into a narrative about our teaching. It’s impartial, to be sure, but it’s something. The comments that students add to teaching evaluations are another place we see that impact in narrative form, although these are so skewed by the context of actually sitting in the class that it misses the longer term impact of how that course may have changed someone.

For the diminishing few of us on the other side of the tenure-hurdle, think about those letters of recommendation we write for junior scholars.  Whether we’re writing for someone to get hired, promoted, or granted tenure, what we’re doing when we craft those letters is creating a narrative about the candidate’s impact on their corner of the academic world so far. Of course, we augment that with quantitative data, “this many articles over this span of time,” and “these numbers in teaching evaluations.”

The fact is, we already combine transactional and transformational metrics in academia in the way that we do peer evaluations. What we need to consider in academia is expanding how we think about ‘impact’ and realize the way that we already use both quantitative and qualitative measures to evaluate and assess the impact of our work.

More than this, we need to reclaim storytelling and narrative, augmented by the affordances of digital media, to tell stories of impact that make a difference.