It's been a while, but I am finally (sort of) getting back to addressing some feedback that my colleague and I got on an article we are working on with regard to MOOCs.  My colleague, Zaharias, thought it would be a great idea to sit down and make an (initial) typology of issues around the development of MOOCs. The abstract was accepted for a special issue of a journal, but our final version was not.  This I found a bit odd, but I've taken the peer reviewer's comments to heart, and I am thinking of ways of addressing them.  Some, in my mind, are valid.  I "live" in my head, so for me terms like cMOOC, xMOOC, FSLT12, CCK11, MobiMOOC, connectivism and so on are second nature.  I think to some audiences there might be some need to explain what this alphabet soup means.  Other comments that I also took to heart revolve around typos that just sneaked under our radars. After 8200 words, and multiple readings, who could blame us ;-) As a newly minted editor, though, for the CIEE journal I realize how annoying this might be for peer reviewers and editors :)

Anyway, there was some feedback, though, that I take issue with. In order to do this initial typology our literature review spanned over 100 sources ranging from academic, peer reviewed articles, to conference presentations, to news items from Inside Higher Ed and the Chronicle, as well as blog posts from respected leaders in the field, such as Stephen Downes, George Siemens, and people who've run their own MOOCs.  We opted to only use academic literature that was free on the web in an open access format.  I do have access to library research databases, however I find it very odd, philosophically, to study open education and to rely on so many closed access journals in order to do my research.  I could definitely see the peer reviewer's point that we should go after more (closed access) academic articles  to increase the pool of our data, however I take serious issue with the following advice that we got:

The collected materials in this manuscript are not high quality document owing that most of them are informal documents. The authors should collect academic high quality papers to survey and analyze.

Just because something is not in an academic, peer-reviewed, journal doesn't mean that it does not have some sort of quality, and it does not deserve attention.  There have been quite a few blog posts out there that were pre-cursors to academic articles that someone in an academic article down the road deemed to have sufficient quality to publish. Furthermore, if someone has published in peer reviewed articles, and regularly gives conference presentations and keynotes, don't their blog posts carry more weight than mine or someone who's a novice in the subject?  And, last but not least, what happens in professional publications such as the Chronicle and IHE never makes it to academic publishing (see for example the issues with FOE MOOC and #massiveteaching). These things need accounting and addressing, and if they never make it to publication, for one reason or another, they become invisible to educational research. This seems like a big failure to me.

Another thing that piqued my interest is the idea of laying down the groundwork.  I understand that there needs to be some level of definition of terms in academic research, and when I eventually do a dissertation I expect that I will be defining a lot of terms before I get to the heart of my own experiments.  That said, in an academic article, which usually has a limit on the amount of words you can put in it, how much space do you use for definition of terms, and how much do you devote to the heart of the matter? How reasonable is it to expect that, in the age of Google and in light of your providing of a full reference list, that you don't necessarily need to briefly describe terms like xMOOC, cMOOC, CCK, and PLENK? At the end of the day, I am not averse to defining such terms (in addition to any existing citations) but I do wonder how this is counted against the authors of papers when there are defined word limits for articles?

Your thoughts? Any other peculiarities of peer review you have a bone to pick with? :)

Image from: http://www.phdcomics.com/comics/archive/phd081508s.gif