An Open Letter to ISI Web of Knowledge, Google Scholar, and SciVerse Scopus

A bit of context first. This letter arose as a result of discussions regarding citations practices in meta-analyses on Twitter several weeks ago. We all agreed citations of work within meta-analyses should be counted, but noted that many modern citation counting services disregard any citations not in the main body of a paper. Appendix citations are dropped, however. And due to a variety of restrictions, we often only cite the papers we use as data in appendices, and so are not counted. Our discussion resulted in the following letter that sixty-eight scientists around the globe have signed. I look forward to further fruitful discussion with the organizations above (who have been contacted) and hope that this situation is corrected.

Update 1: Within a few minutes, Chris from Web of Knowledge contacted me to make sure he had the details correct, and has forwarded this to the WoK Dev team. We’ll see if they get in contact.

Update 2: More signatures added on Nov 24, 2013. We’re now at 72.

Update 3: Despite also hearing from SCOPUS that this was being forwarded on to their product manager, no word back. Also, no word from GS, despite this being sent directly to members of their dev team by common contacts.

Dear Staff & Support Teams,

We are writing as a community of researchers in ecology, evolution and conservation biology who rely on the citation metrics you provide. We love your service, and, frankly, depend on it for a wide variety of reasons (e.g., identifying research trends, finding relevant papers, evaluating the influence of papers/researchers). However, in one respect your counts are becoming increasingly inaccurate. We have come to realize that citations – sometimes hundreds of them – that we have made in the appendices of our own published papers are not being counted by your service. We felt obligated to bring this issue to your attention, as it affects the accuracy and reliability of your statistics.

We are a group of researchers who conduct meta-analyses: analyses of published data, typically gathered by extracting data from published scientific papers. The standard practice in our and many other fields is to include citations to works that we use for data – but in the case of meta-analyses in particular, these citations do not necessarily appear in the main reference section of our papers. As meta-analyses regularly get their data from 50+ papers, and journals often have strict space limits and limits on the number of references, the citations are often included in supplementary material, most of which are online only (SOMs). These appendices are subject to the same level of peer review as the core article.

As far as we understand, citations in SOMs are not recorded by you. Hence, the papers we cite do not receive the proper citation credit, and the citation lists you provide for our own papers are incomplete. The failure to consider references in SOMs – particularly in the case of meta-analyses – severely reduces the accuracy of the citation counts you present for each paper. It is also poses a huge ethical problem for us because it means that we do not give proper credit, in the form of a citation, to the many superb scientists whose high-quality work is the jumping-off point for our own analyses. In turn, these colleagues of ours may be less inclined to be charitable when we come asking for data contributions to future meta-analyses.

We ask that you consider including citations garnered from supplementary online materials in academic journals.

If there is any way that we can facilitate this by contacting journals and editors in our own fields, we are more than happy to help. We feel very strongly about correcting this problem, and hope it can lead to some tangible benefits for both you and countless numbers of scientists.

Thank you, and we look forward to engaging in a productive dialogue about this issue.

Signed,

Jarrett Byrnes
Department of Biology
University of Massachusetts Boston
Boston, MA 02125

Ross Mounce
Department of Biology & Biochemistry
University of Bath
Bath, BA2 7AY
UK

Alexander Bond
Department of Biology
University of Saskatchewan
Saskatoon, Saskatchewan, S7N 5E2
Canada

John Griffin
Department of Biosciences
Swansea University
Swansea, Wales, SA28PP
UK

Mark Anthony Browne
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Gavin Simpson
Institute of Environmental Change and Society
University of Regina
3737 Wascana Parkway,
Regina, SK S4S 0A2
Canada

Michael P. Taylor
Department of Earth Sciences
University of Bristol
Bristol BS8 1RJ
UK

Julia Stewart
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Mary Hunsicker
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Robert Lanfear
Ecology, Evolution, and Genetics
Australian National University
116 Daley Road, Acton
Canberra, ACT 2602
Australia

Ethan White
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Morgan Ernest
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Noam Ross
Department of Environmental Science and Policy
University of California at Davis
1 Shields Avenue
Davis, CA 95616
twitter: @noamross

Trevor A. Branch
School of Aquatic and Fishery Sciences
University of Washington
Seattle, WA, 98195
United States
twitter: @TrevorABranch

Philip Martin
Centre for Ecology & Hydrology,
Wallingford,
Oxfordshire.
OX10 8BB.
UK.
twitter:@_PhilMartin

Kyle Edwards
Kellogg Biological Station
Michigan State University
3700 E. Gull Lake Dr.
Hickory Corners, MI 49060

Jonathan S. Lefcheck
Virginia Institute of Marine Science
The College of William & Mary
Gloucester Point, VA 23062

Terry McGlynn
Department of Biology
California State University Dominguez Hills
1000 E. Victoria St.
Carson, CA 90747
USA

Karthik Ram
Museum of Vertebrate Zoology
University of California, Berkeley
Berkeley, CA 94720
USA

Daniel Falster
Biological Sciences,
Macquarie University NSW 2109,
Australia

Bradley Cardinale
School of Natural Resources & Environment
University of Michigan
Ann Arbor, MI 48103

Ignasi Bartomeus
Postdoctoral Researcher
Department of Ecology,
Swedish University of Agricultural Sciences,
SE-75007 Uppsala, Sweden
@ibartomeus

Sean Tuck
Department of Plant Sciences
University of Oxford
South Parks Road
OX1 3RB

Helen Phillips
Department of Ecology and Evolution
Imperial College London
Silwood Park Campus
SL5 7PY

Mark Westoby
Dept of Biological Sciences
Macquarie University, Sydney NSW 2109 Australia

Thomas White
Dept of Biological Sciences
Macquarie University, Sydney, NSW 2109
Australia

Gregory Carey
School of Biological and Chemical Sciences
Queen Mary, University of London
London
E1 4NS

Adam Algar
School of Geography
University of Nottingham
Nottingham NG7 2RE
UK

J. Emmett Duffy
Virginia Institute of Marine Science
The College of William and Mary
Gloucester Point, VA 23062-1346

Eduardo S. A. Santos
Departamento de Ecologia
Universidade de São Paulo
Rua do Matão, 321 – Trav. 14, sala 243
Cid. Universitária – São Paulo, SP
05508-090
Brazil

Lauri Laanisto
Institute of Agricultural and Environmental Sciences
Estonian University of Life Sciences
Kreutzwaldi 5, 51014, Tartu
Estonia

Emilio M. Bruna
Department of Wildlife Ecology and Conservation &
Center for Latin American Studies
University of Florida
Gainesville, FL 32611-0430

Julian A. Velasco
Laboratorio de Análisis Espaciales
Instituto de Zoología
Universidad Nacional Autonóma de México
México, D.F. CP. 04510
Phone: (55) 5622-8222 ext: 47880

Timothée E. Poisot
Theoretical Ecology Group
Université du Quebec a Rimouski
Quebec Centre for Biodiversity Sciences
Rimouski, QC, Canada

Steven J. Cooke
Biology Department
Carleton University
Ottawa, ON, K1S 5B6, Canada

Mick Watson
Director of ARK-Genomics
The University of Edinburgh
Edinburgh, UK
EH25 9RG

Xiao Xiao
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Joshua King
Biology Department
University of Central Florida
Orlando, FL 32816

Andrew D. Steen
Department of Microbiology
University of Tennessee Knoxville, TN 37996

Chris Harrod
School of Biological Sciences
Queen’s University, Belfast
BT9 7BL
UK

Allen Hurlbert
Department of Biology
University of North Carolina
Chapel Hill, NC 27599-3280

Mike S. Fowler
Department of Biosciences
Swansea University
Swansea, SA2 8PP
UK

Dieter Lukas
Department of Zoology
University of Cambridge
Cambridge, CB23EJ
UK

Kate Boersma
Department of Zoology
Oregon State University
Corvallis, OR 97331

Ramón E. Martínez-Grimaldo
Instituto de Biología
Universidad Nacional Autónoma de México

Fernando T. Maestre
Universidad Rey Juan Carlos
Departamento de Biología y Geología
Escuela Superior de Ciencias Experimentales y Tecnología
C/ Tulipán s/n, Móstoles, 28933
SPAIN

Andrew B. Cooper
School of Resource and Environmental Management
Simon Fraser University
Burnaby, British Columbia, CANADA V5A 1S6

Josef C. Uyeda
Institute for Bioinformatics and Evolutionary Studies
University of Idaho
Moscow, Idaho 83844

Elita Baldridge
Department of Biology and The Ecology Center
Utah State University
5305 Old Main Hill
Logan, UT 84341
United States

Franciska T. de Vries
Faculty of Life Sciences
Michael Smith Building
The University of Manchester
Oxford Road
Manchester
M13 9PT
United Kingdom

Aidan M. Keith
Centre for Ecology & Hydrology
Lancaster Environment Centre
Library Avenue
Bailrigg, Lancaster
LA1 4AP, United Kingdom
@Aidan_M_Keith

Christophe Thebaud
UMR “Evolution & Biological Diversity”
CNRS & Univ Paul Sabatier
F-31062 Toulouse
France

Rafael D. Zenni
Department of Ecology and Evolutionary Biology
University of Tennessee
Knoxville, TN
USA

Natalie Cooper
School of Natural Sciences
Trinity College Dublin
Dublin 2, Ireland

Philippe Desjardins-Proulx
Université du Québec à Montréal
Quebec Centre for Biodiversity Sciences
Rimouski, Québec
Canada

Lars Gamfeldt
Deparment of Biological and Environmental Sciences
University of Gothenburg
Gothenburg, Sweden

Mark Hahnel
figshare
4 Crinan Street
London
N1 9XW
UK

Jessica Couture
National Center for Ecological Analysis & Synthesis
University of California, Santa Barbara
735 State Street, Suite 300
Santa Barbara, CA 93101-3351

Jarrod Cusens
Institute for Applied Ecology New Zealand
Auckland University of Technology
31-33 Symonds Street
Auckland
New Zealand

Steph Borrelle
Institute for Applied Ecology New Zealand
Auckland University of Technology
31-33 Symonds Street
Auckland
New Zealand

Nick Isaac
NERC Centre for Ecology & Hydrology
Wallingford OX10 9QA
UK

Sally Keith
ARC Centre of Excellence for Coral Reef Studies
James Cook University
Townsville, QLD 4811
Australia

Sarah Supp
Department of Ecology and Evolution
Stony Brook University
650 Life Sciences Building
Stony Brook, NY 11794-5245

Jessica Blois
School of Natural Sciences
5200 N. Lake Rd.
Merced, CA 95343
USA

Mike Whitfield
Botany, School of Natural Sciences
Trinity College
College Green
Dublin 2
Ireland

Richard J. Butler
School of Geography, Earth and Environmental Sciences
University of Birmingham
Edgbaston
Birmingham, B15 2TT

Jens Kattge
Max Planck Institute for Biogeochemistry
Hans Knoell Str. 10
07745 Jena
Germany

Scott Chamberlain
Biology Department
Simon Fraser University
Burnaby, BC, Canada V5A 1S6

Blaire Steven
Bioscience Department
Los Alamos National Laboratory
Los Alamos, NM, USA, 87545

Santiago Soliveres
Universidad Rey Juan Carlos
Departamento de Biología y Geología
Escuela Superior de Ciencias Experimentales y Tecnología
C/ Tulipán s/n, Móstoles, 28933
SPAIN

Francis Brearley
School of Science and the Environment
Manchester Metropolitan University
Chester Street
Manchester
M1 5GD
UK

Michiel van Breugel
Smithsonian Tropical research Institute
Av. Rossevelt 401
Balboa, Ancon
Panama, Panama

Dylan James Craven, PhD.
Synthesis Centre for BioDiversity Sciences (sDiv)
German Centre for Integrative Biodiversity Research (iDiv)
Leipzig, Germany

PeerJ Turns One!

x-posted from openpub

One year ago, one of the more intereting experiments in open access publishing – PeerJ – launched. It’s model of membership rather than paying by the article is still something people are grappling with – it’s just so different, so delightfully disruptive. Not only that, but PeerJ has stepped in to fill the void in providing a biology preprint server (which we have used), as well as coming up with a more intuitive interface for commentary on preprints and published work – along with supplying reviews alongside published work. A number of other great open access journals have tried one or more of these innovations, but few have tried them all *at once*. Not only that, but I have the feeling they’re not going to stop there.

After all, if they’ve tried this many new things in year one, I, for one, want to know what year two is going to hold…

Best Peer Review Experience Ever

So, I recently submitted a piece regarding the future of scholarly publishing in Ecology & Evolutionary Biology. Simultaneous to posting, I put up a preprint in PeerJ Preprints and also put it on Google Docs for line by line commentary (which you are welcome to give!). I asked in both places that commenters identify themselves, unless they felt deeply uncomfortable.

OMG the experience has been amazing!

At PeerJ you can comment on the main page of the article, and others can rate it – which is fantastic – and I’ve gotten some wonderful feedback there (thanks Lars!)

The Google Doc experience has been even more fascinating, given the ability to put in line by line comments.

One of our reviewers is using the Google Doc for their comments. It has made it easy to see what they are saying, respond to things that I think are relevant (or I’ll just change some of the text in the next draft for bigger changes), and have an interactive experience with the reviewer. It absolutely fabulous.

I’ve been really fascinated by the idea of how collaboration can improve peer review ever since reading Leek et al.’s 2011 piece Cooperation between Referees and Authors Increases Peer Review Accuracy. I’m delighted that one of our reviewers has embraced that ethos, and in so doing, I can see how this will really help with future publications if not just Ross Mounce, but everyone embraced this model. Very cool!

A Preprint Experiment: Four Pillars and a Foundation for the Future of Scholarly Publishing

x-post from the OpenPub Project blog

So, we got together, had two working group meetings to discuss the future of scholarly publishing in Ecology, Evolutionary Biology, and the Earth and Ocean Sciences. What were were thinking that entire time?

We’ve just submitted a piece that brings together our broad ideas (some of which have been seen before), but, simultaneous to publication, we’ve also decided to put up a preprint. Why? Simply put, immediate access is one of our four pillars of the future of scholarly publication. Once you feel something is ready for public consumption, put it out there! We’ve been delighted to watch the evolution of PeerJ Preprints, so we’ve placed our piece there.

Byrnes et al. (2013) The four pillars of scholarly publishing: The future and a foundation. PeerJ PrePrints 1:e11 http://dx.doi.org/10.7287/peerj.preprints.11

This immediate access to the piece goes hand in hand with another of our four pillars. Open Review. We want to know what you think. And now. We hope you give us feedback over at the preprint. Or, if you want to give us more detailed annotated comment, we’ve put it in a comment-open Google doc. Highlight something you disagree with. Argue with us. We welcome it! We’d ask that you put your name with the comment. We want a discussion, as discussion will improve this manuscript and help us shape our argument rather than just one-way commenting. This will also allow *you* to get full recognition for your comments, and we will include this in future acknowledgements.

So, enjoy the piece – our commentary is not a straight experiment-analysis-discussion piece, but rather part of a broader ecosystem of scholarly products that we feel are important to get out there. We look forward to hearing what you think of the piece!

I want to know what *YOU* think about review, preprints, and publication

As part of the OpenPub project, we’re soliciting folk to send us videos about their experience with the scholarly publication process. We want to use these to try and crowdfund the development of OpenPub – our preprint server with robust tools for discussion and interaction. Interested? Check out the full request over here and/or email me!

Post-Publication Commentary – a Google Hangout Discussion

After the fantastic NCEAS working group on the future of scholarly publishing in EEB (big success – stay tuned for more news on that front!), I participated in a really interesting online discussion about Post-Publication peer review with Carl Boettiger, Jamie Ashlander, and Scott Chamberlain. The conversation was great, and I think brought up a lot of interesting points – both that echoed some discussions in our group and went far beyond them.

There’s a lot of grist for the mill here – what is the roll of post-publication peer review, do we have an efficient system to conduct it, can we do better, and if so, how (and how can we do it simply instead of having to build some whole new platform)?

Take a gander, and I’d love to hear more of your thoughts about how we can better accelerate the pace of the scientific discussion in an efficient way.

Open Haus Conclusion: Peer Review or Bust!

Friday’s NCEAS discussion about changing the future of scientific publishing was fascinating. We had a wide variety of views about the current system, what it’s adding to our papers, if the problems really are problems, and why we got here. But there was one thing that came up again and again, that even the most skeptical status quo folk in the crowd agreed was where the true value of the current journal system lies –

Editorial peer review.

Not a single person in the audience denied that their papers were not hugely improved by the peer review process, as well as the editor’s comments and revisions that came with it.

And that was it, really. Any other model you please was deemed a non-starter by many if it didn’t include editorial peer review. The one thing which generally we all do as part of service (except at higher EIC levels – usually). There was some other talk of niceties – professional layout, etc., but it was also pointed out that much of that we can now do ourselves on our desktop.

So that would seem to be the gauntlet thrown. If you want to change the system, the new system still has to involve strong peer review – guided by editors or guided by the crowd or whatnot was not discussed (but, interesting to ponder). Without it, solutions will be viewed with skepticism or even derision (gauging by reactions). But with it, there is room for a viable alternative. Food for thought.

Open Haus: The Future of Scholarly Publishing

Today at The National Center for Ecological Analysis and Synthesis we’re having an open discussion about the future of scholarly publishing. I may post some notes from it later, but, Stephanie Pau and I have compiled a list of thought questions and helpful links to help folk prep. And, as it was an interesting gathering of linkage, I thought it mide be useful more broadly. So, below is the text of my email and the useful links. I realize that I am blogging my email. Is this a new low? Perhaps. Enjoy!

Hello, all! For today’s open house, we’ll be talking about the growing hubub regarding academic publishing and the relationship between scientists and publishers. Is it time for a change? What is the way forward?

This is a big topic, and not all of you may be aware of the things that have gone down over the past few months that are bringing this to a head. As such, Steph and I have put together a list of links that give you an introduction to all of this as well as some questions to bear in mind while perusing them. Don’t worry, they’re all pretty quick reads – short blog posts or an Xtranormal video which is brilliant.

Also exciting, we’re going to have a few guests who are involved in scholarly publishing from campus (and maybe beyond).

Marty Einhorn (KITP) for perspective on http://arxiv.org and the physics community’s take on this
Josh Schimel (EEMB) who is on the ESA publications committee
Chuck Bazerman (Education) a consultant for http://hypothes.is/

See you at 4pm in the lounge. And, as the weather has been gorgeous, if we want to continue the conversation after 5, Tony Ray’s?

Open House – Is it time for a change in scientific publishing?

Questions to Ponder

1. Do we need a change our model of scientific publishing? Why?

2. What needs to change?

3. Are scientists/ecologists ready for a change? Are we too conservative or slow to adopt change in general, open access in particular? I.e., good in theory, doesn’t work in practice? (http://www.nytimes.com/2012/01/17/science/open-science-challenges-journal-tradition-with-web-collaboration.html?ref=science)

4. What are the differences between commercial (e.g., Elsevier) and non-profit journals (e.g., ESA) that affect the exchange of scientific information (http://bjoern.brembs.net/comment-n820.html)? There does not appear to be a difference in quality as measured by the number of citations (http://octavia.zoology.washington.edu/publishing/ecology_citationprice.html)

5. Do commercial journals offer us something that non-profit journals do not? Prestige? What about differences between ESA and PloS (high costs put on author) models?

6. If the exchange of information is better served by open access, should we refuse to review for commercial journals?

7. Is there a difference between exchange of information for the sake of the discipline and personal academic success?

Resources on Open Access
http://oad.simmons.edu/oadwiki/Social_media_sites_about_OA

The Issue at Hand
An Introduction with some Humor
http://www.youtube.com/watch?v=GMIY_4t-DR0

The Research Works, H.R. 3699 Act & Responses from Scientists
http://thomas.loc.gov/cgi-bin/query/z?c112:H.R.3699:

http://www.michaeleisen.org/blog/?p=807

http://blogs.scientificamerican.com/evo-eco-lab/2012/01/06/scientists-fight-for-access/

http://osc.hul.harvard.edu/stp-rfi-response-january-2012

http://www.monbiot.com/2011/08/29/the-lairds-of-learning/

http://www.guardian.co.uk/commentisfree/2011/aug/29/academic-publishers-murdoch-socialist

ESA’s statement on Open Access back in Jan (that some on Ecolog-L were not too happy about)
http://www.esa.org/esablog/ecology-in-policy/esa-policy-news-january-13-2/

A Pledge to Not Publish in Elsevier Journals (e.g., TREE) with a lot of folk signing on
http://thecostofknowledge.com/

Further Meditations
Academic Spring?
http://www.slate.com/articles/technology/future_tense/2012/02/federal_research_public_access_act_the_research_works_act_and_the_open_access_movement_.html

Comments from Michael Hochberg
https://sites.google.com/site/perspectivesinpublishing/our-mission

Do publishers add value? Nature says yes.

http://www.nature.com/nature/journal/v481/n7382/full/481409a.html

http://www.michaeleisen.org/blog/?p=873

Publishers need us more than we need them
http://dougbelshaw.com/blog/2012/01/21/you-need-us-more-than-we-need-you/#.TyAbbuNSSWU

http://dougbelshaw.com/blog/2012/01/22/journals-academic-and-the-ivory-tower/#.TyAbieNSSWU

Oh, just go and read Michael Eisen’s blog already. I mean, he co-founded PLoS!
http://www.michaeleisen.org/blog/

Solutions?

Federal Research Public Access Act, or, Scientists Strike Back. #FRPAA
http://cyber.law.harvard.edu/hoap/Notes_on_the_Federal_Research_Public_Access_Act

http://www.michaeleisen.org/blog/?p=925

http://newsbreaks.infotoday.com/NewsBreaks/Bill-Introduced-for-Open-Access-to-Federally-Funded-ResearchFRPAA-Revisited-67078.asp

Beyond Academic Journals
http://dougbelshaw.com/blog/2012/01/25/beyond-academic-journals/

What Math and Physics have been doing for years
http://arxiv.org/

Nature Preceedings
http://precedings.nature.com/

Faculty 1000 new open access publication:
http://blogs.nature.com/news/2012/01/f1000-launches-fast-open-science-publishing-for-biology-and-medicine.html

Something Wholly New?
Gratuitous self-link

http://futureofscipub.wordpress.com/

People and hashtags to follow on twitter

#openaccess

#frpaa

@mrgunn

@openscience

@mbeisen (Michael Eisen)

@phylogenomics (Jonathan Eisen)

@researchremix

@FakeElsevier

A Vision for the Future of Scholarly Publishing

In many ways, the Research Works Act has been a blessing (see excellent link round up here). It has taken the moderately complacent but always grousing scientific community and whipped our feelings about the current state and cost of scientific publishing into a white hot fury. Ideas are bandied about, critiques given, and people begin to take action.

So what is the way forward? Certainly we are not getting away from journals in the near term. Or ever, really, as they are fabulous final curated repositories scientific results. They are the end point and gold standard. And I think we’re all coming to the conclusion that a PLoS-like model is a great way to go. Science must end up in an open access repository at the end of the day.

But a final resting place aside, what should the future look like so that research results can be disseminated rapidly and openly? How can we fold in peer review as a part of the process, as it is one of the hallmarks of the scientific quality control.

So I’ve been dreaming. A vision of the future of scientific publishing. What if arXiv, reedit, PLoS, pubcreds, slashdot’s commenting system, figshare, Data One, and Web 2.0 had a baby? This lead to an idea – a concept – a proposal.

So, here’s my vision of the future. It’s not the only vision, and there is substantial room for discussion, but, it’s a start… Consider this a SciFi musing on scholarly publishing.

I sit down with my morning cup of coffee and log into SciX. I am presented with several options on the main screen:

Read Papers
Submit a Paper
Revise a Paper
Review the Reviewers

So, what happens when I click on each of those? Let’s follow each one, one by one.

OK, so, I click on “Read a Paper”. I’m taken to a screen that shows different scientific disciplines and a search box. I click on my discipline of interest, as I’m just browsing. I am presented with a list of subdisciplines, a search box, and a list of papers below that with several sorting options (# of reviews, submission date, review score, etc.) I click on my subdiscipline, and am presented with just a list of papers which I can sort by various criteria and, again, a search box. I sort so that I can see the latest submissions and find one that piques my interest. I click on it, read it, maybe even view some of its supplemental videos or such. I have a strong opinion about it – I think it’s good, but has a few flaws that need to be corrected before it should be accepted by the community. The paper already has one review filed. I read it, and it’s OK, but it misses some key things. So I click review.

I write a brief review, just like a normal paper. I click that my review should remain anonymous. I also need to select one option from the following list:

This paper is fraudulent/not a paper (flag for review).
This paper is not acceptable in its current form (reject).
This paper is good, but requires some large revisions (major revisions).
This paper is acceptable, but requires some changes (minor revisions).
This paper is acceptable as is.

As I think there are some serious flaws, but like the work, I select the major revisions option.

At that moment, it just so happens that I get an email from SciX. The email contains the latest papers in my chosen disciplines and sub-disciplines that have received at least two more acceptable reviews than rejection reviews. I.e., reject is a score of -1, major revisions is a score of 0, and any acceptable score is +1.

OK, great. Let’s move on. Let’s say I wanted to submit a paper. The submission process is the same as any other site except, today when I click on submit, a screen comes up that denies me the ability to submit. It reads that I currently do not have a review to submission ratio of 3:1. I’m all good on reviewing reviews (more on that later), but I need to file more original reviews myself! I grumble about it, bemoaning my days in grad school when I only had to keep up with 1:1, but it’s right, so I go and file one more review before I come back and submit my paper in the usual way – except for that embedded video. After submitting, I like to the doi on my website and CV.

OK, it’s several weeks later. I have received two major revisions reviews on my paper. I’ve done the revisions, and thought carefully about the reviewers responses. So I go back to SciX and click on Revise a paper. I upload the revised version. I then upload an individual response to each reviewer. They are both anonymous, so I don’t know who they are. However, once I hit ‘resubmit’, they are sent an email. It tells them that I have responded to their reviews and submitted a revised version. They have two weeks to look at the revision and the responses. If they do nothing, the paper will be marked as “acceptable as is”. If they wish, though, they can go and submit a secondary review. This review also includes an option of “Did not respond to my review at all.” If both reviewers select this for more than two rounds of resubmission, my paper is booted and I have to start over again.

A few weeks later, both reviews come back positive, and my paper is included in the next email out. I have also received two additional ‘acceptable’ reviews of the paper with no comments attached. I list the score on my CV. I also hit “submit to journal” and select PLoS One. The system generates the proper submission, and attaches the review history. All I do is fill in a cover letter.

I have another paper to submit, but, I know I’m down on my number of “reviewing the reviewers”, so, I click on that option. I am brought to a screen with five reviews. For each review, I also have the title and abstract of the paper. For each review, I am asked to select one of the following options:

Review is fraudulent/someone padding their bank/unrelated to paper (flag)
Review is cursory. Email reviewer for more detailed review.
Review is acceptable, but laced with inappropriate invective. Count as half-review.
Review is fully acceptable.

I quickly go through and click acceptable for all of them, except one that merely says “No.” with “Reject” selected. Clearly, not fair to the author or the community.

With this finished, I go and submit my next short paper. It’s a brief note, but one that I feel important to get out into the literature.

After all of this, I go to my profile page. I check my list of papers, note their current scores and number of downloads and citations, and update those on my website and CV.

So, that’s it. That’s my simple vision. Open access and transparency from hitting the ‘publish’ button to reading and writing reviews. And a reputation based economy so that papers are only marked accepted when the weight of reviews say so, but anyone can still look at them and the comments that others have made about them.

Thoughts?

UPDATE I’m tremendously excited about F1000′s announcement of their new F1000 Research which is being discussed across the interwebs. I fear that their model of post-publication peer review will end up suffering the same fate as PLoS One, though – comments on highly controversial or touted articles, but most of the rest going without comment or notice. The above vision solves that problem.

See also (things I have found after writing the above):
Gowers’s How might we get to a new model of mathematical publishing?
Gowers’s more modest proposal
This excellent thread at Math 2.0
Nikolaus Kriegeskorte’s excellent The Future of Scientific Publishing

The Story Behind the Paper: Climate Change and Kelp Forest Food Webs

ResearchBlogging.orgYay! First paper of my postdoc is out in the August 2011 issue of Global Change Biology!

Woohoo!

So, what have I been doing for the past few years of my life?

In brief summary: Kelp. Food webs. Climate change. A potent combination.

And if you want the punchline without digging into the rest of this article, it is this: the diverse, complex food webs of southern California kelp forest will likely be greatly simplified if climate change leads to big storms every year.

I could give you all of the gory details of the paper, how I arrived at that conclusion, the science-y nuggets, etc. Instead, I thought I’d give you the slightly longer, more human, but more meander-y version of how this paper was created – the story of this particular story. It’s not something we always talk about in science. Science isn’t always a nice linear process. What we set out to do it not always what ends up happening. In the end, though, we want a nice linear story that leaves the juicy bits of exploration out. And the process behind this paper was, indeed, intellectually juicy.

So, how does a paper on kelp forests, food webs, and climate change, come to be?

Like all projects in my life, this was not one I was expecting to do. It’s kind of a theme for me. I came to work at the Santa Barbara Coastal Long-Term Ecological Research site to work on potential feedbacks between the diversity of life on the seafloor and grazing pressure by urchins. Essentially, I wanted to test a theoretical model that I’d put together with colleagues in a 2007 paper. I was convinced that feedbacks between diversity and function were going to be the next big thing. And I still think they’re pretty neat. The experiment was pretty fun, led me to revisit some old concepts in new ways, and ultimately produced some great data which is going to be submitted soon.

Five year running average of extreme (i.e., storm-driven) winter non-tidal residual wave heights from the San Francisco Tide Guage. Starting ~ 1945, we see an increase. Figure adapted from Bromirski et al. 2002 J. Climate.

But early on in all of this, the project PI called me to his office and laid out the following.

The SBC LTER has a buttload (metric, not Imperial) of data. They’ve been sampling thirty-five 80 m2 transects at reefs along the Santa Barbara Chanel coastline every summer for nearly a decade, counting the heck out of everything. At the same time, we’ve noticed two interesting things in the climate literature: 1) climate change projections say that the strongest storm of the winter should get bigger in the future, and 2) if you look at the data, the largest winter waves in California have gotten bigger over the last 50 years.

We know that big waves rip out kelp to life on the seafloor.

We don’t know what will happen if kelp is ripped out every year, or lost altogether.

The LTER had started a project to simulate this annual storm disturbance – big 200m2 plots where we went through and trimmed the giant kelp every winter with hedge clippers. But, coming up on year 2, no one had an idea of how to put the whole story together.

Hello, opportunity, how’s it going?

My facebook network. My mom is actually fairly well connected - but mostly to my High School cluster.

I’d long been fascinated with a network approach to studying communities. Basically, you can visualize life on the seafloor by thinking about Facebook. No, really! There are a ton of tools out there you can use to visualize your network of friends, with the links between them showing friendships, and you at the top. There’s a ton of information there. Who are the hubs? Who connects disparate groups of friends? How does the size of, say, your group of college friends influence the number of other random groups of friends you’ve accumulated through life?

And who has your mom friended, anyway?

OK, now, instead of friendship, think of your friends as eating each other. And you’re at the top!

OK, ok, now replace the people in your network with different species. And you’re a shark. Or giant squid. (or theoretical giant mutant Pycnopodia). Voilá! Food web! And all of that structure and complexity, it has real meaning describing the stability and function of a community of organisms. (well, ok, the function part is what I’m tackling in my current postdoc at nceas)

So, I went back to my PI and said, OK, hey, let’s take a look at how changing the annual frequency of storms can shift the network structure of kelp forest food webs. It would be an indirect effect, so we can use my favorite tool, Structural Equation Modeling, for the analysis. And we can even bring in two awesome other bits of data – transect-level wave height projections from the Coastal Data Information Program and awesome new measurements of kelp beds right after storm season taken by satellite (and developed by Kyle Cavanaugh – an awesome grad student at UCSB who uses Landsat images to count kelp).

He said, sure! But, I might want to see about that food web. You see, no one has actually put together a full kelp forest food web. Or even one for the 250 species that we sample. So, can this be done? Really?

And thus began a 6 month quest. Of living and dying by google scholar. Of talking to experts. Of driving up and down the coast to marine labs, riffling through their libraries of unpublished masters theses, or appendices to undergraduate student reports. Just to find out, who indeed, eats who?

It’s that kind of basic natural history that is necessary to inform sexy fun theory-based analyses. And without it, the sexy-fun-stats-nerdery is really meaningless.

I emerged with a nice solid web, a good sense of uncertainty, and a decent idea of how I’d put these models together. The next part was shockingly simple. Use plyr to smoosh our data with a master kelp-forest food web to get individual transect-level food webs (e.g., what the structure of a place with two seaweeds, an urchin, and a lobster, versus a full-blown hyper-diverse kelp forest)? And then use Structural Equation Modeling to fit models that looked like so:

Path diagram of an SEM showing how waves indirectly influence the species richness and linkage density of a kelp forest food web.

This is all very interesting, and one can contrast the strength of different indirect pathways by which waves influence kelp. It was not immediately intuitive, however, as to what this means for the future of kelp forests under a climate change scenario with annual large storms. Fortunately, as I was fitting Structural Equation Models, which are really just a system of linear equations, I could turn my models into dynamic simulations.

Yes yes. Lots of coding.

I then used these simulations to make predictions about how increasing the annual frequency of large storms will affect the network structure of kelp forest food webs. I could reproduce the table of results for you, and discuss each individual result, but I think the following image more or less sums it up

Basically, frequent large storms will simplify food webs in the end. What’s interesting, though, is that just one storm after years of calm – our current scenario – may actually increase complexity. Everything gets a little mixed up as sunlight streams in and lets suppressed algae establish a beachhead, even while top predators may decrease in diversity. And, shockingly, results from our large-scale field experiment – at least the first two years – appeared to match this pattern beautifully.

And thus, blammo! Publication!

Well, OK, no. Honestly, after the initial results it took a few meeting with my co-authors to get everyone on the same page. In no small part this was due to the atypical methods I was using. Also, while the final paper has about nine or ten different measures of community complexity in it, my initial analyses looked at about thirty. I had some winnowing to do in order to establish a good story. Good clear story is king. What is a scientific paper, after all, than good storytelling backed up by data and then confused with jargon.

After we all got on the same page (and I had tried my story out via a few talks and posters), I wrote it up for Science. Because, well, why not. Even so, it took multiple rounds of revision, before submission. And sweet sweet rejection. Thus followed attempts to submit to three different other journals (What? I wanted to try and be in one of the glossy magazines! I’m thinking’ about my career, here, and other postdocs will back me up on this!) And a major re-write for the format of each. Then, finally, I realized that GCB really was a logical and perfect fit for the piece, and the reviews I got were most helpful in clarifying the last few pieces.

In the end, I’m a pretty proud Papa on this one. I think it’s a nice solid piece of science. It’s got a massive chunk of natural history in it, filling what I see as a key gap. It uses some fancy-pants statistics – and allowed me to go on a deep statistical odyssey in learning the ins and outs of some arcane parts of SEM, such that I’m now an SEM package developer in R. And it coupled the analysis of a smokingly hot large-scale observational dataset (go-go LTER power!) with an intense and awesome mongo-effort field experiment (Clint, Shannon, and Christine, you guys are underwater animals!) It’s basically everything I want in a paper. And yet, it all coalesces around a single story:

The diverse, complex food webs of Southern California kelp forest will likely be greatly simplified if climate change leads to big storms every year.

BYRNES, J., REED, D., CARDINALE, B., CAVANAUGH, K., HOLBROOK, S., & SCHMITT, R. (2011). Climate-driven increases in storm frequency simplify kelp forest food webs Global Change Biology, 17 (8), 2513-2524 DOI: 10.1111/j.1365-2486.2011.02409.x