Monthly Archives: January 2012

A Vision for the Future of Scholarly Publishing

Posted on by
16

In many ways, the Research Works Act has been a blessing (see excellent link round up here). It has taken the moderately complacent but always grousing scientific community and whipped our feelings about the current state and cost of scientific publishing into a white hot fury. Ideas are bandied about, critiques given, and people begin to take action.

So what is the way forward? Certainly we are not getting away from journals in the near term. Or ever, really, as they are fabulous final curated repositories scientific results. They are the end point and gold standard. And I think we’re all coming to the conclusion that a PLoS-like model is a great way to go. Science must end up in an open access repository at the end of the day.

But a final resting place aside, what should the future look like so that research results can be disseminated rapidly and openly? How can we fold in peer review as a part of the process, as it is one of the hallmarks of the scientific quality control.

So I’ve been dreaming. A vision of the future of scientific publishing. What if arXiv, reedit, PLoS, pubcreds, slashdot’s commenting system, figshare, Data One, and Web 2.0 had a baby? This lead to an idea – a concept – a proposal.

So, here’s my vision of the future. It’s not the only vision, and there is substantial room for discussion, but, it’s a start… Consider this a SciFi musing on scholarly publishing.

I sit down with my morning cup of coffee and log into SciX. I am presented with several options on the main screen:

Read Papers
Submit a Paper
Revise a Paper
Review the Reviewers

So, what happens when I click on each of those? Let’s follow each one, one by one.

OK, so, I click on “Read a Paper”. I’m taken to a screen that shows different scientific disciplines and a search box. I click on my discipline of interest, as I’m just browsing. I am presented with a list of subdisciplines, a search box, and a list of papers below that with several sorting options (# of reviews, submission date, review score, etc.) I click on my subdiscipline, and am presented with just a list of papers which I can sort by various criteria and, again, a search box. I sort so that I can see the latest submissions and find one that piques my interest. I click on it, read it, maybe even view some of its supplemental videos or such. I have a strong opinion about it – I think it’s good, but has a few flaws that need to be corrected before it should be accepted by the community. The paper already has one review filed. I read it, and it’s OK, but it misses some key things. So I click review.

I write a brief review, just like a normal paper. I click that my review should remain anonymous. I also need to select one option from the following list:

This paper is fraudulent/not a paper (flag for review).
This paper is not acceptable in its current form (reject).
This paper is good, but requires some large revisions (major revisions).
This paper is acceptable, but requires some changes (minor revisions).
This paper is acceptable as is.

As I think there are some serious flaws, but like the work, I select the major revisions option.

At that moment, it just so happens that I get an email from SciX. The email contains the latest papers in my chosen disciplines and sub-disciplines that have received at least two more acceptable reviews than rejection reviews. I.e., reject is a score of -1, major revisions is a score of 0, and any acceptable score is +1.

OK, great. Let’s move on. Let’s say I wanted to submit a paper. The submission process is the same as any other site except, today when I click on submit, a screen comes up that denies me the ability to submit. It reads that I currently do not have a review to submission ratio of 3:1. I’m all good on reviewing reviews (more on that later), but I need to file more original reviews myself! I grumble about it, bemoaning my days in grad school when I only had to keep up with 1:1, but it’s right, so I go and file one more review before I come back and submit my paper in the usual way – except for that embedded video. After submitting, I like to the doi on my website and CV.

OK, it’s several weeks later. I have received two major revisions reviews on my paper. I’ve done the revisions, and thought carefully about the reviewers responses. So I go back to SciX and click on Revise a paper. I upload the revised version. I then upload an individual response to each reviewer. They are both anonymous, so I don’t know who they are. However, once I hit ‘resubmit’, they are sent an email. It tells them that I have responded to their reviews and submitted a revised version. They have two weeks to look at the revision and the responses. If they do nothing, the paper will be marked as “acceptable as is”. If they wish, though, they can go and submit a secondary review. This review also includes an option of “Did not respond to my review at all.” If both reviewers select this for more than two rounds of resubmission, my paper is booted and I have to start over again.

A few weeks later, both reviews come back positive, and my paper is included in the next email out. I have also received two additional ‘acceptable’ reviews of the paper with no comments attached. I list the score on my CV. I also hit “submit to journal” and select PLoS One. The system generates the proper submission, and attaches the review history. All I do is fill in a cover letter.

I have another paper to submit, but, I know I’m down on my number of “reviewing the reviewers”, so, I click on that option. I am brought to a screen with five reviews. For each review, I also have the title and abstract of the paper. For each review, I am asked to select one of the following options:

Review is fraudulent/someone padding their bank/unrelated to paper (flag)
Review is cursory. Email reviewer for more detailed review.
Review is acceptable, but laced with inappropriate invective. Count as half-review.
Review is fully acceptable.

I quickly go through and click acceptable for all of them, except one that merely says “No.” with “Reject” selected. Clearly, not fair to the author or the community.

With this finished, I go and submit my next short paper. It’s a brief note, but one that I feel important to get out into the literature.

After all of this, I go to my profile page. I check my list of papers, note their current scores and number of downloads and citations, and update those on my website and CV.

So, that’s it. That’s my simple vision. Open access and transparency from hitting the ‘publish’ button to reading and writing reviews. And a reputation based economy so that papers are only marked accepted when the weight of reviews say so, but anyone can still look at them and the comments that others have made about them.

Thoughts?

UPDATE I’m tremendously excited about F1000′s announcement of their new F1000 Research which is being discussed across the interwebs. I fear that their model of post-publication peer review will end up suffering the same fate as PLoS One, though – comments on highly controversial or touted articles, but most of the rest going without comment or notice. The above vision solves that problem.

See also (things I have found after writing the above):
Gowers’s How might we get to a new model of mathematical publishing?
Gowers’s more modest proposal
This excellent thread at Math 2.0
Nikolaus Kriegeskorte’s excellent The Future of Scientific Publishing

A Need to Understand Climate Change’s Indirect Effects

Posted on by
Reply

We know that warming, storms, drought, acidification, and the myriad of other effects of climate change will impact natural ecosystems. Most of our studies have concentrated on direct effects, though. For example, if you change temperature, you alter herbivore grazing rates. But what about indirect effects? For example, I’ve found that increased intense storm frequency may remove kelp which will have an indirect effect on the structure of kelp forest food webs.

So, I did a little experiment. I went to Web of Knowledge and searched the following term: “climate change” AND “impact”. I got 21,310 entries. Then I searched again using this query: “climate change” AND “impact” AND “indirect effect”.

The search returned 35 entries.

Surely, this must be a mistake. So instead of “indirect effect” I went with just “indirect”. 506. Better. If I took out the word impact I went up to 1,202. So, at maximum, 5.6%.

OK, maybe this was because I was looking at EVERYTHING. So I filtered it down to just Environmental Sciences and Ecology. “climate change” AND “impact”: 9,248. “climate change” AND “impact” AND “indirect”: 173. Removing impact got me to 689. Only 7.5%.

I’m guessing there are other careful ways of filtering, but, either way, I’m pretty surprised that even at this point, the study of the indirect of climate change still accounts for so little of our knowledge. Pretty interesting. Although I’m heartened by the fact that this literature seems to be increasing exponentially.

Running R2WinBUGS on a Mac Running OSX

Posted on by
16

I have long used JAGS to do all of my Bayesian work on my mac. Early on, I tried to figure out how to install WinBUGS and OpenBUGS and their accompanying R libraries on my mac, but, to no avail. I just had too hard of a time getting them running and gave up.

But, it would seem that some things have changed with Wine lately, and it is now possible to not only get WinBUGS itself running nicely on a mac, but to also get R2WinBUGS to run as well. Or at least, so I have discovered after an absolutely heroic (if I do say so myself) effort to get it all running (this was to help out some students I’m teaching who wanted to be able to do the same exercises as their windows colleagues). So, I present the steps that I’ve worked out. I do not promise this will work for everyone – and in fact, if it fails at some point, I want to know about it so that perhaps we can fix it so that more people can get WinBUGS up and running.

Or just run JAGS (step 1} install the latest version, step 2} install rjags in R. Modify your code slightly. Run it. Be happy.)

So, this tutorial works to get the whole WinBUGS shebang running. Note that it hinges on installing the latest development version of Wine, not the stable version (at least as of 1/17/12). If you have previously installed wine using macports, good on you. Now uninstall it with “sudo port uninstall wine”. Otherwise, you will not be able to do this.

Away we go!

1) Have the free version of XCode Installed from http://developer.apple.com/xcode/. You may have to sign up for an apple developer account. Whee! You’re a developer now!

2) Have X11 Installed from your system install disc.

3) Install http://www.macports.org/install.php and install from the package installer. See also here for more information. Afterwards, open the terminal and type

echo export PATH=/opt/local/bin:/opt/local/sbin:$PATH$'n'export MANPATH=/opt/local/man:$MANPATH | sudo tee -a /etc/profile

You will be asked for your password. Don’t worry that it doesn’t display anything as you type. Press enter when you’ve finished typing your password.

4) Open your terminal and type

sudo port install wine-devel

5) Go have a cup of coffe, check facebook, or whatever you do while the install chugs away.

6) Download WinBUGS 1.4.x from here. Also download the immortality key and the patch.

7) Open your terminal, and type

cd Downloads
wine WinBUGS14.exe

Note, if you have changed your download directory, you will need to type in the path to the directory where you download files now (e.g., Desktop).

8 ) Follow the instructions to install WinBUGS into c:Program Files.

9) Run WinBUGS via the terminal as follows:

wine ~/.wine/drive_c/Program Files/WinBUGS14/WinBUGS14

10) After first running WinBUGS, install the immortality key. Close WinBUGS. Open it again as above and install the patch. Close it. Open it again and WinBUGS away!

11) To now use R2WinBugs fire up R and install the R2WinBUGS library.

12) R2WinBugs should now work normally with one exception. When you use the bugs function, you will need to supply the following additional argument:

bugs.directory='/Users/YOURUSERNAME/.wine/drive_c/Program Files/WinBUGS14'

filling in your username where indicated. If you don’t know it, in the terminal type

ls /Users

No, ~ will not work for those of you used to it. Don’t ask me why.

Food Web Structure and Changing Diversity at Two Levels

Posted on by
2

This is part of a larger series of open notebook posts about how food web structure modifies the effects of predator extinctions. For an introduction and list of other posts, see here.

OK, last but on two-level food webs for the moment. I’ve examined how food web structure can change the effects of predator or prey extinctions on both top-down and bottom-up control. A number of folk (including me) have theorized that changes in diversity at two trophic levels should interact – that the consequences of predator diversity loss should change as prey species are lost.

So bearing in mind our master food web and the little 2-level sliver of it we’re thinking about, let’s interrogate this idea. (And yes, that use of the word interrogate goes out to Scott Richmond).

Let's zoom in on one part of the general food web

Thinking about Extinction in Our Framework Thus Far

Thinking about what I’ve put together thus far, I’m not so certain that changing diversity at two trophic levels should influence predation or energy transfer beyond our understanding what’s happening with one trophic level. The simple probabilistic equations that I’ve shown to describe energy transfer and predation both rest on thinking about the average consequences for individual species. Each equation rests on taking a mean value of the probability of, say, being eaten over all prey when predators go extinct. If prey are going extinct as well, that should’t affect the outcome.

Why? Think about it this way. Let’s say you have a food web of 3 predators and 3 prey, and each trophic level is losing one species. For prey species a, the probability that it will be eaten does not change. This is because implicit in asking the question of what are the consequences of extinction for species a, we are asking what are the consequences for species a in all food webs in which it exists. Thinking further, what is, say p(eaten) for a species that does not exist? It’s not 1, but it cannot be 0 either. We just don’t think about it. This argument works as well when thinking about energy transfer.

So, I’d argue, that to understand p(eaten) we simply use the equations derived to understand p(eaten) under predator loss and to understand p(energy) we use the equations derived to understand energy transfer under prey species loss.

Well that chain of logic is uncomfortable. I don’t like where it led at all. I guess tacitly it suggests that maybe the variance of p(eaten) and p(energy) should somehow change… But I haven’t so much thought about variance other than thinking it would work in a similar way to means. Maybe I’m missing something. What is the proper way to calculate variances here? How do simultaneous extinctions affect this variance?

Still, even for the mean value of p(eaten) I’m no so sure. Let’s go draw some webs and see if this plays out.

Webs show that my Logic is Correct. Great.

Let’s start with our 2 predator, 2 prey food web with 1 of each going extinct.

Aaaaand – yeah, those results match exactly with what would be predicted from p(eaten) and p(energy) looking at predator or prey loss independently. The variance is larger – doubled, actually (from 0.125 to 0.25). Interesting. What about something more radical, say, a 3 predator, 3 prey web with 2 predators and 1 prey going extinct.

Yup, still the same as the single-level results, although, here the variance only increases slightly (by a factor of 1.03125).

So, clearly, the single-level results are true for the mean. The variance is still…yeah, I don’t quite have that figured out.

Comparison with the Experimental Literature

So, this result, that you can predict the average effects of changing diversity at two trophic levels at the same time by looking at the results for changing diversity of just one trophic level – does it agree with the experimental literature? Let’s think about one of my favorite examples – Lars Gamfeldt’s excellent 2005 Ecology Letters piece.

In this paper, Lars (LARS!) shows that he wishes I was working on the paper we are collaborating on rather than writing this entry.

Sorry, rather, Gamfeldt shows that prey and consumer species richness can interact. The key quote from the abstract is “…prey richness did not increase resistance to consumption when consumers were present. Instead, our results indicated enhanced energy transfer with simultaneous increasing richness of consumers and prey.”

I find this heartening. Here, p(eaten) was determined by consumers, as predicted. The second statement is curious as well and hearkens to Figure 4 of the paper where total biovolume (predators and prey) is clearly the highest when all 3 predators and prey are present. This is clear evidence that energy transfer into this food web is at its highest here. It drops, though, as consumer richness, but not prey richness, changes. Which, actually, we’d predict based on our in initial examination of energy transfer in the presence of predator loss alone. So…Gamfeldt’s results do appear to echo what I’ve shown here. And for anything with less than 3 consumers shows a consistent relationship for producer loss.

Ah ha. So…I admit, intuitively, I still think that under loss at both levels, p(energy) and p(eaten) should be products of the results from both the prey and predator equations together. But they don’t appear to be (otherwise for the 2-2 web with 1 loss at each level, we’d have p(eaten) and p(energy) = 0.5625). Hrm. This bears more thinking – at least for p(energy) why one does not have to incorporate diversity at both trophic levels. Clearly there’s something a little more complex that needs to be represented in a general equation for p(energy | Er, Ep) though. And likely p(eaten) as well. Hope to come back to that later.

That, and I’m starting to (unsurprisingly) see that some meta-analysis to compare predictions to observed results is going to be necessary, and that figuring out the right metric is going to be non-trivial.

Prey Loss in Different Food Web Structures: We’ve Been Here Before

Posted on by
1

This is part of a larger series of open notebook posts about how food web structure modifies the effects of predator extinctions. For an introduction and list of other posts, see here.

OK, only two more entries (I think) on simple two-level food webs before we jump into the great unkown (and you’ll see how unknown it is). So far I’ve been talking about the consequences of losing predator species for predation and energy transfer. But, what about losing prey? And what about losing both? In this entry, I’m going to show that we already know how to think about prey loss and food web structure. We just have to stand on our head. So, keeping our “Master Food Web” in mind, and that we’re zooming on on a particular component, let’s think about loss of prey.

Let's zoom in on one part of the general food web

Who will be eaten?

First, the obvious nice result. If prey go extinct, this does not change the probability that they prey trophic level will be under control. No predation links have been deleted. Therefore, p(eaten) is still 1.

Huh? Yeah, really. Think about it. This is an obvious answer, but, well. It’s a rather nice one!

What’s the probability that energy will get to predators?

So, energy transfer is the thing to focus in on. Sure, energy will still enter the prey trophic level, but the probability of energy getting to some of the predators after some prey go extinct may now be 0. Oops!

The wonderful thing is, p(energy) can be defined using exactly the same framework as p(eaten). We just have to stand on our heads. Let’s first look at a familiar 2-predator 2-prey web with 1 extinction.

It’s quite similar to what we’ve seen before with predator loss with a mean p(energy) of 0.75 across all prey extinction scenarios. However, what’s different is that we’re now interested in what PREDATORS have no prey, not what prey have no predators. This implies that we can merely flip the equations from before, replacing predators and prey as follows.

Let’s assume Er extinctions of our resource species (i.e., prey), Sr is our maximum resource species richness, and Di from before now becomes the out degree of predator i – their number of prey. We can simply revisit our earlier framework for the following two equations.

p(energy | Er) = 1-dh(0; Di, Sr-Di, Sr-r)   (4)

p(energy | E_{r}) = 1-sum_{D=1}^{S_{r}}{p(eaten|E_{r}, D)F(D)}     (5)

Simple, no? And no extra explanation needed!