I’ve Got the Power!

There is nothing like that pain of taking a first look at fresh precious new data from a carefully designed experiment which took months of planning and construction, 8 divers working full days, and a lot of back-and-forth with colleagues, and then you find absolutely no patterns of interest.

Yup, it’s a real takes-your-breath-away-hand-me-a-beer kind of moment. Fortunately, I was on my way to a 4th of July dinner with my family up in the mountains of Colorado, so, I wasn’t quite going to commit hari-kari on the spot. Still, the moment of vertigo and neausea was somewhat impressive.

Because, let’s admit it, as much as science is about repeated failure until you get an experiment right (I mean, the word is “experiment” not “interesting-meaningful-data-generating-excersise”), it still never feels good.

So, round one of my experiment to test for feedbacks between sessile species diversity and urchin grazing showed bupkiss. Not only was there no richness effect, but, well, there wasn’t even a clear effect that urchins mattered. High density urchin treatments lost about as much cover and diversity of algae and sessile inverts as treatments with low or no urchins in them. What had gone wrong? And can this project be salvaged?

Yeah, I can see a pattern here.  Sure...

Yeah, I can see a pattern here. Sure...

Rather than immediately leaping to a brilliant biological conclusion (that was frankly not in my head) I decided to jump into something called Power Analysis. Now, to most experimental scientists, power analysis is that thing you were told to do in your stats class to tell you how big your sample size should be, but, you know, sometimes you do it, but not always. Because, really, it is often taught as a series of formulae with no seeming rhyme nor reason (although you know it has something to do with statistical significance). Most folk use a few canned packages for ANOVA or regression, but the underlying mechanics seem obscure. And hence, somewhat fear-provoking.

Well, here’s the dirty little secret of power analysis. Really, at the end of the day, it’s just running simulations of a model of what you THINK is going on, adding in some error, and then seeing how often your experimental design of choice will give you easily interpretable (or even correct) results.

The basic mechanics are this. Have some underlying deterministic model. In my case, a model that described how the cover and diversity of sessile species should change given an initial diversity, cover, and number of urchins. Then, add in random error – either process error in each effect itself, or just noise from the environment (e.g., some normal error term with mean 0 and a standard deviation). Use this to get made-up data for your experiment. Run your states and get parameter estimates, p values, etc. Then, do the same thing all over again. Over many simulations of a particular design, you can get an average coefficient estimates, the number of times you get a significant result, etc. “Power” for a given effect is determined by the number of “significant” results (where you define if you’re going with p=0.05 or whatnot) divided by the number of simulations.

So that’s just what I did. I had coefficient estimates (albeit nonsignificant ones). So, what would I need to detect them? I started by playing with the number of replicates. What if I had just put out more cages? Nope. The number of replicates I’d need to get the power into the 0.8 range alone (and you want as close to 1 as possible) was mildly obscene.

So, what about number of urchins? What if instead of having 0 and 16 as my lowest and highest densities?


Shown above are two plots. On the left, you see the p value for the species richness * urchin density interaction for each simulated run at a variety of maximum urchin densities. On the right you see the power (# of simulations where p<0.05 / total number of simulations). Note that, for the densities I worked with, power is around 0.3? And if I kicked it up to 50, well, things get much nicer. As soon as I saw this graph, the biology of the the Southern California Bight walked up behind me and whapped me over the head. Of course. I had been using a maximum density that corresponded to your average urchin barren. The density of urchins that can hang out, and graze down any new growth. But that's not the density of urchins that CREATED the barren in the first place. Sure enough, looking at our long-term data, this new value of 50 corresponded to a typical front of urchins that would move through an area and lay waste to the kelp forest around it. Which is what I had been interested in the first place. *headdesk* So, round two is up and running now. The cages are stocked with obscenely high densities of the little purple spikey dudes. And we shall see what happens! I am quite excited, and hopeful. Because, I've got the Power! (and so should you!)

Going Topless with Urchins

There’s nothing so satisfying as pulling back and seeing your brand new experiment out there in the water.

It’s been a crazy week or three getting this up and running, but now my first big postdoctoral experiment is soaking in the water, with urchins grazing away.

I’m testing some ideas regarding how diversity mediates the impact of disturbance by urchin grazing, and vice-versa – how disturbance by grazing can alter diversity. In essence, I’m testing a model of a community feedback process based on a framework whipped together by Randall Hughes, myself, and a few other fabulous co-authors.

But even though your ideas may be high-up and lofty, they always meet some interesting realities on the ground. Reality point 1 – my god, we built a lot of large cages.

This is about 1/4 of the cages before deployment. The rest were in the water. Thank fod for cheerful undergrad labor (fueled by brownies made from scratch – the key is to underbake them, and use a combination of eggs and egg yolks for extra gooey-ness) They look like such simple cheap affairs – some garden fencing, some PVC, some netting around the bottom…and then there’s about 1 ton worth of chain and half a ton of rebar stuffed into them. Subtidal work: unless it’s heavy, the waves will sweep it away.

Reality 2 – sometimes, you’ve gotta do it topless. Yes, the cages have no tops. This would seem the height of insanity if you want to keep something INSIDE. However, urchins appear to not like bendy flexy things. Sure, they’ll crawl up to the tops, but then they get to that wave strip at the margin, and freak out and freeze up. I’ve watched it. It’s kinda odd. And those cages that did have a top on them? That top, even if it’s mesh, creates a LOT of lift. So, a small wave washes by, and suddenly the cage top becomes an airplane wing. Unless you’ve added a huge amount of weight to your cage (see above), you may well never be able to find your cage again.

Reality 3 – nature is variable. Well, duh. See the two cages with two very different species compositions, som providing more or less biomass. I mean, the whole premise of this experiment was to use a natural gradient in species diversity as a treatment. But sometimes adding or subtracting one species can make a huge difference. Sampling (Reality 4 – ID-ing to the species level in the field on SCUBA gets pretty tedious after one hour, let alone 4 or 5) was pretty interesting, showing that large differences were indeed generated by both position on the reef, local topography, etc, as well as whether, say, tiny sea cucumbers had colonized a plot, whether the plot was full of lush Pterygophora, or the presence of the squat thick gorgonian Muricea.

Reality 5 – hungry urchins are hungry. And devious. Upon addition of urching to plots, they zoomed over to any brown algae (particularly the aforementioned Pterygophora or any juvenile giant kelp) and began munching in earnest. Some ran for the sides of the cages (and a few managed to squeeze out – Reality 6, the best laid plans of underwater mice and men… I’ll be doing some replacements this week with larger urchins). But the instand voracious consumption was really quite impressive.

I’m pretty stoked, and deeply curious as to how this will turn out. I’m sure there will be cursing, frustration, and bizarre results in the future, but for now, SCIENCE! Love it!