Friday, January 13, 2017

Responding to NIH's Rigor & Reproducibility Requirements

The NIH's new Rigor and Reproducibility requirements are in full swing according to many study section reviewers. Reviewer reports of the discussions and emphasis being given to these requirements are reinvigorating the discussion amongst NIH funded researchers and those aspiring to be funded by NIH around the best way to respond to these requirements.

It's useful to consider the origin of these requirements. This NIH initiative was in part a response to several articles that came out a few years ago that reported a surprising number of research projects published in top journals couldn't be reproduced or had fundamental flaws.

These new requirements fall into four general categories:

Scientific Premise:
Scientific premise refers to the body of completed research and data (by the proposing PI and others) that form the basis or the justification for what the PI is proposing as a next logical step. The NIH wants to make sure that the research they're funding stands on reliable data and/or fills in necessary gaps in the current research.

Rigorous Experimental Design:
According to the NIH website, "Scientific rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. This includes full transparency in reporting experimental details so that others may reproduce and extend the findings." Thus, this is where the NIH is hoping to remedy funding research that cannot be reproduced; if experiments are designed rigorously then they remove the question marks that keep other researchers from being able to replicate.

Relevant Biological Variables:
Historically, NIH funded research that used a disproportionately high number of male animals. This had an unintended consequence on the results of such research not taking into account sex in various experiments where it might make a difference. To remedy this, the NIH is now asking for researchers to account for both sexes in research and to provide better justification if only one sex is being used in research. Although better inclusion of females in experiments is a priority, the NIH has identified other variables they want justified in grant proposals.

Authentication of Resources:
Authentication of resources is simply a requirement by the NIH to make sure the chemical and biological resources you use in your experiments are reliable.

Dr. Jennifer Kemp, the Director of the Research Office in our Department of Medicine recently offered some tips on how to address these requirements. Even if your research has always met these requirements, you need to be more explicit about them in your grant proposals. According to Dr. Kemp, each of these requirements should be addressed under a subheading naming it in your proposal. Scientific Premise should be addressed in the Significance section, Rigorous Experimental Design and Relevant Biological Variables should be addressed in Approach, and Authentication of Resources should be addressed in a new attachment.

To learn more about the NIH's requirements and Dr. Kemp's suggestions, please see the resources below:

Rigor and Reproducibility - NIH
Update on NIH Grant Proposal Requirements - Dr. Jennifer Kemp

Tuesday, December 13, 2016

Reviewers are on a need-to-know basis

I recently reviewed a faculty member's grant application. It was a cool project! I could understand the importance, I had examples of application, I got the gist of how it would be carried out, etc. Only trouble was, there were quite a few paragraphs of technical information that I couldn't understand. Normally, this would be expected since many grant applications must include technical language to show reviewers, who are experts in the field, that you're on the cutting edge. However, the intended sponsor for this grant stated that they had laypeople on their review committee. So, I as a layperson reviewing this grant was a comparable reviewer.

All of this to say that when it comes to grant-writing, less is more. Your challenge is to write to your audience (your reviewers) as concisely as possible to clearly convey the great importance of your project and offer just enough justification and technical details to convince them you will be successful... and that's it! So, seems simple enough, but there are variables to consider as you decide what needs to go into and stay out of your grant application.

Who are your reviewers?
To figure out what your reviewers need to know, you need to first have a sense of who they are. Are they laypeople or experts? Are they in your field or not? Are they familiar with you or your area? To answer these questions, explore the website of your sponsor. Some sponsors will list their reviewers and some will describe the background of their reviewers. Sponsors will also describe the level of detail and background information they expect in their program announcement or grant application guide. Not surprisingly, to know what to tell them, we must first know who reviewers are.

What do they need to know?
Once we have a sense of who our reviewers are, we can then start to formulate what they need to know to make a decision on your project. No matter who they are, you always want to emphasize the importance of your work and the impact your work. You must also convince your reviewers that you will be successful in carrying out your project.

How do you best convey your project?
The language you use to convince your reviewers will vary based on your reviewers. For reviewers who are experts in your field, you'll want to demonstrate your knowledge of the field and relevant scholarship. However, make sure that you are careful that your discussion of the field doesn't get too tangential to your project. Stay focused, show your expertise, and bring it back to your compelling case. Start with your case and end with your case. For laypeople, you'll also want to convince them that you and your project are the way to go, but you'll need to use jargon-free and plain language to show them.

It may sound harsh to say, but at the end of the day, if your grant proposal doesn't speak to your reviewers in whatever way, it's your fault. If you didn't know who they would be or what they needed to know, it's of no use to blame them for that. If your grant is rejected, use the reviewer comments to better understand them as an audience so you can better write for them when you resubmit.

Confessions of a Grant Reviewer - Margaret Ring
Crafting a Sales Pitch for Your Grant Reviewer - Robert Porter

Monday, December 5, 2016

So, what do you do? (planning your holiday pitch)

Tis the season of the office holiday party! This may mean large department/university parties or lunches and/or it may mean office parties with your spouse or partner. Either way, you're bound to be asked..."So, what do you do?" We all get the question, we all expect it, but if you're anything like me it still often blind-sides you. This is a particularly difficult question for researchers, because what you do can be complex and hard to explain for different audiences. Sure, you can take the easy way out with, "I'm a Professor of Physics," receive an impressed look from your conversation partner, and ensure no further questions are asked, or you can use the opportunity to practice your research pitch with different audiences. Hey, depending on the party, maybe you're pitching someone who could be a resource.

So, this week I offer some tips on creating your holiday pitch!

Assess your audience
When headed to a social event, give some thought to who will be there and what their interests are. Will there be other researchers and academics at this party? Will there be entrepreneurs? Foundation representatives? Considering your audience beforehand can give you a headstart in planning your pitch, but don't be afraid of asking a question or two of the person you're talking with, such as, "Have you ever heard of the theory of X?" Or, "Are you familiar with Y disease?" Depending on the answer(s) you get to these precursory questions, you can skip over parts or give a brief explanation to set yourself up to describe your research.

Stay focused on impact
Key to giving your holiday pitch is to stay out of the weeds. To best explain your research, your conversationalist will likely be most interested in the importance of your work. What's the end result? If your research works to cure cancer, why not start there and offer more specifics as you go or as they ask? By focusing on the impact, your pitch will be clearer and more compelling.

Do not use jargon
I attended my spouse's office party this weekend (he works for a Tech startup) and when asked what I did, I simply said "I'm in research development." Most people thought this was just great, and the conversation didn't go much further, which was fine with me. But, I found myself wondering what picture they had in their head of research development. I'm quite certain that they weren't picturing me developing grant-writing seminars or writing this blog. Research development is jargon where you really don't know what it is if you aren't working in it and even then it's a pretty institution-dependent field. I say all this because I used jargon at this party to get out of any conversations about my work and I get the sense that some academics might do the same, drop a big word, offer no explanation and just wait for a topic change. However, you're doing amazing research, so share it with people, don't let the conversation drop prematurely. Remember, this is practice!

Offer scenarios
In an effort to avoid jargon and to better relate to your audience, think about scenarios or metaphors you can use to explain your research. For instance, saying, "Have you ever used a fit bit before?"
When someone says yes, you go on, "Well, I develop the technology that measures how far you've walked in a day." This is a very impact/application-focused way of describing what you do that most people can relate to.

Now, I'm not trying to trick you into a whole month of extra conversations about work that will make you a social outcast at next year's parties. Always, gauge what your conversation partner is interested in. If they aren't interested in really understanding your research, then let it go, ask them what they do and try another prompt next time to spark interest. Perhaps, next year, folks will be tracking you down to catch up and find out what interesting research you've been doing since you talked last.

Do schools kill creativity? - Sir Ken Robinson (This is a nice example of how to pitch your research and a funny bit about talking about research at parties)
Answering the dreaded "So, what do you do?" question - 99U

Monday, November 28, 2016

Grant Timelines

I don't know about you, but for me this time of year flies faster than any other. I feel like as soon as I've thrown away my jack-o-lantern, there are holiday decorations everywhere you turn. This reminded me of grant timelines (what doesn't!?!). Grant application deadlines often feel very far away, and for more novice grant-writers they may set aside a program announcement as they work on other seemingly, more pressing work. Yet, with a grant deadline on the horizon, the clock is ticking, even if you can't hear it yet.

Below are some things to consider as you create a timeline for your grant-writing.

Grant development is more than just writing a grant
Part of the reason that researchers don't always give themselves enough time to write their grants is that they don't consider all of the pieces of good grant development. For instance, taking the time to research the sponsor and the program to which you're considering applying takes significant time. Just looking through a sponsor's grant application guidelines can be time intensive as they are sometimes over a hundred pages long! Also, as I've discussed before, it's very important to give yourself time to contact and work with a Program Officer (PO) to give you the best chance for success.

A solid grant development timeline is six months
Realizing how much is involved in addition to writing your grant proposal, ORDE recommends a six month grant development timeline. Although you won't begin writing your proposal till closer to three months before, you must begin assessing the fit with a sponsor, working with a PO, and developing your project early on.

The due date should not be the day you turn it in
As you develop your timeline, do not simply look at the due date and count backwards. The problem with this strategy is that planning to turn in your grant application on the deadline is really testing fate in a number or ways. First, you're not alone if you're planning to turn it in on the due date. Sponsor and university systems are flooded with submissions on a big deadline, and things can go wrong. For instance, with NIH applications, they must clear two systems error free before they are considered accepted at the NIH. If you wait till the due date to move through these systems and receive an error that may take some time to correct, you'll really be down to the wire. Also, it's not unheard of that sponsor or university sites go down especially under heavy traffic. So, play it safe and get your proposal in at least a day ahead of the due date.

You must rely on others
Another reason why you really should not wait till the last minute is that you're not the only person involved in your grant application. Working with your grants administrator and submitting your application to the Office of Grants and Contracts (OGC) can take more time than you might anticipate. In addition to carving out additional time for these processes, you should also check in with your grants administrator as early as possible to alert them to your proposal and timeline. Also, double check the time and process required by OGC to avoid any hang ups.

Grants are getting increasingly competitive and some sponsors will reject any application that doesn't follow the rules or that isn't a good fit. Giving yourself the time to avoid these problems will be well worth it!

ORDE Grant Development Timeline

Monday, November 14, 2016


As we look toward the Thanksgiving holiday next week, I thought it would be a good opportunity to blog about the opportunities for incorporating gratitude in grant development, or what I'm coining, grantitude. Within fundraising offices, gratitude is an essential part of the equation. Annual Funds will hold thank-a-thons in their phone banks and in the major gift realm, there is a whole professional field focused on stewardship, or folks who are tasked with thanking donors and continuing to cultivate them toward other gifts.

Is this done because these institutions and professionals are just overwhelmed with gratitude? Well, not to be skeptical around Thanksgiving, but no, thank-a-thons and stewardship happen because they've been shown to generate more giving.

Now, it's true that fundraising is different from grant development, however, we can glean some strategies from fundraisers that are applicable.

Always say thank you
At the heart of stewardship is showing appreciation to anyone who gives you money. So, when you receive a grant of any kind, make sure to figure out where to give thanks and do it. Send a thank you note or email to a PO that worked with you in the process, or send a note of thanks to any folks in leadership positions at the agency that it would be appropriate to contact for a quick thanks. This is a nice thing to do and shows appreciation for those who spent time and ultimately money on you and your work, but on the strategic side, it allows you to stand out from your competition. How many researchers think about sending a thank you or showing gratitude for a grant? Probably, not a whole lot, so if you do it, it may give you an edge or at least get your name in front of people you want to know who you are.

Cultivate relationships
These thank yous are a part of continuing the relationship(s) you have with an agency. Chances are, your first grant is not the only one you'll ever try to get from a sponsor, so it makes sense to build your relationship. Aside from saying thank you, make sure you are a good steward of their money - get those pesky progress reports to your PO on time and follow-up with any requests or questions that a sponsor has. Being nice to work with may work to your benefit the next time you go to submit a grant application.

Show grantitude to all involved
Even though they don't have direct responsibility for your being funding, don't forget to show gratitude to reviewers. Now, I don't mean sending them thank you notes (you don't know who they are anyway). I'm saying that in any resubmissions, when responding to reviewer comments, show gratitude for their work in reviewing your proposal, and be gracious in your revisions and explanations. Don't be argumentative; it won't get you anywhere good.

This hopefully gives you few ideas on employing grantitude in your process. The resource below gives other related strategies. Have a happy thanksgiving and may the grant-makers continue to smile on you. :)

Grant Management - Stewardship - The Grant

Monday, November 7, 2016

The Science of Team Science

Many of today's biggest research questions cannot and will not be answered or solved by a single, lone-wolf researcher. Questions around climate change, diseases, genomics, etc. call for a brilliant and diverse set of researchers and thinkers...working together. Yet, academic institutions, where so many of these researchers are employed, are traditionally geared to support and promote independent researchers.

In the midst of this conundrum, several scholars have begun working together to develop a field that looks at how scientists can best collaborate and be productive. This field is referred to as Team Science, and the study of Team Science is referred to as The Science of Team Science (SciTS).

Team Science looks at several barriers or opportunities to promote effective collaborative science. According to NCI's description of Team Science, they include the following:
  • Funding opportunities
  • Institutional infrastructure and resources
  • Organizational rules particularly around tenure and promotion
  • Team processes: table-setting, early agreements, publication ownership, and a feedback loop on how collaboration works for everyone
  • Interpersonal dynamics
  • And collaborative skills among scientists
SciTS has been digging into these barriers to discover how Team Science can be best developed and promoted. For instance, recent SciTS articles have come out that look at how women are under-represented in team science, particularly in co-authored research and continued collaborative relationships with other Scientists.

Below are SciTS and Team Science resources for you to access if you're trying to better understand how you can effectively collaborate with other researchers while navigating the structural realities that can sometimes be barriers.  

The Science of Team Science Website
Team Science Toolkit - NCI
Difference in collaboration patterns across discipline, career, and stages - PLOS Biology Journal
Rosalind's ghost: Biology, collaboration, and the female - PLOS Biology Journal

Monday, October 31, 2016

Sponsor Data Sharing Requirements

As I've discussed in past blogs, major federal funding agencies are demanding that applicants incorporate data management plans into their proposals. But there is much concern among researchers around the expectations for data sharing in these plans.

The National Institutes of Health (NIH) and the National Science Foundation (NSF) have addressed and clarified applicant expectations to some extent. Even if you aren't interested in applying to the NIH or NSF, it's worth watching how they are developing requirements, as other federal and non-federal agencies will likely follow suit. The points below outline some of the general positions these agencies are adopting:

Program-Directed Policies
Not surprisingly, data and best practices for data sharing is really discipline-specific. Best practices around privacy, sharing, access, and preservation vary wildly by field. Therefore, when you go to look at the general FAQs at the NIH or NSF, many of the responses are, to paraphrase, "it depends." The NSF repeatedly answer these data questions saying that many of the practices will be dictated by one's "community of interest." For these agencies, this basically means that reviewers within a directorate or study section will assess whether a proposal's data management plan meet the standards of their research community.

Addressing sharing and preserving
After the recent reports describing how few research projects could be reproduced, based on the data and methods that remained, the NIH and NSF are demanding more rigorous data management practices. This is partly to resolve the reproduction dilemma, but it is also so that agencies can get more bang for their buck. If new researchers can access the data from past funded projects, they can use it to produce even more research and analysis. This can mean more research for less money. With that said, agencies are asking PI's to share as quickly as is reasonable. The NIH asks for PI's to share data as soon as their publications have been accepted. They also want the data that they funded to produce to be kept in good shape for others.

Talk to POs
I probably could make this a heading in almost every blog I write, but it fits here too! Data and data sharing are complex and the questions surrounding data are often very specific. So, who better to counsel you on how to address these complexities than your friendly neighborhood Program Officer! However, as with all PO conversations, make sure you do your homework and understand your agency's data policies and requirements before reaching out to your PO.

In ending, if you start to hyperventilate when considering all the new requirements being asked of you by funders around data, remember, they're still working it out too, so work things out together!

NSF Data Management FAQs 
NIH Data Sharing FAQs