NIH Updates: Rigor and Reproducibility

You've probably heard the discussion around NIH projects and the questions around rigor and responsibility taking place in the last few years. But what does this mean for your next grant application?

The updates around rigor and reproducibility focus in four areas: premise, design, variables, and authentication.

Premise: Sure, PIs have always needed to show preliminary data or results when applying to the NIH, but now you need to take it a bit further. In addition to discussing your preliminary work, you must also discuss the strengths and weaknesses of it. You must now vet the foundation of your project in your grant application.

Design: The NIH is expecting more rigorous experimental design; their definition of rigorous includes the use of methods that are reproducible. As they state, "Robust results are obtained using methods designed to avoid bias and can be reproduced under well-controlled and reported experimental conditions." (NIH website, 2016)

Variables: A couple years ago, Sally Rocky, former Director of Extramural Research at the NIH, reported that a good majority of projects funded by the NIH did not include female animal models. In response to this particular overarching bias, the NIH now expects that sex as well as other relevant variables be accounted for in project design in funded grant applications.

Authentication: According to Hughes (2007), "between 18-36% of cell lines might be misidentified or cross-contaminated." Thus the NIH "expects that key biological and/or chemical resources will be regularly authenticated to ensure their identity and validity for use in the proposed studies" (NIH website, 2016). For AMC researchers, the Barbara Davis Center's Molecular Biology Service Center
does offer authentication services.

The NIH put the following diagram together to look at the various reasons and implications for their new rigor and reproducibility. The link and other resources are available below.

Resources:
NIH New Grant Guidelines Diagram
Updated Application Instructions
Presentation from the Department of Medicine - Jenny Kemp, PhD

New investigator grants

ORDE has spent the last several weeks updating our New Investigator e-book, which is just out. I thought I would focus the blog on some things to consider when looking for a new investigator grant program. Below are four questions you'll want to answer before applying to any new investigator grant program.

How does the agency define new investigator?
If you're a new investigator, you're a new investigator, right? Well, maybe. Different agencies define new investigator differently. Some are looking at how many years since you received your terminal degree(s). Some are looking at how long you've been in your research position. And, some are looking at whether or not you've received major funding previously.

Is it a mentored grant?
Some new investigator programs are the same sort of research-focused programs as those that are not for new investigators. And others are considered mentored awards, where in your proposal you must address your own career development plan, and also identify a mentor who will work with you throughout the award period. The NIH Career Development or K awards are generally framed in this way. While K applicants must identify a research project in their proposal, the larger focus is on the candidate, their mentor, and their career development. On the other hand, the NSF's CAREER program is research-focused. While applicants are wise to show how their CAREER project fits in with their and their department's larger research goals, this is peripheral to the research project itself. Other agencies run the gamut.

Do your past grants affect eligibility?
At some agencies, the new investigator programs are targeted at bringing very early career investigators and their research up to speed. Thus, if you have shown that you are competitive for major funding previously, this could make you ineligible for some new investigator awards. For instance, at the NIH, if you have secured major funding as the PI, e.g., received an R01, you lose your new investigator status and would not be a good candidate for a K award. But, for the NSF CAREER program, about half  of CAREER awardees have received previous awards from the NSF and it puts them in a better place to compete for the CAREER and certainly does not make them ineligible.

What are the goals of the program?
The questions above really all lead to this question. Before you decide whether or not to apply for a new investigator grant, you must first understand the goals of the agency and the new investigator program. Is the agency hoping to create new independent investigators with their program by funding career development? Or is the agency looking to promote those newer investigators who have already proven that they are independent and productive researchers? When you understand the program, you can consider if it is a good fit for you at your current stage.

Resources:
New Investigator e-book - ORDE
ORDE Funding resources

The proposal abstract: Sell your idea early

Last week I attended the annual research development conference (NORDP), and had a chance to hear from my favorite grant coach, Robert Porter, PhD. Dr. Porter was touting the importance of the grant abstract for selling an idea to reviewers at the outset. More specifically, for NIH applicants, he was referring to the Specific Aims and for NSF applicants, the Project Summary. He made the point that when reading grants, reviewers make up their minds about a proposal in the first two paragraphs or at least in the first couple pages. Thus, it is essential that your core argument fall in the opening of your proposal.

Dr. Porter recommended the following three paragraph formula for writing your abstract:

Paragraph 1: Lay out your problem

1. Grab the reader's interest
2. Explain why it's important
3. Summarize the state of the art and limitations
4. Describe challenges to solving the problem

You want to begin your proposal by driving home the urgency for your project. Explaining what's wrong, what needs to be done, and setting up the importance and need for your research is essential. Porter suggests that you use words like "exciting," or "unprecedented" to convey confidence. However, I find that this language often comes across as hyperbole. It's always better to show how your project is exciting or unprecedented instead of just saying it is.

Paragraph 2: State your solution

1. Describe your concept and credibility
2. Describe the project's fundamental purpose

This is the section where you want to put the meat on the bones of your argument. You've communicated the need and excitement for your project, now you have to give your reviewers confidence that you can actually deliver something substantial.

Paragraph 3: Create a vision

1. Show how your work will advance the field
2.  Envision the world with the problem solved

This is really the icing on the cake that directly relates your vision for your project with that of the agency to which you're applying. Show the big picture of your project; let them know that this is the just the exciting start to even larger results. But, at the same time, never forget that your agency also has a vision and a plan for what they want to do. Your vision should be aligned perfectly with theirs and you should say how they are aligned directly.

Resources:
Crafting a Sales Pitch for Your Grant Proposal - Robert Porter
Six Critical Questions to Launch a Successful Grant Proposal - Robert Porter