Thursday, May 21, 2015

Top Ten Frequently Made Mistakes (in grants)

As a grant developer's tribute to David Letterman's final days on the Late Show, I thought I'd offer my own top ten list of common mistakes grant writers make:

10. They are overly technical
Although, many grant reviewers will be experts in your discipline, depending on the agency, you're likely to have some reviewers who are not as familiar with your specific research area. Because of this, it's wise to cut down on jargon and overly technical descriptions of your project.

9. They offer too much detail
When writing a grant, it's important to offer reviewers enough detail on your project so that they can understand it and get a sense as to whether it's do-able, but you don't want to give them more than that. You have limited space to make your case, so cut out extra details in favor of making your overall argument more compelling.

8. Graphics are not readable
A picture is worth a thousand words (as they say), but not if you can't understand it.  When using graphics in your grant, make sure they look professional, are clear and easy to understand, and are referenced in the body of your grant.

7. There is no white space
Even though most grant writers find it difficult to fit their ideas into grant page limits, it's a bad idea to cram in as much text as you can. You want your grant to be as visually appealing as possible and appropriate white space and break up of ideas make your grant easier to read.

6. They are poorly organized
Although many writers get started by just writing and brainstorming, make sure at some point you step back from your free writing and craft an outline for your grant to make sure your final proposal is clear and logical.

5. The narrative is not compelling
Although researchers' projects are incredibly exciting to them, it's important to take the time to understand what makes the research exciting and what might make it especially exciting to the agency to which you're applying? Remember to focus on what's most important to your audience.

4. The aims/objectives are not clear
Certainly, you must be crystal clear on your research objectives before crafting a grant, but even when you are, it's a good idea to vet your aims or objectives with your peers, mentors, and even laypeople to make sure they are equally clear to your reviewers.

3. They submit at the deadline

Grants take a lot of work and a lot of time. So, it's not uncommon for applicants to be putting their final submission together at the last minute. However, doing this puts you at great risk. Systems go down and errors pop up, and you don't want to lose your shot at funding over a minutes-late submission.

2. They don't fit with the agency's mission
The grants climate is competitive. You may have heard the saying, "flat is the new up" in reference to federal granting agency budgets. Because of this, only the grants that are a perfect fit for an agency's mission, vision, and approach will have a chance of being funded.

1. They do not read the program announcement
It may seem weird that this is the number one suggestion. Yet, as Robert Porter found in 2009, 60% of grants are not even reviewed because they are a poor fit or they do not follow directions. And, anecdotally, we frequently come across PI's who are well into writing their grant and have not yet read the agency's program announcement.

The Grant Development Lifecycle - NORDP 2015 Presentation
What Do Grant Reviewers Really Want Anyway? - Robert Porter

Friday, May 15, 2015

Understanding Private Funders: A Look at the Robert Wood Johnson Foundation

As stated on their website, "The Robert Wood Johnson Foundation is the nation's largest philanthropy dedicated solely to health" (RWJF). Because of this, the RWJF is certainly worth knowing if you are a health researcher, but even if you are not, it's useful to understand the RWJF to get a sense of how private funding agencies can work similarly and differently than the federal agencies with which we tend to be more familiar.

Understanding the big picture vision of a private agency is vital. Although, it is important to understand the big picture for any agency, private agencies tend to set a a more narrow focus for their funding. For instance, the vision set by RWJF is "Building a Culture of Health," which although they define this culture of health very broadly, they have nailed down four interconnected and measurable action areas:
  • Making Health a Shared Value
  • Fostering Cross-Sector Collaboration to Improve Well-Being
  • Creating Healthier, More Equitable Communities
  • Strengthening Integration of Health Services and Systems
Find more details on these action areas in the President's 2015 Message.

Another way that private funders like RWJF can be unique is in the role of their Program Officers (POs). Some smaller foundations do not have POs, and for those that do, the responsibilities and processes of those POs can differ from what we expect from POs at the NIH or the NSF. For instance, the RWJF assigns a PO to each call for proposal and supplies an email address for that PO in the call. PI's are encouraged to email questions to the PO and the PO will offer live webinars at times to allow him/her to answer questions for everyone at once. However, the RWJF does not supply comments or feedback on proposals and the PO does not discuss declined proposals.

The RWJF also tends to be a proactive agency in terms of the connections it makes with researchers in the field. Last week Benjamin Miller, Assistant Professor in Family Medicine and RWJF-funded researcher spoke at ORDE's Know Your Agency Lunch on the RWJF to share his experience.  He talked about how the RWJF reached out to Dr. Miller's colleagues to find out what he was like to work with before contacting him directly and asking him to submit a proposal. Since being funded, the RWJF has served as a partner and a resource for Dr. Miller. This gives us a glimpse into the RWJF's approach and also the vast network they have and use to move their mission and vision forward.

As with any agency, it's important to understand the mission/vision of the agency, their history, and what sorts of projects they've been funding as you work to understand if your research would fit. Use the links below to begin to understand if the RWJF might be a good fit for your research.

ORDE Know Your Agency Brief: RWJF

Friday, May 8, 2015

Developing a Concept Paper

Well before you begin writing a grant, you should have a concept paper. Concept papers are one to two page overviews of a research project or idea that an investigator uses to vet and tailor their project to be a good fit for a particular funding agency. 

A concept paper can be used as a tool to allow a researcher to hone a particular research idea, but they also serve as a tool for marketing your research and networking with potential collaborators or Program Officers. You can keep concept papers on hand at conferences to give to folks you're interested in partnering with, or email them to a Program Officer to get a sense of the fit of your project for their directorate, study group, or program.

Heilmeier's Catechism is a good format to use for your concept paper.  The catechism's creator, George Heilmeier is the former Director of the Defense Advanced Research Projects Agency (DARPA) who recognized that answering the following questions are important when making a good case for funding.

What are you trying to do?
Give the reader an overview of what your project is about. Don't go into great detail but give him/her a good sense of the project.
How is it done today, and what are the limits of current practice?
Offer some context for your project.  Give enough background that the reader understands where your project fits in the larger story.
What's new in your approach and why do you think it will be successful?
Explain why what you're doing is unique, why it's important, and why it needs to be done now.

Who cares? 
Who benefits from your research and how? What are the financial, social, health, or scientific benefits from you project?
What are the risks and the payoffs?
What happens if things don't go as you planned, and why is your project worth taking those risks, i.e., what is the payoff and why is it worth the risk?
How much will it cost? How long will it take?
What are the midterm and final "exams" to check for success?
How will you know that you've been successful? What sort of assessment is necessary to show success or impact?

Heilmeier Catechism: Nine Questions to Develop a Meaningful Data Science Project
Writing a Concept Paper - University of San Francisco

Friday, April 24, 2015

Some Tips on NIH Grants

This week, ORDE hosted an NIH panel with faculty from our Denver campus who have been funded by the NIH through a variety of institutes and mechanisms. The NIH is a large organization made up of 27 institutes and centers and using a variety of different funding mechanisms. Because of the size and complexity of the NIH, it can be hard to nail down, especially for early career investigators, but our panelists offered the following universal NIH advice:

Speak with your Program Officer:
Because the NIH includes so many institutes with their own ways of doing things, it is essential that you get to know your Program Officer to understand the nuances of the institute, get advice on study sections, and to discuss your project fit. The panelists also suggested talking with your Scientific Review Officer (SRO) when working on a resubmission. The SRO manages the peer review process for a particular study section and can often give you clarification on your grant review and the comments you receive.

In an NIH grant, fill up the page:
Dr. Laura Argys, Professor of Economics, NIH funded researcher, and long-time NIH reviewer urged PIs to not leave large sections of their grant blank.  This is because if reviewers wonder why you didn't discuss a particular piece of your project, they're less likely to let the PI off the hook since they had room to go into it. The next suggestion she offered was to allow for white space in your grant.  Make sure that the final grant is easy to read and maneuver.

Reinforce what's important:
Often, only three reviewers read your grant and they have a lot to get through and will skim/read your grant quickly.  This makes it important for you to clearly state what you're doing and why it's and important and to restate these important elements throughout your grant.  That way if a reviewer misses something important the first time, they get it again at another point. Even if they catch it the first time (and remember, they may be jumping around and not starting at page one), they'll understand its importance when it is reiterated in the next section.

Give reviewers clear cues:
NIH grants are scored on their significance and their innovation, so when you are describing these elements of your project, use the phrase "The significance of this project is..." or "This project is innovative because..." The three reviewers that read and present your grant to the rest of the study group will appreciate this seemingly blunt description of these important pieces of your grant, because it gives them the tools they need to understand and hopefully advocate for your project.

Have a layperson review your grant:
Although your project may be very technical and complex (most NIH projects are), you still want a layperson to understand your abstract and specific aims - this means you have been quite clear and you can count on a diverse set of reviewers to get what you're doing quickly and easily. They'll likely be thankful!



Friday, April 17, 2015

Grant Reviews: Making your project better?

This week we held our second seminar on Grant Resubmissions on the Denver campus. Dr. John Swallow spoke to our group about how he approaches resubmissions. And although, the thought of needing to write and re-write your grant is enough to make anyone melancholy, Dr. Swallow had a different perspective.

He told us that someone had once told him that on average a PI must submit a grant four times to be funded by the NSF.  Dr. Swallow, Professor and Chair of Integrative Biology, embraced that statistic, whereas others might have decided it wasn't worth submitting in the first place with those chances.  Dr. Swallow submitted his CAREER grant to the NSF all three times before it was funded on the third application. But because he expected the need to resubmit, Dr. Swallow worked to use the feedback he received from reviewers to improve not only his grant, but his CAREER project. This meant that when he was finally funded, the project that he carried out was much better than what he would have done had his first CAREER application been funded.

We've all heard the adage: feedback is a gift. Yet we oftentimes do not feel like we've received a gift when someone has constructive criticism for us. Although Dr. Swallow talked about the positive approach he takes to reviewer comments and resubmissions, he also talked about his frustration when he hears that he has not been funded.

A couple of weeks ago, our blog focused on deciding to resubmit, and we suggested talking to your Program Officer as you make that decision.  Yet, you want to make sure that before you talk to your PO that you have let go of your frustration so that you don't inadvertently take it out on your PO - that would be a bad move.  It doesn't do you any good for a PO or any colleague to have a bad taste in their mouths about you when they're reviewing your grant because you vented your frustration to them in a moment of weakness.

Similarly, when you respond to reviewer comments, you want to make sure that you keep a positive tone, reminding reviewers of what they liked about your grant (if they're the same reviewers) or telling your new reviewers what the first group liked about your grant.  Although you can't be in the room when reviewers discuss your grant, you want to do everything you can to set a positive and excited tone around your project.  If you're frustrated and defensive, even a great grant won't fair as well.

Social Psychologist, Amy Cuddy, studies how behavior can influence thinking and attitudes, and in her Ted Talk she discusses how you can not only "fake it till you make it" when you're not feeling confident, you can fake it till you become it. So, in acting confident (even when you don't feel it), you can trick yourself and everyone else into believing.

I think this principle can be translated into grant resubmissions. Even if you are feeling bad about your grant when it's not funded, by "faking" enthusiasm and excitement to improve your grant, you can not only convince your reviewers of this excitement in the next round, you can actually make your project even better, just like Dr. Swallow has done.

Thursday, April 9, 2015


Last week, ORDE held a seminar on the National Science Foundation's (NSF) Faculty Career Development Award (CAREER). Participants asked a host of good questions, and we've shared many of them below along with some responses and resources for further edification.

How do I reach out to/work with the Program Officer (PO)?
Identifying and working with a PO is important for understanding if your project is a good fit for an agency. The NSF and the CAREER program is no exception.  As Robert Porter suggests in his article, Can We Talk? Contacting Grant Program Officers, begin by sending an email to the PO for the directorate in which you're interested. Briefly, describe your project and ask to schedule a follow-up call.  Have a one-page description of your project ready in case the PO asks for it. You don't want to make them wait once you've sparked their interest.

How long can the letter of collaboration be?
The CAREER program announcement for 2015 and 2016 has few changes from the previous program announcement, but one of these changes is the requirements for the letter of collaboration.  The NSF has limited the letter of collaboration to one sentence. They actually require applicants to use the following sentence and just to fill in the blanks:

"If the proposal submitted by Dr. [insert the full name of the Principal Investigator] entitled [insert the proposal title] is selected for funding by the NSF, it is my intent to collaborate and/or commit resources as detailed in the Project Description."

As the sentence suggests, you should include the details of your collaboration in your project description.

What other career-oriented grants are offered at other agencies?
Many agencies offer career grants that are intended for early career investigators, but they vary widely. Some of these are mentored grants meant to move the investigator to independence (e.g., some of the NIH's K grants). Others are for independent investigators launching their research career (e.g., the NSF CAREER award). You can get a sense of some of the different career grants from ORDE's New Investigator's funding e-book.

What are success rates by directorate?
It is difficult to find success rates for the CAREER grant by directorate, but Dr. Sonia Esperanca offered some data in her presentation for the NSF Denver conference in the Spring of 2014. According to her chart on slide 12, in 2013, Computer & Information Science and Engineering (CISE) had the highest success rate (near 25%) and the lowest was Social, Behavioral, and Economic Sciences (SBE) (near 8%).

How important are preliminary data and publications when applying?
Certainly, this is hard to answer in a straight-forward way, but we asked Professor Michael Jacobson, who recently served as a Program Officer for the NSF. He suggests that competitive applicants should have some preliminary data when submitting their CAREER, because the PI should be able to show their expertise and would likely not be working on a brand new project. In terms of publications, Dr. Jacobson thinks that some are necessary, and those that show independence from one's mentor are looked at more favorably.


Friday, March 27, 2015

Deciding to Resubmit

At many agencies, resubmitted grants have a higher success rate than first-time grant submissions. Yet many researchers are deterred from resubmitting when reviewer comments and critique are difficult to swallow. The most successfully funded researchers have usually received as many no's as they have yes's and often more. However, when you receive a "no," you have a decision to make. If you decide to resubmit, you want to move as quickly as you can to revise and resubmit.

As you consider resubmitting and what you might do in your resubmission to enhance your chances of success, consider the following questions and suggestions:

What level are the suggested changes?
Getting comments from reviewers that suggest you clarify a section of your grant or make minor changes to your methods are very different from comments that suggest a flawed hypothesis or a poor fit between your research goals with the agency's funding priorities. Determining if reviewers are excited about your project and whether changes you make can move you from a not funded to funded in the next iteration is key to deciding if you should revise and resubmit to the same agency.

Is there a better agency fit?
Sometimes in reading reviewer comments, you may get a sense that there is a fundamental disconnect between your project and the agency's mission or goals. If this is the case, you may want to begin to search for agencies whose mission might better align with your work.

Which comments hold water?
One of the most frustrating aspects of reviewer comments is when you get contradicting opinions or comments that seem out of left field. Despite initial reactions to comments, after you've taken a couple of days to mull them over, go back to your proposal and honestly weigh which comments can make your project better and which can't and why they can't.  For those that can help you improve, be grateful for them and begin incorporating them. For those that are not helpful, see if there are ways you can improve your proposal to make your decisions and line of thinking more clear.  Perhaps a reviewer misunderstood aspects of your proposal, which led to their questionable comment. Are there ways you can revise to avoid such confusion by future reviewers?

Must you respond to all comments?
Of course some of your reviewer comments will be good to respond to or incorporate into your grant resubmission, but the question of whether you need to respond to all comments depends on whether the agency to which you're submitting allows a response statement in your resubmission.  Agencies such as the NIH request an introduction to the grant that outlines your responses to your summary statement.  In  cases where you must respond directly, it's wise to respond to all of the comments, especially when you will have the same reviewers for your resubmission.  If, like at the NSF, all grants are considered new even if they are resubmissions, you needn't respond to each and every comment in your grant if it does not make sense to do so.

Should you talk with your Program Officer?
The answer to this question is almost always - Yes! But, with resubmissions, make sure that you're not angry or trying to defend yourself before you pick up the phone.  Once you're ready to have the conversation, do call/email your Program Officer.  Oftentimes, your Program Officer was in the room during the review of your grant and they can offer you some clarification, advise you on changes you're considering making, and even help you make the decision as to whether you should resubmit.

To learn more, register for one of our upcoming seminars on Grant Resubmissions.

Resubmitting Your Grant - Wayne State University
Resubmission of the Grant Proposal - Chapter from Writing Dissertation and Grant Proposals (Chasan-Taber, 2014)