Friday, May 20, 2016

Getting and giving peer feedback

Peer feedback is really a crucial tool for faculty researchers. It is core to the review processes for publications and for grant proposals. But even beyond this, peer feedback is essential for getting your manuscript or your grant proposal into shape before you submit. However, giving and receiving feedback is oftentimes an overlooked skill, but one that can make all the difference for you and your colleagues when employed well.

At ORDE, we recommend that PIs get three internal reviews of their grant and integrate the feedback before submitting. One of your reviewers should be a layperson and the other two should be experts in your field, but not familiar with your project (you don't want your peer reviewer intuiting things in your proposal that aren't there; you want them to identify what's missing).

Getting feedback
When asking for feedback from your peers, be specific with the kind of feedback you want. I've found that when asking for feedback on my writing, if I don't explain what sort of feedback I'm looking for, my reviewers tend to resort to their grammar school training and focus on punctuation and spelling rather than advising me on clarity.

Along with giving your reviewers instructions on what you want them to look for in your proposal, also give them enough time to adequately accomplish the task. If you're waiting till the last minute to pull things together, and then dropping it on your colleague's desk, not only will you frustrate that colleague and likely get sub-par feedback, but you probably won't have time to really use the feedback they do give you. On that same point, give your colleague a heads up. Ask them if they can review your proposal weeks in advance, tell them when you will send it to them, and agree on when they can have feedback back to you.

Lastly, be sure to thank your peer(s) for investing their time in you and your proposal. Make sure you offer to return the favor and let them know how important their feedback was to you. And, circle back around when you get that grant to thank them for their role in your success.

Giving feedback
When your colleague comes back around to ask you to review her proposal, some of the same tips apply. First of all, ask your colleague to plan ahead to ensure you have adequate time to review the proposal and she has adequate time to incorporate useful suggestions. Also, ask your colleague what she wants you to look for. Is she still tweaking her idea? Is she open to methodological suggestions? Does she want feedback on clarity or persuasiveness? If she asks you to proof it, you may want to suggest that they find an editor for that.

Giving feedback doesn't seem like a difficult concept, but many folks are uncomfortable with giving constructive feedback to a friend or colleague. You don't want to hurt any one's feelings after all. I actually think the reason people stick to grammar and spelling feedback is because they're nervous about offending a colleague. Yet, constructive feedback is essential to get better. However, good editors or reviewers understand how sensitive people are about their work. The editing adage, "murder your darlings" is often used, because getting feedback on your latest pride and joy can feel quite personal.

So, to deliver your necessary but perhaps sensitive feedback, some suggest a sandwich approach. Begin by telling your colleague everything you liked about their proposal, then move into some constructive critique, and end with more positive. In using this approach, it's important to be sincere in what you found as strengths. And, instead of dividing up the positive attributes, share all of them at the front end and finish by describing how excited you are about their project.

Also important to giving good feedback is being specific with your critique, and making sure your advice is usable. We don't call it constructive criticism for nothing. Negative feedback that is not useful or can't be applied is just mean.

I included a discussion of getting and giving feedback, because to be successful, you need to do both. If you make it a habit to ask for feedback but never find time to give it, then it won't belong before your peers are too busy to look at your proposal. Reciprocity is the name of the peer feedback game.

Giving and Receiving Feedback: A Guide to the Use of Peers in Self Assessment - UTS
The Delicate Art of Giving Feedback - Harvard Business Review

Friday, May 13, 2016

Developing your writing practice

I recently heard one of our faculty members compare writing to exercising. In that, they're both hard to do either one consistently. But both are good for you and you need to practice them to develop your skills. She warned that, like shirking your daily run when planning to run a marathon, you'll be in a lot of pain when you try to write a major piece, be it a manuscript or a grant proposal, when you haven't been writing consistently.

I thought this was an appropriate topic as the semester ends, and for many faculty researchers, their daily schedules change for the summer. Certainly, summer is a good time for a little break, but make sure you don't pay for that break with your writing!

Don't wait to be inspired
I'm happy to report that in all of my bad habits, procrastination isn't one. Yet, I must admit that when I have a major writing project on the horizon, I often go through a short spurt of paralysis. I'm not sure how I want to approach the piece, or it feels a bit insurmountable. However, I recognize when I'm doing this and am able to move through it. I do this in one of two ways. If I really am clueless as to my approach, I turn to the literature. You can't write well, if you haven't read the relevant literature and know what you're contributing. The second way is to just start writing. Be careful not to get stuck in the reading where you just keep looking for the next article instead of starting to write.

Do outline and rewrite
As I've mentioned in previous blogs, free-writing can be a good way to get yourself going even if it's really bad in the beginning. However, if you write your first draft completely off the cuff, oftentimes you have a lot of re-working and re-writing to do. Now, you'll always have re-writing to do even if your first draft is awesome. But, if you've written 20 pages without looking back once, who knows what you're working with (including you). By all means, start free writing, but try drafting the introduction of your paper and then working with that to decide what makes sense and what doesn't, what should you keep and what is off topic. Use this initial writing and brainstorming to create an outline for your piece. Look at the nuggets that come out of your initial thinking and writing and decide how you're going to build on those.

Try to write in chunks or sections and make sure you're on track. Having your piece organized in sections makes it easier to re-write and revise your writing. As you finalize your writing, make sure that it flows and those sections fit together. Also, try to avoid correcting your work at a proofing level (grammar, spelling, and punctuation) too early. Stay at a higher level of editing/revising until you think you've got a final draft.

Do consider your writing practice
As you consider your writing practice for the summer, think about what you're doing now. When do you do your best writing? Are you a morning person or a night owl? Try to plan your daily writing around when you're at your best. Also, look at the bigger picture. What is the flow of your week? What days do you have meetings? What days can you spare uninterrupted time to write? Also, be honest about your bad habits. If you procrastinate or get paralyzed on a new project, recognize it and make a plan to get through it. Take all these things into account and make yourself a summer plan. Identify when you'll write and when you'll exercise. :)

How to Develop a Daily Writing Practice - Mattan Griffel
16 Ways to Improve Your Writing Skills  - Dan Shewan

Friday, May 6, 2016

Research collaboration makes it happen

We in ORDE sometimes meet with faculty that have an abundance of great ideas for research for which they are seeking funding. But these great ideas are not always fully developed or in alignment with agency goals. Given the traditional lone-wolf approach in research, many PIs think that if they haven't figured out a whole project by themselves then they won't do it. However, more and more, the most exciting and competitive projects are collaborative. But, if you're not sure how to form these collaborations, here are some tips.

Know your strengths and weaknesses
Before you reach out to collaborators, be clear on where your expertise lies and where it is limited. This will help you to pitch yourself and your project to potential partners and help you to identify the right partner(s).

Identify potential team members' strengths and weaknesses
Once you're clear what your role should be in a project and what roles and needs you must fill to make it come to fruition, you're ready to look for collaborators. Talk to your colleagues and make connections in the areas you have needs. When you identify a potential collaborator, set up an initial conversation to vet them.

Be ready to share
If you want a collaboration to be successful, you shouldn't go into conversations assuming that you're the boss and your collaborator will just provide what's needed. If that's how you want to run things, then you're really looking for a consultant on your project, not a collaborator. Most researchers will not be willing to invest themselves in a project that does not feel like it is theirs. So, be prepared to not only share your idea, but adapt the idea with your new partner(s).

Be ready to assert yourself
On the flip side, a collaboration shouldn't feel like a hand off of your idea to another. Some researchers feel their project is co-opted especially when they seek collaboration with a more seasoned PI. To combat this, be ready to assert yourself and demand a true partnership in the project.

Know what's in it for each researcher
An essential piece to a good collaboration is setting each collaborator up for success. When beginning a collaborative project, it's important to get all motivations out on the table to make sure they're complementary. Also, this is the time to discuss order of authorship of subsequent publications, and who will provide what resources and time to which parts of the project. This can help you avoid confusion or conflict later on.

But is collaboration worth it?
Collaboration is a lot of work, but it seems to be the direction we're going. According to Adams (2012):
  • Co-authored pubs tend to get cited more
  • The first paper with 1,000+ authors was published in 2004
  • The U.S. collaborates with China the most on 3-4% of its Science papers
The Rise of Research Networks - Adams (2012)

Friday, April 29, 2016

The mystery of funding decisions

This week, the National Institute on Aging (NIA) put out a blog detailing their complex use of multiple funding lines to make funding decisions in various priority areas. The NIA does this to offer transparency and clarity to researchers. Basically the funding line identifies the minimum score that will be funded in a particular area, such as Alzheimer's and all grants scored better than the funding line are then funded.

The NIA blog joked that to understand their funding system you "need an advanced degree in mathematics," but that got me thinking about the wide range of ways that different agencies make funding decisions. These decisions are often made using some semblance of the following approaches:

Scores/Peer Review
Many agencies have a rigorous peer review process to assess the science or the approach proposed by the researcher. Some agencies, such as the NIH and NSF, recruit reviewers with the right expertise to assess the proposed science in their grant applications. However, even at these institutions, there are oftentimes grant reviewers who are not experts in your precise area reviewing your grant. This makes it important to strike a balance in your grant writing that speaks to the experts reading your grant, but also is clear and compelling to those who are not experts in your area.

When it comes down to it, not all grants that score well in peer review are funded, and in fact some grants that receive slightly worse scores may get funded ahead of the most stellar if they are a better fit for the agency's priorities. Although PIs sometimes look at agencies as piggy banks, that is certainly not how those in an agency see themselves. Instead of seeing their responsibility as handing out money to do the best research, they see themselves as the stewards of a mission with distinct goals in which they are invested. Thus, it is essential that grant applications incorporate the mission and goals of an agency for them to realize success.

Who you are as the researcher, or who composes your research team needs to be the right fit for the project you're proposing. As we often tell our researchers, you must show reviewers that you are the best person to conduct the research you're proposing. Unfortunately, it's not enough to come up with a truly great research project. You need to have that and you need to have the dream team or be the dream PI to carry it out.

It's also true that in the grants world, for many agencies, it's who you know. Some of our PIs who have been funded by the Department of Defense (DoD) suggest that this is a big part of the DoD's funding decisions. The DoD likes to fund researchers that they know and that they know do good work. This speaks to why it's so important to reach out to and work with Program Officers at agencies when they are available to you. Working with a PO gives you a leg up in understanding what an agency wants, but also lets the PO know you, which can also give you an advantage in some cases.

At many agencies, the Director is the one who gets to make final funding decisions and is charged with making the best decision for the agency. This is true for Program Directors at the NSF and Institute and Center Directors at the NIH. Certainly, this is frustrating to PI's when a Director makes an ultimate decision that does not fund them, but the Director has a purview of all of these other mechanisms and can make the best decision for the agency to further their mission.

Not all agencies use all of these approaches to determine funding, so it's important for you to do your homework on the agency, their mission, goals, and their processes even before you sit down to write your proposal.

Resources (examples of how funding decisions are made):
Transparency and funding lines - NIA Blog
Grant Review Process - National Endowment for the Arts

Friday, April 22, 2016

Office of Naval Research

Last week, we were pleased to host two program officers from the Office of Naval Research (ONR). The ONR is one of multiple branches of the US Department of Defense that sponsors research related to their mission. More specifically, ONR interests include mathematics, computer science, electronics, machine learning/intelligence, sensors, communications, ocean engineering and acoustics, materials research (including biomaterials), non-destructive evaluation, cognition, biometrics, computational neuroscience, decisionmaking, gut microbiology, microbial fuel cells, force health protection, stress physiology, aviation technology, unmanned air systems, turbine engine technology, and many more.

Linda Chrisey, one of the program officers who visited both the Denver and Anschutz Medical Campuses is responsible for the following research areas at ONR:

Synthetic Biology (for sensing/information processing, electrobiosynthesis).  She funds work in microbes and 1-2 projects in plants and fungi.  Typically non-biomedical, except for syn bio as applied to gut microbiota (below)

Gut Microbiology (Gut-Brain-Axis, role in behavior/cognitive performance, effects of stressors such as circadian rhythm or sleep disruption, environmental changes such as altitude/O2 levels, rapid cycling of brown fat, synthetic biology manipulation of gut microbiota).

Microbial Fuel Cells (non-biomedical applications. Microbial fuel cells for powering of devices in remote locations (undersea, riverine); microbial electrochemical systems for shipboard waste treatment)

Marine biofouling and its control (interkingdom signaling that influences biofouling community development; mechanisms involved with bioadhesion/settlement by macrofoulers.)

Although it is generally a good idea to talk with a program officer before submitting a grant application to the funding agency, this is especially important at the ONR. Dr. Chrisey encouraged new researchers to call her (or the appropriate PO) or send a brief (one or two paragraphs) description of their project before moving into the proposal development process.

The ONR's proposal development process:
1) PI determines suitability of proposed project to ONR mission, programs, and specific topic areas
2) PI contacts program officer for assistance and questions about applying for funding
3) Program officer requests white paper
4) PI receives informal feedback from program officer, encouraging or discouraging full proposal
5) PI submits proposal
6) ONR Evaluation Panel reviews proposal
7) ONR scientific community makes final funding recommendations
8) Recommendations are forwarded to ONR Contracts/Grants Office for negotiation/award
9) PI’s institution receives award notice

Know Your Agency Brief: Office of Naval Research
ONR Grant Proposals

Friday, April 15, 2016

Make me care

Andrew Stanton, Screenwriter for films such as Toy Story and WALL-E, in his TED talk suggested that a core tenet for storytelling is to "make me care" or rather, make the reader care. This, I would also suggest, is a core tenet for grant-writing as well. Stanton talks about how a good story draws an emotional investment from the audience by using intrigue to make a promise to the audience that engaging with the story will be worth their time.

How then can we as grant-writers capitalize on these same principles to make our readers, and more importantly our reviewers, care? I suggest that we begin by clarifying and developing the case. The overarching case for your research project is composed of two parts: the problem and the solution. You can help your reader care by highlighting both pieces early on and creating a contrast. You want your reviewer thinking, "This problem is terrible and we have to do something about it!" And then you want them to follow that hopefully emotional response to the problem with that of excitement at your solution.

You may feel like this doesn't apply to you, because the problem you're trying to solve isn't that bad or the solution embedded in your project does not solve the whole problem. This is fine; most projects have some work to do on both of these fronts. On the problem side, start looking further out. Why is it that you were drawn to this area? If this problem isn't confronted what could happen? Who are the people affected by this problem (see the storytelling blog from a few weeks ago). If you do have a big, bad problem to work with, don't skip the accentuation of that problem in your grant. If your work will help cure cancer, don't just assume everyone will understand the full significance of the problem. Take the time to share how many lives are lost, how many lives affected, how many dollars spent, etc. so that the reviewer is immersed in the problem.

If your solution only partially confronts the problem, join the club. Most research projects make incremental gains against a problem, but your job is to show your reader why your incremental gain has to happen. What happens if we don't continue down that path you're on? What are the consequences? Whose consequences are they ultimately? What are the potential breakthroughs that you're working toward and what will be the end game?

Peter Frederick in his book, Persuasive Writing: How to Harness the Power of Words (2012), describes his Boo/Hurray Theory as a form of persuasive writing to create the contrast between the problem and the solution in a grant. He suggests, that the grant-writer structure their introduction as follows:

There is a problem/opportunity
It is big enough to justify the funding requested
No one else has come up with an adequate solution
We have an idea for that solution
We can’t just do it because there are major barriers
Funding can overcome the barriers in these ways
If we overcome the barriers and develop the solution, the benefits will be significant for everyone we’re trying to help (Frederick, 2012, p. 148)
Although, Frederick jumps back and forth between the boo and the hurray more than I would probably advise doing, the way he contrasts the problem and solution using this strategy make good sense. He is suggesting that we lead our reviewer through these steps to provoke an emotional investment. In short, we work through this to make them care.

Frederick, P. (2012). Persuasive writing: How to harness the power of words. Harlow, England: Pearson.
Andrew Stanton's TED Talk
Clear and Compelling: Persuasive Scientific Writing Prezi- Naomi Nishi

Friday, April 1, 2016

Visual Displays

This week, I wanted to offer you some more practical tips and ideas for creating and using visuals in your grant applications. Visual displays can be used to help you analyze your results and clarify your thinking, some may help your reader understand your results, and some can do both. Below I discuss some different visual display options.

Matrices can serve as an excellent tool for organizing and cross-analyzing information. I've seen them used in education research proposals where the researcher communicated the tasks, outcomes, and assessment plans by research goal. They're also great for showing time lines in a proposal and outlining due dates for key deliverables. These sort of matrices can help both the PI and the reviewer understand the project and its organization. However, matrices lose their effectiveness when they are too big, and include so much information that the reader can't get a gist of what it means from looking at it briefly. Also, if a matrix gets too complex (e.g., it is trying to cross analyze more than two categories), the reader can get lost in it and at that point a visual display does more harm than good.

Comparative images
I have seen some quite compelling comparative images in proposals. When PIs have lab results that are self-evident and they can show a picture with their test results next to the control, this can be powerful for the reader. Of course, this means that the images must have a clear contrast for them to be striking for the reviewer. Also, consider the knowledge base that interpreting your images will require. If you have mass spectrometer results, but your review panel includes lay people, you may want to reconsider or you may need to include a bit more explanation to allow all of your readers to understand why the images are so remarkable.

Conceptual model
One of the first challenges that confront a grant reviewer when reading a proposal is to get an overall sense of what the PI wants to do. The research project is often complex and can be challenging to understand how it all fits together even for someone in the same field. A conceptual model for the project included early on in a proposal can offer the reader a tool for making sense of your project visually as well as through prose. Basically, a conceptual model is a visual representation of your project and it's goals; think of it as a map of your plan that will give your reader a big picture before they start digging into the nitty gritty. Using a conceptual model, you can show how your research goals, aims, and/or hypotheses fit together and give a sense of the results you expect as well as their impact.

Decision model
I've always been captivated by "choose your own adventure" books. As a kid, I was terrible at them and my character always died right away, but I still loved the idea. Even today, I'm always struck by how many problems or projects can be illustrated using a choose your own adventure style. A decision model is similar to this concept in that it is a flow chart that shows where and how you will choose the path of your research project. When you want to show that even though there are undecideds within your project, you will achieve important results and meet your goals regardless of the path, decision models can help you do that. Of course, a pitfall is that in using a decision model, you are bringing attention to the unknowns in your project, and depending on your plan and how comprehensive your back-up plan is, you could feasibly cast doubt in the minds of your reviewers, so use decision models carefully.

Effective Visual Design in Proposal Writing - Allegra Johnston
The Incorporation of Visuals into Grant Proposals - Michigan State University