About this time last year I was starting to think about my NSF CAREER grant application. With my recent positive news and the process still fresh in my mind, I thought I would write about my experience and maybe provide a little advice. There is a lot of advice out there that is no-doubt more valuable than mine, from program officers, senior people, etc., but recency might lend my account some value.
I will break this down by advice, discussing my process throughout. Click on the linked tips below for details (or just scroll down):
1. Take a Lot of Time at the Early Stages
2. Contact Partners Early
3. Make Use of Formal and Informal Resources
4. Writing and Formatting; Sharing and Making Things Look Good
5. Include Experts in Your Advisory Board
6. Responding to Questions
7. Lean on Your Community
1. Take a lot of time at the early stages.
When I work on any grant, I first start with the blurb. This is a short statement of the problem, the opportunity, and the proposed solution. When I help mentor graduate students in fellowship applications, like the NSF GRFP, I advise them to write a short paragraph that covers (1) The problem—start broad and then narrow, (2) What people have done about it before and why it isn’t enough, (3) What is the particular opportunity that will allow you to address shortcomings from #2, and (4) How will you do that?
I worked on the blurb for three months. In February, I knew I wanted to do something with experience sampling and nudges (based on frustration with not knowing what students’ motivation was in the moment and with what we had found in previous work about students making decisions that might not be the greatest). I had a conversation then with one of my current NSF program officers about new ideas. She gave me some general feedback about what they look for in CAREER grants and things that she would question in a proposal about a topic such as mine.
| I sat on the idea for over a month until my university had an NSF CAREER workshop. A lot of the info in the workshop was for first-time grant-writers, but it inspired me to work more on my blurb. I find that, especially at the blurb stage, I need to alternate between talking through and writing through my ideas. This was my opportunity to talk to myself in writing (my notes and questions at right reflect some of the things discussed at the workshop and how they apply to my particular research).
I now had a title and a stronger idea for the blurb. I started a Google Doc to refine my blurb. I used a notebook to keep track of larger ideas. I shared the Google Doc frequently with about eight academic friends who I bounce ideas off of.* |
Notes I took during a CAREER workshop |
*A note: Academic peers can be great resources for idea bouncing, but it is important to return the favor. I try to provide feedback often. It isn’t just tit for tat though, as both the feedback receiver and giver get something out of it–I improve my own writing by helping others with theirs.
At this point I had been working on my ideas since February and by the end of April, this blurb is what I had to show for it. It may not seem like much, but to me, this is the backbone of any proposal, it needs to tell a clear story and drive the direction of the narrative. The blurb wasn’t perfect, but it was a start for something to send out to more senior colleagues and to my program officer.
4/24 blurb
The measurement and influence of motivation in situ
There is a growing recognition that students need skills that aren’t measured by traditional achievement tests to succeed in challenging domains, such as mathematics. Both researchers and practitioners have implemented programs to measure social-emotional learning and student motivation (e.g., Hough, Penner, & Witte, 2016, describing California’s use of social-emotional outcomes under ESSA). Additionally, there has been progress in influencing these dispositions at the subject or course level. For example, interventions that increase students’ perceived utility of mathematics can improve their mathematics self-confidence, effort, and achievement (e.g., Brisson et al., 2017). Although these gains in measurement, influence of, and attention to motivation provide promising insights and new directions for research and policy, they may ignore important opportunities to understand and affect motivation at a finer grain size–one situated within the myriad interactions students have with learning content during their school day. Within this project, I focus on the measurement and influence of motivation in situ–targeting student self-efficacy, value, and choice within the digital mathematics platform, Spatial Temporal (ST) Math. By using data-mining and experience sampling methods to gather information on in-the-moment student motivation and choice, I will be able to present a more nuanced interpretation of how learners experience and interact with digital mathematics content. Further, by including embedded interventions–similar to “nudges”–I will test how motivation-supportive messages within the platform can influence both choice and academic success. A series of experiments within this project will build toward an integrated system of messaging that can be applied beyond ST Math–one that can be used across the broad swath of digital platforms for mathematics learning to support adaptive student motivation, choice, and success in mathematics.
I sent the blurb to five or six senior researchers in my field to ask their opinion—some over email, some over lunch, bringing a paper copy of the blurb. I then sent it to the program officer I spoke to in February. She asked for a one-pager, which is basically the blurb, plus some more method detail, and a statement of intellectual merit and broader impacts. She sent that one-pager to the main CAREER program officer in my directorate, who scheduled a phone call with me for May 7. He gave me some more specific advice and suggested someone who would be good for my advisory board. Between his advice and the advice of my senior colleagues, I continued to refine the one-pager and look for more potential advisory board members. At the end of May, I emailed potential advisory board members, attaching the now updated version of the one-pager, below. Some were on board immediately, others had some clarifying questions and we had a number of emails before they volunteered. All were gracious with their time and advice (more about advisory board members below).
THE MEASUREMENT AND INFLUENCE OF MATHEMATICS MOTIVATION IN SITU
This project leverages the unique affordances of digital mathematics curricula to understand and influence student motivation for mathematics as students engage with content and make choices directly relevant to their learning. Motivation for mathematics has been shown to predict learning and achievement outcomes, but most prior research on mathematics motivation has occurred at a broad level, such as understanding and improving student mathematics course-taking (e.g., Harackiewicz et al., 2012). Although these gains in measurement, influence of, and attention to mathematics motivation provide promising insights and new directions for research and policy, they may ignore important opportunities to understand and affect motivation at a finer grain size–one situated within the myriad interactions students have with learning content during their school day. Within this project, I focus on the measurement and influence of motivation in situ–targeting student self-efficacy, value, and choice within the K-8 digital mathematics platform, Spatial Temporal (ST) Math. By using data mining and experience sampling methods (ESM) to gather information on in-the-moment student motivation and choice, I will be able to present a more nuanced interpretation of how learners experience and interact with digital mathematics content. Further, by including embedded interventions–similar to “nudges”—I will test how motivation-supportive messages can influence both choice and academic success. A series of experiments within this project will answer questions about the dynamic nature of student mathematics motivation and learning and build toward an integrated system of messaging that can be applied beyond ST Math–one that can be used across the broad swath of digital platforms for mathematics learning to support adaptive student motivation, choice, and success in mathematics.
I propose to use data collected within ST Math to first understand student experiences and choice within the platform for the purpose of identifying potential areas for intervention. Three main types of data will be collected: (1) data generated from student interactions with ST Math, including student scores and choices as they navigate the platform; (2) student answers to ESM questions regarding student self-efficacy, value, and affect for and about the game content as they experience it; and (3) broader measures of motivation and achievement, both within and outside of the game.
The project will proceed in three phases: (1) embed experience sampling questions and enhanced choice options within ST Math, pilot these measures, and develop protocol for data extraction and analysis; (2) examine associations between situated student motivation, choice, and academic outcomes, both in and outside of ST Math; develop nudge interventions based on these associations; (3) iteratively test interventions in a series of A/B experiments within the platform.
Intellectual merit: Results of this project will advance the theoretical and analytic bases for understanding motivation and choice situated in a real-world learning context. By combining top-down theory-driven analyses with bottom-up data mining techniques, this project will contribute synergistic new methods to the fields of educational psychology, mathematics learning, and educational game design. These new approaches have transformative potential, not only to understand student motivation and behaviors, but to inform improved teaching and learning in mathematics and with digital curricula.
Broader impacts:This project addresses the important national concern of strengthening the mathematics competency of our students through enhancing their mathematics learning and motivation. By investing in innovations to understand and improve student actions within platforms such as ST Math, we can reach large numbers of children with programs that approach maximal effectiveness with each iteration. Digital platforms that enhance both student content learning and motivation can have impacts that extend beyond the particular content—providing students a foundation to succeed in challenging mathematics throughout K-8 and beyond. [Back Home]
My project involved a software developer and three school districts. I had originally intended it to be two school districts, and as soon as I had a blurb, I contacted the two districts and the developer. For the districts, it can take weeks or months for approval to go up the chain and get to a point where they are ready and able to send a letter of collaboration (there are specific requirements for this letter, so make sure to provide a template for your collaborative partners). For the developer, they had a subcontract as part of the grant, which involved even more paperwork, including a formal budget. This meant there were two channels of communication with them: With me, regarding the content of the proposal and the scope of their work, and with my grants office, regarding the specifics of the paperwork (that I know very little about).
A couple of weeks before the grant was due, I realized I was not using my money efficiently. I needed to visit at least one district to observe in classes and conduct cognitive interviews, but both my districts were in other states and including the travel was costly. Luckily, I had a local district partner who I knew would return letters quickly, and I was able to add this third school district on very short notice. [Back Home]
3. Make use of formal and informal resources.
Formal: As I’m faculty at a research-focused college within an R1, I have certain institutional resources that make grant writing much easier and more efficient. Paramount among these is a pre-awards office who handles paperwork and budgeting. As soon as I had a rough sketch of what I wanted to do (May, after I had formed a blurb and outlined what the research might look like), I contacted the research development director to work on a budget. When working with grants staff, they can often put together a spreadsheet from your description of what you need (they will know how much students and postdocs cost, etc.). This spreadsheet then goes back and forth as the budget is adjusted, often up until a few days before the due date. An aside: The CAREER budget is very tight—five years is a long time to make 500K to 1M stretch, especially when you have subcontracts or districts that need funds (remember: about half of this money is going to go to “indirect” costs).
Grants staff also gathers all the other paperwork that is needed: Data Management Plan, Facilities Statement, BioSketch, Current and Pending Awards List, Collaborators List, Postdoc Mentoring Plans, Budget Justification, and more. If you have such an office or staff, no need to try to reinvent the wheel or search on your own for how to put these documents together. Ask for examples and/or templates.
As I mentioned above, one formal resource you may have is a department, college, or university-wide NSF CAREER workshop or office hours. Attend these and the webinar offered in early summer by each NSF directorate. My directorate (EHR) offered theirs mid-June. You may also have a formal mentoring program at your university or funds available to take a trip to D.C. to meet with program officers (this is one way to get program officer feedback if you don’t have relevant current grants; cold emails to program officers are also fine). Even if you don’t know that these opportunities are available at your university, ASK. When you are serious about bringing in funds, your department chair might find a way to come up with seed money to increase your chances.
Informal: PhD students (and early career faculty) differ in the strength of their networks. Some have famous well-connected and very hands-on advisors and others have virtually no advisor at all. There is no need to accept either as the limits to our mentor network. I have found that senior scholars are extremely generous as long as I am respectful of their time, approach them about a topic they care about and want to discuss, and have an attitude of openness regarding sharing my own materials and experiences. As I wrote about when discussing my blurb, I relied heavily on conversations with mentors and peers to help hone my ideas. If you have no connection to famous researcher X, you likely have connections to peers or graduate students that know them or are trained by them. Some of my most helpful feedback comes from grad students trained by someone outside my immediate network—with grants, I know I need to write clearly enough that they can understand.
CAREER-specific materials are also available through informal means. In early summer, I searched on the NSF CAREER website for funded awards that were close to mine in topical or methodological focus. I identified four recent awardees and cold-emailed them introducing myself and asking if they would be willing to share their proposals. In my email, I offered to share any of my own resources in return. All four responded within three days with copies of their narratives (Thank you!). Although amazing, these exemplars didn’t help me with the specific content of my proposal (that’s unique to each applicant), but I was able to see the range of options for how to structure the proposal, what to write about the integrated education/research plan, and how much detail was necessary for specific methods. I am more than happy to pay it forward and share my proposal narrative with anyone who emails me.
I also spent a lot of time googling, especially in those early months while I was working on the blurb (perhaps one day someone will find this blog post that way). Lots of winners, mentors, and reviewers have written about their experience and provide tips for CAREER hopefuls. I tend to gather as much information as I can and then try to distill to the common points. It’s easy to fall down the rabbit hole though, as information online about the CAREER is endless. I eventually identified the point of diminishing returns and moved toward more closer, specific resources than Google could provide. [Back Home]
4. Writing and formatting; Sharing and making things look good
Because my writing is so social and involves asking for feedback frequently, I find that Google Docs is the best platform for my narrative as it is forming. I have not found any other platform that is as easy to share and solicit feedback with when your feedback-givers may have varying levels of familiarity with other platforms. Google Docs though, does not make for pretty documents—I learned this after spending too many frustrating hours formatting and having to reformat for every tiny change. The way I handle this is to let my readers know at the top of the doc that things are not yet formatted and to include a lot of placeholders (e.g., {image goes here}).
Screenshot Showing My Narrative in Word |
My favorite day in grant writing is when I can take the document off of Google Docs and format it in Word. I know, I should probably be using LaTex or R Markdown, but there are only so many things we have time to learn on the tenure track. I find it easiest to use tables to organize my graphics and text. I often create models within PowerPoint, as in the example from my theoretical framework, at left. |
Because I research a commercially-available software, there are a number of glossy graphics I can include in my proposal, and I think they help to break up the text and give the reviewers a sense of the platform I will be researching (I apologize that much of this blog post is largely uninterrupted text).
Sometimes it takes me time to find the right tool for the graphics I need. I struggled to make nice flow charts for a while before a colleague pointed me to draw.io, which I used to make the figure below (the stars were inserted after-the-fact). Because space is tight and I include a lot of graphics, I forego perfect APA style for my table and figure headings. The most important thing is that the narrative is clear and easy to read for the reviewer, and these headings don’t really offer much in this situation. Even though it takes more space, I do prefer to use APA-formatted in-text citations over endnotes. Again, I try to think from the perspective of the reviewer: Using in-text parenthetical citations allows reviewers to immediately see the body of literature in which I’m situating my work. This information is important—I don’t want them to have to scroll or flip for it.
5. Include experts in your Advisory Board
As junior faculty, I include senior scholars (often distinguished professor level) on the Advisory Board (AB) for any grant I submit. The rules for letters of commitment for AB members make clear that they are not endorsing the project; however, having the right people shows the reviewers that you know who the experts in your field are and where you might have weaknesses in your team.
For the CAREER, there is no “team” exactly, it’s just the PI, so a good AB is even more important. When I spoke with the program officer in May, he noted that my project was going into EHR, so it had to really focus on a STEM topic. I couldn’t just be using math as a means to study motivation, but had to think about how what I was doing was embedded in mathematics. AB members with solid math education backgrounds would help to address this potential concern. The program officer had a specific recommendation, who I contacted. In looking for another math ed person, I thought about the content of the platform I study, ST Math, and invited someone who had expertise in virtual manipulatives for math learning.
When I thought through all the areas my proposal covered (math motivation, experience sampling, specific elements of motivation theory, computer science/data mining), I realized I would need six people instead of the normal four I generally go for on regular NSF grant proposals. Often, AB names are just listed within the narrative, with further detail maybe provided in the budget justification. This is great to save space, but again, thinking about the information the reviewers want to know, I chose to dedicate space within my narrative (a full page) to providing detail on the expertise of my board. I can’t know for sure, but I think it paid off—almost all my reviewers mentioned the strength of the AB. [Back Home]
The first step in getting an NSF grant funded is almost always receiving questions from the program officer. In December, I received an email from mine with a note to “please see attached concerns” about my proposal. Although this doesn’t sound positive, it usually is. The “concerns” were nothing to take lightly though. I had one week and three pages to answer NINE meaty questions about my proposal theory, methods, and logistics. One thing in particular that they saw as weak was my plan for integration of research and education. I had dedicated over a page to this in the narrative and had felt confident about what I had written. I had a phone call with the program officer and she gave me some insight into what I needed to do. I tried to embed the educational activities better into the research and my timeline.
I sent in the answers a week later and heard nothing. Soon after that, the government shut down and there was no chance to hear. Once it reopened in February, I was thrilled to receive another email from the program officer, this time just asking for a phone call. Unfortunately, there were a number of issues remaining.
| I had not done as good a job at integrating education and research as I had thought. In the conversation with the program officer, something finally clicked and I understood I was supposed to show the bidirectional relation between teaching and research. The big question was: How would the educational activities inform the research over the course of the five years? I was able to put together a table to answer this question. | Snippet From My Revised Education Plan |
The program officer also asked for some other small changes to the budget and mentoring plan, which were easy enough to do, and ideas on what should go in the abstract that is shared on the NSF website. Receiving the questions and communication was exciting, but anxiety-inducing. I set aside most other activities for a few days in December and for over a week in February to make sure I could respond well and in a timely manner. [Back Home]
The entire process, from initial idea to award took a year. More than any other grant, I felt very personally invested in this one—it was something that represented me alone, whereas my other grants involved teams of researchers. That personal investment can wreak havoc on the nerves. It helped to have others who were also applying to share information with or who could relate to the particular CAREER waiting game anxiety. Some of the same academic friends who gave me feedback in the early stages were still there as I waited or wrangled with the program officer’s questions. An online community of other academic moms who were also applying for the grant served as additional support.
In general, all my academic endeavors benefit from community: To improve research ideas, to discuss teaching and mentoring, to illuminate the hidden curricula that is part of academic processes, and to commiserate. The CAREER application process was no different. I am grateful to my aca-friends online and in person for their company through our academic journeys. [Back Home]
If you would like further details on the content of my CAREER award, please visit the Project Page.
Thank you for reading! I’m happy to answer questions and if wanted, I can put a Q&A section on here as well.
Dr. Rutherford's Lab 