Postdoc Best Practices Proposal | Paul G. Allen School of Computer Science & Engineering

UW CSE's original CI Fellows Best Practices proposals are posted below.

Taking Collective Responsibility for the Postdoc Experience at the University of Washington

Summary

At the University of Washington’s Department of Computer Science & Engineering (UW CSE), as at other major programs across the nation, we have experienced a dramatic growth in the number of postdocs in recent years. And, as is the case with most of our peers, we have failed to develop processes by which the department as a whole—vs. individual faculty members—assumes a reasonable measure of responsibility for the experiences that these scholars have while they are members of our community and for their success in moving on to the next stage of their careers.

UW CSE faculty involvement in the Computing Innovation Fellows Project—in the conception of the project, serving as a PI, serving on the steering committee, serving as a source of CIFellows, serving as the host of CIFellows, and reflecting upon the unique attributes of the program that made it so successful—has made it clear to us that we can, and must, do a far better job of taking collective responsibility for our postdoctoral scholars.

Three of the five Co-PIs on this proposal are themselves postdocs. The UW CSE postdoc community took responsibility for reflecting on their experiences—good and bad—and working with the faculty to formulate a set of sustainable practices that we believe will result in a significant improvement in the postdoc culture in UW CSE and elsewhere.

We are eager for, and fully committed to, wholehearted participation in a distributed experiment coordinated by CCC that we expect will have tremendous benefit for our field.

1 Introduction

The University of Washington’s Department of Computer Science & Engineering (UW CSE), like many programs nationwide, has experienced a dramatic growth in the number of postdocs. Ten years ago, we had but 2. Today, we have 27. To place this growth in context, over the same period our faculty has increased by 24%, student enrollment by 50%, technical staff by a factor of 2.5, and research expenditures by a factor of 3. Growth in postdocs blows away these otherwise significant increases.

Exponential growth—whether in technologies, companies, or academic programs—almost always catches us unaware, and is thus accompanied by processes that lag by several years, and therefore by several orders of magnitude. This is undeniably the case with (UW CSE) postdocs.

In the case of our Ph.D. students, we pride ourselves on taking collective responsibility for their experiences and success. We have a full-time staff advisor, an annual two-day orientation, an annual welcoming party, weekly receptions (“TGIFs”) with the faculty, monthly lunches with the Chair, an active graduate student organization with specific activities and responsibilities, an annual Review of Progress for all students involving the full faculty, an annual autumn faculty meeting at which all graduating students are discussed (so that all faculty can assist in placing these students), a clear roadmap for progress through the program, a well understood set of expectations for faculty regarding how students are to be mentored and treated, careful consideration of mentoring when faculty members undergo annual reviews, reappointment or promotion, and a habit of trumpeting the successes of our Ph.D. alums.

In the case of our postdocs, little of this exists.

Postdocs are not the same as Ph.D. students, of course—they and their mentors can be expected to have considerably greater independence. However, the gap in our processes is far too great.

This proposal aims both to bring UW CSE’s support for postdocs in line with recently-developed community best practices and to experiment with several new proposed best practices. We will follow up our implementation of these practices by evaluating their effectiveness using surveys, interviews, and quantitative evaluation of postdocs and their collaborators, and report to the community on the results.

This proposal is a collaborative effort between UW CSE’s faculty and postdocs, and was in large part crafted by postdocs themselves. Three of the five Co-PIs are postdocs. Our proposed activities are informed by a series of discussions between postdocs and faculty and a recent survey of the department’s postdocs.

One of the key findings of our survey is that postdocs are generally satisfied with the support they receive from their postdoc mentors, but a significant fraction are not satisfied with the level of support from the department. There is currently no department-level support infrastructure for postdocs (in contrast to junior faculty and graduate students), leaving mentoring largely at the discretion of individual faculty. This proposal includes several steps towards building such an infrastructure. These include appointing a faculty member as a program coordinator to act as a third-party ombudsperson and ensure that each postdoc is receiving adequate support from their mentors, and conducting periodic reviews of postdoc progress.

A second part of our proposal is based on the observation that, in addition to mentoring, postdocs need an environment in which they can develop an independent research program. Towards this end, we plan to experiment with a program under which postdocs can apply for small grants from a pool of department funding, giving them experience with managing and funding their own research. We will also attempt to improve postdocs’ access to undergraduate and graduate research collaborators.

In the final part of our proposal, we present a plan for implementing these practices sustainably—with adjustments as necessary—should the evaluation throughout the grant period show evidence that their continued implementation is merited.

2 Current Practices and Their Effect

To gain an understanding of what needs to change for postdocs within the UW CSE community, we examine the current practices proposed and/or implemented for postdocs in UW CSE and how their effect is perceived by the postdoc community.

We do this in three steps:

1. We summarize previously published best practices for postdocs in Computer Science and other fields. We identify what they cover and to what extent they make sense to the growing postdoc community in UW CSE.

2. We conduct a case study of the postdoctoral position in UW CSE and compare it to the existing ideal.

3. We report on a survey conducted among current UW CSE postdocs to gauge how the currently implemented practices are perceived by the individuals most directly affected.

2.1 Published Best Practices

Two existing documents outline suggestions for best practices for postdocs in the sciences [3] and, specifically, in computer science [1]. Concern about postdocs in computer science is very recent and is largely a reaction to the large recent growth in postdoc positions in this field [2].

The existing publications lay out a high-level description of the postdoctoral experience, by treating topics such as position and benefits, goals and expectations, performance evaluation, and career development. These are not necessarily specific to the field of computer science. In fact, even the more specific proposal [1] is generic enough to be applicable to any postdoc. We survey each of these to provide an overview of the proposed practices.

2.1.1 Position and Benefits

Current best practices call for the postdoctoral position to be created such that institutional recognition, status, and compensation commensurate with the expected contributions of the postdoc are awarded. Access to health insurance
and retirement benefits should be provided, regardless of the mentor’s funding sources.

We believe this call includes that the salary and other benefits should be reasonable to support a family and meet expectations for a minimum salary after five or six years of graduate studies, so that one is not forced to choose other career options for these reasons.

Most proposals call for a limit (of approximately five years) to be set for the total duration of postdoctoral appointments including time served in the same position at any previous institution). The intended benefit is to advance the postdoc’s career early and effectively; a prolonged appointment can hurt the postdoc’s prospects, rather than benefit them.

In addition, the duration of an appointment shall be adequate for the successful achievement of its intended goals. A minimum duration of two to three years is typically adequate. Contracts shall not be unnecessarily limited to a term less than the adequate amount of time. At the same time, the postdoc should be given the opportunity to terminate his or her appointment early, should career advancement necessitate a move.

The department shall invite the participation of postdocs when amending further standards or conditions to the appointment.
A postdoc appointment should be a desirable training opportunity, instead of merely a holding place for the next career step, or, worse, cheap labor. It is important to make the postdoc feel that his or her position is valued, and to allow the postdoc’s participation in the terms of his or her own appointment.

A significant fraction of the postdocs at top US universities are international scholars - some are graduates from US universities and some are graduates from universities in other countries. Under a temporary contract, many of the universities have certain restrictions for visa supports which may limit the personal and professional travels of international postdocs. It is important that the university be proactive in supporting the postdoc in professional travel, for example to conferences, which often take place internationally. For example, enable the postdoc to travel internationally by making sure the “travel validation” on J-1 visas, which many international postdocs hold, stays current.

2.1.2 Goals and Expectations of the Postdoc Appointment

As postdoc backgrounds, goals, and experiences are very diverse, existing proposals recommend that both postdocs and their prospective mentors write down a list of goals for the postdoc’s appointment, at or before the beginning of the postdoc. This list should then be discussed between the postdoc and his or her mentor.

This practice strives to create a common ground for the postdoc’s and the mentor’s goals and expectations. It can even serve as a ground upon which the postdoc can decide whether s/he wants to take up the appointment at all. The discussion should be carried out on equal ground and give the postdoc room to modify the mentor’s goals, within reasonable guidance of the mentor.

2.1.3 Periodic Evaluation

Given that the postdoc’s goals have been discussed and fixed at the beginning of the appointment, postdoc and mentor should meet regularly to discuss and assess whether the goals are being worked towards. This discussion might include changing the goals if it is determined that they are obsolete with respect to the postdoc’s current situation.

The department should conduct a recurring survey to gauge the postdoc experience. The survey can be released annually. This practice ensures sustainability of the proposed practices and informs the department of the potential for change of the practices due to a changing environment.

2.1.4 Career Development and Job Placement

Departments should mentor postdocs to help them develop their careers. Such mentoring may include general aid with job applications and interviewing, one-on-one counseling sessions with a career advisor to help the postdoc decide his or her next career move, and organized informal meetings with the mentor to discuss career prospects.

The department should make sure that the postdoc is well prepared for his or her next career step before the end of the postdoc’s appointment, similarly to how the department ensures that its graduate students are prepared.

2.2 The Postdoc Position in UW CSE: A Case Study

The practices published thus far certainly apply to postdocs in UW CSE and we strongly agree with them. But are they put into practice? As a case study, we compare a postdoc experience in UW CSE to the practices presented in the previous section.

Currently, the UW CSE community includes 27 postdocs - a number that has increased dramatically in recent years. Postdocs typically stay between 2 and 3 years, and most contracts are for a duration of 2 years and then extended. However, there is no upper limit to the total duration of a postdoc appointment.

UW CSE provides each postdoc with an appointment as Research Associate. Research Associates receive compensation within the average pay bracket for similar positions in computer science at other US universities (according to Glassdoor.com), and receive health and retirement benefits. The salary is roughly half of a comparable salary in industry for a person with equivalent training and experience.

With the exception of retirement benefits, the status of Research Associate is equivalent to that of a Visiting Scientist, which is a “placeholder” position for visitors to the department of all ranks (professor to visiting graduate intern). As such, postdocs are not involved in departmental decisions that might potentially affect their appointment. In fact, postdocs have no representation in departmental decisions.

Duties of the postdoc position and assessment of their fulfillment are at the discretion of the postdoc and his or her mentor. There is no formal schedule for evaluation of the postdoc’s progress, nor is there a formal requirement for a written description of the postdoc’s duties.

Unlike with faculty or graduate students, there is currently no independent third party responsible for ensuring the adequate mentoring of post-docs, nor any kind of institutional support infrastructure. All mentoring happens between the postdoc and his or her mentor, who is in almost all cases also employing the postdoc. Furthermore, no evaluation of the postdoc experience is conducted by the department. There is little information about postdocs inUWCSE available to the outside world. A webpage exists, listing postdocs in the department, which is updated only sporadically.

Mentoring on job prospects is either at the mentor’s discretion or done via workshops and classes that are consolidated among postdocs and graduate students. No mentoring tailored to postdocs is provided, except at the mentor’s discretion.

UW CSE does not have structured support specifically geared towards postdocs to gain and administer research relevant resources, such as recruiting students to work on their projects. There is a quarterly “research-night,” which gives graduate students, postdocs, and faculty the opportunity to pitch research ideas to undergraduates during a poster reception. Postdocs have to compete with faculty and graduate students in this scenario. This can be seen as an opportunity to learn how to recruit students as future tenure-track faculty. On the other hand, postdocs are members of the UW CSE community for only a short amount of time and as such need access to student talent quickly to facilitate their projects. At the same time, they might not be as effective pitching their ideas, as UW’s culture and what students expect is still new to them.

The experience, background, and expectations of postdocs joining UW CSE are very diverse. Postdocs join from other US universities, from abroad, and (rarely) from UW CSE’s own graduate program. Each postdoc potentially has had a very different graduate school experience and, depending on the academic system in his or her home country, different expectations of a postdoctoral career. To this end, the university provides an International Scholars office to help international postdocs with their needs. So far, however, the department does not automatically offer to pay for the non-trivial costs involved in obtaining and maintaining a visa.

To summarize, while basic provisions, such as salary, benefits, and duration of appointment, are in accordance with current best practices, many other of the proposed practices are missing. Postdocs have no representation in the department, it is not ensured that expectations of the position are clear from the start, and no assessment of their fulfillment is ensured from any side. Finally, career support is delivered only in limited form via advice by the postdoc’s mentor.

2.3 Survey

We conducted a survey of UW CSE postdocs to determine overall satisfaction with the current practices. The survey as well as the summarized responses are reproduced in Appendix A.

18 out of 27 active postdocs in the department responded to the survey, a 2/3 response rate. The survey itself focuses on the lesser developed parts of the postdoc experience in UW CSE, which fall into three groups:

1. assessment and fulfillment of goals and expectations,

2. career development, and

3. periodic evaluation.

Questions about salary and benefits were not asked.

2.3.1 Goals and Expectations

We began by asking postdocs what they expect to learn throughout their postdoc appointment. The survey shows that most postdocs expect and actively seek out opportunities to gain more research-related experience during their time by publishing research results (83%), writing grants (61%), and mentoring students (72%). Teaching is not as high a priority (39%).

The first three opportunities are typically provided as part of the postdoc appointment in UW CSE. The last one is typically not expected in UW CSE. Given that UW CSE is trying to prepare postdocs to become research faculty, this distribution is healthy.

None of the postdocs aspire to carry out another postdoc. This is understandable, as a postdoc should in all cases be a temporary training position and not a permanent career state. It has to be ensured that this expectation stays this way.

When asked whether their own expectations of the postdoc appointment were clear at the outset, the majority (79%) of postdocs believe this to be the case. Slightly less (70%) agree that their mentor’s expectations were clear and even less (43%) agree that the department’s expectations were clear. One postdoc points out that “it was clear that I would be doing research and little teaching (agreed from both sides). It was less clear what else I would be doing or what the support infrastructure would be.”

While the situation generally seems to be good, perhaps a written, mutual statement of the expected goals of the postdoc - as implemented in the CIFellows Project - would improve the postdoc experience. When asked about this directly, the vast majority (85%) of postdocs agree.

Things clearly have to change at the departmental level to provide a more streamlined postdoc experience, clearly stating the department’s goals. Specifically, when asked about support from the department, 66% of postdocs agree that they receive enough support from the department, but a significant number generally (27%) or strongly (7%) disagree with the level of departmental support. Specific points of complaint are the mechanisms for information dissemination, which seem to preclude postdocs from valuable departmental information. Postdocs also point out that guidance from the department on how to allocate time and what the postdoctoral culture in UW CSE is like would be
helpful.

Our survey shows that postdocs in UW CSE either strongly (67%) or generally (27%) agree that their mentor is supporting their goals. One postdoc (6%) strongly disagrees, signaling that perhaps a departmental review of mentorial support is in order to ensure postdocs receive uniform support. Furthermore, when asked whether an independent third party would be valuable, such as an ombudsperson that postdocs can address about their progress and goals, the overwhelming majority (93%) agree.

2.3.2 Career Development

A rather large number of postdocs (40%) are unsatisfied with the provided mentoring on job prospects. Some explicitly point out that they feel feedback on job prospects is missing or insufficient. This clearly has to be improved.

To determine what kind of career counseling should be provided, we asked postdocs about the type of career they aspire to take up after their appointment has ended. 83% of UW CSE postdocs aspire to pursue either a tenure-track faculty or academic researcher position. 11% aspire to an industrial research position, with the remaining 6% (1 postdoc) wanting to take up an industrial engineering position instead. Thus, more counseling for academic faculty careers should be provided.

Gaining independence from their mentor is one of the important steps in a postdoc’s training and perhaps one of the biggest changes from being a graduate student. To what degree this occurs is mentor-dependent in UW CSE. Our survey shows that 73% of postdocs strongly agree that they have enough independence from their mentors. One postdoc strongly disagrees. Mentor independence should be encouraged as part of the postdoc’s career development to ensure it is uniformly provided.

2.3.3 Periodic Evaluation

We asked postdocs directly whether periodic departmental evaluation and some form of follow-up would be desirable and the majority (69%) agree. We conclude that periodic evaluation of the postdoc experience by the department is thus a good idea.

2.3.4 Summary

In summary, while the climate of postdocs in UW CSE seems to be good, there are certainly a number of things that can be improved, such as a better information policy between postdocs and the department, and the availability of an independent third party that postdocs can address.

Furthermore, postdocs point out repeatedly that access to student talent is scarce. 33% say they have insufficient access to resources and many comment that their access to (graduate) student talent is limited.

Some postdocs point out that their mentor is on sabbatical, which they did not know before starting their appointment. Advisors should be more proactive about their near future travel plans when offering to hire a postdoc, who depends upon their mentorship.

3 Proposed Additional Best Practices

The best practices proposed in the existing articles [2, 1, 3] are largely supported by the postdocs and their mentors at UW CSE. Based on our findings via the survey conducted, as well as meetings with postdocs and faculty, we find that this existing set is not complete and at times too broad to capture the needs of postdocs in UW CSE. In this section, we propose three additional best practices that serve to better support postdocs in UW CSE. We relate each proposal to our survey findings and describe their intended benefit to the postdoc.

3.1 Access to Funding

Funding for postdocs varies from position to position. Some postdoc mentors guide their postdocs through seeking and managing research funds; others simply fund postdocs through existing grants and shield them from funding concerns; and some allow postdocs the freedom to fund their own endeavors.

To do their research effectively and demonstrate they can do this independently, postdocs should have access to some of their own funds via their host departments, instead of being fully funded by their mentors.

We propose that it is helpful for postdocs to compete over a pool of independent research funding that is maintained by the department. Obtaining research funding would be subject to roughly the same procedures as is obtaining a government research grant. This practice helps postdocs attain more independence from their mentor and teaches them how to behave in the competitive environment of tenure-track faculty. At the same time, by having only postdocs compete over this money, we shield them from the stark competition with faculty over government research grants.

3.2 Access to Personnel

Perhaps even more important to research success is access to personnel. In discussions and in the responses to our survey, postdocs expressed a desire for greater access to students, both graduate and undergraduate. In particular, there should be no competition between postdocs and mentors over student talent. Instead, the mentor should help the postdoc in finding collaborators early, given the short duration of the postdoc appointment.

A way to foster collaboration with graduate students may be to assign the postdoc to help on an already on-going research project with some of the mentor’s graduate students. If the topic of the project is aligned closely enough with the postdoc’s research ideas, these students might be more likely to collaborate with the postdoc in the future.

Furthermore, faculty should make an effort to advertise the postdoc’s project to graduate students and give the postdoc the opportunity to advertise his or her ideas widely within the department. This has to be done early and effectively, such that students interested in the postdoc’s research can contribute to the project for a fair amount of time. A postdoc’s mentor should advertise a new postdoc joining the group and his or her research ideas before the postdoc has joined.

Undergraduate students can be another useful resource to help increase research productivity. UWCSE has a “research night” each academic term (quarter) at which established researchers in the department can recruit undergraduates to work on their projects. Including postdocs in this kind of event ensures that they have access to the same undergraduate
talent pool as faculty.

The benefits of postdoc-undergraduate collaboration run in both directions. Postdocs can be good advisors to undergrads: they have recent experience to offer to undergrads who may be interested in the academic track. Postdocs may also have more time to offer undergrads than a professor could offer.

3.3 Periodic Evaluation

We note that answers to our survey were not uniform. This points to the fact that each postdoc’s experience and needs within our department are currently unique and largely driven by their respective mentors. For example, it is clear that some mentors put more emphasis on developing career-building skills than others. Also, mentors have different approaches to their involvement in the mentoring process.

To provide a more uniform experience to postdocs across the department, the orchestration of their experience should not be isolated to their individual mentor. At the same time, we do realize that each postdoc’s needs vary widely, based on their experience, goals, and research subfield. To this end, we propose that each postdoc’s progress should be discussed and evaluated at the departmental level. This can occur, for example, in a faculty meeting, similarly to how the progress of graduate students is discussed.

The emphasis should be on discussion, rather than evaluation. The intent is to provide the best support to each postdoc individually to help each achieve his or her potential and goals. A document about the goals of the appointment, jointly written by the postdoc and the mentor, can be used as a basis for the discussion of each postdoc’s performance and experience. The intended outcome of this discussion is to inform the mentor whether the experience of their postdoctoral mentees is facilitated enough by the mentor’s application of the proposed best practices. The mentor can choose to adjust his or her mentoring style based on the outcome of the discussion.

4 Plan of Implementation

We in UW CSE commit to implementing and evaluating the best practices proposed in the previous sections. The implementation will directly affect the ever-growing number of postdocs in our department.

We are not going to change departmental practices that are already serving postdocs well. For example, the average duration of the postdoc appointment in UW CSE of two to three years is already matching best practices and is not in danger of becoming prolonged. However, we will remind faculty to be wary of employing postdocs for longer than this amount of time. Also, given our survey results, it is highly unlikely that postdocs will take up another postdoc after their current appointment is finished.

The proposed changes for implementation involve the definition and evaluation of goals of the postdoctoral appointment and postdoctoral involvement in the department’s business, access to research personnel, a research training program, networking, and career counseling. Lastly, we are going to discuss applicability to the greater field and how we plan to collaborate with other awardees.

4.1 Goals and Evaluation

The department will change the involvement of postdocs in the department’s business by appointing a postdoc liaison to the department’s Chair and Executive Committee with full access to information, full input on decisions, and responsibility to communicate openly with the UW CSE postdoc community.

In addition we will appoint one faculty member to act as ombudsperson for postdocs. Postdocs can talk to the ombudsperson about any matter related to their position that they feel they cannot discuss with their mentor. (We have an analogous ombudsperson for commercialization activities: a faculty member whose responsibility is advising faculty and students, and ensuring that student rights are respected.)

We will require faculty to discuss the goals of the postdoc appointment with each new postdoc candidate and write down these goals in a joint document together with the postdoc appointee as part of the appointing process. There will be a yearly review of the goals between the mentor, the postdoc, and the ombudsperson to see whether the goals are worked towards and to review whether the documented goals are still in accordance with the postdoc’s reality.

Also, we will discuss the progress of each postdoc appointee at a faculty meeting, analogous to the annual review of progress of each graduate student. According to our third proposed practice, this discussion serves to inform each mentor about whether their individual application of the best practices and mentoring style is effective to the success of each of their postdoc mentees.

“Graduating” postdocs - those going on the job market - will be discussed among the full faculty at the same time that graduating Ph.D. students are discussed: the full force of the faculty will be devoted to advocating for the appropriate placement of both Ph.D. students and postdocs.

4.2 Access to Research Funding

Using departmental gift funds and Industry Affiliates membership fees, we will establish a pool of research funding for which postdocs can compete. For example, the cost of an undergraduate research assistant (more than 50% of our undergraduates participate in supervised research) is exceedingly modest. This will provide postdocs with a degree of autonomy, and with valuable experience.

4.3 Access to Research Personnel

To broaden each postdoc’s access to research personnel, we are going to encourage faculty to advertise the postdoc’s research projects to graduate and undergraduate students. Also, we are going to include postdocs in prospective graduate student orientations. During these orientations, postdocs can give talks to pitch their projects to prospectives alongside faculty.

We are also going to encourage postdocs to participate in “undergraduate research nights” by advertising the event to them at a departmental level. During these research nights, postdocs can pitch their project in a poster session to undergraduate students.

4.4 Research Training Program

The environment in UW CSE is already such that postdocs are heavily involved in designing research projects, authoring quality technical papers, delivering effective research talks, and pitching research ideas. They learn these skills by participating in the day-to-day activities of the various research groups. This includes practicing their skills within the department and receiving feedback from faculty, other postdocs, and graduate students.

Whether postdocs are involved in writing grant proposals or managing their own independent research, however, is currently dependent upon their mentor. To make postdocs uniformly less dependent upon mentors and to encourage them to write grant proposals, we intend to make small grants available that postdocs can compete for without involving their mentors or other faculty as primary PI/co-PI.

We intend to make workshops available for postdocs that help them acquire the skills needed to craft effective grant proposals. This way the postdocs will be able to acquire skills for successful grant writing which will possibly strengthen their resume for academic job search, as well as have funding to support their independent research by acquiring resources and hiring or supporting undergraduate and graduate students.

4.5 Networking

The departmental culture in UW CSE is already highly collaborative and postdocs are encouraged to mingle with and learn from other postdocs and faculty. We believe that networking within the larger research community is best done at conferences within the field. This requires postdocs to travel. We commit to ensuring enough funds are available for each postdoc to travel to at least 2 conferences a year.

4.6 Career Counseling

The standard duration of a postdoctoral appointment is 2 to 3 years, and applications for a full-time faculty position typically consume on the order of 3 months, which further shortens the effective time to enhance the postdoc’s resume. Therefore, the postdoc would need to utilize the first 1-2 years of the appointment by enhancing his or her publication record, acquiring additional skills like teaching, mentoring and grant writing, and by building a stronger network in his or her research community. We intend to hold a quarterly orientation (to accommodate as many postdocs as possible since they may have different start dates) to advise postdocs about managing their time and provide them with an overview of what awaits them throughout their appointment.

We will encourage postdocs that are already on the job market to meet frequently to compare notes and experiences, and to give each other advice. These meetings should also include others at the institution on the job market in that year, such as graduate students. We believe that it is important to keep these meetings within the circle of those postdocs and graduate students that are actively seeking a job.

In addition, we will invite speakers from outside the department to talk about their career choice and job search procedure. We are going to invite both postdocs and graduate students to these talks.

Faculty should take pride in nurturing postdocs and demonstrate their postdocs’ success to the outside world. We will encourage faculty to post a successful followup job placement of their postdocs on their own webpage. In addition to the mentor’s posting, we will require the department to advertise postdoc job placements on their publication infrastructure, in the same vein as Ph.D. placements are advertised.

4.7 General Applicability to the Community, and Collaboration

The proposed practices can be implemented in any academic institution and should be helpful to all of the field.

In order to discuss our gained information and collaborate with other postdoc-employing departments, we plan to host panels and invite interested departments to discuss the state of postdocs and relevant measurement data. These panels can be held online and should occur at least once a year. UW CSE has experience with hosting such online panels.

To disseminate the results of our evaluation to the community, we plan to author joint publications with other awardees of the grant to publish in a high-impact, overarching publication venue, such as an article in the Chronicle of Higher Education, Communications of the ACM, or IEEE Computer. We also expect to talk about the state of our effort at the CRA Conference at Snowbird and raise the visibility of our efforts.

We also envision postdocs to give talks to the community about their experiences. These case studies can serve to encourage other departments to adopt the successful best practices out of our experiment.

Finally, we are happy to form consortia with other awardees and coordinate on the formulation, implementation, and evaluation of further best practices.

5 Evaluation

We intend to evaluate the proposed practices along three axes:

1. Qualitative, by conducting surveys of postdocs, mentors, and graduate students.
2. Quantitative, by evaluating postdoc performance metrics, such as number of publications, number of visited conferences, and awarded grants.
3. Investigative, by interviewing postdocs and mentors, as well as allowing outsiders to review the postdoc culture in the department.

We discuss each in this section.

We propose that all awardee PIs collaborate to evaluate results and use these as one source of baseline data for a joint document to be published (see Section 4.7). They should also use them to adjust the proposed best practices in the future.

5.1 Surveys

We intend to conduct semi-annual surveys involving the postdocs and their mentors. These surveys are primarily aimed at evaluating the experience of the postdoc appointment from both sides. Do postdocs feel good about the quality of their experience and hopefully better than before? Do mentors feel good about their experience with postdocs given the new set of practices? How did the availability of resources and personnel change with the implementation of the proposed practices? An exit survey will also be performed at the end of each postdoc appointment, asking postdocs and mentors about their total satisfaction with the postdoc appointment and the implemented practices.

Additionally, for each postdoc-graduate student collaboration, a semi-annual survey will be conducted with graduate students, to gauge how their experience of working with postdocs changes over the course of implementing these best practices and how their perception of postdocs changes in general. Do students see the postdocs they are working with as important contributors to their research group and the research enterprise in general? Do they believe that working with postdocs is a worthwhile undertaking for them? Do they themselves aspire to take up a postdoc appointment after they graduate?

Conducting these surveys semi-annually over the period of the grant of 3 years will yield a total number of 6 data points without becoming too much overhead for the postdocs and department to carry out. We do not believe it makes sense to collect this data more frequently versus the time it takes for the new practices to start being effective.

We intend to follow up with graduated postdocs half a year after the end of their appointments to investigate how helpful they feel their postdoc was for their current career and how they feel at that point in time about their postdoc appointment.

Finally, we intend to survey applicants for postdoc appointments to investigate whether they would choose a postdocin UW CSE over a postdoc at another department and whether this is due to some of the implemented best practices. Similarly, we will investigate whether they prefer a different job, such as a junior faculty position or a job in industry, over a postdoc appointment, and why. This survey will likely be carried out for each applicant, after the applicant has either accepted or declined an offered postdoc appointment.

5.2 Quantitative Evaluation

We intend the department to record the following quantitative data for each postdoc: the number of research publications, grant proposal applications, and awarded grants, conference and workshop visits and talks delivered, number of students mentored, and number of research projects supervised or co-supervised.

This information will be collected yearly by the department and serves to augment the data gathered via surveys, by providing hard data on how well each postdoc is performing within the department over time. This data will be used to evaluate the best practices and give hints on when each practice should best be invoked to support the postdoc during his or her career, by evaluating when postdocs were most productive along each of the skill sets we intend to train them in. For example, training of research skills, such as authoring papers and giving talks, might best be conducted early in the postdoc’s appointment when this still fresh in their heads due to a possible previous graduate school experience, while career advice might best be given towards the end of the postdoctoral appointment when this is most valuable due to an upcoming job search.

5.3 Interviews and Reviews

Finally, we plan to conduct interviews with postdocs and mentors to gain qualitative insight into their experiences and to receive direct feedback on the proposed practices. We intend to do this once a year with every postdoc and mentor. Such interviews may last anywhere between 30 minutes to an hour and should yield answers we can quote in research publications. We intend to investigate directly how satisfied postdocs and mentors are with each implemented practice. Ethnography - assessment - is a critical component, and one with which we have expertise from other projects.

We also intend to involve people outside the department to evaluate the postdoc experience. We believe this to be another opportunity for collaboration among awardees of the grant. Awardee institutions can evaluate each other to investigate how the implementation of the proposed practices differ within each department and how that impacts the postdoc experience.

6 Sustainability after Grant Period

We are committed to ensure sustainability of the - potentially adjusted - best practices after the duration of the grantwith reasonable financial overhead to the department.

We see this as entirely feasible.

The majority of what we have proposed involves putting practices and processes into place. The startup costs dramatically exceed the costs of continuing these practices and processes once they have been established and debugged. Activities become “routine” - part of the general “cost of doing business” rather than an added expense. Faculty duties, for example, become committee responsibilities.

To continue to allow postdocs access to research and career training opportunities, such as grant writing workshops, orientations, and a travel budget to facilitate networking, we propose to charge each postdoc mentor a yearly overhead “tax” of appropriate size (for example, $3000-$4000) for each employed postdoc. This overhead is small compared to overhead charged for supervised graduate students and should thus only minimally impact the day-to-day activities of postdoc mentors.

Funds to support independent postdoctoral research grants - if determined to be a successful practice - may be allocated via academic and professional societies and government institutions, such as the CRA, CCC, NSF, ACM, and IEEE. We intend to work with these societies and institutions to make such funds available. If this is not successful, departmental gift funds and Industry Affiliate funds will continue to be used, filling this gap. If the value is demonstrated, the funds will be available.

Our proposed implementation and evaluation includes measures to ensure that every postdoc receives a high-quality experience, such as the discussion of each postdoc’s performance in a faculty meeting and a thorough evaluation via surveys, interviews, and quantitative evaluation of each postdoc by the department and by outsiders to the department. We intend to continue the discussion of postdoctoral progress in the faculty meeting and to conduct periodic surveys and quantitative evaluation within the department, perhaps at a reduced annual frequency. We also encourage collaborating departments to continue the proposed cross-departmental evaluation, at a reduced frequency. For example, once every several years.

Despite the invested time, these measures will likely involve a travel budget to invite a small review board of collaborators to the department to conduct the cross-departmental evaluation, as well as a minimal staff budget to conduct the survey-based and quantitative evaluation on a yearly basis. Travel budgets for such evaluations are typically available in departments and should thus minimally impact any department’s business. Also, such cross-departmental evaluation might be conducted as part of the departmental assessment process that occurs typically every few years and would thus be consolidated with the budget allocated for this process. We project a staff budget for the creation and evaluation of an annual survey to not weigh heavily either.

We intend measures like the specification and evaluation of goals for the postdoctoral experience not to be a requirement after the grant period. This way the institution will be able to help postdocs with all levels of experience and maturity, maximizing benefits for postdocs as well as their mentors, and minimizing time and effort spent in official formalities. In any case, their implementation requires solely a time commitment of the involved faculty.

The duties of the postdoc ombudsperson is a natural faculty committee responsibility, analogous to chairing our graduate student review of progress committee, or our commercialization oversight committee; establishing the “routine” will be a significant undertaking, but sustaining it should not be.

7 Conclusion

Our field is experiencing an explosive growth in the number of postdocs. We find ourselves at a crossroad. We can choose to follow the path of fields such as the biomedical sciences, in which many individuals move from postdoc appointment to postdoc appointment in a holding pattern that seems designed to serve the interests of senior investigators far more than the career development needs of the postdoc. Or we can choose to blaze a new trail, in a limited-duration postdoctoral experience that is oriented towards serving the interests and needs of the postdoc.

Make no mistake - this is a choice. Right now, UW CSE, like most other Computer Science programs, is thoughtlessly choosing the former course.

The Computing Innovation Fellows Project has shown that the other path is feasible, and is beneficial to all parties. The current award process will enable a set of programs - hopefully including UW CSE - to expand on the CIFellows experiment, testing and institutionalizing best practices in postdoc mentoring that make Computer Science a model for other disciplines.

The time to act is now. We doubt that UW CSE is far behind most other programs in our postdoc practices, but the survey that we have presented identified a number of significant shortfalls, such as a lack of a clear definition of goals for the appointment, a lack of access to resources and personnel, a lack of external evaluation and intervention, and a lack of independence of postdocs from their mentors.

To improve the situation, we have crafted this proposal. Within it, we have identified the most important existing best practices. These include:

1. a clear definition of the position, including adequate salary and benefits, as well as a minimum and maximum time limit of the appointment.
2. Written mutual expectations and goals of the postdoctoral appointment.
3. A periodic assessment of whether the goals are being worked towards, both between postdoc and mentor, as well as by an independent third party.
4. A comprehensive training program for the postdoc’s career development.

We have also proposed a set of three new best practices that are particularly crafted to serve the needs of postdocs in our field:

1. Access to independent funding for each postdoc to ensure more research independence from his or her mentor.
2. Independent access to research personnel, such as graduate and undergraduate student talent, to aid the postdoc in carrying out independent research.
3. Periodic evaluation of each postdoc’s progress to ensure a more uniform experience among all postdocs.

We have proposed an implementation of these best practices within the UW CSE department and laid out a plan to effectively evaluate their merits. We intend to collaborate with other interested departments to jointly discuss andrefine our findings, and to disseminate them to the community via publications in overarching venues, such as theChronicle of Higher Education.

We also presented a plan for the sustained implementation of those practices that are shown to be beneficial, after the conclusion of the grant. The plan is designed to apply to any department, not just to UW’s.

We would like to point out, again, that our proposal has been, in large part, crafted by the postdocs in our department. Three of the five PIs of this proposal are also drawn from these same ranks.

There is a need for change. We intend to act now.

References

[1] Anita Jones. Computer science postdocs - best practices, 2013. http://cra.org/resources/bp-view/best_practices_memo_computer_science_po....

[2] Anita Jones. The explosive growth of postdocs in computer science. Communications of the ACM, 56(2):37–39, February 2013.

[3] National Academy of Sciences. Enhancing the Postdoctoral Experience for Scientists and Engineers: A Guide for Postdoctoral Scholars, Advisors, Institutions, Funding Organizations, and Disciplinary Societies. National Academy Press, 2000.

A Questions and summarized responses of a survey conducted in UW CSE

We conducted an anonymous survey with the following questions and received 18 responses from the postdocs in UW
CSE. The count of answers for different questions are given in parentheses *.

Q1. What is your primary goal as a postdoc?

Tenure-track faculty (14 out of 18)
Academic (non-tenure-track) researcher (1 out of 18)
Industrial research (2 out of 18)
Non-research industrial work (1 out of 18)
Another postdoc (0 out of 18)
Other (please specify) (0 out of 18)

Q2. What do you hope to gain from your postdoc?

Additional publications (15 out of 18)
Grant-writing experience (11 out of 18)
Teaching experience (7 out of 18)
Mentoring experience (13 out of 18)
Other (please specify) (4 out of 18) (knowledge about a new area, visibility in the research community, self-estimation about whether one is eligible to be a professor)

Q3. My access to resources (people, funding) has been sufficient.

Strongly agree (3 out of 18)
Agree (10 out of 18)
Disagree (2 out of 18)
Strongly disagree (0 out of 18)
N/A (0 out of 18)

Q4. I have enough independence from my postdoc mentor.

Strongly agree (11 out of 18)
Agree (3 out of 18)
Disagree (0 out of 18)
Strongly disagree (1 out of 18)
N/A (0 out of 18)

Q5. My postdoc mentor and UW CSE have supported my goals.

Postdoc mentor

Strongly agree (10 out of 18)
Agree (4 out of 18)
Disagree (0 out of 18)
Strongly disagree (1 out of 18)
N/A (0 out of 18)

UW CSE

Strongly agree (6 out of 18)
Agree (4 out of 18)
Disagree (4 out of 18)
Strongly disagree (1 out of 18)
N/A (0 out of 18)

Q6. Expectations for my postdoc were clear at the outset.

Postdoc mentor’s expectations were clear

Strongly agree (6 out of 18)
Agree (4 out of 18)
Disagree (4 out of 18)
Strongly disagree (0 out of 18)
N/A (0 out of 18)

My expectations were clear

Strongly agree (7 out of 18)
Agree (4 out of 18)
Disagree (3 out of 18)
Strongly disagree (0 out of 18)
N/A (0 out of 18)

UW CSE’s expectations were clear

Strongly agree (4 out of 18)
Agree (2 out of 18)
Disagree (3 out of 18)
Strongly disagree (2 out of 18)
N/A (3 out of 18)

Q7. I’m getting enough feedback on my job prospects.

Strongly agree (4 out of 18)
Agree (5 out of 18)
Disagree (5 out of 18)
Strongly disagree (1 out of 18)
N/A (0 out of 18)

Q8. Would it be helpful...
... if you and your postdoc mentor were required to mutually formalize expectations at the outset of yourpostdoc?

Yes (11 out of 18)
No (2 out of 18)

... to informally talk to someone other than your postdoc mentor about your progress and goals?

Yes (12 out of 18)
No (1 out of 18)

... if there were some departmental followup on your goals and progress, either periodically or at the end of your postdoc?

Yes (9 out of 18)
No (4 out of 18)

Q9. What have you learned since starting your postdoc *that you wish you’d known at the beginning*? Feel free to add any other brief free-form comments or suggestions here. (how to involve and hire undergraduate and graduate into research projects of the postdocs without involving mentors, how to better market oneself in the job market, how to utilize the two/three years time the best possible way)
* In some questions, multiple answers were allowed. Also any question could be skipped without entering an answer

Addendum for Option A – Postdoc Best Practices National Coordination Role

We propose to have the University of Washington also act as a national coordinator for the Post-doc Best Practices Program. We see the role of the coordinators as two-fold: (1) bring together all the CCC grantees in the program so that they are aware of each other’s activities and reduce duplication of effort by leveraging each other’s work, and (2) disseminate the work of all the grantees through workshops, activities at conferences, web sites, and publications. To accomplish this we propose to add two additional months staff support for our program coordinator. We already have a person ready and willing to take on this role, our professional masters program advisor, who has many years’ experience managing a program for over 120 graduate students. We expect to be ableto begin coordination activities immediately. Among these we will include establishment of metrics and evaluation of results from all program grantees to feed back to the programs.

Coordination

We see our coordination role as one of bringing together grantees in the program to start the process of leveraging each other’s work in an effective manner. We expect to begin with a one-day meeting co-located with the CIFellows program annual meeting in May 2014. We will use that time for each grantee to present their objectives, determine overlaps, identify missing elements, and begin the process of adjusting our activities in concert. We expect this meeting to be annual with presentation of activities and evaluation results since the last meeting. In later meetings (after the first), we expect to also invite other, non-grantee institutions to participate and act as critics and adopters of our collective work. Meeting participants will discuss the presented activities and results and, if merited, how they can be replicated at other sites or how they can be modified to evolve the program over time. We see this as a light-handed facilitation rather than a strict command structure. Some overlap is likely warranted as well when different approaches are proposed and could provide an opportunity for evaluation of different strategies.

Dissemination

Dissemination will be a central part of our coordination role. Many institutions with only a few postdocs can benefit greatly from observing our activities and adopting the practices that are likely to work best in their particular environment. We will work with a select few representatives to understand the best way to package our findings, such as creating manuals or guides for postdoc mentoring under different circumstances. We will also ensure the visibility of the program by proposing workshops or panels co-located with flagship conferences. We will be working with other grantees to prepare a major presentation at the Snowbird conference and an extensive article for Computing Research News or other widely disseminated publications in the community.

Addendum for Option B – Funding to Assist in Developing Postdocs’ Independence

We propose an innovative program to provide post-doctoral researchers at the University of Washington with the opportunity to propose their own research projects and to employ undergraduate research assistants that they will supervise directly. The intent of this program is enable postdocs to pursue a line of investigation independent of their mentor/PI. In addition, we expect it to help them develop skills in preparing a research proposal and in mentoring students.

Available Pool of Undergraduate Research Assistants

The University of Washington Department of Computer Science & Engineering has a long tradition of undergraduates involved in research activities with a large fraction (over 25%) being involved in projects for either research credit or pay. In addition, we have an expanding 5th-year MS program. These students are all involved in research activities as part of their 4th or 5th years. Thus, we have a large supply of undergraduate research assistants available with a strong track record of these students co-authoring and presenting papers at workshops and conferences.

Benefits for the Postdoc

For the postdoc, we believe this type of research program will have several important effects. First, it will allow thepostdoc to develop research directions independent of their mentor’s. Although the proposal will be relatively short (we expect something on the order of 5 pages), it will have all the elements of larger NSF-style proposals including placement of the investigation with respect to related work in the discipline and an evaluation plan. Thus, we expect it to be a good start at proposal writing. Second, by employing an undergraduate research assistant, the postdoc will have the opportunity to mentor a student who is truly their responsibility. Again, this is important preparation for understanding how to recruit, motivate, and engage a student to do their best work and learn the basics of research, presentation, and writing for publication. Finally, this funding opportunity will allow postdocs to start a somewhat
riskier project if they choose, since the project may be conducted in parallel with better-established research for the mentor. The ability to turn a “wild” idea into published research is a key differentiator for faculty candidates.

Logistics

We plan to run the program with an annual deadline in early Spring Quarter so that students can be recruited for the summer and the following academic year. Our ACM chapter has a quarterly “Research Night” when faculty and graduate students can present work for which they are seeking undergraduate assistants. This program will ensure our postdocs also take part in this activity and provide them with one avenue for recruiting. Funding will be on the order of $10K per project. We are seeking funding from CCC to seed the program, with the department providing the bulk of the funding in an ever-increasing proportion leading to 100% department support after 3 years. We anticipate funding approximately 10 projects at the $10K/project rate each year. This will provide enough funding
for hourly pay for an undergraduate working up to 20 hours/week for the duration of the year as well as some funding to have the student attend a conference or workshop. The department will assign a committee of three faculty members to review the project proposals. Ideally, the top projects will have the highest degree of independence and new direction from the postdoc’s PI and current project.

Evaluation Metrics and Methods

We propose to expand our evaluation to more effectively demonstrate the effectiveness of our proposed “best practices” as well as their adoption and effect at other programs around the country. We will seek to develop a database of CSE post-docs through voluntary participation. We will begin by contacting department chairs and ask them to propagate our message to their post-docs and have them register in our system. Registered post-docs will be asked to complete initial and annual surveys about their career goals, mentoring, and research. Metrics will include quantitative measures such as the post-docs integration into the research community (conference attendance, presentations both inside and outside their departments, collaborations with students and faculty, publications, etc.) as well as how well their preparation for their eventual careers is progressing given their initially stated goals. In addition, we plan to conduct a series of interviews to provide qualitative data on their post-doc experiences and, in particular, their sense of community in the research enterprise at local and global levels.

To obtain meaningful results, we plan to assign registered post-docs to treatment and control groups based on the extent to which participating institutions have implemented the suggested post-doc interventions. This may be done at the grain-size of universities to avoid having members of both groups represented in a single institution. We will then compare the statistics from the treatment and control groups at strategic points over the course of the grant. Our hypothesis is that post-docs in the treatment group will show increased maturity and sense of community as well as having a larger number of specific citable accomplishments. A further measure will be the degree to which they feel they accomplished their initial goals and/or how their goals evolved.

To ensure proper survey design and statistical analysis we are adding to our proposal a co-PI, Elizabeth Litzler of UW’s Center for Workforce Development.

The Center for Workforce Development (CWD) at the University of Washington has a long history of formative and summative program evaluation and mentoring training, both of which combine to complement the work proposed here. CWD is an external evaluator for many NSF-funded programs, including ADVANCE, LSAMP, and the STEM Talent Expansion Program at UW. Dr. Elizabeth Litzler is the Director for Research at CWD and a sociologist skilled in both quantitative and qualitative analysis who currently conducts the National Center for Women & Information Technology (NCWIT) external evaluation. She will provide evaluation expertise to the project team. Particularly, Dr. Litzler will help the team create a logic model to express the expected outcomes of the project and how the project plans to get there; help in the design of survey and interview instruments to ensure that the appropriate things are being measured; and finally, assist in the analysis of data to assess whether the expected outcomes have been met. The addition of the control group will enable a stronger understanding of impact; analyses will test whether the control and treatment groups differ on key variables. Appropriate bivariate and multivariate analyses will be used for survey analyses. Annual evaluation reports will be written and provided to the project team to summarize the results from the analyses.