INTRODUCTION

The collaborative efforts of multidisciplinary clinicians and scientists have historically driven advances in the field of radiation oncology. However, modern-day delivery of radiotherapy has become tremendously complex, a characteristic which typically results in increased specialization. While specialization allows deeper understanding of individual elements of a complex problem, it inherently discourages multidisciplinary collaboration. Without purposeful cultivation of our ability to effectively communicate and collaborate with one another, we risk compromising future multidisciplinary benefit. In 2014, an NIH-sponsored educational program entitled “Integration of Biology and Physics into Radiation Oncology (IBPRO)” was first delivered at Wayne State University. IBPRO was a six-day course designed to foster multidisciplinary research and clinical collaboration among radiation oncologists, medical physicists and radiobiologists and was delivered every year from 2014–2018. The characteristics of this course along with preliminary results from participant feedback have been published elsewhere (1–2). Here we present the final measurable outcomes from the course and assess the value of this initiative to the profession of radiation oncology. In other words, we address the question: “Did it work?”

MATERIALS AND METHODS

IBPRO course participants from all five years were surveyed to quantify the effectiveness of the course. The survey was sent approximately one year after completion of the final course in 2018. Thus, respondents had between one and five years after the course to generate productivity associated with their participation. The survey was identical to the preliminary survey described elsewhere (2) and was sent to 240 total participants. Survey questions included the number of institutional protocols submitted, research grant applications submitted and funded, published manuscripts, research presentations, collaborative research projects, and collaborative working relationships “that they would attribute, in whole or in part, to their participation in IBPRO.” In addition, participants were asked whether participation in IBPRO resulted in changes in patterns of clinical practice, and whether they now approach clinical dilemmas more collaboratively (“never”, “occasionally” or “often”).

RESULTS

A total of 95 participants responded (40% response rate), including 21, 15, 20, 22 and 17 responses from participants in years 1–5 of the course, respectively. While the respondent's name was optional in an effort to maximize participation, 29 of the 95 respondents provided their name and contact information. Of these, only 12 participated in the initial survey. Nearly two-thirds of respondents reported ongoing collaborative working relationships generated by IBPRO, and nearly 50% of respondents identified specific ongoing collaborative research projects that they would attribute, in whole or in part, to participation in the course. Research metrics resulting from the course are shown in Table 1. The 95 respondents produced a total of 45 institutional protocols and submitted 52 research grant applications attributable to participation in IBPRO, of which 19 had been funded at the time of the survey. They also attributed 96 research manuscripts and 106 research presentations to their participation. While the total number of published research manuscripts attributable to participation in IBPRO is quite high, a number of these manuscripts included multiple IBPRO participants. Thus, this number does not represent the total number of unique manuscripts resulting from IBPRO. One example is the ongoing series in the Journal of Applied Clinical Medical Physics entitled “Three Discipline Collaborative Radiation Therapy (3DCRT) Debate Series.” This debate series was inspired by the IBPRO course and includes a radiation oncologist, medical physicist and radiation biologist on each debate team. The first of these was published in January 2019 (3). This series resulted directly from IBPRO and continues to foster collaborative debate on cutting-edge clinical and research topics associated with radiation oncology.

TABLE 1

Responses and Percentages for Post-Participation Survey Questions Regarding Collaborative Research Output Attributed, in Whole or in Part, to Participation in IBPRO

To evaluate the total quantitative research output generated, we have defined “quantitative research metrics” to include institutional protocols developed, research grant applications submitted, manuscripts published and scientific presentations delivered. A total of 63 out of 95 (66%) respondents reported generating at least one of these quantitative research metrics and these participants reported an average of nearly five such quantitative research metrics per respondent. Table 1 provides the mean number for each metric averaged over all respondents. However, when averaged over only those who participated in each metric, the mean values become 2.0 for institutional protocols, 2.0 for research grant applications submitted, 2.2 for research manuscripts, and 2.9 for research presentations. In addition, 48% of respondents stated that specific information presented at IBPRO changed the way that they practice, and 95% of respondents practicing in a clinical setting stated that, since participation in IBPRO, they have approached clinical dilemmas more collaboratively.

DISCUSSION

The data presented here represent a very substantial contribution to radiation oncology research by a relatively small number of researchers within a relatively short period. It is difficult to estimate the entire contribution since participants producing the most significant collaborative research may have been the most likely to respond to the survey. However, while mean values for each metric may be lower if data could be collected from all participants, the total absolute output would certainly be larger. Thus, the cumulative collaborative research productivity resulting from IBPRO is even larger than that presented in Table 1. Moreover, both collaborative efforts and resulting output take time to mature and it is reasonable to assume that additional output has taken place since data collection, particularly since many of the respondents had participated in the course only approximately one year prior to their response to the survey. Thus, these data represent a very conservative estimate of the tangible outcome associated with IBPRO, and we can expect overall totals for these metrics to continue to increase. A number of participants have been exceptionally successful in generating research productivity associated with IBPRO, the most productive of whom are identified by the upper end of the range of each metric in Table 1. From the optional respondent information, it appears that the majority of the respondents to this final survey are first-time responders and were therefore not included in the initial survey report (2). Thus, these new data represent output from many additional participants, rather than continued output sampled from the same productive researchers. This supports the claim that the impact of this course is expected to continue to increase over the next several years.

While we present a considerable quantity of collaborative output, this should be evaluated in context. The participants in IBPRO represent a group of professionals interested enough in advancing the science of radiation oncology and establishing research collaborations that they competed for the opportunity to devote six days to an immersive interdisciplinary collaborative learning environment. One may ask how much collaborative output should be anticipated in this time from a highly selected cohort of 240 collaboratively-minded professionals. Re-evaluating the results through this lens, one third of respondents did not generate a quantitative research metric attributable to IBPRO, and one third did not identify an ongoing collaborative working relationship generated by IBPRO. It would be interesting to understand the reasons why some participants did not establish collaborative relationships and/or generate quantitative research attributable to IBPRO and others were remarkably productive. Unfortunately, we are not able to identify these reasons nor are we able to evaluate how collaboratively productive these researchers would have been without participation in IBPRO. Furthermore, due to the nature of self-reporting and voluntary surveys, we are unable to fully evaluate the results of the course from the entire cohort of participants. Despite the lack of a suitable benchmark for comparison, results from the majority of IBPRO participants responding to the survey certainly appear quite impressive.

CONCLUSIONS

The IBPRO course generated a substantial body of collaborative research within a relatively short period through an intervention involving a relatively small number of radiation oncology professionals. Based on survey data, two-thirds of IBPRO participants generated at least one of the following: an institutional protocol, research grant application, research manuscript, or research presentation. These were attributable in whole or in part to participation in the course. In total, respondents identified 45 institutional protocols and 52 research grant applications submitted, and 94 manuscripts and 106 presentations attributable to participation in IBPRO. However, the total collaborative research production associated with this course is difficult to estimate due to incomplete survey data from participants. The data in Table 1 represent a remarkable contribution to the profession of radiation oncology resulting from a single six-day collaborative educational intervention. In addition, the vast majority of participants solved clinical problems in a more collaborative manner, and clinical practice patterns were affected for nearly one half of all participants after IBPRO. Many collaborative working relationships generated by this course continue to actively drive research productivity. Additionally, the creation of a new debate series in a professional journal is just one of the many enduring legacies of this course. In the introduction, we posed the question of whether this intervention worked. We believe that the data presented here demonstrate that indeed it did. IBPRO serves as a model for our ability to leverage collaborative learning in an educational intervention to foster multidisciplinary clinical and research collaboration. It has already had a profound impact on the profession of radiation oncology and will continue to bear fruit far into the future.