Preparing Residents for Future Practice
This educational project and study occurred during the July 2011–June 2012 academic year. We administered a preintervention questionnaire (see online supplementary appendix) to assess residents' needs and baseline attitudes toward patient portals. The questionnaire asked residents to self-report their usage of email with patients and to perform a brief self-assessment of communication skills. We assessed resident attitudes toward the portal by asking for their response to a series of statements concerning physician workflow, patient empowerment and satisfaction, and professional liability resulting from messaging. Residents took the initial questionnaire after the 1 h lecture, but before either case-based small group sessions or use of PatientSite. Respondents used 4-point Likert scales for agreement with the statements (strongly agree, agree, disagree, strongly disagree), which were dichotomised to agree or disagree for analysis.
We enrolled hospital-based residents in the portal (n=130) beginning in October 2011, following the large group lecture, which served as their orientation to the portal. Faculty preceptors were available to answer questions from the residents. Preceptors also automatically received a copy of messages that patients sent to residents. Residents were ineligible if their preceptor did not participate in PatientSite (n=5), or they were an intern doing a preliminary medical year (n=11) (figure 1).
(Enlarge Image)
Figure 1.
Flowchart of the PatientSite Educational Project Evaluation.
Patients enrol into the portal through a participating provider, and typically learn about the portal from their doctor, a waiting-room brochure, or word of mouth. We did not notify patients of the opportunity to register for the patient portal in any systematic way, but left this to the discretion of the residents, as is done at the faculty practice. In most cases, enrolment occurred after discussion about the portal at a physician visit.
A postintervention questionnaire was administered at the end of the academic year, roughly 9 months after enrolment began. This questionnaire repeated questions from the original survey in addition to asking for self-reported attendance at the lecture and small group session. Of those respondents who used the portal, we also asked about any missed or emergent messages sent by patients, and missing data (n=2) were considered negative. The surveys were administered electronically, and were linked with the preintervention survey.
We performed a matched pairs analysis for the preintervention and postintervention data using McNemar's exact test. In a posthoc exploratory subgroup, we stratified responses to the questionnaire by whether or not residents reported using PatientSite to communicate with patients during the intervention. We also recorded PatientSite usage statistics for resident empanelled patients, and also looked at response turnaround time for patient-initiated messages to residents. Because surveys were anonymous, we did not link survey responses to usage statistics. To determine whether prior experience with e-mail or with the curriculum predicted usage of the portal, we performed multivariable logistic regression, including prior email experience, training level and curriculum participation in the model. Residents who were not based at the participating clinic site were excluded from the regression analysis. Statistical analysis was performed using SAS V.9.2 (SAS Institute, Cary, North Carolina, USA).
This project received an educational exemption by the Institutional Review Board of Beth Israel Deaconess Medical Center.
Methods
Needs Assessment and Preintervention Survey
This educational project and study occurred during the July 2011–June 2012 academic year. We administered a preintervention questionnaire (see online supplementary appendix) to assess residents' needs and baseline attitudes toward patient portals. The questionnaire asked residents to self-report their usage of email with patients and to perform a brief self-assessment of communication skills. We assessed resident attitudes toward the portal by asking for their response to a series of statements concerning physician workflow, patient empowerment and satisfaction, and professional liability resulting from messaging. Residents took the initial questionnaire after the 1 h lecture, but before either case-based small group sessions or use of PatientSite. Respondents used 4-point Likert scales for agreement with the statements (strongly agree, agree, disagree, strongly disagree), which were dichotomised to agree or disagree for analysis.
Resident Enrolment
We enrolled hospital-based residents in the portal (n=130) beginning in October 2011, following the large group lecture, which served as their orientation to the portal. Faculty preceptors were available to answer questions from the residents. Preceptors also automatically received a copy of messages that patients sent to residents. Residents were ineligible if their preceptor did not participate in PatientSite (n=5), or they were an intern doing a preliminary medical year (n=11) (figure 1).
(Enlarge Image)
Figure 1.
Flowchart of the PatientSite Educational Project Evaluation.
Patient Enrolment
Patients enrol into the portal through a participating provider, and typically learn about the portal from their doctor, a waiting-room brochure, or word of mouth. We did not notify patients of the opportunity to register for the patient portal in any systematic way, but left this to the discretion of the residents, as is done at the faculty practice. In most cases, enrolment occurred after discussion about the portal at a physician visit.
Postintervention Survey
A postintervention questionnaire was administered at the end of the academic year, roughly 9 months after enrolment began. This questionnaire repeated questions from the original survey in addition to asking for self-reported attendance at the lecture and small group session. Of those respondents who used the portal, we also asked about any missed or emergent messages sent by patients, and missing data (n=2) were considered negative. The surveys were administered electronically, and were linked with the preintervention survey.
Data Analysis
We performed a matched pairs analysis for the preintervention and postintervention data using McNemar's exact test. In a posthoc exploratory subgroup, we stratified responses to the questionnaire by whether or not residents reported using PatientSite to communicate with patients during the intervention. We also recorded PatientSite usage statistics for resident empanelled patients, and also looked at response turnaround time for patient-initiated messages to residents. Because surveys were anonymous, we did not link survey responses to usage statistics. To determine whether prior experience with e-mail or with the curriculum predicted usage of the portal, we performed multivariable logistic regression, including prior email experience, training level and curriculum participation in the model. Residents who were not based at the participating clinic site were excluded from the regression analysis. Statistical analysis was performed using SAS V.9.2 (SAS Institute, Cary, North Carolina, USA).
This project received an educational exemption by the Institutional Review Board of Beth Israel Deaconess Medical Center.