Novel capstone program evaluative tool for optimizing resources, minimizing faculty workload, and increasing student professional readiness

Introduction: : Capstone programs are a vital part of undergraduate engineering education and allow students to develop in industry-critical skills in teamwork, project management, research and development, and communication, but these programs often demand an extraordinary level of faculty effort and a high frequency of interaction between faculty mentors and student teams. Faculty workload challenges are compounded when programs involve multidisciplinary projects or operate at smaller institutions. Since there is not one ideal teaching modality that results in sustained student satisfaction, holistic career preparation, and faculty workload reduction, novel program evaluation methodologies are needed to optimize the impact of capstone programs through strategic decision-making and resource allocation. The BioDesign Experience (BDE) Analysis Tool is a two-part, anonymous survey tool, delivered at the beginning and end of each capstone program. The BDE Analysis Tool assesses a cohort’s perceived level of preparation across six critical areas of professional and industry readiness which allows faculty mentors to (1) customize capstone program elements to meet the needs of the cohort and (2) evaluate the program effectiveness. Initial data from the 2024-2025 academic year indicate increases in readiness and confidence across all six areas queried, with student-perceived readiness in scientific literacy, and intellectual property knowledge, and communication skills showing the largest changes. This survey tool can serve as a model for the analysis of complex or multidisciplinary teams that rely heavily on teamwork and collaboration skills to reach goals, operate with limited budgets, and involve a high student to faculty ratio.

Materials and

Methods: : The BioDesign Experience (BDE) is an interdisciplinary device development program, initially piloted at Johnson & Wales University during the 2023-2024 academic year, that offers challenging research experiences, cross-disciplinary skill building, and a professional development program to holistically prepare undergraduate students for STEM graduate programs or careers in industry. This team design course brings biomedical, robotics, and electronics engineering students together with students from the biology, computer science, and product development programs. The BDE Analysis Tool was employed during the 2024-2025 academic year to ensure continuous improvement, optimize resource allocation, and better meet the needs of the cohort. The BDE Analysis Tool is a 52-item anonymous survey, delivered at the beginning and end of a student’s time in the program, covering six critical areas; (1) interpersonal skills and professional networking, (2) written communication, (3) oral presentation and communication, (4) interdisciplinary collaboration and teamwork, (5) readiness to pursue graduate education or enter the workforce, and (6) the medical device development process including project management, regulatory affairs, and IP knowledge. Students self-assess and answer with a score from one to five, with one indicating strong disagreement and five indicating strong agreement with the statement. The broader BDE program includes numerous activities to support learning across the areas assessed, such as intellectual property literacy modules, field trips to biotech startup pitch competitions in the region, and scientific communication coaching sessions. Initial data were analyzed to determine the areas of greatest need for the cohort, namely oral communication, medical device industry knowledge, and networking skills.

Results, Conclusions, and Discussions:: The BDE Analysis Tool is a student self-assess survey, with 52-items formulated as statements. Students answer with a score from one to five; one indicating strong disagreement, five indicating strong agreement. Statements are organized into five areas critical to professional preparation with a sixth, discipline-specific area. Example statements include “I understand what a research poster is and can assist in preparing one for a conference presentation,” and “I can keep my portions of project and materials organized enough for others to understand or use.” Discipline-specific questions relate to medical device design, regulatory affairs, and intellectual property literacy and include statements such as “I understand the difference between validation and verification,” and “I understand the four main types of intellectual property and how they might apply to a particular project.” Initial data from the 2024-2025 cohort revealed students felt weakest in their medical device industry knowledge, oral communication, and networking skills. Faculty effort and resources were optimized to support this unique cohort of students, and final student responses revealed increases in these areas. The faculty mentor increased opportunities for students to present projects including a student-led, university-wide presentation workshop, prior to the university annual symposium. Additionally, the faculty mentor increased in-person training on industry topics, added field trips to biotech startup pitch events, taught patent and trademark searching tactics, and hosted two graduate school application preparation workshops. Since students indicated confidence with interpersonal and teamwork skills, the mentor developed two novel forms for progress reporting, to be uploaded at regular intervals by all team members, allowing the mentor to monitor and compare development of these skills, without devoting in-class time. BDE student teams were extremely effective, delivering four poster presentations at the 2024 BMES Annual meeting and nine prototypes at the JWU Student Research, Design, and Innovation Symposium in April of 2025. Incorporating self-assessment tools into capstone programs allows faculty mentors to optimize resources, adapt training materials, increase team agility, and develop new opportunities to support professional preparation. More broadly, varying the discipline-specific questions allows for scalability and use of this tool in analyzing graduate engineering and medical programs.

Acknowledgements and/or References (Optional):: Research reported in this presentation was supported in part by the Rhode Island Institutional Development Award (IDeA) Network of Biomedical Research Excellence from the National Institute of General Medical Sciences of the National Institutes of Health under grant number P20GM103430.