Study design and participant
This study employed a one-group, quasi-experimental mixed-methods design with pre- and post-assessments. The study focused on an online Digital Health and Informatics Course for undergraduate nursing students in China. All students who enrolled in the course at our university, a medical university located in Guangzhou, southern China, were invited to participate in the study. To be eligible, students had to agree to participate and successfully complete the six-week course. Students who expressed disinterest or were already engaged in other digital health learning or programs were excluded from the study.
Course details
Development of course
A multidisciplinary team of experts in digital health, informatics, and the medical field was established at a medical university in Guangzhou, southern China. The team consisted of three nurses, two doctors, two health informatics specialists, and a teacher with extensive experience in designing and conducting medical courses for nursing students. The team held monthly meetings to discuss the development, implementation, and evaluation planning of the course.
The course was developed in three stages. Firstly, a comprehensive list of digital health and informatics areas was generated based on previous education in this field [16, 17]. This list was used to define the most relevant topics for nursing students. Secondly, the team of eight experts reviewed the objectives, learning activities, and assessment tools for the course. Based on this review, a proposed course with five core topics was devised, which all eight experts agreed upon. Thirdly, the course procedure was enhanced by incorporating multimedia learning materials such as illustrations, photos, animations, and videos. This was in accordance with the Multimedia Learning Theory [19]. Through group meetings and discussions, the multidisciplinary team reached a consensus on the final course module and procedure.
Digital health and nursing innovation topics
The course was named as the Digital Health and Informatics course and was held at a medical university in Guangzhou, China, for eleven days in six weeks. Details of the overview and content of the course are shown in additional file 1. The content of this course focused on five key topics: digital health informatics, nursing informatics, emerging technologies for eHealth solutions, patient data security and privacy, and eHealth. The timetable and focus of each topic are shown in Table 1.
In the final week of the course, students were assigned group work-based learning activities to present a critical reflection on the following three questions:
-
(1)
Given the recent emergence and certification of health informatics professionals, will there continue to be a role for nurses in informatics within the next decade?
-
(2)
Should there be a specific professional designation for nurses with informatics expertise? What will the role of the Informatics Nurse look like in the future?
-
(3)
In the face of the evolving sophistication of technology, will there still be a need for nurses with informatics expertise?
During the group work, students were encouraged to read materials such as scientific papers and textbooks related to digital health and nursing informatics. Following each group’s presentation, the teachers provided feedback on students’ performance and on the material they were presenting, thus enabling them to review their strengths, areas that needed improvement, their development and learning, and to reconsider their learning processes.
Teaching members
For the Digital Health and Informatics course, teachers were eligible if they had extensive experience in digital health and informatics, software engineering, information management, and knowledge management. They were also eligible if they had previous teaching experience in software engineering and enterprise systems development, or if they had conducted wide-ranging research in the areas of information sharing, healthcare informatics, artificial intelligence, machine learning, and digital health.
In this course, all lectures were delivered by two teachers, all of whom hold a Doctoral degree and have a proven academic or professional background in the fields of software engineering, artificial intelligence, and information systems, or digital health research, nursing, public health, and implementation science. Additionally, both teachers have obtained Teacher Qualification Certificates, which demonstrate that they possess the basic teaching skills necessary to perform educational and teaching activities in higher education. Furthermore, both teachers have in-depth knowledge of the course content, aligning with the course’s subject expertise. Moreover, they have more than three years of teaching experience and were extensively involved in digital health course planning and education administration. As a result, they are capable of delivering engaging online lessons, promoting interactive student online participation, and maintaining open discussions between students and teachers.
Course procedure
The course commenced on July 23, 2022. The students who enrolled in the course were sent a link via email. In order to take part in the study, they were required to provide informed consent and complete the electronic baseline questionnaires. Following each lesson, students were granted access to the course through various modalities, including offline and online access to downloadable lesson videos for offline viewing on their electronic devices. Participants had the flexibility to watch the modules in their preferred order. The course structure for each topic is outlined in Table 1.
Evaluation
The assessment of the impacts of the course was conducted using a mixed methods approach to evaluate (1) knowledge and comprehension of the key digital health and informatics topics, (2) the self-assessment of nursing informatics competencies, and (3) the students’ satisfaction with the Digital Health and Informatics course. Table 2 summarizes the below-described outcome and outcome measurements.
Research materials
The demographic questionnaire
The questionnaire was used to collect students’ demographic data, including age, gender, year of bachelor study, and experience with nursing informatics use.
Knowledge and comprehension of key digital health and informatics topics
To evaluate the knowledge and comprehension of key digital health and informatics topics, an online quiz with a total score of 100 points and focus group discussions pre-and post the course were designed.
The quiz consisted of a total of 25 choice questions that were developed by the multidisciplinary team of experts. These questions were considered important learning topics and were based on relevant lecture materials. Before finalizing the quiz, a pilot version was tested by ten nurses who had graduated from the same medical university within one year and had experience with digital health learning. The pilot test aimed to improve the content, length, and understandability of the quiz. The final version of the quiz focused on eliciting students’ knowledge and comprehension of digital health, health informatics, and nursing informatics including definitions, nursing’s early role, and nursing informatics competencies (see additional file 2).
In addition to the quiz, all students were invited to participate in focus group discussions pre- and post-course. The focus group topic lists were developed based on examples from similar studies and research team discussions. The focus group discussion questions aimed to assess participants’ knowledge and comprehension on informatics and digital health such as definitions and emerging technologies. The discussion also explored the importance of informatics and digital health as well as the role of nurses in nursing informatics and digital health implementation (See additional file 3).
Self-assessment of nursing informatics competencies
All students were invited to participate in an online nursing informatics competency survey pre- and post-course. According to previous literature [20], nursing informatics competencies include not only computer-related skills, but also the knowledge and attitudes needed by nurses to complete specific informatics tasks. The online survey consisted of two parts (See additional file 4).
• Part one provided instructions for completing the survey.
• Part two included a validated Chinese version of the Self-assessment of Nursing Informatics Competencies Scale (SANICS) [21] developed by Yoon [22]. The scale consisted of a total of 28 items, covering three domains: computer technology, information technology, and information knowledge. The Cronbach’s alpha of the Chinese version of SANICS was 0.931 [21]. Five-point Likert-type criteria was applied (1 = not competent; 2 = somewhat competent; 3 = competent; 4 = proficient; 5 = expert), with a higher total score indicating a higher level of nursing informatics competency. The Chinese version of SANICS items were categorized into 5 sub-scales: role of clinical informatics (Factor 1; items 1–5), basic computer knowledge and skills (Factor 2; 6–16), applied computer skills (Factor 3; 17–20), wireless device skills (Factor 4; 21–24), and nursing informatics attitudes (Factor 5; 25–28). The five domains and examples of items are presented in Table 3.
Satisfaction with the Digital Health and Informatics course
Following the implementation of the course, all students were invited to join an online survey using a performance-focused course evaluation form (See additional file 5). The survey aimed to gather feedback on students’ learning experience and obtain specific comments regarding the course. Also, students were invited to take part in focus group discussions on the course evaluation. The focus group discussion questions were as follows:
-
“What do you like about the course?”
-
“What do you dislike about the course?”
-
“Do you have any suggestions on the future improvement of the course?”
Data collection
Quiz and survey
Prior to the study, participants were provided with information regarding the purpose of the study. They were asked to complete web-based questionnaires in the form of an online quiz, SANICS surveys, and course evaluation forms. The surveys were conducted between June and July 2022. A link containing a password to access the private survey questionnaires was sent to each student’s individual email inbox. Participants were informed that their participation in the study was voluntary and that choosing not to participate would not affect their learning or assessments. They were assured that they could withdraw from the study at any time without any negative consequences or impact on their academic grades. Furthermore, their privacy and confidentiality would be protected, and all participants provided written consent to participate. Participation in the online poll was also voluntary and anonymous.
Focus group discussions
A total of five pre- and post-course focus group discussions were conducted with all students to explore their knowledge and comprehension of key topics in digital health and informatics, as well as their satisfaction with the course. The face-to-face focus group discussions were conducted by one researcher (HS, PhD, female). The interviewer had received training and possessed extensive experience in qualitative research. Each focus group discussion lasted approximately 50–60 min and was recorded with the participants’ consent. The recordings were later transcribed and used as textual data.
Data analysis
For the quantitative data, survey data were exported from SPSS version 23 (IBM, Armonk, NY, USA) for analysis. After data cleaning, frequency descriptive statistics were utilized for categorical variables. Descriptive statistics such as the mean, standard deviation, median, and range of linear variables were calculated, along with frequencies and percentages of categorical variables. We compared the difference of the SANICS scores pre- and post-course using paired t-test analysis. P-values < 0.05 was considered statistically significant.
For the focus group discussion data, transcripts were imported into Atlas.ti for Windows version 7.5.18 (Scientific Software development, Berlin). Qualitative content analysis was performed inductively using the following steps: (1) open coding, (2) categorization, and (3) theming. Rigor was enhanced by repeatedly reading the transcripts, keeping a record of the analytic decision trail, and through crystallization with multiple researchers engaging in discussions of evolving categories and emergent themes. In terms of students’ knowledge and comprehension of key digital health and informatics topics, related quotations were compared to identify the changes in the same themes extracted pre- and post-course. For instance, the theme of emerging digital health technologies was extracted from pre- and post-course focus group discussions. We will compare the differences in relevant quotations, such as whether students mentioned more types of technology after the course.
Additionally, based on the focus group discussion data and responses to two open-ended questions in the course evaluation form provided by students, we analyzed their evaluations and suggestions regarding the course. Data saturation was achieved as being the point at which no new or relevant information could be identified through the iterative, preliminary analysis of the data [23]. After the first two focus group discussions, a preliminary analysis using the proposed codes was performed, and a data saturation grid [23] was developed to determine if saturation was reached. The data saturation grid consists of a report of the occurrence of themes and codes (displayed in rows) during each focus group (displayed in columns) in a tabular format. In the grid, saturation is considered reached when the grid column for the current focus group indicates no new information emerged for that particular theme or code. We found that in the fifth focus group discussion, data saturation on all themes and codes was achieved (data saturation table included as additional file 6).
Ethics and consent
This study was assessed and approved by The University Ethics Committee of Guangzhou Medical University (Reference Code: L202303012). All methods were carried out per relevant guidelines and regulations. Informed consent was obtained from all participants.
link