Breaking Barriers: Blended Learning Transformation in Software Testing at Technological University Dublin
- Software Testing
- Extrinsic Motivation
- Industry-Standard Tools
With rising costs of living and rent in the Greater Dublin area, many students remain at home while attending university and have a long commute to university. This has had an impact on attendance. The primary goal of this transformation was to allow students to engage even when they cannot be physically present. In addition, the module aimed to give students an authentic experience of working in teams, which in the software development industry, in particular, is becoming more and more blended in nature.
Context
The lecturer first designed this module in 2022 under the assumption of an in-person mode of delivery. The module is delivered to full-time third-year undergraduate students of Technological University Dublin (TU Dublin). The module is mandatory on the BSc in Computer Science (Infrastructure) and one of a range of options available to students in the BSc in Computer Science and BSc in Computer Science International. As third-year students on a computer science programme, students are expected to have some familiarity working with technology and to have some experience in the Python programming language. The module gives students the skills and tools they need to apply a rigorous testing regime to their projects, which is particularly helpful for the final year project students are expected to complete in fourth year. As a practically oriented module, students are graded using 100% continuous assessment, with students required to submit two individual assignments and a multiple-choice quiz.
Due to timetabling constraints, the lingering impacts of the Covid-19 pandemic on student behaviour, and the rising cost of living in Dublin leading to significantly increased average commute times for students, attendance was lower than expected in the module’s first delivery in 2022. However, feedback from students indicated that the lab work, in particular, was very worthwhile and paid dividends in their other modules. With this in mind, the lecturer decided to redesign this module for blending to allow students to participate in labs remotely.
This case study describes the changes made to the model to begin a blended mode of delivery starting January 2023.
People Involved
The lecturer delivering this module has a strong interest in blended learning techniques and is a member of the university’s Hy-Flex Community of Practice. As the university campus was closed during Covid, the lecturer has significant experience in delivering online-only content, but less experience in the mixed-mode delivery characteristic of blended learning.
The Teaching Assistant (TA) on this module is a PhD student in TU Dublin and has deep knowledge and practical experience in the area of Software Testing. Traditionally, the student group would be split in two for the labs with the lecturer leading one lab group and the teaching assistant leading another. This was the TA’s first experience of teaching on a blended module.
Between all three programmes, there were 37 students taking this module. As TU Dublin participates in a large number of double degrees and exchange programmes, primarily taking place in third year, the class has a larger-than-average ratio of non-native English speakers. Student ages ranged from 19-22 meaning there were no mature students (as defined by the Higher Education Authority, Ireland) in this class.
While this course is mandatory for some students, a survey of the students who chose the module revealed that they did so for the following reasons
- Interest in the subject (software testing)
- Applicability to the Final Year Project
- Usefulness in future job applications
- Interest in the programming language used (python)
Students had the option of attending labs in person or online using Microsoft Teams, access to which is provided by the university). All content was delivered through the university Virtual Learning Environment (VLE) and was available both on campus and off. Students required internet access to participate but were given the opportunity to complete the lab-work asynchronously if they were unable to join the lab session. As computer science students, all participants had sufficient technological skills to benefit from the blended learning techniques.
Fictional Name: Sarah Demographics: 19-22 Irish | | Fictional Name: Stefan Demographics: 19-22 Non-native speaker
| |
Profession: Full time undergraduate student | Motivation for joining the course: Interested in acquiring the knowledge of software testing to apply to her Final Year Project next year | Profession: Full time undergraduate exchange student | Motivation for joining the course: Mandatory module for the exchange programme |
Their challenges as a student:
| Their challenges as a student: – Has excellent English but sometimes finds that instructors speak too quickly and with an unfamiliar accent – Is new to the university and does not know many of his classmates | ||
How did the transformation help them: – Allowed her to engage without needing to be physically present in the classroom | How did the transformation help them:
| ||
Before the transformation
During the initial (2022) delivery, the module took place fully in person. Labs were held in person at 09:00 on Monday mornings. Many students had no further lectures or labs until 16:00 on Monday afternoon. Feedback from students indicated that this, coupled with the fact that some had commute times in excess of two hours, was a major factor in their decision not to attend lab sessions.
Within industry, software testing is usually done by larger organisations and is inherently team-based. One of the lecturer’s aims in designing the original module was to give students an authentic experience of the software development process. All lab work was undertaken in teams of four, with students working in pairs within their teams. Students initially disliked the idea of working in teams, but as the module progressed and they became more used to working with each other they enjoyed it more and more. It was observed that students who worked in the same team each week had a more positive experience of the teamwork aspect of this module than students who were often switched to another team due to non-attendance of their team-mates.
Anecdotal evidence suggests that younger students are more extrinsically motivated than mature learners. Feedback from the 2022 cohort revealed that some students based their decision not to attend in part on the fact that lab work was not directly assessed. The lecturer was reluctant to begin assessing lab work directly, as, in his experience, this approach can have a negative impact on the students’ enjoyment of the session, and ultimately reduce the learning involved. Students of other modules had previously raised concerns with the lecturer about the fairness of assessing teamwork, and often responded negatively to team projects which are graded. A strong correlation between lab attendance and module attainment was observed in 2022, and it is the lecturer’s belief that engagement with lab work is an essential pre-requisite for students to enjoy and succeed in the module.
The main goals of the transformation were:
- To remove barriers to participation in lab work for students with timetabling or commuting constraints
- To extrinsically motivate students to engage with the lab work without directly allocating marks to the outcomes of the lab work
- To improve the student experience of teamwork in the module
The transformation
Lab work was delivered in-class more or less similarly to the previous year. To facilitate the blending, a group call on Teams was initiated at the beginning of each lab session and students were given access to this through a link on their VLE. In an effort to increase the immersiveness of the experience for on–line students, a webcam was used so that students at home could see what was going on in the classroom.
The Transformation:
Lab work was delivered in-class more or less similarly to the previous year. To facilitate the blending, a group call on Teams was initiated at the beginning of each lab session and students were given access to this through a link on their VLE. In an effort to increase the immersiveness of the experience for on–line students, a webcam was used so that students at home could see what was going on in the classroom. As fewer students attended in person, delivery took place in a single classroom and instructors took turns with one instructor assigned to monitor the on-line session while the other focussed on the students in the classroom. Students were encouraged to use their microphones to ask questions but preferred to use the chat functionality in the virtual classroom. The online-instructor either answered questions directly or relayed them to the in-person instructor who would answer them for both on-line and in-person attendees to hear.
Initially, the aim was to allow students to complete the tasks asynchronously if they wished. Whereas previously students often completed tasks using pen-and-paper or by demonstrating code to the instructor, tasks were updated to require students to post the results of their work to YouTrack, (a project management / bug-tracking system used widely in industry) to facilitate asynchronous delivery. A short video was recorded for each lab walking students through the exercises to ensure that they had understood the brief.
This did not work out as well as expected, as many of the on–line teams who did not attend the live lab session did not complete their lab work. Student feedback revealed that many teams were having difficulties finding a time to suit all team members, and consequently, the work was not completed.
From week 4, all teams were asked to attend the live session, whether on-line or in-person. This significantly increased the lab completion rate.
Changes in Activities
In an effort to extrinsically motivate students, a lab task was added for each team to create a mind-map of the topics covered during the lecture. This was a guided task, as many students preferred to be given some direction rather than left to work on an open-ended project. The lecturer and TA selected the best submission each week which was announced at the beginning of the following week’s lab session. Students enjoyed the low-stakes competitive nature of this task, and the response from some teams was outstanding.
A nice side-effect of this was that at the end of semester, all students had a ready-made study-resource which helped them prepare for the class test. This cohort’s performance on the test was an improvement on the previous year’s, which may in part be due to the effort they put in to producing these materials.
Changes in Assessment
Students are required to complete two individual assessments as part of this module. As feedback on the teamwork was generally very positive, the lecturer gave the class the option to instead complete a team-work assignment on a real-world problem taken from a Google coding challenge. Somewhat surprisingly, students voted overwhelmingly in favour of the individual assignment. This is most likely due to the fact that while students enjoy low-stakes teamwork they become more apprehensive when they feel that their attainment depends at least in part on the performance of others.
Early Termination
While the transformation worked well, in Week 7, it was brought to the lecturer’s attention that students on one of the three programmes had a module directly after software testing which was delivered in-person only. A small number of students attended the software testing labs online and then did not attend their in-person delivery for the following module. When asked about their non-attendance they said that they did not have time to travel into campus from their homes after the software testing module. Removing barriers to student engagement was a core principle of this exercise and in blending, and it was very important that the exercise not be seen to have the opposite effect. At this point, it was decided in conjunction with school management to end the option of remote attendance for the class as a whole.
After the transformation
Feedback from the students was generally very positive. Separate surveys were distributed to students who participated online and in-person. Roughly 30% of the online attendees felt that they would have had a more engaging experience in the classroom, but every one of them felt that the option was very important, as they would not have attended in-person due to their own personal circumstances. This was borne out after the remote option was withdrawn as overall class attendance dropped.
When delivering hybrid sessions with both on–line and in-person attendees it is important that the on-line option does not negatively impact the experience of students who attend in person. In-person attendees reported unanimously that they did not feel that the on-line option had a negative impact on their learning experience. This was possibly a result of the division of labour between the TA and lecturer. As one instructor monitored the virtual classroom while the other monitored the physical classroom, issues with divided attention were avoided.
The students praised the authentic experience of using industry software, and learning about software testing using the tools they will likely encounter when they begin their careers in software development.
Many online students reported difficulties in organising meeting times when working on tasks asynchronously, though one team actually enjoyed the experience and appreciated the flexibility it offered them. It is likely that asynchronous delivery is better suited to the more mature students in this cohort, and that in many cases it is unlikely to work well for young undergraduate students with less experience in independent time management.
Students’ attainment on the end-of-semester class test improved this year, which may, in part, be due to the time spent building collaborative study resources as part of their lab work.
Normal 0 false false false EN-GB X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:8.0pt; mso-para-margin-left:0in; line-height:107%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri",sans-serif; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi; mso-ansi-language:EN-GB;} Technologies Employed
Bongo is a solution integrated into the VLE adopted by the university. It provides an environment where the lecturer can broadcast their classes, record the sessions, and interact with the students via web chat. Having one instructor dedicated to monitoring the VLE chat and another dedicated to monitoring the physical classroom
Figma and Stormboard are online tools for collaborative design and brainstorming. They allow students more flexibility in creating a document that standard word-processing tools and can easily be adapted to create mind-maps and other study resources. The lecturer was unfamiliar with Figma before the transformation, but although students were guided through using Stormboard they were given the flexibility to use any tool they liked, and Figma was selected by one team for their collaborative work. One of the benefits of teaching a very technically literate cohort is that they are often aware of additional online tools, and giving them the freedom to bring their own technology can be very beneficial.
YouTrack is a project management and bug-tracking tool used widely in industry. It is designed to allow teams to collaborate and works well for managing team-based software development projects.
Reflections
Challenges Faced
It was unfortunate that the blending exercise was terminated prematurely. The lecturer would have hoped that students with timetabling commitments which required them to be on campus would not have taken the option to attend on–line. If this had been foreseen, however, it could have been avoided. A lesson learned from this is that it is important to consider the impact that blending one module may have on other modules on the programme. It should be made clear to students that the opportunity to blend should only be taken up where it does not impact other modules.
Care should also be taken in offering an option for asynchronous delivery, particularly when teamwork is involved. Students benefit from the structured experience of timetables and in-person deliveries. Giving students the option to complete work in their own time gives them much greater flexibility, but also gives them much more responsibility over managing their schedules. While some students thrive in this environment, not all do. The instructor believes this approach would be better suited to mature students.
The online option increased overall engagement. Students who would otherwise have only rarely attended engaged fully with the module. This increased engagement of online students did not come at a cost to the in-person learners. Students appreciated the authentic experience of using industry-standard collaboration tools to work with their teams, and reported in a terminal survey that they particular enjoyed the practical orientation of the module.
The weekly low-stakes competition for best mind-map worked very well and many students put a lot of effort into this task. It sometimes feels like awarding marks for work is the only tool available to lecturers to extrinsically motivate undergraduate students. This approach has the unwelcome side-effect of increasing the stress and decreasing the enjoyment of the task, particularly when teamwork is involved. In future years, the lecturer intends to explore online badges as a more concrete motivator for these low-stakes exercises.
What went well?
Alternatives
Adding digital achievement badges for the team with the best project, and other aspects of module engagement may be a more tangible reward and will be explored in future deliveries. Many teams reported using Discord for their teamwork. Discord is a voice, video and text app popular with gamers. Students enjoy the platform, and many have set up unofficial chat rooms for modules and programmes within the university. It would be interesting to see if incorporating Discord as an ad-hoc communication platform for project teams would increase engagement in future.
Costs
Some initial work was required in tailoring assignments to fit the YouTrack platform. This was, however, beneficial to students and well worth the time investment. Additional was required to record videos for each lab. Recording videos can be very slow if the instructor strives for a perfect delivery. The instructor had experience in recording videos and had become much quicker at doing so, as they are more willing to let the recording flow even when script mistakes were made. Instructors new to blended learning may find video recording to have a high cost in terms of additional workload, but this pays off as the videos may often be re-used and the time spent recording becomes comparable to the time spent delivering the material in front of a class. All software used in this transformation offers a free tier so there were no additional financial costs incurred as part of this experience (assuming the infrastructure is in place e.g. a Virtual Learning Environment, webcams etc.).