[et_pb_section fb_built=”1″ fullwidth=”on” _builder_version=”4.16″ background_enable_image=”off” locked=”off” global_colors_info=”{}”][et_pb_fullwidth_header title=”Evaluation” _builder_version=”4.27.2″ title_font=”|700|||||||” title_text_align=”center” title_font_size=”58px” title_letter_spacing=”-2px” background_color=”rgba(0,0,0,0.45)” background_image=”https:\/\/ascnet.ie\/techmate\/wp-content\/uploads\/2024\/11\/pexels-ron-lach-10638075-scaled.jpg” background_blend=”overlay” min_height=”242px” custom_margin=”||||false|false” custom_padding=”67px||67px||true|false” custom_padding_tablet=”” custom_padding_phone=”” custom_padding_last_edited=”on|desktop” global_colors_info=”{}”][\/et_pb_fullwidth_header][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.16″ _module_preset=”default” min_height=”643.8px” custom_padding=”||0px|||” global_colors_info=”{}”][et_pb_row column_structure=”1_2,1_2″ _builder_version=”4.16″ _module_preset=”default” custom_margin=”-42px|auto||auto||” global_colors_info=”{}”][et_pb_column type=”1_2″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.2″ text_font=”Teko|600|||||||” text_text_color=”#ffffff” text_font_size=”16px” text_letter_spacing=”2px” background_color=”#e0011a” width=”100%” width_tablet=”” width_phone=”100%” width_last_edited=”on|tablet” max_width=”100%” max_width_tablet=”100%” max_width_phone=”” max_width_last_edited=”on|tablet” custom_margin=”||30px||false|false” custom_padding=”7px|18px|7px|18px|false|false” locked=”off” global_colors_info=”{}”]<\/p>\n
Evaluation of the Toolkit<\/p>\n
[\/et_pb_text][et_pb_blurb image=”https:\/\/ascnet.ie\/techmate\/wp-content\/uploads\/2024\/12\/3.png” alt=”Woman sitting at a laptop” _builder_version=”4.27.2″ _module_preset=”default” custom_margin=”||10px||false|false” box_shadow_style_image=”preset2″ global_colors_info=”%91%93″]<\/p>\n
Pilot evaluation of TechMate took place as part of the personalised feedback study<\/a> at TU Dublin.<\/p>\n [\/et_pb_blurb][et_pb_toggle title=”Show More” open_toggle_background_color=”rgba(0,0,0,0)” closed_toggle_background_color=”rgba(0,0,0,0)” icon_color=”#000000″ use_icon_font_size=”on” open_icon_color=”#000000″ open_use_icon_font_size=”on” module_class=”et-readmore-toggle” _builder_version=”4.27.2″ _module_preset=”default” custom_padding=”0px|0px|0px|0px|false|false” custom_css_main_element=”display: flex;||flex-direction: column;||align-items: flex-end;||||||” custom_css_toggle_title=”order:2;||padding: 0px 20px 0px 0px !important;” custom_css_toggle_content=”order: 1;||padding: 0px;” border_width_all=”0px” global_colors_info=”{}”]<\/p>\n In this study, lecturers in computing department were interviewed and their views and opinions collected on the usability and usefulness of\u00a0personalised feedback action<\/a>. Following this pilot, TechMate is being updated, and a formal evaluation will follow. It is proposed that the evaluation will assess 9 usability and usefulness attributes and take place in a facilitated session, using a think-aloud method for user tasks and a semi-structured interview.<\/p>\n [\/et_pb_toggle][\/et_pb_column][et_pb_column type=”1_2″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.2″ text_font=”Teko|600|||||||” text_text_color=”#ffffff” text_font_size=”16px” text_letter_spacing=”2px” background_color=”#e0011a” width=”99.3%” width_tablet=”” width_phone=”100%” width_last_edited=”on|tablet” max_width=”100%” max_width_tablet=”100%” max_width_phone=”” max_width_last_edited=”on|tablet” custom_margin=”|-395px|30px||false|false” custom_padding=”7px|18px|7px|18px|false|false” locked=”off” global_colors_info=”{}”]<\/p>\n Evaluation of Actions in the Toolkit<\/p>\n [\/et_pb_text][et_pb_blurb image=”https:\/\/ascnet.ie\/techmate\/wp-content\/uploads\/2024\/12\/Two_people_computer.png” alt=”Two people at a computer” _builder_version=”4.27.2″ _module_preset=”default” custom_margin=”||10px||false|false” box_shadow_style_image=”preset2″ locked=”off” global_colors_info=”%91%93″]<\/p>\n Different actions may need to be measured in different ways to show impact. For instance, many\u00a0Policy related actions<\/a>\u00a0rely on measuring enrolment or retention data.<\/p>\n [\/et_pb_blurb][et_pb_toggle title=”Show More” open_toggle_text_color=”#0c71c3″ open_toggle_background_color=”rgba(0,0,0,0)” closed_toggle_background_color=”rgba(0,0,0,0)” icon_color=”#000000″ use_icon_font_size=”on” open_icon_color=”#000000″ open_use_icon_font_size=”on” module_class=”et-readmore-toggle” _builder_version=”4.27.2″ _module_preset=”default” custom_padding=”0px|0px|0px|0px|false|false” custom_css_main_element=”display: flex;||flex-direction: column;||align-items: flex-end;||||||” custom_css_toggle_title=”order:2;||padding: 0px 20px 0px 0px !important;” custom_css_toggle_content=”order: 1;||padding: 0px;” border_width_all=”0px” locked=”off” global_colors_info=”{}”]<\/p>\n Indeed, when\u00a0creating an all-female course\u00a0<\/a>or\u00a0recruiting new female faculty staff<\/a>, the most straightforward way to see whether the action worked is to compare recruitment\/retention numbers before and after the implementation.<\/p>\n Actions such as once-off outreach events, workshops, industry visits can use feedback as an evaluation approach. Feedback on measures such as,\u00a0perception of the field<\/a>, and\u00a0interest in computing<\/a>, could be assessed before and after the event. Alternatively, feedback of\u00a0enjoyment\/satisfaction<\/a>\u00a0from an event can be collected.<\/p>\n Actions which take place in the classroom such as\u00a0pair programming<\/a>, or changing the\u00a0class or lab dynamics<\/a>\u00a0can use same evaluation approaches as actions that take place during the academic term outside of class. The latter include, buddy systems or\u00a0personalised emails<\/a>\u00a0received from the lectures with feedback on assessment. If a short term impact is required, they can be evaluated with measures such as\u00a0sense of belonging survey<\/a>\u00a0or by using feedback with one or multiple measures listed above.<\/p>\n Actions such as\u00a0<\/a>advertising campaigns<\/a>\u00a0or amendment of marketing materials would benefit from knowing the statistical numbers afterwards, rather than asking students for feedback\/opinions\/perceptions.<\/a><\/p>\n Our current work-in-progress suggestion is to select actions for evaluation where evidence of impact is possible in a relatively short term. By that, we mean that the impact can be seen immediately after implementation. These actions could include many in-class initiatives, as well as some mentoring programs.<\/p>\n [\/et_pb_toggle][et_pb_code _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”]