About me

I am an assistant professor at the Software Technology for Learning and Teaching group of the department of Information and Computing Sciences at Utrecht University. The overall goal of my research is to support students in learning programming with high-quality tools and feedback. To achieve this, I investigate tutoring systems and (automated) feedback generation for learning programming, and code quality in education. I am also interested in the implications and opportunities of Large Language Models/generative AI in computing education.

Between 2015 and 2020 I was a parttime PhD candidate at the Open University of the Netherlands, funded by an NWO doctoral grant for teachers.

Until August 2020 I was a (senior) lecturer in Software Engineering at Windesheim University of Applied Sciences.

Highlighted projects

  • Working group on LLMs in CEd

    In this ITiCSE 2023 working group we study several aspects of Generative AI in Computing education, such as student and teacher perceptions, ethics, and LLM performance on programming tasks.

  • Feedback on programming steps

    In an ITiCSe 2022 working group and follow-up project we study how, why and when experts would give feedback on student code while they're solving a programming task.

  • Learning about code quality

    Together with researchers from the software company JetBrains, we study how code quality can be improved in large online learning platforms.

Publications

A complete list of my publications can be found on Google Scholar.

    2024

    • Next-Step Hint Generation for Introductory Programming Using Large Language Models. Lianne Roest, Hieke Keuning and Johan Jeuring. Accepted to ACE 2024.

    2023

    • The Robots are Here: Navigating the Generative AI Revolution in Computing Education. James Prather, Paul Denny, Juho Leinonen, Brett Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton-Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent Reeves and Jaromir Savelka.
    • What Skills Do You Need When Developing Software Using ChatGPT? Johan Jeuring, Roel Groot and Hieke Keuning. Koli Calling 2023.
    • Developers Talking About Code Quality. Jürgen Börstler, Kwabena E. Bennin, Sara Hooshangi, Johan Jeuring, Hieke Keuning, Carsten Kleiner, Bonnie MacKellar, Rodrigo Duran, Harald Störrle, Daniel Toll and Jelle van Assema. Empirical Software Engineering 28, 128 (2023).
    • Exploring the Potential of Large Language Models to Generate Formative Programming Feedback. Natalie Kiesler, Dominic Lohr and Hieke Keuning. Accepted to Frontiers in Education (FIE) Conference 2023. Arxiv preprint .
    • A Systematic Mapping Study of Code Quality in Education. Hieke Keuning, Johan Jeuring and Bastiaan Heeren. Proc. of ITiCSE 2023. Version with complete bibliography . Mapping table.
    • Student Code Refactoring Misconceptions. Eduardo Oliveira, Hieke Keuning and Johan Jeuring. Proc. of ITiCSE 2023.
    • Detecting Code Quality Issues in Pre-written Templates of Programming Tasks in Online Courses. Anastasiia Birillo, Elizaveta Artser, Maria Tigina, Yaroslav Golubev, Hieke Keuning, Nikolay Vyahhi and Timofey Bryksin. Proc. of ITiCSE 2023.
    • Analyzing the Quality of Submissions in Online Programming Courses. Maria Tigina, Anastasiia Birillo, Yaroslav Golubev, Hieke Keuning, Nikolay Vyahhi, Timofey Bryksin. Proc. of ICSE-SEET 2023.

    2022

    • Towards Giving Timely Formative Feedback and Hints to Novice Programmers. Johan Jeuring, Hieke Keuning, Samiha Marwan, Dennis Bouvier, Cruz Izu, Natalie Kiesler, Teemu Lehtinen, Dominic Lohr, Andrew Peterson, Sami Sarsa. ITiCSE Working group reports 2022.

    2021

    • A Tutoring System to Learn Code Refactoring. Hieke Keuning, Bastiaan Heeren and Johan Jeuring. Proceedings of the SIGCSE Technical Symposium on Computer Science Education 2021.

    2020

    • Student Refactoring Behaviour in a Programming Tutor. Hieke Keuning, Bastiaan Heeren and Johan Jeuring. Proceedings of the Koli Calling Conference on Computing Education Research 2020.
    • Automated Feedback for Learning Code Refactoring. Hieke Keuning. PhD Thesis, October 2020.

    2019

    • How Teachers Would Help Students to Improve Their Code. Hieke Keuning, Bastiaan Heeren and Johan Jeuring. Proceedings of Innovation and Technology in Computer Science Education (ITiCSE), 2019. + Supplemental material

    2018

    • A Systematic Literature Review of Automated Feedback Generation for Programming Exercises. Hieke Keuning, Johan Jeuring and Bastiaan Heeren. ACM Transactions on Computing Education (TOCE), volume 19, issue 1, 2018. . Table with reviewed tools.

    2017

    • I know it when I see it: Perceptions of Code Quality. Jürgen Börstler, Harald Störrle, Daniel Toll, Jelle van Assema, Rodrigo Duran, Sara Hooshangi, Johan Jeuring, Hieke Keuning, Carsten Kleiner, Bonnie MacKellar. ITiCSE Working Group, 2017.
    • An intelligent tutor to learn the evaluation of microcontroller I/O programming expressions. Hugo Arends, Bastiaan Heeren, Hieke Keuning and Johan Jeuring. Proceedings of the Koli Calling Conference on Computing Education Research, 2017.
    • Code Quality Issues in Student Programs. Hieke Keuning, Bastiaan Heeren and Johan Jeuring. Proceedings of Innovation and Technology in Computer Science Education (ITiCSE), 2017.
    • Automatically Classifying Students in Need of Support by Detecting Changes in Programming Behaviour. Anthony Estey, Hieke Keuning, Yvonne Coady. Proceedings of the SIGCSE Technical Symposium on Computer Science Education, 2017.

    2016

    • Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. Hieke Keuning, Johan Jeuring and Bastiaan Heeren. Proceedings of Innovation and Technology in Computer Science Education (ITiCSE), 2016. . Extended Version, Technical report. . Table with reviewed tools.

    2014

    • Strategy-based feedback in a programming tutor. Hieke Keuning, Bastiaan Heeren and Johan Jeuring. Proceedings of the Computer Science Education Research Conference, 2014.
    • Strategy-based feedback for imperative programming exercises. Master Thesis. Open University of the Netherlands, June 2014.

    Teaching

    My interest in Computing Education (CEd) comes from, and continues to be inspired by, my extensive experience in teaching computer science and software engineering topics. At UU, I am currently involved in the following courses:

    In the past, I have taught over twenty courses, designed multiple courses, and coordinated several educational programs. Some coursed I have taught are: Artificial Intelligence for games, Discrete Mathematics, C# programming, OO programming in Java, Client-side web technologies, Software Architecture and design patterns, PHP Web Programming, ASP.NET, Functional Design and OO programming, Simulation, Compiler construction, Databases and SQL, Introduction to programming in C++, and Introduction to Logic.

    Thesis Supervision

    I currently supervise two PhD students.

    You can always contact me if you are interested in doing a Bachelor or Master thesis on a topic related to my reserach interests. Concrete projects can be found on my Konjoin profile.

    Other

    Service

    I served in the program committee of several CEd research conferences (ITiCSE, SIGCSE, AIED, Koli, QUATIC), and regularly review for journals (TOCE, TLT, C&E).

    Recent & upcoming talks

    • Invited talk at NeurIPS 2023 workshop Generative AI for Education (GAIED), December 2023.
    • Panel discussion, VERSEN event on Generative AI and Ethics, November 2023.
    • Talk Copilot and ChatGPT in programming education: what do we know from a research perspective? at NIOC 2023 (Dutch Informatics Education Conference).
    • Talk A.I. tools in programming education at the U-talent ‘Docentennetwerk informatica’ meeting, Januari 2023.

    Media

    • Improving computer science education. Feature in I/O magazine article, April 2023.
    • Betere code met automatische feedback en hints. Hieke Keuning. AG Connect, December 2020.