404
+Page not found
+diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/2020meetings/index.html b/2020meetings/index.html new file mode 100644 index 00000000..ba7c9cf9 --- /dev/null +++ b/2020meetings/index.html @@ -0,0 +1,456 @@ + + + +
+ + + + + + + +During 2020 we ran a series of online workshops, with support from the London Mathematical Society, Universitas 21, and the School of Mathematics at the University of Edinburgh.
+You can access recordings and materials from these workshops through the links below:
+ + +Title | +Date | +Location | +Description | +
---|---|---|---|
Effective use of Maxima | +2 Dec 2020, 15:00-17:00 GMT | +Course page - effective use of Maxima | +Learn how mathematics educators use Maxima effectively with STACK, and how to use more advanced Maxima techniques to improve your own STACK questions. | +
Addressing common student errors | +25 Nov 2020, 15:00-17:00 GMT | +Course page - student errors 25-11-20 | +Gain insight into common student errors based on mathematics education research, and get practical experience of using STACK to detect and give feedback on errors. | +
Assessment of proofs in STACK | +7 September 2020, 15:00-16:30 BST | +Course page - assessment of proofs | +Learn about important practical findings from educational research on how to teach mathematical proof effectively, and hear about practical approaches to assessing proof online using tools like STACK. | +
Getting started with STACK | +20 July 2020, 10:00-12:00 BST | +Course page - getting started 20-07-20 | +Learn about the features of STACK and write your first STACK question with experts on hand to help. | +
Addressing common student errors | +3 July 2020, 14:00-16:00 BST | +Course page - student errors | +Gain insight into common student errors based on mathematics education research, and get practical experience of using STACK to detect and give feedback on errors. | +
Quality questions and assessing proof | +19 June 2020, 10:00-12:00 BST | +Course page - quality questions | +Learn about advanced features of STACK that help to make sure your questions work reliably, and get tips from experienced authors on how to write polished questions. In the second half, learn about ways to assess proof using STACK. | +
Getting started with STACK | +15 June 2020, 10:00-15:00 BST | +See 20 July above | +Hear from some experienced users about how they use STACK in variety of courses, then write your first STACK question with experts on hand to help. | +
Page not found
+STACK is an online assessment system for mathematics and STEM, designed to enable students to answer questions with a mathematical expression, such as a polynomial.
+STACK uses the Computer Algebra System Maxima to evaluate expressions.
+Before answers are graded, students confirm their answer is interpreted correctly by the system. Invalid answers, like ones with mismatched brackets, are rejected.
+Teachers write grading trees that mark answers based on mathematical properties, like "is it factored?"
+As an open-source project, users help improve STACK by adding features, translations, bug reports and more. Many users share STACK materials, for example through the ABACUS material bank.
+
+Credits and contributionsABACUS material bank
STACK is trusted by many respected institutions.
+ + + + + + + + + + + + +STACK is trusted by a number of commercial partners.
Antti Rasila, Guangdong Technion - Israel Institute of Technology
+Since 2006, Aalto University has been using STACK to run online assessments within their Engineering Mathematics courses. Initially, the University was running a STACK version developed in-house specifically for the University, but this was later merged back into the main STACK branch in version 3.0. After receiving funding, the material bank Abacus was created, with the purpose of helping institutions collaborate and share STACK questions. The project's agreement was written in a way that ensured question authors kept their copyright, while still encouraging collaboration. Abacus has 30 members and its courses cover most of undergraduate mathematics and many parts of physics.
+The MatTa group ("Matematiikkaa tietokoneavusteisesti"; "mathematics by using computers") was founded in 1990s by lecturer Emeritus Simo K. Kivelä at the Department of Mathematics and Systems Analysis at Aalto University, formerly known as Helsinki University of Technology. The purpose of the group was to investigate the use of computers and information technology to teach engineering mathematics. Initially, the projects involved, for example, using visualisations, multimedia and symbolic computation to make mathematical content more interesting and accessible for students. The group also produced a number of free Finnish language electronic lecture notes with interactive content and a substantial database of traditional pen and paper exercise assignments called Euler. In 2006, the group started to take interest in automatic assessment, as they believed computer-aided methodologies could achieve a practical impact in teaching activities there. There was also hope at the University that Computer Aided Assessment could reduce the number of students assistants required to grade homework, both to save cost and because a sufficient number of qualified graders was not always available. Initially, the group focused on the Maple T.A. system, however it was found lacking in several aspects. The most important of these were performance issues and an incompatibility with various browsers, in particular ones available for the Linux operating system. The high licensing fees were also a problem, as they could undo all the savings that the University hoped to achieve by using the system.
+In spring of 2006, the group started gaining interest in STACK, which has an open source license. This license would allow the University to use its in-house software development skills to improve the platform where needed, and it would also guarantee that licence fees would not be imposed in the future. After deciding to use STACK, which was running its standalone version 1.x at the time, the group spent the summer of 2006 modifying the software to better suit the University's needs. This included replacing the rendering code of mathematical formulas to support browsers other than Microsoft Internet Explorer, translating the software to Finnish and a Shibboleth based integration with the Finnish national Haka single-sign-in system.
+After modifying the system, the University started testing STACK. The initial test was with the university's Basic Course in Engineering Mathematics KP3. A. Rasila wrote four STACK assignments for each week of the course, all of which were relatively simple questions involving numerical or algebraic input. The system had numerous small technical issues, but the students were generally happy with the system as evidenced both by student feedback and general usage statistics [1].
+Encouraged by the positive results, it was decided to test the system more comprehensively to convince the teaching community that automatic assessment was worth of the effort in teaching engineering mathematics. It was important to show that the online assessment could be used by teachers with little programming skill, and that it would not cause a large increase in their workload. This was crucial for the long-term success of the project, as MatTa's previously developed materials were often only used by a handful of teachers, most of whom where developers themselves. Furthermore, the group needed to demonstrate that the system had real cost savings, as well as good learning outcomes compared to traditional types of assignments. The University set up an experimental course "Discrete Mathematics", where STACK would form a significant portion of the final grade [2]. Following the positive results of this course, the use of STACK began to spread at the University, and within three years, most basic courses in Engineering Mathematics at Aalto University were using STACK.
+A key ingredient in the success of STACK at Aalto University were the in-house developments by programmer Matti Harjula, also wrote his Master’s Thesis on this work [3]. His initial work included analysing the data of the first 2006 experiment and further developing the system to address any shortcomings. His project was successful, as it led to a system that many lecturers at the department were willing to use, and no major issues were identified in the subsequent use of the system.
+However, it also led to a new problem. Independently from Aalto University, the British STACK developers had been developing STACK 2.0, which was very different from the 1.x series Aalto University had been using for their work. The in-house developments by Aalto University were not easily implemented into STACK 2.0, but merging the Aalto University code back into the trunk of STACK was still necessary for a number of reasons. Firstly, a unified code base would speed up the overall development of STACK, and the Department at Aalto University did not intend to permanently commit many resources to maintaining the system. Secondly, there was interest from other Finnish universities to try the system, however they preferred to use the “official” British version of STACK, which would prevent them from using the materials developed at Aalto University. Because of this, the Finnish and British developers agreed to join forces in combining their work into STACK 3.0. During the STACK 3.0 merge, experimental features of STACK were also being added to the Aalto University version, as documented in section 7.8 of [4]. These features were related to several pedagogical ideas, detailed in [5,6,7].
+There were two key advancements in 2015. Firstly, STACK 3.0 was completed and Aalto University had finalised their plans to the deploy the system as a replacement to their old in-house version. Since the University was now using the official STACK version, it was easier to collaborate with other Universities on STACK material. Secondly, the consortium of the seven Finnish technology universities called for proposals in developing collaborative practices in education. A draft proposal for joint development of electronic materials for mathematics education was proposed and selected as a pilot project with over half a million euros of funding. A. Rasila was chosen as the national coordinator of this project, which become known as the Abacus consortium [8].
+Abacus is a database of teaching materials, mainly STACK questions, covering most basic topics in undergraduate mathematics. Its mission statement is to help lecturers find high quality and free learning materials, achieved through facilitating the sharing of online teaching resources between institutions. National and international collaboration is at the heart of the project, as it helps standardise the platform, minimise the necessary contribution of each institution and contribute to the overall market share of the platform. Language differences between courses is not a concern, since adding translations to individual questions are almost trivial compared to the expense of actually programming the mathematics e-content.
+In drafting the initial consortium agreement, Abacus drew from two other related projects, namely a 2009 e-learning collaboration project with the Bavarian Virtual University VHB, which was funded by the later discontinued Finnish Virtual University, and the MatTa material bank Euler.
+In projects largely developed through public and institutional funding, fair use is a common problem. Some partners may feel they are contributing more than the rest, and hence will not be interested in collaboration and instead push for in-house development. Furthermore, collaborators who develop materials without being paid are often unhappy when someone else makes even minor monetary gains from their work, which has led to a widespread use of non-commercial licenses for academic work. While understandable, these restrictions hinder the acceptance of platforms like Abacus, and lead to divisions in the community. Furthermore, non-commercial licenses can be confusing, as the exact nature of commercial activity is not clearly defined in academia. For example, is "free" work done by a salaried lecturer commercial use? Finally, it can be necessary for institutions to share the sources of assignments due to international differences in notation, course content and language. On the other hand, a pure open source model is not particularly attractive for e-assessment materials, since students should not be able to search online for the teachers' answers.
+To solve these problems, the Abacus consortium agreement was written such that there are no strict legal restrictions on what kind of institutions can join the consortium or what kinds of materials can be included. The materials are shared in a way such that each partner gets substantial rights to develop them based on a license similar to open source, but is not allowed to share the source code with third parties. The original developers retain their copyright of the materials and are allowed to give them other licenses besides the one required by Abacus. Since new members are reviewed by the consortium's steering group and are required to make a one-time contribution to the consortium, the agreement encourages collaboration and guarantees that existing partners benefit from the growth of the consortium.
+As of July 2019, the Abacus consortium consists of 30 members, 17 of which are from Finland, as well as three from Estonia and two from both Germany and Norway. Other countries present in the consortium are China, France, Ireland, Portugal, Ukraine, and the United Kingdom. There are ongoing discussions with several other potential partners, mainly from Europe. The database currently contains problem assignments for most bachelor level courses in mathematics, as well as a substantial number of physics assignments. Although Abacus accepts contributions in any language, most materials are available in English. Translations of the remaining Finnish-only materials are ongoing, and are expected to be completed in 2020.
+[1] A. Rasila, M. Harjula, and K. Zenger. Automatic assessment of mathematics exercises: Experiences and future prospects. In ReekTori 2007: Symposium of Engineering Education, 70-80. Helsinki University of Technology, Finland, Teaching and Learning Development Unit, http://www.dipoli.tkk.fi/ok, 2007.
+[2] A. Rasila, L. Havola, H. Majander, and J. Malinen. Automatic assessment in engineering mathematics: evaluation of the impact. In ReekTori 2010: Symposium of Engineering Education. Aalto University, Finland, Teaching and Learning Development Unit, http://www.dipoli.tkk.fi/ok, 2010.
+[3] M. Harjula. Mathematics exercise system with automatic assessment. Master's thesis, Helsinki University of Technology, 2008.
+[4] C. J. Sangwin. Computer Aided Assessment of Mathematics. Oxford University Press, Oxford, UK, 2013.
+[5] H. Majander and A. Rasila. Tutkimus suuntaamassa 2010-luvun matemaattisten aineiden opetusta, Experiences of continuous formative assessment in engineering mathematics, 197-214. Tampereen yliopistopaino Oy - Juvenes Print, 2011.
+[6] A. Rasila, J. Malinen, and H. Tiitu. Automatic assement and conceptual understanding. Teaching Mathematics and its Applications, 34(3):149-159, 2015.
+[7] T. Pelkola, A. Rasila, and C. J. Sangwin. Investigating Bloom's learning for mastery in mathematics with online assessment. Informatics in Education, 17(2), 363-380, 2018.
+[8] A. Rasila. E-assessment material bank abacus. In Proceedings of EDILEARN16, 8th Annual International Conference on Education and New Learning Technologies, July 2016.
Michael Kallweit
+At the Ruhr-Universität Bochum, students from all disciplines are offered digital mathematics tasks in an e-learning course for self-study in order to prepare and revise. These questions make intensive use of the randomisation and differentiated feedback options of the STACK question type. Some tasks have been supplemented by adaptive exercises, which guide the students through complex tasks. These adaptive tasks have had great success, are shown to have the same level of student engagement as paper-based tasks and be a better predictor of exam performance.
+At the Ruhr University Bochum, digital exercises have been used in courses at the Faculty of Mathematics since 2011, and since 2013 this has included STACK. In 2015, a cross-curricular e-learning course was set up, in which students can repeat and process selected topics for exam preparation by themselves. The course takes advantages of Moodle's gamification features: students gain points for engaging with quizzes, contributing to a "level". They also earn badges for completing material in a short time or with a high mark. Additionally, students have access to digital tutorial support.
+In addition to these features, the University wanted to add learning exercises that adapt to students' needs.
+In online assessment, weekly digital exercises are often done in "examination mode", that is, where students complete tasks in a given timeframe without interacting with the teachers in the meantime. Normally students are only allowed one attempt at the question, and differentiated feedback is only available for the students after the deadline. However, experience shows that students rarely use the detailed written feedback provided and are satisfied with just seeing their score. This observation is consistent with the students' responses to manually graded paper-based homework.
+Additionally, often valuable reflective loops that promote learning are omitted. As highlighted by Hattie's meta-analysis [2 and 3], it is important to increase students' interactivity with the feedback on their solutions. According to [4], good feedback should show concrete possibilities of how gaps in skills can be closed by students.
+Contrast this with tutorial lessons, where teachers often help students find a solution path. In the form of minimal assistance, teachers will intervene to help students achieve steady progress in dealing with a problem. If the students take an erroneous path to a solution, the teacher not just corrects the error, but will instead take the student on the path through the task. After being steered on the right path, the student will often fix the error by themselves. This procedure is known as instructional scaffolding: with just minimal support from the teacher, the students can master the individual tasks themselves, and can actively reflect on difficulties and mistakes during the process to close gaps in their knowledge.
+The developers at Ruhr-Universität Bochum wanted to bring this concept to online assessment: to build an online assessment system that can carry out instructional scaffolding to help students find a solution path. STACK was ideal for this, as its features in providing specific feedback depending on the properties of students answers provide a great foundation for an adaptive system.
+Instead of being confronted by a teacher in a tutorial group, students can process adaptive digital self-learning tasks. Following the above principles, students follow an adaptive path of intermediate step tasks in small steps after submitting an incorrectly answered task. The intermediate steps focus on concrete knowledge and skills that must be combined to solve the original task.
+The adaptive methods were tested in entrance exams at secondary level II schools, through diagnostic error analysis tools [5,6,7]. The goal of this was to isolate the underlying errors behind why students incorrectly process digital tasks. Using digital tasks with STACK and a complex composition of digital tasks, the researchers achieved a considerable error detection rate of about 90%.
+Adaptive self-learning tasks follow the scaffolding principle. The error analysis and subsequent presentation of intermediate steps take place within the potential response trees of a STACK question, and offer added value compared to traditional digital tasks:
+Moodle currently only offers limited possibilities for adaptively designing a sequence of tasks in tests. Hence, the adaptive tasks were introduced at the level of individual STACK questions. For each question, the potential response tree analyses the student's solution, and if they have made an error, shows a link to the starting point of an individual path. By including an externally stored Javascript file, the next intermediate step exercise becomes visible only by clicking on a button. The analysis of the input for this intermediate step is also stored in a response tree which adaptively defines the subsequent tasks. By repeating steps and integrating additional ones, the adaptive procedure can simulate real tutorial support.
+The following elementary example for matrix multiplication illustrates the central principles of the adaptive task format:
+If the input of the solution is incorrect, the student is informed that they can work on the task again in guided intermediate steps via the "Weiter" ("Next") button.
+The first intermediate step then opens directly below the actual task, in this case asking students to identify the rule that is used when calculating matrix products. Subsequent steps will be available each time a step is correctly completed and the student clicks "Next"
+The entire adaptive path with all intermediate step tasks has the following structure.
+It is also possible to ask students to make a decision about which direction they want to take.
+Branches at which students make decisions about which direction they want to take are also possible. Overall, the feedback can be structured in a variety of ways according to Zech's hierarchy levels, with a focus on motivation, strategy or content help [8].
+When comparing the University's new online assessment to their old, multiple-choice based system, several improvements become apparent. In multiple choice questions, you can only have a finite number of "distractors", but in STACK you can give feedback on an unlimited number of answers. In particular, the developers compared a question about expanding (2x+3y)^2, and found that while the old system had only 9 distractor options, the STACK system had registered 41 different answers over its lifetime, which could be grouped into 28 different types of mistakes. This is a major improvement on the number of mistakes students can receive specific feedback about.
+Additionally, the developers wanted to compare the performance of paper-based and digital homework. Across all courses, they did not find any significant difference in how many tasks students complete between the two systems. This is encouraging; the University saves resources by not relying on human markers for homework, and this confirms that they have not lost any functionality or student engagement by moving towards online assessment. For the course "Mathematics for Chemistry", the correlation between paper-based homework scores and exam scores was found to be 0.56, but the correlation between online homework scores and exam scores was 0.63. Similar results were found for the other courses at the University. Hence, online quiz scores are a better predictor for exam scores.
+One of the main challenges of STACK integration was keeping up with new versions. It was important to convince administrators that going through the trouble of updating STACK was worth it, as new updates and features could be important to question authors.
+Additionally, there was a challenge in distributing the questions to new users, since Moodle's options for managing and distributing questions were found lacking. To solve this, a database for managing STACK questions in moodleXML files was created, which included more sophisticated search options. For each question, the system generates an overview pdf file with a short description, information on available languages, screenshots and an automatically generated overview of the potential response trees. About 700 questions are in the database, and because of its usability, it is easy to convince new users that it is simple to find good STACK questions.
+It was helpful to have a strong community of STACK users to help with projects. For example, one of the lecturers held a seminar to teach effective STACK usage to students training to become school teachers. The students of this class became a valuable community of STACK users willing to help with other projects. Additionally, it was valuable that STACK had been translated to German, as this allowed the University to easily spread STACK to other German institutions and build good partners.
+The STACK courses at Ruhr-Universität Bochum continue to be improved on. Future additions may include using more sophisticated features in questions, such as equivalence reasoning inputs and interactive JSXGraph visuals.
+[1] M. Kallweit and E. Glasmachers. Adaptive selbstlernaufgaben mit STACK. 2019.
+[2] J. Hattie. Visible learning: A synthesis of over 800 meta analyses relating to achievement. Routledge, 2009.
+[3] J. Hattie and H. Timperley. The power of feedback. Review of Educational Research, 77(1):91-112, 2007.
+[4] D.J. Nicol and D. Macfarlane-Dick. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2):199-218, 2006.
+[5] R. Bruder, N. Feldt Caesar, A. Pallack, G. Pinkernell, and A. Wynands. Mathematisches grundwissen und grundkonnen in der sekundarstufe ii. W. Blum et al. (Hrsg.), Bildungsstandards aktuell: Mathematik in der Sekundarstufe II, pages 108-124, 2015.
+[6] M. Schaub. Einsatz des elementarisierenden testens im ein- und ausgangstest des online vorkurses vemint. Beiträge zum Mathematikunterricht 2018., pages 1567-1570, 2018.
+[7] F. Zech. Grundkurs Mathematikdidaktik. Theoretische und praktische Anleitungen für das Lehren und Lernen von Mathematik. 1978.
Interview with Chris Sangwin
+At the University of Edinburgh, online assessment has been consolidated with in-house support, mostly with STACK, for the majority of year one and two mathematics modules and for many other mathematics and general science courses. A dedicated learning-technologist post was created to help course organisers implement online assessment. Replacing the current online assessment with human marking is currently estimated to save the School over 6100 hours of work marking students' work each year.
+The School of Mathematics at the University of Edinburgh had, over the years, grown to use a number of separate online assessment systems. However, there was growing concern that using separate systems created a number of problems, for example students had to learn the syntax of many different systems. It was also problematic that students needed to purchase access to the online systems provided by publishers. At the same time, student numbers in the UK continued to grow [1], and there were growing expectations from students for more frequent formative assessment in their courses. This motivated the school to bring all assessments in-house with STACK.
+STACK was first used in Edinburgh at a small scale for the Lothian Equal Access Programme for Schools (LEAPS) summer school in 2016. Following the success with LEAPS, STACK was used in larger courses such as Introduction to Linear Algebra, a year one module with over 600 students, in the 2016-17 academic year. The University's primary learning environment is Blackboard "LEARN", so students gain access to a dedicated STACK service via the LTI protocol through their normal LEARN pages.
+Having identified the need for additional support, the School of Mathematics created a dedicated "Learning Technologist" post. The primary goal of this post was to transform existing (largely paper-based) problem sets into online assessments and to develop alternatives to existing external online assessments. The Technologist used the following process to transform these assessments.
+Before the year started, current forms of assessment were reviewed, and suitable STACK questions were written together with course organisers.
+During the academic year, quizzes were made available, and their usage monitored.
+When the year was over, this monitoring data was analysed and questions were updated where appropriate.
+Creating online assessments is significant additional work, but once the quizzes have been created they require minimal work to maintain and can last for the lifetime of the course. Because of this cost-benefit relationship, employing a Learning Technologist has been an effective strategy in ensuring change actually took place.
+In 2019, many courses have online assessments using STACK. A course will typically have weekly or fortnightly assessment, many using a combination of flipped classroom "reading quizzes" (RQ), formative "practice quizzes" (PQ) and summative "assessed quizzes" (AQ). Some courses also make an optional mock online exam available to students. The following table provides an overview from the 18/19 year, including the number of students, how often quizzes were given out (per week W or semester S), the average number of questions per quiz (#Q/quiz) and the total number of questions (#Qs).
+Year | +Course | +Students | +Quizzes | +#Q/quiz | +#Qs | +
---|---|---|---|---|---|
1 | +Introduction to Linear Algebra | +604 | +2 RQ/W | +2-3 | ++ |
+ | + | + | 1 AQ/W | +6-8 | +110 | +
1 | +Calculus and its Applications | +594 | +1 RQ/W | +4 | ++ |
+ | + | + | 1 AQ/W | +10 | +150 | +
1 | +Proofs and Problem Solving | +344 | +1 RQ/W | +2 | +20 | +
1 | +Engineering Mathematics 1a | +393 | +3 PQ/W | +5 | ++ |
+ | Mathematics for Natural Sciences 1a | +138 | +1 AQ/W | +7-10 | +240 | +
1 | +Engineering Mathematics 1b | +401 | +3 PQ/W | +3-5 | ++ |
+ | Mathematics for Natural Sciences 1b | +136 | +1 AQ/W | +7-10 | +200 | +
1 | +Mathematics for Physics 1 | +193 | +5 AQ/S | +5 | +25 | +
1 | +Mathematics for Physics 2 | +202 | +5 AQ/S | +6 | +30 | +
1 | +Fundamentals of Algebra and Calculus | +113 | +5 PQ/W | +10-20 | ++ |
+ | + | + | 3 AQ/W | +6-12 | +950 | +
2 | +Probability | +312 | +1 AQ/W | +3-5 | +36 | +
2 | +Several Variable Calculus and DEs | +287 | +1 PQ/W | +5-12 | ++ |
+ | + | + | 1 AQ/W | +5-10 | +180 | +
3 | +Honours Algebra | +202 | +9 AQ/S | +1-8 | +46 | +
3 | +Combinatorics and Graph Theory | +60 | +2 RQ/W | +2 | ++ |
+ | + | + | 4 AQ/S | +1-6 | +45 | +
3 | +Symmetry and Geometry | +36 | +1 AQ/W | +4-7 | +45 | +
4 | +Galois Theory | +27 | +1 RQ/W | +2 | +22 | +
PG | +Fundamentals of Optimization | +190 | +3 AQ/S | +4-8 | +16 | +
PG | +Introduction to Probability and Statistics | +23 | +1 PQ/W | +6-8 | ++ |
+ | + | + | 5 AQ/S | +4 | +62 | +
Over the course of this project, the Learning Technologist estimates that developing a fully functioning quiz of 8 questions took about two person-days, or 16 hours of work, to create. As an example, consider the large first-year course "Introduction to Linear Algebra", with around 600 students in 65 tutorial groups. Replacing half of the weekly hand-ins with online assessments has saved each tutor over one hour of marking per week. Hence, over 65 hours of work is saved each week as a result. In addition, the students now complete more than double the number of practice problems, providing them with enhanced formative feedback which would be impossible to resource otherwise. Since online quizzes can be reused each year, this will be a consistent saving for as long as the course remains.
+Overall, it is estimated that STACK saves the University over 6100 hours of work each year [2].
+The goal was to have the first 4-6 weeks of quizzes ready before the beginning of a semester. However, as the term progressed, it was not unusual for the last few quizzes to be ready only "at the last moment". This did not give the question authors a lot of time to review their questions, but does reflect the realities of teaching.
+It was also important that the course organiser were fully involved in the authoring process. They had to give clear and explicit guidance on the learning objectives and help review the mathematical content.
+Finally, when working with large question banks, organising became difficult. Large Moodle question banks are tricky to browse, and maintenance becomes tedious when questions are duplicated between similar courses.
+The most significant factor in the success of this project was the dedicated Learning Technologist post. This was essential, as it assured course organisers had practical support, and that online quizzes could have a consistently high level of quality.
+It was helpful to be able to work closely with STACK developers when designing quizzes, in a way that is difficult with non-open-source programs. For example, the "numerical" input type which helps students enter answers at a pre-specified level of numerical accuracy, was designed to help cope with the difficulties of assessing vague student answers in statistics questions [3].
+The School of Mathematics continues to expand the use STACK at the University of Edinburgh. Future challenges involve modifying STACK to suit more conceptual courses, such as group theory and real analysis, as well as expanding into statistics and computer programming using the CodeRunner system [4]. There are also plans for wider use of STACK in summative assessments, with the possibility of using it in online examinations.
+[1] Patterns and trends in UK higher education 2018. Universities UK, September 2018. ISBN: 978-1-84036-409-5.
+[2] C. J. Sangwin and K. Zerva. Developing online learning materials to support undergraduate education at the University of Edinburgh. Mathematics Today, 2019.
+[3] K. Zerva. Developing STACK assessments in Edinburgh, 2017-2019. In Contributions to the 1st International STACK conference 2018 in Furth, Germany. Zenodo, 2019.
+[4] R. Lobb and J. Harlow. Coderunner: a tool for assessing computer programming skills. ACM Inroads, 7(1):47{51, March 2016.
Interview with George Kinnear and Richard Gratwick
+STACK is used in the completely online course "Fundamentals of Algebra and Calculus". The course is designed to prepare students for Higher Education (HE) study at the University of Edinburgh, where incoming students have a wide range of mathematical backgrounds. The main barriers were the risk of students using online answer engines to answer questions, and the lack of community. The organizers attempted to address this by clever question design and creating "autonomous learning groups". The course was enabled by University support, both in the form of a dedicated learning-technologist post and assistance from colleagues. Subsequent diagnostics test results indicate this course was successful.
+Approximately six hundred students typically study first-year mathematics courses at the University of Edinburgh. They have a wide range of mathematical backgrounds and attainments. "Fundamentals of Algebra and Calculus" (FAC) was introduced as an additional course, approximately covering SQA Advanced Higher Mathematics and A-Level Further Mathematics, to address the needs of this diverse group. Since +some incoming students do not have the chance to take these higher qualifications before coming to university, the course fulfils a role as part of the University’s Widening Participation strategy, enabling access to mathematics courses for a wider range of students, e.g. students admitted through contextual admissions. This also was an attempt to address a concern from the University that one reason for undergraduate non-continuation was a lack of preparation for the mathematical components of students’ degree programmes, particularly those students on non-mathematics degrees within the College of Science and Engineering. Increasingly, the second-semester year 1 "Calculus and its Applications" course had been adapted to address these problems. However, staff and students felt that it recently contained too much material. FAC aimed to relieve some of the pressure on this course.
+FAC was delivered as a completely online course to make it scalable, since the above issues are not unique to the School or the University. FAC plays a role in addressing the so-called “mathematics gap” of attainment, preparing students from a wide range of mathematical backgrounds for HE study, and as such could easily see demand both across the University and beyond.
+FAC is essentially a course in mathematical techniques, assessing routine computation. STACK precisely lets teachers assess such questions automatically and give tailored feedback to students. Quite sophisticated questions can be implemented, for example “give an example” type questions, asking students to provide examples of e.g. a quadratic with given roots, or a sequence with certain monotonicity or boundedness properties. These are important for "retrieval practice", that is, encouraging students to recall previous topics.
+The course is worth 10 ECTS credits, and consists of approximately 200 hours of work for the student. The course is delivered in ten weekly units, each comprising a number of quizzes intended for students to work through during that week. Each week ends with a 90-minute "Practice Quiz" the student can take an unlimited number of times, and a 90-minute "Final Test" only allowing one attempt. The test is either given a grade of fail (0-80%), mastery (80-95%) or distinction (95%-100%). The high pass mark encourages students to master a topic before moving on. The final grade is determined by combining the results of the 10 "Final Tests" (worth 80%) with a final 2-hour test covering the whole course (worth 20%).
+The development involved a substantial up-front time investment: writing the material in these quizzes, including over 900 STACK questions. Although some existing questions were borrowed from other courses, the majority of these questions were authored from scratch. The quizzes interleave textbook-style exposition with videos of worked examples, interactive applets and practice questions, and as a result, the questions have a polished and varied look. Two lecturers were involved, each responsible for five units, with final cross-checking and occasional extra contributions.
+The design of the course was heavily influenced by educational research. Firstly, the course uses "faded worked examples", that is, presenting a sequence of problems with different amounts of the solution already worked out. For example, a topic might begin with a full worked solution of a problem, followed by a worked solution with the final step as a STACK input, followed by a worked solution with the last two steps as STACK input, and so on. There is evidence that this is helpful for students [2].
+Additionally, given the literature on the importance of "retrieval practice" [3], FAC gives plenty of chances for students to recall what they have learned, for example by having quizzes draw on skills from previous weeks. Weeks alternate between algebra and calculus, spacing out practice of the two topics.
+Furthermore, the lecturers decided not to make feedback for the final tests available to students until after the deadline, since there is evidence suggesting delaying feedback is helpful for learning [5].
+Finally, FAC gives students a chance to create strong "example spaces" [4], that is, sets of examples that a student can recall for a given topic. Example spaces are developed through "give-an-example" style questions, which are easy to write in STACK.
+FAC uses a lot of graphs and interactive applets to make questions visually appealing. Visual intuition is an important part of learning about functions, so it is appropriate for graphical components to play a significant part in this course. FAC uses both JSX graphs (which can be coded directly in STACK) and external plugins, like GeoGebra. The advantage of JSX graphs is that they are self-contained; there is no need to worry about changes to external programs, or licensing rights. There is, however, a large cost-benefit analysis to consider. JSX graphs are time-consuming and have a steep learning curve, while external applets are often quick to set up and have simple drag-and-drop features.
+The six hundred students taking the first-semester year 1 "Introduction to Linear Algebra" course sat a +diagnostic test (also delivered through STACK) in September 2018. In January 2019, the students sat the test again with the same questions but different random variants. In the September test, students also enrolled on FAC scored on average fifteen percentage points lower than their peers. This is not surprising; FAC will have been recommended to students who felt like they needed more practice in mathematics fundamentals. However, in January after studying FAC, they had gained these fifteen percentage points and were scoring in line with their peers. This is quantitative evidence that FAC has done exactly what was hoped: removed the discrepancy in attainment between the two groups of students.
+The lecturers also wanted to know if taking FAC had improved student performance in other maths courses such as "Introduction to Linear Algebra" (ILA) and "Calculus and its Applications" (CAP). Of students scoring similarly in the September diagnostics test, students who took FAC were scoring better than their peers in online quizzes for CAP. They also scored at a similar level to students who did not take FAC in the exams for ILA and CAP.
+While it is true that authoring a simple STACK question is ultimately straightforward, there was nonetheless a technological hurdle to be cleared. One lecturer was entirely new not only to STACK, but to Maxima and any form of online assessment, and therefore was confronted with quite an intimidating prospect. However, they ended up authoring hundreds of questions without trouble, and are now confident in undertaking significantly more demanding questions. This hurdle was easier to overcome given STACK requires no programming expertise.
+The lecturers were mindful of the possibility that students could be working through assessed quizzes in one browser tab, with a tool like WolframAlpha open in another. While ultimately this does no benefit to the student, and it remains their responsibility to ensure submitted answers are their own work, the potential of this kind of abuse should be minimised. This just required a bit of cunning at the level of question-setting: turning “find the derivative of this given function at this given point” into “at what point is the value of the derivative of this given function this given number?”
+The limitations of an online course include the difficulties of forming meaningful staff-student and student-student interaction. The lack of real staff-student interaction made it hard to gauge whether the level and volume of material was suitable – anecdotal feedback indicates this was misjudged in at least one of the weekly units. Student-student interaction is important to foster a sense of community. This was addressed by setting up “autonomous learning groups” in which the students could study, in person, once a week, without staff input. Furthermore, students could ask questions in the online forum Piazza, and get in-person help in the MathsBase study room if needed.
+The University had created a dedicated learning-technologist post to help design the online assessment and write STACK questions. Their work was invaluable for authoring some of the questions, often given only a one-line sketch of the desired question.
+The capacity in STACK to clone questions and export questions from other courses was also very useful, making it possible to produce multiple related questions, sharing, for example, a grading structure, without duplicating any work.
+Finally, it was an advantage that there is substantial “in-house” support at the University, since STACK is now used across the entire first-year curriculum and increasingly beyond. Hence, there are a number of members of staff with experience of authoring questions, and the students are familiar with it.
+A lot of data has been collected in FAC's first year. Quiz responses will be analysed to determine if any questions should be modified. There are also plans to have interviews with students on the effectiveness of the course, which will help identify areas that need work.
+The course organiser will be looking at improving the "autonomous learning groups". Since groups were given weekly tasks and asked to upload solutions to a shared Dropbox, the lecturers can see how much students interacted with these groups. Initially, the engagement was very strong, but half-way through the course, a lot of groups seemed to stop meeting. The course organiser will look at ways to improve turnout for these groups.
+The course continues to be worked on. The lecturers are considering options for more interactive questions, for example using equivalence reasoning to assess line-by-line arguments, or interactive JSX graphs that ask a student to, for example, "drag a vector so it is perpendicular to another vector".
+The success of FAC paves the way for similar methods to be used in different courses. Other schools have shown interest in creating similar online courses, and some students may find it helpful to have an online course teaching even less advanced maths.
+[1] George Kinnear. Delivering an online course using STACK. In Proceedings of the STACK Conference, 2018.
+[2] A. Renkl, R. K. Atkinson, U. H. Maier, and R. Staley. From Example Study to Problem Solving: Smooth Transitions Help Learning. The Journal of Experimental Education, 70(4):293-315, 2002.
+[3] H. L. Roediger and A. C. Butler. The critical role of retrieval practice in longterm retention. Trends in Cognitive Sciences, 15(1):20-27, 2011.
+[4] P. Goldenberg and J. Mason. Shedding light on and with example spaces. Educational Studies in Mathematics, 65(2):183-194, 2008.
+[5] H. G. Mullet, A. C. Butler, B. Verdin, R. Borries, and E. J. Marsh. Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition, 3(3):222-229, 2014.
Yasuyuki Nakamura
+Sangensha LLC., Japan
+Takahiro Nakahara
+Typing mathematical expressions into mobile devices can be time-consuming. To solve this problem, developers at Nagoya University are developing a "flick interface" for STACK, similar to the popular Japanese keyboard mode. Students can pick a common integer or variable and flick in a direction to quickly turn, for example, into . A survey of usability suggests students prefer this input type to their traditional keyboard.
+As smartphones become increasingly used in schools and universities, it is important to find a reliable way to input mathematical expressions. Inputting mathematics on a computer is not a problem, but on smartphones and other mobile devices it can be much more time-consuming. For example, to type an answer to the question "Expand (x+2)(x+3)", students have to enter the expression x^2+5*x+6
into the answer space. On a mobile device, this requires several switches between the alphabetical and numerical views of the keyboard, resulting in 19 key touches for this simple answer.
Recently, a flick keyboard for Japanese characters has become very popular in Japan, especially amongst young students. Japanese has three character systems. Hiragana is mostly used for simple and native Japanese words, katakana mostly for foreign words and kanji for all other words. The keyboard shows only a limited number of hiragana[^hiragana] characters, but students can hold and "flick" on a key to choose a similar character. Characters are grouped together by sound, for example grouping together na, ni, nu, ne and no. The keyboard will then suggest related kanji and/or katakana characters that share similar sounds. It was the popularity of this keyboard that motivated the developers to create a similar interface for inputting mathematics, as a STACK input type.
+The flick interface was developed by Yasuyuki Nakamura and Takahiro Nakahara, funded by a grant from the Japan Society for the Promotion of Science. Like the Japanese character keyboard, students are faced with a small set of common inputs. Students can switch between numerical values and common variables by pressing "123" and "xy", respectively, or see a full keyboard view by clicking the keyboard logo. Pressing on a number reveals some common manipulations, and students can then flick in one direction to turn, for example, "2" into "2x". The input is compatible with normal keyboards, and will automatically be selected for the type of device being used.
+The interface is programmed in JavaScript to minimise the dependency on mobile device operating systems. The interface uses MathDox to describe expressions, and the input is then converted to the Maxima format by a conversion filter previously created by the developers. Developing an interface suitable for all different screen sizes was one of the biggest challenges of the project, but now the interface is suitable for most devices of sizes around 5-6 inches.
+The flick interface is estimated to significantly reduce the number of keypresses required to type an expression. The following table shows an estimated comparison for a few common expressions [1].
+Mathematical expression | +Direct input taps | +Flick input taps | +
---|---|---|
+ + | +19 | +8 | +
+ + | +36 | +13 | +
+ + | +23 | +7 | +
To analyse the usability of the interface, the developers conducted a usability test at Nagoya University, Japan. 29 students where asked to enter five common maths expressions by using both a traditional keyboard and the new flick input keyboard. After they entered these math expressions, they were given a survey on usability and satisfaction levels. The survey questions were based on the following five parameters, originally from Jakob Nielsen’s five goals of usability [2]: learnability, efficiency, difficulty or ease in making corrections, rememberability, and the intent to reuse. For each parameter, students were asked to give each input type a score from 0.0 to 5.0. The following table shows the results [1], including both the averages, as well as the standard deviations in parentheses. The survey suggests that the usability and satisfaction levels are higher when the flick input method is used to enter mathematical expressions, rather than the direct input method.
+Question | +Direct input score | +Flick input score | +
---|---|---|
It is easy to learn how to input math. | +3.0 (1.2) | +3.5 (1.0) | +
I can input math quickly and easily. | +2.6 (1.1) | +3.2 (1.3) | +
It is not confusing and easy to correct. | +3.0 (1.2) | +3.1 (1.1) | +
I remember the method that I learnt at the rehearsal. | +3.0 (1.1) | +3.1 (1.1) | +
I will use this method to input math the next time | +2.9 (1.3) | +3.2 (1.4) | +
The next step for the developers is to merge the flick interface into STACK as a new input type. This will include refactoring the flick interface code and working with STACK developers to best implement it into the question type.
+[1] Y. Nakamura and T. Nakahara. A new mathematics input interface with flick operation for mobile devices. 15(2), 2017.
+[2] J. Nielsen. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993.
Interview with Fred Neumann and Jesus Copado
+STACK was integrated into the ILIAS learning management system to support projects for learning content. This was done by creating an integrated question type directly in ILIAS. The biggest challenges were decoupling the dependencies on Moodle, and an increasing number of user queries following a quickly growing community.
+ILIAS is one of the most popular Learning Management Systems (LMS) used in German universities. It has been available under an open-source license since 2000, and since then has grown in popularity in a number of countries, especially Germany and Switzerland. In the last decade, the ILIAS community started a special-interest group related to mathematical assessment. The main driver for this was the "optes" project, a federally funded project to create open learning content for mathematics and natural science. It was key for this project to have a sophisticated question system that uses CAS to evaluate students' input. In 2013, the group compared different systems for CAS based questions and finally decided to use STACK.
+STACK was initially built for the learning management system Moodle, and it would not be trivial to implement STACK into ILIAS. A crowd funding initiative was started to port the STACK plugin from Moodle to ILIAS, and the Innovation in Learning Institute at the Friedrich-Alexander-University Erlangen-Nuremberg (FAU) was asked to implement and maintain it. This would also be an advantage to FAU, as it could then establish STACK as an innovative element of e-learning and e-assessment at the University.
+It was important to create an integrated question type that works like all the others in ILIAS, instead of trying to dynamically connect a separate STACK platform to ILIAS. Initially, a 2-day meeting with lead STACK developer Chris Sangwin was held at Loughborough University in December 2013, where the basic architecture and principles of the question type were laid out. In the following months, the ILIAS question type was created.
+The developers set up a Moodle installation with STACK, examined the code and analysed how it works. Then the ‘library-like’ Moodle core was extracted and integrated into the template of an ILIAS question plugin. The techniques used followed the open source nature of STACK and were supported by the fact that STACK uses the same technology (PHP/mySQL) as ILIAS.
+Initially, the only way to add questions was to create them in Moodle, then import the moodleXML file into ILIAS. However, authoring provisions and an authoring interface were later created in ILIAS, using the GUI classes.
+From the beginning, the ILIAS community has shown a great interest in using the integration of STACK. To date, nine separate institutions have participated in the crowd funding of the plugin. Over time, several workshops have been organized by the community. A German user manual and workshops to teach the creation of STACK questions was created at FAU and ported to an online module by the University of Göttingen. In 2018, the first international STACK conference was held at the Innovation in Learning Institute, bringing together around 65 participants from the Moodle and ILIAS communities.
+As hoped, STACK is now being used extensively at FAU for innovative e-assessment. The ILIAS installation at FAU contains around 3000 questions from over 100 different authors. Additionally, a local ‘STACK user group’ was founded with 40 members from various disciplines, including mathematics, natural sciences, engineering, and economics.
+Based on external and internal funding, it is possible to permanently provide new major versions of the plugin for each major ILIAS release every year, as well as several minor bug fix versions in between. The core of STACK used by the plugin is also updated every year. The ILIAS questions are fully compatible with the Moodle version and can be exchanged between the two Learning Management Systems through the moodleXML format.
+The biggest challenge for the initial integration of STACK into ILIAS was the decoupling of all dependencies from Moodle in the STACK core. Currently, around 50 patches are maintained for that. Furthermore, there were major differences in ILIAS and Moodle regarding the definition and display of feedback and various test player options. The completely different GUIs of Moodle and ILIAS meant the whole authoring interface of STACK had to be redesigned.
+When the community of STACK users in ILIAS grew, the number of bug reports related to exotic edge cases of STACK usage also grew. This could, for example, be issues relating to questions with a huge number of inputs. Furthermore, an increasing community of users brought in very different opinions about the styles of a question, its validation and feedback. Dealing with these issues was a significant challenge.
+A barrier that had to be evaluated is time. The initial work in integrating STACK into ILIAS took significant effort, and the project has to be continually maintained as bug fixes and project features are applied. It is estimated that, on average, 70% of a full-time employee's time is required to maintain the STACK developments for ILIAS. This is currently partially funded by the ILIAS community and the Innovation in Learning Institute.
+Server performance was an issue when it came to using STACK in large institutions. Maxima was designed as a desktop application, and so lacks a set of “good practice” recommendations for hardware and configuration options depending on the expected load. To solve this, there are some initiatives such as the GoMaxima project from HAW Hamburg and an initiative at the University of Göttingen for a docker swarm of Maxima pools.
+The meeting with the lead STACK developer was a major help in getting the project off on the right track. The comprehensive STACK documentation was also invaluable for understanding the question type.
+Additionally, the support from the ILIAS special interest group was invaluable. This group helped sketch the authoring interface in ILIAS, write documentation and organise the first workshops. It also helped moderate the discussion of feature requests from the ILIAS community and organise crowd funding for the maintenance of the plugin. Furthermore, since the Innovation in Learning Institution is not a mathematics institution, it was invaluable to get help from mathematicians to write good demonstration questions.
+Finally, the port was supported by the common technology of ILIAS and STACK and the fact Moodle and ILIAS have very similar quiz structures, and both treat STACK as a question plugin type. Implementation differences aside, the main principles are the same.
+The ILIAS question type will continue to be updated as STACK, Moodle and ILIAS get updated. In particular, the way question types in ILIAS are implemented will change in a future ILIAS patch, which means the question type will have to be adapted accordingly. There are also plans to make STACK less dependent on the Moodle library. If this is successful, the ILIAS plugin will have to be largely rewritten (since it depends on the Moodle library), however the developers are optimistic. From writing the question type the first time around, they have gained invaluable experience. Furthermore, the authoring interface will not have to be rewritten.
+There are many wishes from the ILIAS community for new features to be implemented. Some of these are step-wise feedback, as well as more control of the feedback style, for example the colour of feedback boxes. These may be added to the STACK question type in the future.
+Finally, the maintenance relies heavily on crowdfunding managed between institutions (typically the Innovation in Learning Institution and another University). This means contracts have to be written between the heads of each institution, which can be difficult to manage. There are therefore considerations to transfer the maintenance of the question type to an ILIAS service provider, as it would then be easier to get crowdfunding contracts from Universities.
Ian Jones
+At Loughborough University, STACK has been implemented across many disciplines. Courses are created by module leaders, who prepare paper-based problems, and academics and students who implement these problems in STACK. To promote STACK to lecturers across the University, a showcase module page was developed that makes it easy to navigate and browse good STACK examples across disciplines.
+STACK was installed at Loughborough University in 2014. Since then, a programme of development and dissemination led by Loughborough’s Mathematics Education Centre has resulted in widespread use across the University. The University has encouraged multidisciplinary take up beyond the mathematics department, and now 57 modules across six degree programmes include STACK content. STACK is used for assessment in topics including mathematics, business, statistics, chemical engineering, physics and foundation programmes.
+Initially, questions were developed in conjunction with module leaders. Typically a module leader provided the developers with paper-based problem sheets. The developers, who included academics as well as PhD and undergraduate mathematics students working on projects, adapted the problems into STACK questions. The example below illustrates how the developers paid special attention to making use of STACK’s unique feedback feature to provide personalised feedback to students.
+Alongside development, the STACK content was promoted and disseminated to lecturers across the University via internal workshops. This included workshops to train module leaders in developing their own content. Additionally, a module page was developed, viewable to all teaching staff, that acts as a showcase for STACK content.
+The showcase module page is designed to be easy to use and navigate, even for beginners. This is because newcomers often find navigating, selecting and exporting questions off-putting. To overcome this, all questions within a topic have been wrapped into a downloadable .zip file, and only a few navigable example questions have been provided. The download screen for business-based STACK questions is shown below. Business questions have been a particular success, and a new project to develop further STACK content for the School of Business and Economics begins in Autumn 2019.
+STACK development continues at Loughborough. The University continues to be particularly interested in promoting STACK in areas where it is not commonly used. This development includes developing STACK content for degree programmes beyond mathematics, such as statistics for psychologists, as well as areas within mathematics that are traditionally difficult to assess automatically. A particularly exciting area being explored is assessing proof comprehension, and an example prototype question is shown below.
+Santiago Borio, IDEMS
+Michael Obiero Oyengo, Maseno University
+IDEMS International is working with Maseno University to implement online assessment for their mathematics courses. Developers from IDEMS and Maseno University created online quizzes for two courses during the pilot of the project, running on servers at the University of Edinburgh. The main challenges of the project were related to poor student access to WiFi and time pressure for the question authors. The feedback from the students was mostly positive, and the pilot paves the way for more similar work in the future. This includes online assessment for many more courses at Maseno University, as well as collaborations with other African universities and secondary schools.
+IDEMS International (Innovations on Development, Education and the Mathematical Sciences) is a Community Interest Company working to support different development causes, mostly in Africa, in fields of education and mathematical sciences. The non-profit organisation generates open education resources and offers services to Universities and other institutions. One of their projects was to help implement Computer Aided Assessment (CAA) in African Universities, of which Maseno University was chosen as a pilot.
+Maseno University, Kenya, has a large number of students in its undergraduate mathematics courses, many courses having 800 enrolled students and some more than 1000, but there is little support for lecturers in terms of marking. This puts pressure on their ability to reliably do weekly assessments for students. By changing the continuous assessment from a paper format to an electronic system, the hope was that lecturers would have more opportunities to concentrate on their teaching and supporting students in other ways.
+There were a number of reasons for why STACK was chosen for this project. Firstly, there was already some familiarity with the STACK system within IDEMS, as one of the contributors had previously worked with STACK question authoring. Secondly, the main STACK developer offered assistance, including the opportunity to run the service on servers at the University of Edinburgh. Finally, having compared STACK with other systems, the developers found that the flexibility in answer tests, recognition of mistakes through potential response trees and the ability to have questions that require numerical or algebraic input for answers made STACK more powerful than the alternatives.
+The developers from IDEMS and Maseno University created online STACK quizzes for the courses "Calculus I" and "Introduction to Linear Algebra". In the case of "Calculus I", the main lecturer provided a written quiz structure and list of questions, so the main work was just typing these into STACK and implementing randomisation. In contrast, the quizzes of "Introduction to Linear Algebra" were written mostly from scratch, only using a few questions from the STACK sample materials.
+Initially, students are given a syntax quiz to learn the STACK syntax, and then in each of the following 10 weeks they are given a formative "mastery quiz" and a summative "test quiz". The mastery quiz can be taken as many times as the students wants, but they must get a score of 80% in the mastery quiz to unlock the respective test quiz, where students are only given one attempt. In "Introduction to Linear Algebra", this restriction was later reduced to 70%, as lecturers felt students were having a hard time keeping up with materials. The mastery and test quizzes together count for 30% of the students course grade. Towards the end of the course there is also a review week with STACK questions, and in "Introduction to Linear Algebra" students can take an online practice exam.
+After the first year, the students of the two courses were asked to fill out a survey about their experience with STACK. The first few questions regarded accessing the course. Maseno University does not have reliable WiFi, and it is not common for students' accommodation to have WiFi either. As a result, most students had to use either their own or a friends' phone, and purchase a data package to access the course. Indeed, when asked to rate the difficulty of different aspects, many students ranked "accessing the internet" in the range of "slightly challenging" to "very challenging".
+When asked to rate how helpful they found different aspects of the assessments, many students ranked getting feedback, being able to have multiple attempts at mastery quizzes, having weekly quizzes and having the mastery quizzes open all semester as useful. Most students did not seem to have too much difficulty with the STACK syntax, ranking it in a range of "not a challenge" to "moderately challenging". This was surely helped by the attempts to ensure students were learning the syntax, for example through the introductory syntax quiz. However, it also seems there is room for improvement here.
+The students were also given an opportunity to add any general comments or suggestions. The majority of these answers were from students who were happy with STACK and said it was useful for getting a deeper understanding of the material.
+With regards to quizzes, many students requested more than one attempt at test quizzes, with extended deadlines. They requested more mastery quizzes, suggesting they found them helpful, and some students suggested access to harder, exam-style questions.
+There were also some comments about the feedback for questions. Due to time constraints, the calculus course only had feedback in the form of worked solutions, and for linear algebra, students were only given outlines to solutions. Students said that they wanted better feedback, and so one of the goals for the next iteration will be to add tailored feedback to all questions.
+Finally, there was a call to improve accessibility for the visually impaired.
+As is evident from the survey responses, there was a challenge regarding the students' access to internet. However, this did not end up being as big a problem as anticipated, as students seemed able and willing to pay for data packages to access the course.
+Furthermore, due to the short term allocation of courses and lecturers, there was a significant time pressure for writing the quizzes. As a result, there was no time to implement tailored feedback, and a number of quizzes were delivered late. However, this did not seem to discourage students or create major issues in the completion of quizzes.
+A major enabler for the project was the ability of IDEMS to assist the University in question authoring. The questions were written by Michael Obiero Oyengo and David Ambogo from Maseno University and Santiago Borio, Danny Parsons and David Stern from IDEMS. Additionally, the main STACK developer C. Sangwin helped give crash courses in using STACK, which sped up the learning process. The course was also run on a server at the University of Edinburgh, which meant the developers did not have to worry about setting up a server and could focus entirely on question authoring. Finally, the STACK sample questions helped speed up the question authoring process, as the developers could use them as templates when writing similar questions.
+Following the success of this pilot course, IDEMS is looking to expand into similar projects. The first step will be improving the two initial courses. Once the analysis of the data from the pilot course is finished, it will be possible to identify the main areas that need work. These two courses have been transferred to an IDEMS server, so they can be easily used for online assessment at other universities. The server will be built to cope with universities with many students, as IDEMS expects to use it for some courses with more than 1000 students.
+A number of Universities have expressed interest in this project, including universities from Kenya, Ethiopia, Rwanda, Tanzania and Uganda. In particular, the mathematics department at Bahir Dar University, Ethiopia has agreed to investigate implementing a number of IDEMS' courses. In August 2019, IDEMS is holding a workshop, where representatives from these universities will discuss STACK and how to best implement e-assessment at their institutions. The 5-day workshop will include talks on the advantages of using online assessment, as well as practical work to prepare lecturers who want to use IDEMS' online course material. The workshop is funded by a grant from the Commission for Developing Countries (CDC) of the International Mathematical Union.
+IDEMS is also helping Maseno University create online materials for several of its other courses. In the 19/20 term, Maseno University will run the improved "Calculus I", as well as five other courses with online assessment: "Descriptive Statistics", "Introduction to Probability Theory", "Basic Mathematics", "Calculus II" and "Vector Analysis". For these courses, IDEMS will be training some lecturers to author their own questions, which will increase their control of the online quizzes. As with the two pilot courses, these courses are planned to be piloted at Maseno University first, then refined based on feedback and made available as an OER for other universities.
+Finally, IDEMS is looking at using STACK in secondary schools. There are plans to develop an e-book for African schools with accompanying STACK questions, including an offline system for schools with poor WiFi. IDEMS is also evaluating opportunities to develop an open assessment system for English schools.
David Hilsen
+The commercial textbook publisher, Physics Curriculum & Instruction have developed STACK questions to accompany a Physics textbook [1]. These make significant use of random variants and intermediate problem-solving steps. It was important to have feedback on problem design from academic colleagues, and to have support for significant figures and scientific units in STACK. The question bank will contain 3500 questions by the start of the 2019/20 school year.
+Physics Curriculum & Instruction is a Minnesota-based company providing schools with learning resources for Physics. With a lot of open-source options becoming available, putting resources into developing a commercial textbook can carry a significant risk. Physics Curriculum hopes to address this by directing their development efforts towards online physics educational resources. It was important to have a system that allowed them to address the wide variety of needs of instructors and that utilised randomization of numerical values. This makes it nearly impossible to search online for the answer to a particular problem; a major concern for publishers.
+A large collection of online STACK questions were developed to accompany the book Physics Fundamentals [1]. Access to the online homework package can be purchased along with the book for an extra fee. The online questions are managed in the online learning environment Physics LE, which runs on Moodle. Moodle was picked for its Enrolment key feature and its Adaptive mode question behaviour that allows students to make multiple question attempts with immediate feedback.
+The team had four people authoring STACK questions, and two additional people working on Moodle and the cloud servers. Authoring questions required an average of just over one hour per question with a significant portion of that time devoted to quality control and testing. Ultimately the development sums to a significant financial investment, but also one that leaves an end product of high quality.
+A significant focus in the authoring of questions was on intermediate problem-solving steps, and which ones would be most beneficial to help the student formulate a problem-solving strategy. It was important to go beyond a system in which students simply enters a final numerical value that is marked right or wrong with no additional feedback. The intermediate problem-solving steps used are of the following type:
+The service went live in January 2019. Since then, 15 schools, both high schools and colleges, have urchased the online question package. Instructors gave positive feedback, especially appreciating the multiple-part question structure where students are given specific feedback.
+Initially, there was no provision in STACK for handling significant figures, and limited support for physical units. This was a big barrier for Physics question authoring, and hence a more robust system was developed for this purpose. Additionally, high school students sometimes struggle with the syntax for equation input, especially those with minimal computer background. This continues to improve within STACK and may not be a concern in the future.
+Instructors and colleagues provided feedback from an outside perspective indicating where students might have difficulty. This encouraged the authoring of additional PRT nodes, for example checking for a particular misconception.
+There are many additional schools evaluating the Physics Curriculum online homework system, and additional schools are expected to join. Furthermore, the question bank continues to grow. By the start of the 2019/20 school year, Physics Curriculum plans to have 3500 STACK problems in their system, many of these multiple-part. There are also plans to partner up with OpenStax, a company producing high quality College Physics textbooks, to bring them an affordable online homework solution as well.
+[1] V. P. Coletta. Physics Fundamentals. Physics Curriculum and Instruction Inc., Lakeville, +Minessota, 2nd edition, 2010.
Tim Lowe, School of Mathematics and Statistics, +Tim Hunt, Information Technology.
+The Open University is the UK’s largest academic institution. In 2017/18 there were approximately 175,000 students studying with the University, mainly part time, making approximately 65,000 full-time equivalent students. Students combine their study with work, family, caring and other responsibilities. Students study from home, at a distance, guided by the University’s Moodle-based VLE and using a combination of online and printed materials. Students are supported by a tutor who provides individual support to a group of typically 20 students and usually offers a combination of face-to-face and online tutorial support.
+The School of Mathematics and Statistics offers a number of undergraduate qualifications in mathematics (with a total intake of approximately 1300 each year) and a taught MSc, which is the largest such course in the UK. In addition, students throughout the University can take one or more mathematics modules in support of their main subject of study. There are approximately 13,000 student-module combinations within mathematics and statistics each year.
+STACK is an ideal tool to support the distance-learning of mathematics, as it enables students to practice key techniques from home whilst receiving immediate feedback on their answers. Students can attempt different randomised variants of questions to support their learning and development.
+STACK was first used in the curriculum in 2014 with the launch of a new introductory calculus module: MST124 "Essential Mathematics I", which is currently studied by approximately 2,800 students per year. Since then, STACK has been taken up by many modules at all levels. The main use is for formative “practice quizzes” allowing students to practice the important techniques taught. At lower levels, it is also used for summative assignments, which both encourage students to use the formative quizzes to prepare for the assignment and practice their mathematics, and help students keep on pace with their study by providing deadlines. At the postgraduate level, a series of linked-STACK questions has been developed to guide students through more complex mathematical arguments in the calculus of variations [1].
+The School of Mathematics and Statistics have been supporting colleagues in the School of Engineering and Innovation who use STACK to support their teaching of mathematically-based topics. The School is also using STACK to support students between formal module study, and in preparation for study as a key component of online “Revise and Refresh” support materials.
+STACK is currently used in at least 10 modules providing at total of 330 CATS credits and reaching over 6,500 students annually. Over one million STACK questions are answered by students each year, which is approximately 18% of all quiz questions answered by OU students.
+STACK is supported by the University Information Technology department as part of the Moodle installation. This consists of 11 load-shared Apache Web Servers supported by 2 MaximaPool servers to service the use of Maxima by STACK. Despite the high level of use, STACK has been robust in performance with no unplanned service outages during the period of use. All question authors are strongly encouraged to include Question tests as part of each STACK question, which helps ensure that the integrity of the questions is maintained over time. Before each STACK upgrade, IT checks the question tests of all STACK questions to ensure none are affected by the upgrade. If any tests fail, the questions are corrected by the appropriate module staff before the upgrade is applied to the live system.
+Students’ engagement with the quizzes is often demonstrated by comments posted in online forums, for example questioning details of the worked solution provided. Students are often not content with just being able to answer every question in a quiz, but continue to try to correctly answer the quiz in the minimum time.
+Student feedback on the use of STACK quizzes has been positive, with many choosing to praise them in the unprompted open comments of the university module feedback survey. Conversely, negative comments are received where a module does not have STACK quizzes. Typical student comments include the following:
+The School currently plans to introduce STACK quizzes into additional modules as they are updated. Future plans include extending the use of STACK to further, higher level modules and to expand the use of STACK to support students in checking their preparedness for the study of various modules.
+[1] T. W. Lowe and B. M. Mestel. Using STACK to support student learning at masters level: a case study. Teaching Mathematics and its Applications: An International Journal of the IMA, 2019.
Miriam Weigel, Katja Derr, Reinhold Hübl
+The optes project uses STACK in their pre-course, designed to help students improve their self-studying skills for their first year at university. The tests were developed and maintained at DHBW Mannheim. Here, students take a diagnostic test, and based on their results are given access to various learning modules, comprising text, graphs, animations, examples and exercises. An analysis of data gathered from the first evaluation phase of the project shows that the pre-course is effective at bridging gaps in school knowledge: at-risk students who end up scoring highly in the pre-course show improved performance in their first year mathematics courses.
+Students entering higher education are a diverse group and many students have considerable gaps in school knowledge, putting additional pressure on their first year at university. This particularly applies to mathematics, as basic skills in this subject are considered a prerequisite to successfully complete a course in STEM subjects. As a consequence, many universities provide preparatory courses in mathematics, either face-to-face, web-based, or in blended learning scenarios.
+The joint research project optes, funded by the German Federal Ministry of Education and Research (BMBF), develops and evaluates learning materials and tools that help students “refresh” their basic knowledge in mathematics and support the development of learning strategies. optes stands for “Optimierung der Selbststudiumsphase”, which translates to "optimisation of the self-study phase".
+Over the course of eight years, optes developed a comprehensive web-based pre-course consisting of diagnostic tests, learning modules and concepts for web-based student support, and tested it at the participating universities. The partner universities include Baden-Wuerttemberg Cooperative State University (DHBW) Karlsruhe, DHBW Mannheim, DHBW Mosbach, University of Applied Sciences and Arts Ostwestfalen-Lippe, Universität Hamburg; Cooperating universities: Julius-Maximilians-Universität Würzburg, University of Education Heidelberg.
+The project put a strong focus on the development of self-tests that enable learners to apply their knowledge and provide them with meaningful feedback. This required a sophisticated CAA system, especially one that allowed students to enter an algebraic input. Since optes results are published under Creative Commons or General Public licenses, it was also important that the system was open source. To find the most suitable CAA system, the researchers analysed and compared many existing mathematical tools and in 2013, STACK was chosen. One of the reasons for choosing STACK, is that STACK uses the Learning Management System (LMS) Moodle and the Computer Algebra System (CAS) Maxima, both of which are supported by large communities. This gave the researchers confidence that STACK would also be used and developed further. Additionally, STACK is flexible enough to allow researchers to design very different types of mathematical problems. The only barrier was that STACK was not available for the LMS ILIAS at the time, a system widely used in Germany. However a crowd funding initiative was successful in gathering enough funds to implement STACK into ILIAS.
+The partner university DHBW Mannheim was responsible for the development of tests and self-assessments. Here, a very basic version of the pre-course was implemented in 2013, and based on repeated evaluations, the different tests and learning modules were successively built and improved upon [2]. While some learning resources already existed in the form of printed scripts and paper versions of tests, the complete course material had to be rewritten and typed into ILIAS. In 2019, the course held ten interactive learning modules, with more than 1500 mathematics test items, 140 of which use STACK.
+Figure 1 shows an overview of the course structure as executed at DHBW Mannheim since 2014. The programme runs in the months leading up to a new semester, and starts with a diagnostic self-test covering the entire pre-course's syllabus. See also recommendations by SEFI mathematics working group [6], and cosh [1].
+Depending on their diagnostics test results, students may access the different learning modules. Each course provides text, graphs, animations, examples and exercises, and at the end of each module, students can take a subject-related test consisting of 10 to 15 randomised questions. Students who want additional support can enrol in e-tutored courses, where their learning process is structured and monitored by e-tutors. Students can then discuss problems and test results with peers and e-tutors, and are required to upload completed exercise sheets.
+During induction week, all participating first year students take a final test at the University’s computer laboratories. Since 2014, more than 2800 students have participated in the diagnostic pre-test and the pre-course at DHBW Mannheim. This corresponds to approximately 560 students per year, which is around 80% of the cohort.
+While the diagnostic self-test aims at informing learners of their level of knowledge in relation to the curriculum, the self-tests provided in the ten different courses are designed to encourage learners to independently practice their skills.
+Figure 2 shows an example of an optes question in ILIAS. Students can click the checkmark next to the input box to have their answers validated by the system. On top of the question, the student can click to pop-out an explanation of the input syntax. The question is graded by clicking on “Rückmeldung anfordern”, which means "Request feedback".
+In this example, the student is asked to give an example of a function with three given zeroes. There are no other constraints, and in particular the function can also have more zeroes. The given zeroes are randomly generated, so the student can restart the test to try a variant of the problem with different constants.
+optes questions focus on providing good feedback to students [8]. Feedback incorporates partial marks, if the student for example entered a function with only one or two of the given zeroes, and visuals, by for example graphing the student's answer (in red) against a correct answer (in blue). The student is also shown the value of their function at the three given points and given a model solution. optes questions are great examples of effective use of STACK's feedback features.
+The major goal of the optes project was to improve students‘ self-study abilities in mathematics related subjects. Using data collected from the first evaluations phase of the project, the researchers studied the influence of taking the pre-course on different variables. They found that the biggest factor in study success was a student's previous knowledge and secondary school scores. However, they also found that the pre-course could be very effective at improving first-year performance. Of the students whose performance were at risk, those who scored highly in the pre-course also did better in their first year of study. These were the students with poor prior study success or little domain-related knowledge. Self-test engagement was also found to have significance, as they were related to both pre-course gains and first-year performance.
+A major advantage of learning management systems is their ability to provide automated feedback to students. Instructors are hence relieved from the load of marking hundreds of exercises, and students appreciate the immediate response of the system. However, writing feedback for web-based mathematical problems is time-consuming, and these efforts may grow exponentially when the questions have multiple correct answers. To help soften this burden, it is important to share STACK questions, not only across pre-course projects but also across universities and e-learning platforms.
+A major enabler for the project was the initiative to integrate STACK into ILIAS. If this crowd funding initiative had failed, optes may have had to choose a less suited CAA system.
+Furthermore, a lot of institutional support enabled this project. While all resources developed by optes are Open Educational Resources (OER), the implementation of the learning material at third party universities or educational institutions demanded staff and technical infrastructure to install ILIAS and STACK. Staff and technical support helped administer the pre-course and adapt the learning material to each university’s needs. Furthermore, the e-tutor system was enabled by lecturers and older students who took the time to learn to use STACK so they could help pre-course participants as e-tutors.
+Since 2019, the rollout of optes to other universities and institutions has begun, and as more and more lecturers use the optes resources, the STACK community in Germany continues to grow. This community is largely represented by the working group "Mathe digital". Lecturers are encouraged to develop and contribute their own questions, resulting in a growing and improving database of available STACK questions.
+The researchers continue to evaluate the data from the project. When this analysis is finished, it will be possible to better identify the strengths and weaknesses of the course structure.
+Like all projects funded by the German BMBF programme "Quality pact for teaching", optes will be finished by the end of 2020. The developed concepts and course material, however, will be used and developed further at the optes partner universities and at all universities and institutions that share the material. Future plans include incorporating adaptive testing and expanding the work to other subjects and topics, such as mathematics for business and psychology courses.
+[1] COSH Cooperation Schule Hochschule. Mindestanforderungskatalog mathematik. 2014.
+[2] K. Derr. Identifying consistent variables in a heterogeneous data set. Electronic Journal of e-Learning EJEL, 15(1):82-93, 2017.
+[3] K. Derr, R. Hubl, and M. Z. Ahmed. Prior knowledge in mathematics and study success in engineering. informational value of learner data collected from a web-based pre-course. European Journal of Engineering Education, 10(3):1-16, 2018.
+[4] C. J. Sangwin and I. Jones. Asymmetry in student achievement on multiple choice and constructed response items in reversible mathematics processes. Educational Studies in Mathematics, 94:205-222, 2017.
+[5] B. Alpers. A framework for mathematics curricula in engineering education. 2013.
+[6] M. Weigel, K. Derr, R. Hubl, and T. Podgayetskaya. Stack-aufgaben im formativen eassessment: Einsatzmoglichkeiten des feedbacks. Zenodo, 2019.
+[7] M. Weigel, K. Derr, R. Hubl, E. Mechelke-Schwede, and T. Podgayetskaya. Inhaltliche und technische aspekte des automatisierten feedback. einsatz des fragetyps stack im formativen eassessment. Beitrage zum Mathematikunterricht 2017, 1185-1192, 2017.
Helena Barbas
+https://orcid.org/0000-0002-2384-8042
+The MINTFIT Math Test is an online test for high school students, or anyone interested in a university STEM degree programme. It is provided by the MINTFIT project based in the universities HafenCity Universität Hamburg (HCU), Universität Hamburg (UHH), University of Applied Sciences Hamburg (HAW Hamburg), University Medical Center Hamburg-Eppendorf (UKE) and University of Technology Hamburg (TUHH) and funded by the Hamburg Ministry of Science, Research, Equalities and Districts (Behörde für Wissenschaft, Forschung, Gleichstellung und Bezirke Hamburg, BWFGB). The MINTFIT Math Test, as well as the MINTFIT online tests in physics, chemistry and informatics, run on a Moodle system. The STACK plugin offers many benefits to the Math Test, both when used at home and in different university settings.
+The MINTFIT Math Test is a free diagnostic online test that lets participants check their mathematics skills for a successful start to a STEM degree programme. It is part of the MINTFIT tests and courses programme on the subjects mathematics, physics, chemistry and informatics, accessible on the MINTFIT platform https://www.mintfit.hamburg/. Participants get instant feedback on their results after finishing the Math Test. They can then use two online mathematics bridging courses (OMB+ and viaMINT) to fill the gaps in their mathematics knowledge as identified by the test. The MINTFIT Math Test is available in German and in English. For more information see [1].
+STACK offers many advantages in the setting of diagnostics testing for both participants and organisers.
+A STACK question's potential response tree helps diagnose the errors committed by the participants, as it is possible to compare the entered solution with incorrect solutions that arise from known common mistakes. It is also possible to provide partial credit for partially correct answers, or for errors carried forward.
+Additionally, it is possible to generate many random variants of a question. If for example, the question is to find the derivative of a polynomial, the system can randomly draw its coefficients from a specified set. Variants that are either too difficult or too easy can then be deleted by hand. At the universities in Hamburg, the MINTFIT Math Test is used in exam-like situations as a part of university entrance tests, a module requirement or as an opportunity to collect bonus points for the first term exams. The possibility to generate different questions of the same difficulty level is invaluable here.
+The STACK questions in the MINTFIT Math Test were written by a scientific assistant (the author) and a student helper in 2014/2015, supported by an IT assistant. New STACK questions were easily generated by for example copying an existing question with a similar structure and looking up the coding structure of Maxima. This way, with just a small starting set of exemplary questions, it was possible to quickly get comfortable with STACK without any help from other sources or training.
+The questions in the MINTFIT Math Test can be done by mental arithmetic or by calculating using pen and paper – a calculator is not necessary and should not be used (neither at home nor in exam-like situations). However, since the general approach of MINTFIT and the Math Test particularly is the voluntariness of its users, participants are not controlled (with the exception of some settings, like the university entrance test, where they are seated in an exam room). However, they are informed before starting the test that their results are representative for their math skills only if they complete it without using a calculator or looking the answers up on a webpage like Wolfram Alpha.
+About 50 questions in the question pool of the Math Test are implemented with STACK. The MINTFIT Math Test has been accessible since June 2015.
+It is difficult to measure the precise impact of the MINTFIT Math Test on students' study skills. Nonetheless, there is student feedback from a form offered after the test. Of all those who answered the question: "Do you think that the test is helpful for you or freshmen in general?", 66% answered with a clear "yes". Answers were given in a range from 1 ("yes") to 5 ("absolutely not"), with a mean response of 1.5. Of all those who answered the question "Do you feel fairly rated?", 61% answered "yes". With possible answers as in the previous question, this one had a mean answer of 1.62.
+The main challenges of using STACK were of technical nature. Keeping up with the updates of Moodle and STACK means you have to keep a close eye on the appearance of the test. Moodle itself offers many possibilities, so one needs time to make the optimal adjustments there.
+MINTFIT is a project by the universities HafenCity Universität Hamburg (HCU), Universität Hamburg (UHH), University of Applied Sciences Hamburg (HAW Hamburg), University Medical Center Hamburg-Eppendorf (UKE) and the University of Technology Hamburg (TUHH) in Hamburg, Germany. It is funded by the Hamburg Ministry of Science, Research, Equalities and Districts (Behörde für Wissenschaft, Forschung, Gleichstellung und Bezirke Hamburg, BWFGB).
+The MINTFIT test and courses programme will be completed with regard to the content by the end of 2020. From 2021 on, the service is going to be maintained for current and future applications, and tests and courses will be refined and enhanced.
+[1] Barbas, H.; Schramm, T.: The Hamburg Online Math Test MINTFIT for prospective Students of STEM Degree Programs. MSOR Connections, S. 43-51, 2018.
Stephan Bach
+The Ostbayerische Technische Hochschule Amberg-Weiden (OTH - Technical University of Applied Sciences) offers extra-occupational mathematical bridging courses for non-traditional students with little prior knowledge. In a blended-learning scenario, the self-study component includes quizzes with STACK questions. These questions were implemented with a specific focus on the needs of non-traditional students, e.g. providing support for dealing with input syntax and using partial crediting. The questions use several of STACK's input types, including multiple-choice questions, which help establish successful mathematical practice. Both lecturers and students emphasize the importance of STACK for the program's success.
+In Germany, there are various opportunities to enrol at universities to study a vocational qualification instead of a higher education degree.¹ For example, people with certain professional degrees, such as technicians, get a general university entrance qualification. Often these students do not have the mathematical prior knowledge which is expected by universities in STEM and business fields. Furthermore, the widespread bridging courses are usually not enough for this audience, and they require more extensive extra-occupational mathematics courses. At the Ostbayerische Technische Hochschule (OTH) Amberg-Weiden there is a long tradition of preparation courses for students with vocational backgrounds. Over time these courses have been adapted to changes in the legal framework and were further developed between 2016 and 2018 into the program BeVorStudium. The joint project "OTH mind", funded by the German Federal Ministry of Education and Research, developed a blended learning scenario that included STACK quizzes.
+A majority of participants are in their mid-twenties, have a vocational education in the technical area and aim for studies like mechanical or electrical engineering or computer science. Institutions who are offering preparation courses for this audience face several challenges. One is the high number of topics to be covered in just a few months. Another is the limited number of time slots for in-person courses, with participants typically having a full-time job. This makes the self-study component an important part of the course. STACK is well-suited for the self-study environment, helping students apply new notions to routine problems and get individual feedback.
+BeVorStudium consists of two mathematics modules (plus one module in physics). Mathematics I contains topics typically included in middle grade syllabi, in particular elementary algebra. Mathematics II deals with important content from senior grades, especially calculus. In-person courses take place every other weekend, combined with an online based self-study component in between. Students are encouraged to complete regular online quizzes with STACK questions, discussed in detail in [2].
+Each quiz consists of six to ten normally multi-part STACK questions. When designing the quizzes, the following criteria were followed to meet the needs of non-traditional students:
+Partial credits. Partial credit is used extensively to keep students motivated, and reward correct ideas where possible.
+Specific feedback. Questions put an emphasis on "specific feedback" rather than "general feedback". Specific feedback addresses individual mistakes, and provides positive feedback on correct ideas. If a question includes general feedback too, this usually provides an illustration of the correct answer or a hint to the approach but not a model solution. Questions and model solutions can be discussed in class, which, according to the lecturer, usually takes a significant amount of "well invested" time.
+Variety of input types. In mathematical practice it is important to have a variety of material, approaches and perspectives [3]. To support this idea, the quizzes use a variety of input types. Besides a majority of questions with algebraic input there are also interactive JSX-Graphs and regular multiple-choice questions (MCQs).
+Although there are a number of problems connected to the use of MCQs in the assessment of STEM subjects - for example, when dealing with reversible mathematical processes - there are situations when this question format is more appropriate than others [4, 5]. In the preparation program, BeVorStudium MCQs serve various purposes:
+Since Moodle has built-in support for MCQs, it might seem unnecessary to use STACK for these questions. However, STACK comes with its own set of MCQ input types. Besides having a consistent layout in all questions, STACK's MCQs have several features that improve the didactical quality of the questions:
+Developing interesting question formats. STACK enables authors to implement multiple-choice formats which are recommended for university level assessment [6] like multiple True/False or matching questions.
+Using typical STACK features. Many STACK features such as randomization, partial crediting and in particular differentiated feedback can also be used in MCQs. For example, it is possible to give feedback on options of a checkbox question which are not chosen, or to point out contradictions between certain options.
+Implementing multi-part questions. There are typical mathematics problems which not only require a numerical or algebraic answer but also some verbal addition ("What is the type of the extremum?", "Is this answer unique?", "What is the monotonic behaviour of the function?" etc.). These questions are implemented well through STACK MCQ.
+The experience at the OTH Amberg-Weiden is that non-traditional students value in-person classes. It is therefore important to maintain a close connection between online quizzes and in-person lectures. By doing so, a majority of students also complete the STACK quizzes. During the trial period of the course in 2018 on average 97.2% (Mathematics I, N=9) and 62.5% (Mathematics II, N=16) of active participants² completed the STACK quizzes. These percentages were similar in subsequent years.
+The integration of STACK questions led to an increased amount of time used for self-study. In the 2018 evaluation of the module Mathematics II, eight out of ten respondents said they invested more than three hours per week on self-study - in addition to a full-time job and extensive in person lectures. Both students and lecturers acknowledge the importance of STACK as a part of the preparation program. Individual feedback of alumni shows that some participants pass their math exam at the end of the first term with a good or even very good result.
+After three years of using STACK within BeVorStudium, questions have been consistently improved. There are however some approaches for further development:
+Extending the specific feedback. The specific feedback still mainly focuses on deficits ("You have done something wrong"), and should be extended to address correct ideas as well.
+Making exercises problem oriented. Problem orientation is a principle of successful mathematical practice [3], and means to give exercises a superordinate perspective. This can be achieved by adding reflective MCQs.
+Considering different approaches. Currently most of the questions focus on the result rather than the approach. Implementing questions that focus on the approach could help students better understand mathematical methods and identify mistakes. This could be implemented by asking for certain extensions or using the "Equivalence reasoning" input type.
+[1] Kultusministerkonferenz. Hochschulzugang für beruflich qualifizierte Bewerber ohne schulische Hochschulzugangsberechtigung. https://www.kmk.org/fileadmin/veroeffentlichungen_beschluesse/2009/2009_03_06-Hochschulzugang-erful-qualifizierte-Bewerber.pdf, 2009.
+[2] S. Bach. STACK-Fragen zur Unterstützung der Selbstlernphasen in einem Studien-vorbereitungsangebot für beruflich Qualifizierte. In: Contributions to the 1st International STACK conference 2018, DOI: 10.5281/zenodo.2563803, 2019.
+[3] H. Winter. Begriff und Bedeutung des Übens im Mathematikunterricht. In: Mathematik lehren, 2, pp. 4-16, 1984.
+[4] C. J. Sangwin, I. Jones. Asymmetry in student achievement on multiple-choice and constructed-response items in reversible mathematics processes. In: Educational Studies in Mathematics, 94, pp. 205--222, DOI: 10.1007/s10649-016-9725-4, 2016.
+[5] S. Bach. Using multiple-choice questions in STACK -- reasons and examples. In: Contributions to the 3rd International STACK Conference 2020, DOI: 10.5281/zenodo.3945809, 2020.
+[6] M. A. Lindner, B. Strobel, O. Köller. multiple-choice Prüfungen an Hochschulen? Ein Literaturüberblick und Plädoyer für mehr praxisorientierte Forschung. In: Zeitschrift für Pädagogische Psychologie, 29 (3-4), pp. 133-149, 2015.
+¹ The legal framework for the current arrangement was given by a decision of the Standing Conference of the Ministers of Education and Cultural Affairs (KMK) in 2009 [1].
+² Active participants are considered to be those who attended at least half of the lectures.
Konstantina Zerva, Ilyas Nicholson, Adrián Doña Mateo
+ +HELM (Helping Engineers Learn Mathematics) is a collection of 50 workbooks developed by five English universities – Loughborough, Hull, Reading, Sunderland and Manchester – that covers the curriculum of first- and second-year mathematics courses for engineering undergraduates. They were designed as flexible learning resources and have been used by an estimate of 12,000 students in 55 UK higher and further education sites [1].
+During the ongoing pandemic, as more and more teaching moves to online spaces, the universities of Edinburgh and Loughborough, with the help of several other institutions, have undertaken an effort of translating the HELM materials into Moodle quizzes. This effort was supported with funding from the U21 Global Educational Enhancement Fund [2], which mainly enabled interns (including Ilyas and Adrián) to be employed on the project.
+The newly-created Moodle quizzes very much follow the spirit of the original workbooks but provide the added value of interactivity through STACK questions. The result will be released as an Open Educational Resource (OER) on the University of Edinburgh website.
+So far, about half of the workbooks have been made into Moodle quizzes. Most of the quizzes corresponding to workbooks 1–19 and 35–39 were created during the summer of 2020. These include all the exposition and worked examples as description boxes, and STACK versions of most of the tasks and exercises in each workbook. The materials covered by these quizzes include:
+Feedback from students and lecturers was gathered and acted upon during an extensive revision of the material during the summer of 2021, prior to their public release. This also ensured consistency of style and resulted in the creation of new quizzes and addition of interactive graphs to some questions using JSXGraph.
+Currently, there are over 90 quizzes with an average of 6–8 STACK questions each. Most questions have algebraic or numerical inputs, but some are implemented as multiple choice questions, which better suit the nature of the original HELM exercise. Except where it was not possible, question statements were randomised so that students can practice with different versions of the same exercise. Special care was taken to set up relevant tests, to ensure each question is working as expected and to flag issues in future edits.
+The following screenshots provide some highlights from the HELM quizzes.
+The HELM quizzes have been used in the Engineering Mathematics 1A and 1B (360 students) and Mathematics for Natural Sciences 1A and 1B (130 students) courses at the University of Edinburgh during the 2020/21 academic year. The courses also had online lectures and tutorials, and the HELM quizzes were mostly used for self-study. Each week, 3 or 4 quizzes were made available to students, which would serve as an introduction to the topics to be covered. These were intended as formative feedback and thus did not impact the student’s final grade, but they could be used as practice for the assessed quiz due at the beginning of the following week.
+The students’ engagement with the HELM materials was encouraging. In the Engineering Mathematics courses, each quiz typically received between 250 and 300 complete attempts on the week it was released. Feedback was overwhelmingly positive – some students found the HELM quizzes to be the most valuable part of the course. They particularly appreciated being able to work at their own pace and how the quizzes assumed little or no previous knowledge, a trait inherited from the original workbooks. Students also pointed to some areas that needed improvement, such as worked solutions, which were addressed during the 2021 summer revision.
+Bringing new life to the HELM workbooks in the form of STACK quizzes has proved to be a very fruitful activity. The upfront effort to develop STACK questions has added great value to the online learning experience, through question randomisation and immediate feedback. The quizzes created so far cover most of the contents of first year engineering mathematics, applicable also to other subjects, and demonstrate how STACK can be used to teach a wide variety of topics.
+The work is nevertheless far from over. Around half of the HELM workbooks are yet to be translated into Moodle. Several other universities are interested in using the existing materials in their courses, and further progress will be made prioritising the workbooks that would be of most use to course organisers. In addition to this, it would be interesting to explore the use of interactive graphs as question inputs, which would enable certain types of ‘sketch’ questions in the original workbooks to be transferred to STACK.
+Lastly, the quizzes will soon be released as an OER, which will allow students all over the world to use them for self-study.
+[1] https://www.lboro.ac.uk/departments/mlsc/student-resources/helm-workbooks/past-present-future/
+Siri Chongchitnan
+The onset of the Covid-19 pandemic in 2020 urgently forced us to rethink how we assess mathematics at university level. In the summer of 2020, we turned to STACK with the goal of creating self-assessed online quizzes in real analysis that would supplement traditional problem sheets. We wanted to see how far STACK could be used in a heavily proof-based pure mathematics undergraduate course.
+We worked on the module Analysis II - a core first-year module for maths undergraduates at Warwick. The intake was about 400. This is second course in real analysis follows from the foundation in numbers, sequences and series in Analysis I. Teaching materials, including online quizzes, were to be created on the Moodle virtual learning environment. Here is the weekly scope of the course given over 10 weeks.
+Our goal was to create one set of STACK assessment for each week’s material. Each set should take around 30-40 minutes to complete.
+In the previous year, students had to submit problem sheets weekly, However, Covid-related uncertainties meant that we had to cut this number down to only 4 problem sheets, which counted for 10% of the final grade.
+To supplement the above, we created 11 sets of STACK quizzes, two of which were not counted (one on syntax training, and one on Analysis I revision). They were practice quizzes, in the sense that students could do them an unlimited number of times before the deadline, each attempt building on the last. Each quiz had a pass mark of around 85% (not uniform across the quizzes). Passing 6 out of 9 quizzes gave the student a flat 5% for the module.
+We used STACK to create over 40 multi-part questions over 4 weeks. We are grateful for George Kinnear’s talk on JSXGraph at previous STACK conferences, and to Chris Sangwin and Robbie Bickerton, both of whom gave us inspirations to apply their scaffolding and proof-comprehension techniques to Analysis II.
+We used STACK in conjunction with Moodle-type questions (STACK integrated seamlessly with our Moodle virtual learning environment). We particularly found JSXGraph helpful in creating interactive graphics that visualise and explain tricky analysis concepts better than static media could. Highlights include the questions shown in these figures.
+We set the pass mark for each assignment quite high (>80%) but most students achieved >90%. On average students submitted between 2 and 3 attempts per quiz, with over 90% of students passing all quizzes. These engagement statistics are all encouraging.
+When asked: “Apart from the lecture notes and videos, what other resources are useful for the understanding of Analysis II?”, we found that almost 90% of respondents rated the quizzes as useful. As many students rated the problem sheets as useful. This is evidence that when implemented properly, the learning gains from STACK quizzes are comparable to those of traditional pen-and-paper assignments.
+When asked to “Name one thing about the module which has had the most impact on your learning in this course”, a number of students thought the STACK quizzes were the most impactful element of the course. Some comments include:
+“The quizzes were helpful and the infinite attempts ensured that I did not feel pressured while doing them, allowing me to focus on understanding the content."
+“The quizzes have been a very useful interactive learning tool."
+“The alternating assessment format between quizzes and written assignments is nice and helped reduce stress."
+STACK can be successfully applied to a wide range of mathematics content, including pure mathematics at university level as our work has demonstrated. We have shown that it is possible to turn proofs in existing lecture notes and problem sheets into interactive online quizzes, even in pure proof-heavy courses like ours. The interactivity in the STACK quizzes also meant that students understood the proofs much more clearly than when the proofs were recited during lectures.
+Although STACK was the primary tool for question creation, we also found it useful to mix STACK with other native Moodle-type questions for a varied diet. Creating questions on STACK is an ideal co-creation project which can be done as a summer internship. There are long-term benefits for everyone involved.
+All in all, we found STACK to be highly flexible and adaptable to help us achieve our teaching-and-learning goals. At Warwick we are continuing to expand the use of STACK to other maths modules.
Complex analysis is a classical branch of pure mathematics, and one of the high points of human intellectual achievement. The University of Edinburgh complex analysis course, Honours Complex Variables, is typically taken by around 250 students annually, mostly in Year 3 of Mathematics BSc/MMath degrees. The course covers the following topics:
+The course content is based on an established set of course notes (around 80 pages) written by Dr Richard Gratwick, the current course organiser. The notes contain the background theory, proofs, examples and some exercises for the students to practise. +Students were expected to work on them while they were studying each lecture's content. In addition to the exercises there are also tasks used in face to face workshops.
+Traditionally, the course would have had three 50-minute whole-class lectures and one 50-minute workshop each week. In the academic year 2021-22, the course was redesigned in response to constraints imposed by pandemic restrictions.
+The course ran with two 50-minute synchronous digital lectures each week, in which the lecturer used a form of flipped classroom with polling in Zoom, and one 50-minute in-person workshop. The notes were converted from a static PDF to a collection of quizzes in the STACK service.
+Each week the students had 3-4 STACK quizzes, and taking into consideration the design of the quizzes, we estimate that the students would need 1-2 hours to work through each of them. So, each week we expect the students to spend 10-12 hours on the course in total (contact hours, quizzes, workshop, assignments).
+The STACK quizzes were used for practise and they didn't contribute towards the final mark of the course.
This redesign was influenced by the previous work at the University of Edinburgh developing two fully online courses Fundamentals of Algebra and Calculus (FAC) and Introductory Mathematics and its Applications (IMA) [1].
+The resources for Honours Complex Variables were created by Richard Gratwick, Steven O’Hagan, the postgraduate intern Ivona Gjeroska and the undergraduate interns Maddy Baron and Xie Jin Ng, in collaboration with Giampaolo D’Alessandro (University of Southampton). The redesign aimed to be future-proof in the sense that materials created should be suitable for reuse in subsequent years, with iterated improvements, even when a return to fully on-campus teaching is possible.
A typical pattern of content from the resource is shown in following figure. We see a definition followed a short discussion and video clip of a worked example by the lecturer. Then we have a STACK question inviting the student to check their own understanding of the material. In addition, each unit of work started with a video of the lecturer giving a high-level overview and motivation of the material to come.
+Here are some examples of notable questions created for this course.
+In the future, such sessions could easily translate to on-campus activities using peer instruction, which had been an established practice in the School before the pandemic.
+Student feedback on the course was overwhelmingly positive, with one student responding to the end-of-course survey as follows.
+++“Genuinely this course has been the perfect mix of activities for my learning, I’d go as far say to the best organised course I’ve taken in SoM, certainly this year anyway. The notes being delivered in stack are great and much more engaging than a pdf (the supplementary pdf is much more easy to navigate for finding Theorems etc. however), which actually makes me do all the reading before lectures so I gain so much more from them. Stack is good in part because of the instant feedback on most exercises which are immediately relevant to what you’re learning, but also because it breaks the material up well. Stack being the main resource works in perfectly with the 2 lectures delivered a week and the tutorial. SoM should considering delivering all courses in this fashion.”
+
The course organiser, Richard Gratwick, said:
+++“I believe the redesign of this course has been highly successful, and I like to consider the mode of delivery as a prototype for courses of the future. The motivation was to reflect on the innovations forced upon on us during the pandemic and attempt to implement in a considered way those that proved valuable. Recording short videos of examples and calculations allowed me to present these items “dynamically”, rather than as plain text, but freed up time in synchronous lectures for me to engage with more conceptual high-level discussion of the material. The Stack questions for self-assessment were largely based on exercises included in the previous version of the written notes, but students engaged with them much more regularly when they were presented in the online workbooks – my impression was that in previous years these written exercises were largely ignored.
+
Redesigning a 3rd Year course, so that it contains a substantial amount of online components, was a challenging procedure.
+Online assessment it typically used for larger classes in early years of a university mathematics degree, such as calculus or linear algebra. These courses have a strong methods component. As the subject progresses, especially where there is a focus on mathematical proof, it is harder to write online questions which assess the key concepts. As a community we are gaining experience in using quizzes to structure learning materials (rather then traditional lectures), and in supporting more advanced courses. For example the University of Warwick used self-assessed online quizzes in real analysis that would supplement traditional problem sheets. In this course we have demonstrated that STACK can be successfully applied to a higher-level course to promote students' engagement with the content of the course. This is not as straightforward as in calculus and linear algebra, but nevertheless with careful design of materials it is possible to support students online with STACK in advanced courses such as complex analysis.
Maseno University has continually worked with IDEMS International to create online STACK assessment resources for nine courses over the last four years. Lecturers at Maseno, and from other universities within East Africa; Strathmore University, Kenya, University of Dar es Salaam, Tanzania, and Bahir Dar University, Ethiopia, have successfully used STACK in students' assessments as part of formative and summative assessment in the form of CAT contributions in their courses. The School of Mathematics Statistics and Actuarial Science (SMSAS), at Maseno, intends to create STACK quizzes for all of their courses as one of their long-term goals. This is a report on a 5-day STACK workshop held at Maseno in July 2022, from the 4th to the 8th. The goal of the workshop was to share resources developed at Maseno University with learning institutions in East Africa, and establish a community of users who can collaborate on the same to improve mathematics education. The workshop drew a total of 48 participants from various institutions both locally within Kenya and from Tanzania and Ethiopia, and more expressed interest. During the workshop, there was a discussion about an ambitious plan by INNODEMS (IDEMS affiliate) to recruit recent college graduates as interns, and train them in authoring STACK question types so as to help lecturers implement STACK in their courses.
+As reported in a previous Case study report Innovating Education in Maseno, Kenya, Maseno University held a workshop on electronic assessment in mathematics in August 2019, with participants from Kenya, Ethiopia, Rwanda, Tanzania, and Uganda in attendance. The primary goal of that workshop was to train lecturers on how to use STACK in formative and summative assessment of students. Since then, Bahir Dar University (Ethiopia), has used STACK for practice without giving credit to students for engaging with the resources. However, after the 2022 workshop held at Maseno, there have been plans in using STACK at Bahir Dar for formative assessment as from September 2022. Apart from Maseno University, the University of Dar es Salaam (Tanzania), has also successfully used STACK in formative and summative continuous assessment since the 2019 workshop in three of its courses, Numerical Analysis, Calculus of One Variable and Differential Equations for Non-Majors. Some Kenyan universities have been inconsistent with using STACK in assessment, which is what this 2022 workshop hoped to address. In Uganda and Rwanda, there has been no feedback on the use of STACK.
+STACK resources developed at Maseno were uploaded in the IDEMS server to allow participants in the workshop to access them.
+Resources in these courses are also being reviewed to improve the quality of feedback provided to students for the next semester, which begins in September 2022. Probability Theory and Foundations of Pure Mathematics are two more courses under development.
+The 5 day workshop attracted participants from the following learning institutions;
+The main goal of the 5-day workshop was divided into the following objectives;
+This section summarizes the daily events that took place as part of the workshop from 4th to 8th July, 2022. On the first day, a presentation on the features of STACK and its usage at Maseno University was done by Dr. Isaac Owino (The Chair of the Math Department). Lecturers in the workshop were then allowed to peruse the available courses created by Maseno and IDEMS international, identify a course they’d be working on to ready it for delivery in their various institutions, and form collaborative groups during the workshop. Copies of the courses were then created and participants enrolled in readiness for the next agenda, customizing the existing content to suit individual needs. Organizing content in the course, structuring resources into logical topics, and creating various types of quizzes for different types of assessments were done in an open working session, during the second day.
+The better part of day three was spent on hands-on authoring of STACK questions and identifying other textbook questions that can be added to create a bank of questions for various uses. More hands-on activities like exporting questions from other courses followed during the fourth day of the workshop. Idrissa Said, from University of Dar es Salaam, gave a presentation on various step-by-step working questions he has been using to teach mathematics. He offered to provide two of the courses he created, Numerical Analysis, Calculus of One Variable and Differential Equations for Non-Majors, to be added to the pool of courses participants can borrow resources from. Juma, A Master’s student at Maseno who did a study on STACK, also gave a brief presentation on how learners have been engaging with STACK at Maseno, taking a case study of Complex Analysis 1 courses, a third-year course for all mathematics students.
+The last day of the workshop was mainly an open working session with Q&A from the newly recruited AMI interns, and lecturers who were working on their courses in preparation for delivery.
+Copies of the courses developed by Maseno University were created in the IDEMS moodle site. Participants were asked to select course(s) that they would like to develop in readiness for teaching in September 2022 or January 2023 semester. The Courses selected by participants were then created under the category of their university on the IDEMS Moodle site. Participants were given managerial rights in their courses with a short training on course management in Moodle, STACK question architecture and importing questions from other courses from the list of courses they were enrolled in (to which we gave read-only access to everyone).
+Below is a list of courses selected by lecturers from universities that attended the workshop. These courses are what they selected to work on with readily available resources and support when it comes to implementing STACK.
+University | +Course | +
---|---|
Maseno University | +Descriptive statistics | +
Masinde Muliro University of Science and Technology (MMUST) | +Linear Algebra, Calculus 1,Calculus 2, Complex Analysis 1, Complex Analysis 2, Vector Analysis | +
Strathmore University | +Linear Algebra, Calculus 1, Calculus 2, Basic Mathematics, Measure Theory | +
Kaimosi Friends University College (KAFUCO) | +Linear Algebra, Calculus 1, Calculus 2, Measure Theory | +
University of Kabianga | +Descriptive Statistics | +
Turkana University College | +Basic Mathematics | +
Alupe University College | +Vector Analysis, Descriptive Statistics | +
The University of Dar es Salaam (Tanzania) | +Linear Algebra, Calculus 1, Calculus 2 | +
Bahir Dar University (Ethiopia) | +Basic Mathematics | +
Kisii University | +Calculus 2 | +
Kibabii University | +Complex Analysis 1 | +
Rongo University | +Measure Theory | +
The five-day event was interactive, with participants sharing experiences and reflections on what areas they'd like to spend more time on during the following day. We were able to gather the following feedback from the daily reflections, which helped direct the agenda and objectives of that session based on participant requests.
+One of the main challenges we had was that courses for the 2022/2023 academic year had not been allocated to lecturers in most universities. Hence lecturers were not certain that they would teach the course they were developing. Lecturers however expressed optimism that either they could persuade their heads of departments to assign them the courses they were creating, or that they could, if given a new course, develop it with the help of IDEMS and INNODEMS in time for the start of classes. Another challenge was that we could not accept all applications (more than 60) due to limited funding and not having enough facilitators to cater for the large group. We however felt that the number we had (49) was optimal for the available funding and manageable for the 8 facilitators in the workshop.
+Inadequate equipment was also a major challenge. There were some participants who would have liked the workshop to be online. This is because of finances on their part, teaching and departmental duties at their institutions and flexibility offered by online workshops. We however did not have equipment that would have made giving an online workshop possible. Giving an online workshop would have forced us to reformulate our objectives for the workshop because support would be less efficient. We also believed that personal interactions with other participants, facilitators and interns was important in creating a STACK community. This is one challenge we would like to address in the coming years and with adequate funding.
+Inadequate institutional support remains a challenge at Maseno. Even though the workshop had full blessings and support of the Vice Chancellor, getting lecturers in the School of Mathematics to commit to using STACK for online assessment remains a challenge. It would be more efficient if more lecturers are involved through discussions on what kind of support they need to allow them to integrate online assessment with STACK seamlessly.
+One of the main collaborators of the 2022 Maseno STACK workshop was IDEMS International. Apart from providing financial backing, they paid for travel of Georg Osang (one of their employees) to Maseno to help facilitate the workshop as well as train interns on authoring of STACK questions. IDEMS International also provided direct support in the planning and execution of the workshop as well as coordinating travel for international participants.
+The university of Geneva through a research fund for Danilo Lewanski and his research partners was also a key funder for the workshop. Indeed, we were able to increase participation from an initial estimate of 20 to 50 largely because of the funding from the University of Geneva. +INNODEMS recruited 4 graduate students from Maseno as prospective interns who are currently undergoing training on authoring STACK question types and quality control of the existing materials, with the help of IDEMS International. With the support of the African Maths Initiative, this is done with the aim of providing support to various lecturers to integrate STACK in their courses. Furthermore, INNODEMS was also a key supporter in terms of handling finances, providing logistics and Internet supply for the entire week. Aside from the INNODEMS interns who participated in the workshop, there were other employees who attended the workshop and helped in coordinating events and logistics.
+Lastly Maseno University was integral in providing support in the even, through the Vice Chancellor Prof. Nyabundi, who personally opened the event on day one and pledged to give full support of any kind to see it a success in future. In his speech, the Vice Chancellor expressed optimism that the myth mathematics is a difficult subject could be demystified through improvements in the teaching and learning. The Vice Chancellor also expressed his desire to see more STACK workshops held at Maseno University.
+We have learned some valuable lessons in organizing and giving this workshop. One of the main lessons learned is that planning has to start early in order to attract international participation and facilitation. We missed out on having a number of international facilitators because the planning of the workshop was not concretized early enough. Applications for funding have to be made in a timely manner, and even prior to setting workshop structures in place, there must be clear details on available finances and expected expenditures. This year's workshop was largely successful due to funding from IDEMS and the University of Geneva. Application for the funding was not made in a timely manner which almost derailed the organization of the workshop. Management of funding was another important lesson. We managed to have a workshop with 49 participants from three different countries on a budget of less than 10,000USD. Of the participants, only two paid the application fee of less than 200USD, while most participants paid for their own travel and accommodation.
+With this hard lesson learned, we have started organizing the 2023 workshops with tentative venues being Masinde Muliro University Kenya, Bahir Dar university Ethiopia and the university of Dar es Salaam in Tanzania. There is even a discussion of having three local and more impactful workshops in the three countries.
Achim. Eichhorn and Andreas Helfrich-Schkarbanenko
+We present a method to evaluate fill-in-the-blank student answers in STACK using a string metric. To increase the quality of the evaluation, we use two lists: allowlist and denylist instead of a single teacher's answer. We also show a STACK question equipped with a string metric, by evaluating its use in mathematics courses.
+The fill-in-the-blank questions are important from a didactic point of view. But they can be hardly implemented, since typing and spelling errors, synonyms and geuine alternatives have to be taken into account when evaluating the students' answers.
+To automatically mark fill-in-the-blank questions we used one of the string metrics for measuring the distance between two strings: the Damerau-Levenshtein distance [1, 2], which plays an important role in natural language processing. Informally, this distance is the minimum number of single-character edits (insertion, deletion, substitution, transition) required to change one string sequence into the other. Note that this distance is a metric in mathematical sense, in particular it satisfies the triangle inequality. This enables a suitable string evaluation.
+To increase the quality of the assessment, we extended the basic metric function by the adding the components: allowlist and denylist. To have a relative measure of the difference between two strings, we convert the distance to similarity. Applying the similarity on allowlist and denylist we define an acceptance domain for the students' answers. Here we need an empirically determined threshold parameter.
+Note that the presented method is, strictly speaking, not only based on the strings, but also on semantics, because by introducing the denylist and allowlist respectively, a (trivial) semantic graph consisting of two clusters is set up. For the sake of +simplicity, the evaluation of a student's answer does not take place by means of a semantic distance, but using the string distance to the respective cluster as a whole (single linkage, minimum distance, nearest neighbour, see for example [3]).
+If the Damerau-Levenshtein distance between strings and is then the similarity is defined as + +Given lists + + +then + +We then have an acceptance domain in which + +for some chosen similarity tollerance .
+Here are some examples using the strings "Circle", "Triangle" and "Rectangle", together with their Damerau-Levenshtein distance / similarity.
+We asked students for a suitable solution method by using a fill-in-the-blank question when given a differential equation, see subtask a). +This task was used in the winter semester 2021/22 as part of a mini-test for the lecture Mathematics 2. It was completed by 53 students and all student answers were scored error-free.
+We implemented a string metric directly in the computer algebra system MAXIMA and placed the corresponding function in the Question Variables field of the STACK question concerned. The corresponding XML file can be downloaded from this link (June 2022).
+In the bottom figure we see 18 different student answers (in German) which are positioned in a coordinate system according to both similarities and are classified without errors. The radii of the disks represent the number of equal student answers. In total, this task was processed 263 times. The acceptance domain for correct answers is white-marked.
+Note that for this STACK question and the given allowlist resp. denylist, only the consideration of the allowlist similarity would be sufficient for the evaluation. However, there are situations where the denylist is necessary.
+The authors would like to thank Stiftung Innovation in der Hochschullehre for supporting the project "Digitalisierung Didaktisch Denken".
+ + + +This feature has been added to STACK 4.0 in 2022 as an answer test.
+[1] F. J. Damerau: A technique for computer detection and correction of spelling errors, Communications of the ACM, 7 (3): 171-176 (1964)
+[2] Vladimir I. Levenshtein: Binary codes capable of correcting deletions, insertions, and reversals, Soviet Physics Doklady, 10 (8): 707-710, (1966)
+[3] A. Eichhorn, A. Helfrich-Schkarbanenko: Question Answering in STACK Applying String Similarity. Private communication, (2022)
Authors: Prof George Lawi, Mr Santiago Borio, Dr Michael Obiero Oyengo and Mr Juma Zavick.
+The first African STACK Conference for Undergraduate Mathematics took place at Masinde Muliro University of Science and Technology (MMUST), Kenya, from 19 to 23 June 2023. The aim of the conference was to enable African educators to define a roadmap for the transformation of African Undergraduate Mathematics education through STACK. A total of 66 individuals from 16 institutions participated in the conference. The proceedings included five Keynote speeches, 21 presentations, 10 hours of workshop, and a panel discussion with key African stakeholders from various institutions that attended. The conference successfully raised the profile of STACK within MMUST, with the Vice Chancellor openly supporting its institutionalisation in the Mathematics department. Additionally, participants gained awareness and understanding of how STACK is being used and its impact in African and European universities, started designing STACK integration to courses using the IDEMS Open Question Banks, gained insight into potential technical challenges and how to tackle them, and recognised the need for additional support for full curriculum coverage. Finally, initial steps were taken in the creation of an African STACK Community Leadership Team to drive STACK work in the continent and implement the roadmap discussed during the conference.
+Faced with the ever-increasing challenges such as large class sizes, inadequate teaching personnel, and limited teaching and learning resources, amongst others, African universities have been forced to innovate so as to provide a good education to their students. One such innovation has been incorporating STACK assessments in undergraduate STEM courses since 2019. These STACK assessments have enabled students to be given continuous assessments with immediate formative feedback, which has improved learning and student achievements. IDEMS (Innovations in Development, Education and the Mathematical Sciences) International, a not-for-profit UK-registered community-interest-company has collaborated with a group of African universities to integrate STACK to selected courses and develop a set of open question banks, with the long term objective of achieving full curriculum coverage in undergraduate mathematics.
+STACK integration started through a partnership between IDEMS and Maseno University in 2019, with IDEMS authoring questions for two courses for the second semester of the 18-19 academic year, and an additional two courses for the first semester of the 19-20 academic year. The benefits of STACK became apparent to address some of the mentioned challenges and Maseno University held the first IDEMS-run African STACK workshop in 2019, which led to STACK integration in BDU, Ethiopia, University of Dar es Salaam, Tanzania, and MMUST, Kenya. IDEMS supported this through further question authoring and hosting of courses for these universities. In 2022 Maseno University hosted a second African STACK workshop and in 2023 BDU hosted the third.
+In parallel, other African universities integrated STACK to some courses, work that increased in priority due to the need for remote learning during the Covid-19 pandemic. Most notably, The University of Nairobi, Kenya, expanded their STACK integration to include formal assessments and exams in some undergraduate mathematics courses.
+The natural next step to share experiences and define a collaborative effort for increased STACK integration was the organisation of the first African STACK Conference for Undergraduate Mathematics. This conference served as a platform for educators who have used STACK to share their experiences, methodologies, ideas, and the impact of incorporating STACK into their courses, and those who have not used STACK to learn of its potential in promoting learning and research. It also provided an opportunity for key stakeholders in institutions that do not have STACK to explore ways in which they could get support in integrating STACK at their institutions. Additionally, the final day included a panel discussion involving key stakeholders that formed the basis of the establishment of the African STACK Community Leadership Team to implement the key plans defined during the conference.
+The conference brought together educators, researchers, and other stakeholders to explore the challenges, experiences, and potential of STACK in supporting mathematics undergraduate education. A total of 66 individuals from 16 institutions participated in the conference. This includes three Ethiopians, one Togolese, one Namibian, one Tanzanian, one Italian and three British participants, with the remaining from Kenya.
+The first four days were split into keynote speeches, followed by discussions, presentations, and ending with afternoon workshops. The final day included a final keynote speech and a panel discussion with key stakeholders. Presentations were categorised into five themes: Motivation, New Avenues, International Views, MMUST Institutionalisation and Paving the Road for STACK integration, and are outlined below.
+Over ten hours of practical workshops over the first four days gave participants an opportunity to start working or expand their work on STACK integration in their courses. Most workshops were led by Mr Santiago Borio, with support from Mr Juma Zevick, Mr Montognon Wastalas Dogbalou, Ms Celestine Atieno Oliewo and the INNODEMS STACK Intern team. The first workshop focused on exploring the IDEMS Open Question Banks. The second introduced how to use the open questions and how to share resources effectively. The third presented mechanisms to start creating courses for use in the 23-24 academic year. The final workshop was split into two options: basic authoring, led by Prof Chris Sangwin, and finalising course preparation with STACK questions. The overall outcome of the workshop was an insight into authoring and openly available resources, how to access and how to use them.
+The theme of the first day (June 19th) was Motivation. After the opening ceremony, Dr David Stern delivered a keynote speech discussing the challenges faced by African institutions, the use of STACK as a tool to address these challenges, and the importance of empowering educators through technology. He compared STACK with the printing press in terms of making education more equitable, particularly in challenging contexts.
+In the presentations, Dr Idrissa Said Amour shared the challenges and opportunities faced in implementing STACK at the University of Dar es Salaam, Tanzania, while Dr Abdu Mohammed Seid discussed a STACK pilot in Bahir Dar University (BDU), Ethiopia, and presented the plan to use STACK as a tool for preparing students for the national exit exam and the remedial programme. Dr Jared Ongaro shared his experience of using STACK at the University of Nairobi, Kenya, emphasising that STACK is not solely focused on testing memorisation, outlining the challenges he had to overcome, and detailing his and his colleagues’ achievements in integrating STACK for formative and summative assessment. Finally, Mr Santiago Borio conducted interviews with current undergraduate students and recent graduates from MMUST and Maseno University, discussing their experiences taking courses with STACK questions, the positive impacts of using STACK, the challenges faced, and overall feedback.
+The second day (June 20th) highlighted areas that are being explored for potential STACK integrations, with some examples of exploratory work undertaken. Dr James Kaleli Musyoka, vice chair of the International Association of Statistics Education, discussed the use of STACK to introduce basic statistical concepts and skills. He highlighted the need for data skills in a data-centred world and the role of STACK in transforming statistics education. He identified areas for improvement including: prioritising prerequisites, focusing on understanding rather than formulas, and developing technological skills for data handling. He then led a discussion of new courses and innovations; topics included accuracy for engineers, making question-writing easier, addressing specific needs of different groups, and bridging theoretical teachings with practical applications.
+The presentations highlighted two key areas where innovative use of STACK could be highly impactful, particularly in the African continent: statistics education and secondary schools. Ms Celestine Atieno Oliewo demonstrated three types of statistics questions she worked on in partnership with IDEMS and Maseno University and outlined their individual value and various formats. Mr Zach Mbasu, director of INNODEMS, Kenya, discussed the potential use of STACK in Kenyan schools to help support high school teachers bridge the gap on inequitable resources between public schools and private schools under the new Competency Based Curriculum, particularly in the early secondary grades where mathematics is taught by non-specialists. Additionally, Ms Christine Laetitia presented a collaboration between IDEMS, SAMI (Supporting African Maths Initiatives), a UK-registered charity, and INNODEMS to create a bridging course for secondary school mathematics used in the UK and mostly developed in Kenya. Dr Idrissa Said Amour showcased the progress made in STACK, focusing on developing high-quality questions and exploring alternative grading methods. Prof Mandirevesa Martin Mugochi outlined the context in Namibian universities and the potential impact of STACK in The University of Namibia. Finally, Mr Montognon Wastalas Dogbalou and Dr Herine Adhiambo Otieno discussed the use of technology in teaching STEM subjects in Africa, emphasising how STACK allows students to interact and engage with the content.
+During the third day (June 21st), international perspectives were the focus. Prof Chris Sangwin, the founder and one of the main developers of STACK, gave the keynote speech, in which he highlighted his motivation for creating STACK and its components. He presented the long-term objectives for STACK, including its application in various subjects. He also outlined a three-stage cycle for successful course development using STACK: testing, consolidating and improvement of questions using data from student responses. Finally he outlined The University of Edinburgh’s pilot of using STACK for a Linear Algebra exam.
+In the presentations, Mr Juma Zevick outlined an international collaborative study on the impact of feedback provided by STACK in addressing common misconceptions in introductory integral calculus. Secondly, a group of Kenyan interns presented their insights from the INNODEMS STACK Internship programme, supported by IDEMS, and Mr Santiago Borio outlined the programme in detail and presented a potential model of African STACK professionals that organisations and institutions could take for smooth STACK integration, as well as the support that IDEMS could provide. Dr Danilo Lewanski shared the initial steps of STACK integration in The University of Trieste, Italy, outlined the plans for expansion, and shared research opportunities available. Dr Herine Adhiambo Otieno reflected on careful considerations in the implementation, design, and deployment of educational technology in mathematics, particularly focusing on STACK drawing from past research findings and theoretical foundations.
+The theme of the fourth day (June 22nd) was the institutionalisation of STACK in the mathematics department at MMUST. In his keynote speech, Prof George Lawi outlined the MMUST experience over the previous two years and the ongoing journey towards full undergraduate curriculum coverage. He recognised the support provided by IDEMS and the INNODEMS STACK interns which facilitated smooth workflow in transitioning continuous assessments from a small number of pen-and-paper assignments to a fully online system in eight courses, increasing considerably the amount of work done by students and the feedback they received.
+There were five presentations by MMUST lecturers that focused on different aspects of the MMUST STACK experience. Dr Mary Okombo outlined how she used STACK analytics to identify challenging concepts in a course. Dr Fanuel Olege presented student perception of feedback of STACK in general in mathematics courses. Dr Colletta Akinyi analysed the impact of STACK on learner performance in mathematics at MMUST. Dr Duncan Otieno Oganga shared an Analysis of learner engagement with STACK in Analytical Geometry and Advanced Calculus courses. Finally, Dr Annette Okoth discussed the potential of STACK in transforming learner perception in mathematics at low level primary schools in Kenya.
+The format of the final day of the conference (23rd June) was different. Dr Michael Obiero Oyengo opened the day with his keynote speech, in which he drew out the key conclusions of the various presentations, highlighted the challenges he individually faced when returning to Maseno University after completing a doctorate in a top US university and compared them with common challenges faced by African universities in terms of resources and program quality. He also discussed the introduction of STACK at Maseno University and its positive impact on teaching and learning.
+The last presentation was given by Dr Mary Achieng, who outlined the Strathmore University journey integrating STACK and its preliminary outcomes, as well as highlighting key challenges in creating new content. The closing ceremony took place, in which the Vice Chancellor of MMUST recognised the importance of STACK both for the mathematics department and for STEM-related departments, and highlighted the importance of collaborative innovations, creditting Prof Lawi, the whole mathematics department and the organising committee for the successful conference.
+The conference culminated with a panel discussion of key African STACK stakeholders: Prof George Lawi, focusing on STACK institutionalisation, Dr Abdu Mohammed Seid, representing Ethiopian universities, Dr Idrissa Said Amour, representing Tanzanian universities, Prof Mandirevesa Martin Mugochi, representing Namibian universities, Dr Herine Adhiambo Otieno, focusing on Ed-Tech research, Dr Annette Okoth, focusing on STACK for statistics and data science education, Dr Beth Kiratu, representing the Kenya Mathematics Society, Dr Mary Ochieng, focusing on the Strathmore International Mathematics Conference and how the African STACK conference could learn from its model, and Dr Michael Obiero Oyengo, a key driver of STACK integration in East Africa. A key conclusion was the need for an African STACK Community Leadership Team to promote and support STACK integration in Africa.
+The conference provided a platform for meaningful discussions, knowledge sharing, and collaborations, empowering educators and researchers to explore the role of STACK in transforming mathematics education. Participants in the conference gained valuable insights from presentations, workshops, and discussions, expanding their understanding of STACK and its potential. Organisations involved in the conference, such as IDEMS and the Kenya Mathematics Society, benefited from increased visibility, networking opportunities, and potential collaborations with international partners.
+To sustain the momentum generated by this conference, future activities have been envisioned. The linkage between the Kenya Mathematical Society (KMS) and the STACK community will create efficiency in driving mathematics education innovation programs within Kenyan local universities. This collaboration will facilitate the integration of cutting-edge technology and pedagogical approaches to enhance teaching and learning experiences. The continued support of IDEMS and INNODEMS will be vital in supporting the adoption of STACK in Kenya and other African countries. The STACK internship program which is led by INNODEMS and supported by IDEMS is playing a crucial role in the development of STACK resources and in the online support for lecturers.
+Looking ahead, the University of Dar es Salaam and BDU were proposed as the venue for the next workshop and conference respectively. This decision reflects the commitment to inclusivity and the desire to spread the benefits of the conference across different African countries. By rotating the workshop and conference venues, more local educators and researchers from diverse regions will have the opportunity to participate, share their experiences, and contribute to the ongoing integration of STACK in African universities and other educational institutions.
+The conference set the stage for the future transformation of mathematics education in Africa. Through shared experiences and recommendations, a roadmap to guide the implementation of STACK and improve learning outcomes was adopted. Based on the outcomes and discussions during the conference, the following action points were agreed:
+African STACK Community Leadership Team (Dr Michael Obiero Oyengo): A proposal was given to establish a team that would play a key role in coordinating STACK activities within Africa, fostering collaborations, and ensuring the effective adoption and utilisation of STACK. This team would serve as a central hub for sharing best practices and knowledge, and coordinating support for African institutions implementing STACK.
+Suggestions for strengthening collaboration with KMS (Dr Beth Kiratu): The objective to foster a stronger partnership between the Africa STACK community and KMS to accelerate the adoption of maths education innovations in local Kenyan universities and educational institutions was proposed. KMS is committed to take the leadership in organising and coordinating mathematics education activities in Kenya.
+Continued support for the STACK Internship Programs in Africa (Mr Santiago Borio and Mr Zach Mbasu): IDEMS and INNODEMS agreed to continue supporting the INNODEMS STACK Internship programme. The African STACK Community Leadership team will look for mechanisms to expand this internship model in other contexts and counties.
+Improvement of existing Question Banks (IDEMS and INNODEMS): All participants agreed to help maintain and improve the IDEMS Open Question Banks. Collaboration will be essential to do so, particularly in question review, requesting new questions, and providing recommendations for improvement of individual questions. Mechanisms to do so were outlined in the conference workshops.
+Plans for future STACK Conferences in Africa (Dr Michael Obiero Oyengo): There was consensus that there is a need for the African STACK conference to be held yearly. The African STACK Community Leadership Team will take responsibility for deciding host institutions and collaborate with the respective organising committees. The current proposal is for the 2024 conference to be hosted in Ethiopia by BDU and the 2025 conference to be hosted in Tanzania by The University of Dar es Salaam.
+Throughout the conference, several key themes emerged regarding the implementation and adoption of STACK assessments in African institutions. The key points include:
+The conference was made possible by individuals and organisations and deserve clear recognition. First and foremost, the funding organisations: The Swiss National Science Foundation and Université de Genève, The European Mathematical Society, SAMI, IDEMS International and INNODEMS, whose financial support, both direct and indirect, enabled the conference to take place and support a large proportion of the participants who would otherwise not have been able to attend.
+Secondly, the organising committee members, Prof George Lawi, Mr Santiago Borio, Dr Michael Obiero Oyengo, Ms Christine Laetitia, Mr Juma Zevick, Dr Achiles Nyongesa, Dr Everlyne Odero, Dr Frankline Tireito, Dr David Angwenyi, and Dr Danilo Lewanski. Their dedication, expertise, and tireless work in planning and executing the conference were instrumental in its success.
+Thirdly, the keynote speakers, Dr David Stern, Dr James Kaleli Musyoka, Prof Chris Sangwin, Prof George Lawi, and Dr Michael Obiero Oyengo, for sharing their insights and expertise on STACK and its impact on mathematics education. Their contributions enriched the conference and provided valuable guidance to the participants.
+Furthermore, the supporting organisations: MMUST, Maseno University, the Kenya Mathematical Society, IDEMS, INNODEMS, and SAMI, for their collaboration, resources, and commitment to the advancement of mathematics education in Africa.
+Last but not least, all the participants who attended the conference, and particularly the presenters. Their active engagement, valuable insights, and meaningful contributions made the event a resounding success. Together, the collective efforts and commitment of these individuals and organisations contributed to the groundbreaking success of the Africa STACK conference.
+This appendix summarises the finances for the first African STACK Conference for Undergraduate Mathematics at Masinde Muliro University for Science and Technology, Kenya, held on 19-23 June 2023. Full financial details can be found here.
+The conference had four main sources of funding:
+The total amount spent for the conference was KES 1,349,774.54 (USD 9693.24). This can be classified into the following detailed categories:
+Nine individuals from non-Kenyan African countries applied for support for flights to participate in the conference. The funding available allowed for five participants to be supported: Abdu Mohammed Seid (Ethiopia), Yassin Tesfaw Abebe (Ethiopia), Mandirevesa Martin Mugochi (Namibia), Idrissa Said Amour (Tanzania), and Aménito Jean Claude Kohoun (Togo). The total spent for this was KES 571,475.79.
+All sponsored international participants were provided accommodation. Additionally, some local participants and organisers were provided accommodation as it wouldn’t have been possible for them to participate or support the running of the conference otherwise. Accommodation included breakfast and dinner, and additional funds were used for evening tea not included in the accommodation costs and a pre-conference meal on Sunday 18th June. Finally, further accommodation for two sponsored participants and Santiago Borio in Nairobi as part of their return flights was spent. The total spent for this was KES 340,900.00.
+Lunch and morning and afternoon tea were offered by the MMUST catering team for the five conference days. The total amount spent for this category was KES 357,500.00.
+Transport costs included transport for sponsored participants from Kisumu airport and back, transport for organisers and supporting team members (e.g. INNODEMS interns, etc), and payment for a MMUST shuttle driver for transport from hotel to venue and back. The total spent for this was KES 28,250.00.
+The organising and supporting team met daily from Wednesday 14th June to Saturday 17th June and lunch was provided. Accommodation for Santiago Borio for these days was covered. Finally, logistical costs prior to the conference were incurred by Ms Chrisine Laetitia. The total spent in this category was KES 22,014.00.
+Other conference expenses included stationery, welcome packs and certificates, and purchase or miscellaneous items required to run the conference. Additionally, financial costs were covered, which include Mpesa charges and foreign exchange fees. The total spend for this was KES 29,634.75.
+There was a surplus of KES 5,597.61 from the conference funds in the INNODEMS account. The remaining KES 1,680.90 should be accounted for from exchange rate fluctuations. The surplus was considered an administrative fee for INNODEMS for managing the accounts.
+David Stern, Santiago Borio, Georg Osang, David Whittaker, Getachew Mehabie Mulualem, Abdu Mohammed Seid, Mebratu Fenta Wakeni, Michael Obiero
+Higher education in Ethiopia is currently responding to multiple challenges stemming from national reforms. Bahir Dar University (BDU) is rising to these challenges by seizing the opportunity for change to improve the provision of student centred learning and support for maths and related courses.
+The BDU Mathematics Department recognises the leadership role it has, specifically as a role model for other Ethiopian Universities. In March 2023 the BDU Mathematics Department brought together 49 people from 12 Ethiopian universities to identify priority challenges and actions as foundations for an ambitious, yet achievable, approach to improved quality, consistency and results in mathematics education at this crucial moment in time. This event was co-facilitated by 3 IDEMS International Staff and a Kenyan Educational Innovator who brought in STACK expertise at BDU’s request as this was an open source tool they had identified which could add value to their teaching and learning.
+This case-study outlines 3 of 15 reported outcomes of the event, described in detail in a draft report. These three highlighted opportunities stand out because of their urgency with all three needing immediate action. Two of the three correspond to supporting students in preparation for unprecedented exams that will take place within the next 4 months. After August, this moment in time will have passed. Supporting student success at this crucial time has institutional and national ramifications.
+In 2023 only 3% of school leavers achieved passing grades, leaving thousands without the university place they applied for. All Ethiopian universities have been mandated by the Ministry of Education to provide remedial courses for prospective students to enable them to start their degrees.
+Delivery for these courses started in early April and is expected to run for 16 weeks, followed by an exam. There is a huge need and potential to significantly alleviate the expanded staff workload imposed by remedial courses and support student learning through STACK.
+IDEMS and staff at BDU are co-designing a STACK remedial course in maths, in which any university can enrol their students. The course will cover all topics in the remedial course, and provide formative assessment to allow students to gain from personalised feedback and consolidate mastery through repeated attempts of randomly generated versions of each question.
+The STACK course is in development and each new topic will be released on a weekly basis. If the full course is implemented, students nationally will gain targeted learning and support for remedial maths, increasing the chance of students securing their place at university as planned.
+The Ethiopian school exit exam is at the forefront of Ethiopian Universities minds at the moment. An institution's future desirability is will be strongly linked to their current students' performance. In Mathematics such an assessment is almost unprecedented and universities are looking to each other for support to ensure their students are prepared.
+A majority of the mathematics curricula to be tested in exit exams is already available as digital questions from institutions using STACK in Kenya and Tanzania. Importing these existing questions would populate 50-75% of the exit exam blueprints for maths in Ethiopia, reducing development time and costs, and increasing the readiness of universities for exit exams in June 2023. This would serve all BSc/BEd maths students nationally, plus a greater number of others with a maths component to their degree, e.g. accounting, engineering, statistics, data science.
+The same system would also enable new material for preparation and revision for the exit exams to be re-used among Ethiopian universities, sharing the development resources required and increasing the speed at which outstanding curriculum gaps could be collectively filled. By continuing to develop exit exam revision content, 100% of the blueprint would be available for online preparation in time for the 2024 round of exams.
+Maths for Natural/Social Sciences is a core course for all university degree programmes in Ethiopia. Nationally, a significant challenge is for Maths departments to coordinate the teaching and learning of thousands of students, with varying levels of interest and enthusiasm, taking this foundational maths course.
+STACK supports both lecturers and learners simultaneously:
+During the workshop IDEMS staff had the opportunity to interact with a number of members of the BDU Mathematics department who were commissioned by the Ethiopian Ministry of Education to write textbooks for secondary school mathematics. For various circumstances, the decision was made to share these textbooks in electronic format. As such, the textbook authors who attended the workshop were interested in the idea of making the textbooks interactive by integrating STACK questions. An agreement was reached to build a prototype of an interactive textbook for two topics which would demonstrate how students can practice and get relevant feedback on their attempts at questions. IDEMS are working with one of the textbook authors on creating a prototype for the topics of Equations and Vectors which they intend to present within BDU and potentially to the Ministry as an alternative delivery mechanism.
+The workshop was organised and hosteds by Bahir Dar University, Ethiopia, led by IDEMS International, UK, and supported by INNODEMS, Kenya, and Maseno University, Kenya. Financial support for the participation of key stakeholders was provided by the University of Geneva and the Swiss National Science Foundation.
+Konstantina Zerva, Gavin McWhinnie, Maria Christodoulidou
+ +This case study details the work and insights gained from a STACK-based internship during the summer of 2023 at the University of Edinburgh and reflects upon the unique experience of being a dedicated STACK intern. The internship was facilitated by the Employ.Ed on-campus internship program and ran for 12 weeks from June through August. The main aim of the internship was to contribute to the ongoing work in the School of Mathematics of creating and maintaining STACK materials for various undergraduate courses and to further contribute to the ongoing project of translating the HELM workbooks to STACK.
+At the start of this internship, training began with an introductory presentation by Konstantina Zerva, followed by self-study using the materials available online (namely the Quick Start Guide and the Documentation). After being introduced to STACK, the significant range and depth of features available can mean that the training needed to become a proficient question author can pose quite a challenge. Especially during this 12-week internship, it was important to quickly get a strong grasp of the basics. Training continued through the first week with small introductory tasks, such as reviewing and implementing minor modifications to existing STACK question banks.
+From our experience, the most effective approach to mastering STACK is learning directly from experienced STACK users themselves. Throughout the course of the internship, we were lucky enough to receive guidance and input from multiple STACK professionals working at the university. As an alternative, however, anyone is welcome to join the STACK community on STACK’s free online chat platform.
+Throughout the internship, close collaboration was established with course organisers to develop and refine STACK-based materials for various undergraduate courses within the School of Mathematics. Quality checks, question tidying up and revisions were carried out on the materials designed for the different courses, including:
+In particular, new material was created for the 2nd year Probability course, with an example question shown below.
+Furthermore, an active contribution was made to the ongoing project of translating the HELM workbooks into STACK format (see this case study). This included the incorporation of interactive randomised STACK questions and the addition of comprehensive worked solutions for practice exercises. HELM workbooks 19, 20, 23 and 25 were converted and are to soon be tested and put into use by first and second year engineering mathematics courses at the university.
+For the 4th year course Functional Analysis (MATH11135), there was a need to convert LaTeX course notes to html and then upload this to Moodle so that it could be used in conjugation with STACK to create interactive online notes. (For a previous example of this, see the use of STACK in Honours Complex Variables.) This process lends itself to being fully automated, which can be done with a fairly small Python program. Specific problems (with LaTeX to html conversion using Pandoc) encountered whilst developing this program led to the creation of pandoc-math, a Pandoc filter for converting mathematics course notes to html (GitHub repository). By using this tool, the resulting html can then uploaded to Moodle either manually or via a Python script to parse the html and convert it into Moodle XML.
+One of the highlights of the internship was our participation in a 4-day Erasmus+ Meeting at Johannes Kepler University in Linz, Austria. The meeting had a primary focus on the integration of STACK and GeoGebra, leveraging their combined strengths while also emphasising effective didactics in mathematics education.
+During the meeting, mathematics and teaching students from various European universities came together to combine their skills and collaborate on the creation of digital tasks. These tasks incorporated interactive STACK questions along with GeoGebra applets, enhancing the learning experience. At the conclusion of the conference, each team had the opportunity to present their collaborative work to professors and project leaders, showcasing their innovative ideas and solutions.
+A report of the conference is published here.
+The role of a dedicated STACK intern is undeniably an interesting and enjoyable experience. It's been fulfilling to belong to a global community of users and developers and a privilege to create new high-quality materials for future mathematics students. This section highlights some of the positives, interesting aspects and difficulties of this experience:
+STACK interns can greatly help in creating online content for undergraduate courses and are certainly a worthwhile investment! Provided with good training, interns can give attention to creating engaging, high-quality, and robust STACK questions, a task that academics and course organisers might not have the time or capacity to undertake. Often STACK interns are students themselves who have first-hand experience as STACK users, meaning they have their own insight into what is useful and what might be less engaging or even potentially frustrating for students.
+One of the most noticeable difficulties in writing STACK questions for undergraduate STEM courses as a STACK intern or similar, is that there is difficult balance between having the technical knowledge of what is possible in STACK and the pedagogical knowledge and teaching experience needed to create good questions. Often this means there are two people involved in every STACK question: an academic or teaching staff member who writes questions they believe will be possible to implement in STACK, and a teaching assistant or intern who actually creates the question. This can lead to the STACK author on the technical side receiving an (almost) impossible task or creating questions which are technically correct but are flawed from a teaching standpoint.
+STACK question authors tend to have their own approach to writing questions, with their own styles and quirks. This is likely due to the wide range and depth of features STACK has, allowing users to use what features suit them best. However, this can make collaboration difficult and reviewing other author's questions occasionally tricky. A potential solution for this issue is given under authoring collaboratively, on the page describing "authoring workflow" in the documentation.
+Authors: Juma Zevick and Santiago Borio.
+The adoption of STACK, an online assessment system for mathematics and other STEM courses in African universities, has been driven by the need for efficient assessment solutions, limited resources, and increasing student enrollment. To support the implementation of STACK and enhance mathematics education in Africa, IDEMS International and its Kenyan counterpart INNODEMS have collaborated to establish the INNODEMS STACK Internship Programme. This programme has trained a local team of interns who have successfully integrated STACK into undergraduate courses in universities across Africa. Through their training in STACK question authoring, technical support, and professional skills development, the intern team has played a vital role in the effective implementation of STACK in African universities. The programme has yielded positive results in Kenya, at Maseno University, Masinde Muliro University of Science and Technology, and Ethiopian universities where it is used in supporting students through a remedial program at Bahir Dar University. Challenges such as technical complexities, limited resources, and cultural contextual factors have been overcome through collaboration, regular training, knowledge sharing, and adaptability. The next steps involve expanding the internship programme, continuous capacity building, fostering collaboration and knowledge sharing, and securing additional funding to further support the implementation of STACK in African universities.
+ +STACK has gained significant attention and interest in African universities. This growing interest is primarily driven by the system's unique capability to address challenges associated with limited resources, large class sizes, and the demand for effective assessment solutions that enhance student learning. STACK provides African universities a powerful tool to overcome these challenges and improve the quality of mathematics education across the continent. IDEMS International, a UK-registered not-for-profit community interest company, in collaboration with its Kenyan counterpart INNODEMS, has actively supported African universities in adopting and implementing STACK in undergraduate courses.
+ +The initial observations made during the first IDEMS-led STACK workshop held in Maseno University, Kenya, in 2019, and subsequent workshops in 2022 at Maseno university and in 2023 in Bahir Dar University, Ethiopia, highlighted a growing interest and enthusiasm for the adoption and use of STACK in African academic institutions. These workshops served as valuable platforms for knowledge sharing, collaboration, and capacity building, fostering a supportive environment for STACK implementation.
+ +Recognising the potential of STACK to transform mathematics education, IDEMS, in collaboration with Maseno University and Masinde Muliro University for Science and Technology (MMUST) in Kenya, identified priority courses where STACK assessments could replace or complement traditional pen and paper assignments. These assignments were challenging to mark and provide timely feedback for due to the large class sizes, sometimes exceeding 1,000 students, with limited or no additional support.
+ +Initially, IDEMS directly created the STACK resources for these courses. However, as the demand for STACK adoption grew, the need for a local team of experts became evident. To meet this demand, an internship program was implemented, recruiting four Kenyan recent graduates who had prior experience using STACK during their undergraduate studies. These interns, along with two local contractors from IDEMS, formed a team and received comprehensive training from three experienced members of IDEMS to support the integration of STACK in higher learning institutions.
+ +The adoption of STACK assessments in African universities has emerged as a response to the pressing challenges faced by these institutions. With limited resources and the need for efficient assessment solutions, the collaboration between IDEMS International, INNODEMS, and African universities aims to enhance mathematics education by harnessing the power of STACK to improve learning outcomes and provide a more inclusive and accessible educational experience for students.
+ +STACK has gained significant attention and interest in African universities. This growing interest is primarily driven by the system's unique capability to address challenges associated with limited resources, large class sizes, and the demand for effective assessment solutions that enhance student learning. STACK provides African universities with the opportunity to overcome these challenges and improve the quality of mathematics education across the continent. IDEMS International, a UK-registered not-for-profit community interest company, in collaboration with its Kenyan counterpart INNODEMS, has actively supported African universities in adopting and implementing STACK in undergraduate courses.
+ +The initial observations made during the first IDEMS-led STACK workshop held in Maseno University, Kenya, in 2019, and subsequent workshops in 2022 at Maseno University and in 2023 in Bahir Dar University, Ethiopia, highlighted a growing interest and enthusiasm for the adoption and use of STACK in African academic institutions. These workshops served as valuable platforms for knowledge sharing, collaboration, and capacity building, fostering a supportive environment for STACK implementation.
+ +Recognising the potential of STACK to transform mathematics education, IDEMS, in collaboration with Maseno University and Masinde Muliro University for Science and Technology (MMUST) in Kenya, identified priority courses where STACK assessments could replace or complement traditional pen and paper assignments. These assignments were challenging to mark and provide timely feedback for due to the large class sizes, sometimes exceeding 1,000 students, with limited or no additional support.
+ +Initially, IDEMS directly created the STACK resources for these courses. However, as the demand for STACK adoption grew, the need for a local team of experts became evident. To meet this demand, an internship program was initiated, recruiting four Kenyan recent graduates with prior experience using STACK while students during their undergraduate studies. These interns, along with two local contractors from IDEMS, formed a team and received comprehensive training from three experienced members of IDEMS to support the integration of STACK in higher learning institutions. +STACK allows educators to automate assessments with immediate feedback, enabling efficient handling of large class sizes and promoting a more engaging learning experience. The collaboration between IDEMS, INNODEMS, and African universities aims to enhance mathematics education by harnessing the power of STACK to improve learning outcomes and provide a more inclusive and accessible educational experience for students.
+ +The one-year-long internship programme started in July 2022 and was managed by INNODEMS with training provided by IDEMS. To enhance understanding and proficiency in STACK effectively, the team participated in two weekly meetings and practical sessions led remotely by the IDEMS team from the UK and Austria, and two opportunistic week-long in-person sprints. This training provided an opportunity to leverage the collective knowledge and experiences of the team members, enabling them to offer comprehensive support for the implementation of STACK in different learning environments.
+ +The training program was intentionally designed to be dynamic, with a focus on creating content for undergraduate mathematics courses that were considered high-priority due to their significant enrolment and unique role in providing foundational knowledge for other subsequent courses that students are likely to take. This content was promptly made available to students at Maseno University and MMUST as soon as it was developed, ensuring a continuous and responsive learning experience for the students using the STACK platform. Carefully planned workflows, peer review of each new question, and a further review stage by more experienced authors facilitated by carefully adapted project management tools meant that three new courses integrated STACK assessments in the first semester of the 22-23 academic year, with a further 5 courses in the second semester. +The training focused on practical learning and capacity building, with three priority areas: STACK authoring, Moodle management, and the development of professional skills. This led to the following outputs by the intern team:
+ +The implementation of STACK in African universities, supported by the Internship Programme, has yielded promising results and success stories. These highlight the positive impact of integrating STACK into higher education institutions across Africa. The collaborative efforts between INNODEMS, IDEMS International, and local universities have contributed to the effective implementation and utilisation of STACK, ultimately enhancing the quality of mathematics education and supporting student success.
+ +In 2022, Maseno University added four additional undergraduate mathematics courses to its list of seven courses with fully developed online STACK resources developed through the support of the local team: Introduction to Probability and Statistics, Integral Calculus (Calculus II), Linear Algebra II, and Complex Analysis II. These courses have complemented the existing seven courses already developed at Maseno, including Differential Calculus (Calculus I), Descriptive Statistics, Basic Mathematics, Vector Analysis, Complex Analysis I, Measure Theory, and Linear Algebra I. Incorporation of STACK in these courses has enriched the learning experience for students and transformed the way assessment and learning is done at Maseno.
+ +“In 2018, I returned to Maseno after six years of PhD studies in the USA. One of my new responsibilities was teaching undergraduate maths classes and was immediately overwhelmed by the task. My first class was an introduction to Linear algebra that had 1000 students and I did not have any support in terms of teaching assistants. Clearly, giving homework, marking it, and giving immediate personalised feedback was going to be impossible, unless we innovated. The first time teaching the course, I used the traditional paper-based continuous assessment and I had a challenge marking. The next time I taught the course, I used STACK questions and was able to give weekly homework assignments and my students received immediate feedback which promoted learning on their part. Most students were very happy with the intervention and some expressed their hope of having more courses taught and their formative assessment done using STACK. Some students thought that it was too much work since they were used to only a few questions a semester in the form of continuous assessment. On my part as the course lecturer, I was greatly relieved since I could monitor student progress on a weekly basis and tailor my lectures accordingly. Formative assessment was easier and the exam performance for students that used STACK was much improved. I think it is a great tool that can support the teaching and learning of mathematics, especially in our situations where class sizes are too large and the support for teaching is nonexistent.” (Dr. Michael Oyiengo, Maseno University)
+ +Similarly, MMUST has successfully integrated STACK into five courses thanks to the efforts of the local team: Analytical Geometry, Advanced Calculus, Linear Algebra II, Complex Analysis II, and Vector Analysis, as well as using others developed for other universities with matching curricula. These courses now leverage the interactive features and personalised assessments provided by STACK, enhancing student engagement and understanding of mathematical concepts.
+ +“The academic year 2022/23 marked a significant turning point for MMUST in terms of delivery of mathematics courses. The Department of Mathematics was able to integrate STACK in the delivery of 8 courses. The implementation challenges notwithstanding, the support of interns in authoring the questions was incredible. +The presentations by MMUST lecturers at the just-ended first African STACK conference was very revealing in terms of the impact of STACK on the teaching and learning experience. It was evidently clear that the integration of STACK in course delivery is positively correlated with student performance. We can't thank IDEMS enough for the support offered so far. We plan to upscale the integration of STACK in our courses in the coming academic year. In this regard, we are counting on the continued support from the interns.” (Prof. George Lawi, MMUST)
+ +The Ethiopian Remedial Programme has seen significant progress through the implementation of STACK. The intern team, in collaboration with Bahir Dar University, has played a crucial role in developing STACK resources specifically tailored for the program. These resources have provided remedial students with interactive learning materials, formative assessments, and personalised feedback, enabling them to strengthen their mathematical skills and increase their chances of successfully entering university. Full curriculum coverage was achieved by the team in a 6 weeks period. The focus has now shifted to the Mathematics for Natural Sciences and Mathematics for Social Sciences courses that are compulsory to all undergraduate students irrespective of the subject of their studies and will be made available to all Ethiopian universities.
+ +
"Our journey with STACK began after attending a transformative workshop at Maseno University in 2019. Witnessing our students' remarkable progress using IDEMS-developed assessments for the 'Linear Algebra' course inspired us to embrace STACK in our teaching and assessments. We have since integrated STACK into our education system, offering personalized and immediate feedback, enhancing self-paced learning, and empowering our students to succeed. +Our fruitful collaboration with IDEMS led to the successful implementation of a STACK-based Remedial Program, providing crucial support to a large number of students. As we continue our partnership, we are developing an online Mathematics support course for Natural and Social Sciences. The vast number of first-year students enrolled in this course spread across different campuses, demands innovative solutions, and STACK enables us to deliver timely guidance and support. Looking ahead, we are eager to leverage STACK's potential for the national exit exam, a crucial requirement for our final-year undergraduate students. We aim to provide invaluable preparation tools for this significant exam. Together with IDEMS, we are paving the way for a brighter future in mathematics education at Bahir Dar University." (Mebratu Fenta Wakeni, Bahir Dar University) + +Godfrey Wabwire is a dedicated INNODEMS Intern who has been actively contributing to the development and implementation of STACK at various African universities. With a specialization in authoring STACK questions and a strong background in Mathematics education, Godfrey plays a crucial role in providing technical support and managing Moodle servers for the integration of STACK. He has served as a course administrator for Linear Algebra at Maseno University. Passionate about the intersection between technology and mathematics education. Godfrey is particularly intrigued by the transformative potential of STACK. Holding a degree in Education with a focus on Mathematics and Business, complemented by expertise in Information Technology, from Maseno University, Godfrey aspires to pursue further studies in Educational Technology.
+ +“The STACK internship program allowed me to develop my technical skills and expand my knowledge of STEM education, enabling me to create engaging educational content. The program's supportive environment and networking opportunities, along with the mentorship of individuals like Santiago, Georg, Juma, and Wastalas, have paved the way for future collaborations and my pursuit of postgraduate education.” (Godfrey Wabwire, INNODEMS)”
+ +Emmaculate is an Intern at INNODEMS as well, and her main focus has been on conceptualisation and authoring of STACK questions, and Moodle management amongst others. She has also been also been at the forefront of supporting MMUST, Kenya, and Bahir Dar University, Ethiopia with STACK integration. Emmaculate is diligent in providing valuable assistance to local lecturers and students whenever they encounter technical issues related to STACK. +Emmaculate holds a Bachelor's degree in Mathematics and Business Studies with a Information Technology from Maseno University. She finds great satisfaction in engaging with mathematical problem-solving and enjoys participating in fun activities related to the subject.
+ +“Through the internship program, I gained skills in question authoring, technical support, debugging and Moodle management. It helped me gain professional experience, networking opportunities and I was able to understand different cultures in terms of their educational systems.” (Emmaculate Atieno, INNODEMS)
+ +Nixson Holds a bachelor's degree in Mathematical Sciences with IT from Maseno University. He is also an Intern at INNODEMS, where he is trained in Basic Moodle Management and STACK authoring. Nixon played a crucial role in supporting MMUST with STACK integration in one of its courses (Advance Calculus) by managing the IDEMS eCampus site for the university and he has continued to provide his expertise towards the STACK project in other African learning institutions. +Nixson is passionate about technology that has a positive impact on society, which is why he is interested in exploring and learning more about the technical part of STACK and Moodle.
+ +“As an intern dealing with STACK, the program has broadened my perspective on the innovative ways technology can enhance learning outcomes as well as expanding my knowledge on computer-aided assessment for Mathematics.” (Nixon Kiplagat, INNODEMS)
+ +Mary holds a bachelor's Degree in Education Science (Mathematics and Physics) with IT. Mary also has a certificate in Web development and design from Ricatti Business College. She enjoys getting involved in social impact projects like the African STACK Internship program which focuses on improving mathematics education at the tertiary level through a collaborative initiative. +Mary Sayuni is a trained Moodle and STACK expert. She has been supporting MMUST with STACK integration in Complex Analysis and has continued to provide her expertise in other projects as well.
+ +“It [the Internship] was a great learning opportunity in regards to the exposure to different IT skills, maths courses, and the great professional networking involved in all the events of the program.” (Mary Sayuni, INNODEMS)
+ +The implementation of STACK and the internship programme in African universities presented several challenges for the local team. These challenges can be categorised under the following themes:
+ +Technical Challenges - the team embraced the opportunity to learn programming languages such as LaTeX, HTML, and JavaScript, all of which were unfamiliar to them in the beginning. Though the learning curve was steep due to limited prior experience with these tools, through dedication and concerted effort, they progressively gained proficiency, allowing them to skillfully develop and implement STACK resources.
+ +Limited Resources - the interns team faced challenges due to limited resources, such as access to quality hardware and internet connectivity, which affected their speed and efficiency in meeting the growing demand for STACK implementation and support in African universities. To overcome these limitations, the team employed resourceful strategies by leveraging on the existing infrastructure, formed collaborative partnerships, optimised workflows, adapted STACK authoring to the local context, and provided training and capacity building. These approaches allowed them to make the most of available resources, find innovative solutions, and ensure the successful implementation of STACK despite resource constraints.
+ +Familiarisation - as STACK was a relatively new tool in the African educational context, the team faced the challenge of familiarising themselves with its functionalities, features, and best practices. They had to invest time and effort in understanding how to effectively integrate STACK into existing courses, design assessments, and provide appropriate feedback to students. This process required continuous learning, experimentation, and adaptation to ensure the successful utilisation of STACK. A four-layer review of content created ensured the quality of the content created by the team.
+ +Cultural and Contextual Factors - the team had to consider the diverse educational backgrounds, teaching methodologies, and student learning styles across different African universities. Adapting STACK to suit these varied contexts and ensuring its alignment with local educational practices and requirements required careful consideration and collaboration with stakeholders. +Additionally, cultural differences between the UK, Austrian and Kenyan team members, coupled with remote training, meant that initially the learning was slow. Once cultural barriers were overcome, the training became more effective and efficient, leading to the successful development of a team of experts.
+ +The interns played a crucial role in the success of the first African STACK Conference for Undergraduate Mathematics, from 19 to 23 June 2023, held at MMUST, Kenya. They actively participated in the conference, presenting their experiences and insights gained from supporting the implementation of STACK in African universities, and played a key role in the facilitation of the workshops.
+ +During the conference, it became evident that the internship program had immense potential for expansion and replication across Africa. The next steps identified after the conference include expanding the internship program to other African countries, providing continuous capacity building to the current interns to enable them to train and mentor new teams of interns, fostering collaboration among interns, and promoting knowledge sharing within the team. Additionally, there is a need to secure more funding to support the expansion and sustainability of the internship program, to ensure its reach and impact can be extended to more universities across the African continent.
Meike Akveld, George Ionita, Andreas Steiger
+The question in figure 1 is a typical exam question for a first year analysis course for civil engineers at the ETH Zürich.
+The method needed to solve this question is integration using partial fractions and used to be taught intensively in high school. In most high schools throughout Switzerland this is no longer the case. Of course one could discuss whether it is still necessary, in view of technology, to teach this method, but we do not want to go into this discussion here. We simply assume for the time being that this is still a skill that we want to test in an exam.
+In the past we tried two ways of teaching this in a classical lecture. One way was to develop the theory behind it; this was very time consuming and did not really interest most of the students as all they needed in the end was the recipe how to do it. Alternatively we simply gave them the algorithm and did some examples with them in class. This was not satisfying either, as the anti-derivatives that showed up there were not intuitive and maths seemed to have become a tool box (or even black box?) that no one understands, but one just applies, strongly contradicting our beliefs as educators.
+In this Case Study we will introduce a more satisfying solution for us to teach this topic, both for students and for lecturers, using STACK.
+We designed a Moodle Quiz which gives a step by step introduction to the method of using partial fractions to integrate rational functions.
+The quiz assumes very little prior knowledge. Students only need to know the factor and sum rules for integration and the following anti-derivative
++
Students who want to discover this method all by themselves can try to do so, but they will need to invest a fair amount of time (or perhaps refresh their high school knowledge). For others we provide extensive hints and solutions.
+Whatever the students decide to do, they will be guided in a natural way to the more complicated anti-derivatives and thereby develop a deeper understanding of the method and the results.
+As you can see in figure 2, the quiz is divided into many small sections, each focusing on one step at a time. Figure 3 shows the first section which deals with integrating very simple fractions. All a student needs to know and apply is the basic anti-derivative (1). The question on the left should be straightforward, but for the question on the right a simple substitution is required. If the student does not remember how to do this, the two offered hints should help. We used the setup for hints programmed by Michael Kallweit, see [1]. These hints are not visible upon first opening the question, but can be opened by clicking on a button.
+The next series of questions, shown in figure 4, demonstrates the general principle of the quiz. After starting with an easy example the students are led to more general situations. They can always use hints when it is unclear why one question should be seen as a generalisation of a previous one. If the hints do not suffice, the student can also ask for a worked solution and then try the question again. We use gently randomised numbers for new attempts such that some work is required when trying again.
+These STACK questions are part of our so-called Integral Trainer (see [2]). This tool is designed for students to practice and improve their integration skills. The project was conceptualised by two experienced maths lecturers and programmed mainly by our STACK developer.
+The quiz about rational functions consists of 22 questions and covers rational functions with general quadratic denominator or with cubic denominators with real roots, see figure 5.
+None of the questions use the Int answer test, but rather the AlgEquiv answer test. The main reason for this is the constant of integration. Using the latter test the teacher can decide whether or not to give partial credit if only the constant of integration is missing.
+Another delicate situation arises when computing integrals of the form + +for which the discriminant of the de-nominator is see figure 6.
+Is the answer or ? Of course it depends on the discriminant. If , then both answers should be interpreted as correct, but if , then the absolute value is mandatory. Thus we use a certain trick to distinguish between these situations. The idea is to determine a point that indicates whether the absolute value is needed or not. In this case we considered the -coordinate of the vertex of the parabola associated to the student's answer. Then we calculated its logarithm. If this number has imaginary part equal to zero, then the absolute value is not needed, otherwise it is mandatory.
+Unfortunately, the Int answer test did not completely meet our needs. The example above was one of the questions which ignited a technical discussion with Chris Sangwin about the use and limitations of the Int answer test. We hope that this will lead to improvements and calibrations in its design. For more information on how the Int answer test works, we refer to the STACK documentation [3].
+This quiz was so far used in two ways: One lecturer instructed the students to solve the first part of the quiz before coming to class (as with a Flipped Classroom) and then only showed some harder examples in class. Another lecturer explained the method in a traditional style and used the quiz as extra practice material. Both sets of data have not been evaluated yet, nor the effect of the quiz on the exam.
+As a next step we will (certainly) try to evaluate both the data collected in our quiz and the exam results. After we have analysed those, further steps will hopefully become clear.
+[1] M. Kallweit, Programming hints for stack, 2022, last accessed 3 March 2023. [Online]. Available: https://moodle.ruhr-uni-bochum.de/mod/page/view.php?id=1674600
+[2] M. Akveld, G. Ionita, and A. Steiger, "Integral trainer," 2022, last accessed 19 April. [Online]. Available: http://www.math.ethz.ch/stack/
+[3] C. Sangwin et al., “Stack documentation, answer tests,” 2022, last accessed 19 April 2023. [Online]. Available: https://docs.stack-assessment.org/en/ +Authoring/Answer_tests/
Franca Hoffmann, David Stern and Juma Zevick
+The American Institute of Mathematics (AIM) hosted an engaging workshop on Open Source Mathematics Curriculum and Assessment Tools at Maseno University in Kisumu, Kenya, from August 5 to 9, 2024. This event brought together a vibrant mix of educators, researchers, and software developers. The workshop was organised by Joe Champion from Boise State University, Franca Hoffmann from CalTech, Michael Obiero from Maseno University, and Mary Ochieng from Strathmore University, with support from the National Science Foundation (USA). It aimed to improve the use and impact of open educational technologies, including STACK, in mathematics. The event also set the stage for the International STACK Conference at the Technical University of Kenya, starting on August 12, 2024.
+This event adopted the AIM workshop format, encouraging participants to propose open problems and questions before it began. These suggestions ranged from specific issues to broader challenges that could shape the future of the field. The workshop included lectures to equip participants with the necessary background on these problems, followed by parallel working sessions for collaborative problem-solving. It was designed to foster collaboration between mathematicians, researchers in mathematics education and developers of open-source tools, including WeBWorK and STACK.
+The main topics for the workshop were
+In addition to featured talks, participants engaged in focused group discussions. Key topics are summarised below.
+For open-source educational software to be competitive with commercial options, collaboration on development and content is key. WeBWorK is widely used in the US and part of its attractiveness is a well-organised question bank. Questions are tagged by topic and through a user-friendly interface can easily be integrated into worksheets and courses. However, ownership of questions and derivatives is not clearly traced. +For STACK, several question banks have already been developed by different groups (some open, some closed), and additional question banks are rapidly being developed. There is a pressing need for a structured approach for sharing and collaborating on the development of questions in STACK. A concrete plan for a general library of STACK questions, tagged by topics, emerged. However, this is not enough.
+The discussions produced a clear vision: A system for a structured repository of curated material, allowing for multi-layered ownership. Here, by "curated" we mean material that is (i) tagged by topic, (ii) quality-controlled, and (iii) exists within a broader context. For instance, a question may exist within a sequence of questions intended for a specific pedagogical/educational purpose, which exists within a worksheet, topic, chapter, course etc, and may be adopted to the requirements of a given educational level or curriculum. +Ideally, such a repository of materials should be usable across different platforms and educational tools so that the user is not constrained by the choice of tool. The participants discussed how to overcome technological barriers for this goal, and PreTeXt presented itself as a vehicle in this direction.
+Accessibility could serve as a minimum requirement for writing formats (e.g. a screen reader being able to interpret the content). In order for groups maintaining question banks for their own context (at the level of a continent / region / country / institution / department / individual) to take ownership of this process, a structured approach is needed. The group discussed the architecture for this vision, both from a technological and managerial perspective, and concluded that working towards this vision represents an opportunity for novel ideas to emerge for instance for the development of new technology to allow for version control of content and its derivatives across multiple ownership levels. Some of the technological challenges were then taken up by the interoperability discussion group.
+The vision of having question repositories which cut across multiple tools requires technical interoperability beyond current capabilities. For this the workshop participants discussed:
+For agnostic authoring, the potential of markup languages that capture the structure of learning materials, while being both highly human-editable and computer readable, was discussed. The initial efforts in this direction by PreTeXt were discussed at the meeting. PreTeXt documents can be published in multiple formats, as they separate the task of authoring from publishing. PreTeXt could potentially play the role of a question-engine agnostic authoring tool; it currently integrates WebWorK, and the potential of integrating the new STACK Application Programming Interfaces (API) in the future was discussed. While PreTeXt may be able to render questions in a variety of tools, it will come at the price of losing rich tool specific features. It was discussed that there is already a user base for which interoperability and access to a wider bank of material is a priority. Over time, the agnostic authoring capabilities could be extended, potentially enabling open tools to reflect on and extend their own capabilities in the process.
+For author interoperability, a common authoring style is key, and whilst this may mean making choices on the development side for specifying a meta-framework, targeting 80% of automatic correct cross-platform translation seems like a good tradeoff between needed sophistication and quality of desired functionality. +Interoperability requires standardised protocols and APIs; an API for STACK is now available as of recently. For translation of material/markup languages as well as translating content to other human languages, AI may be a useful tool, as long as appropriate training sets can be identified. For interoperability between two platforms, the goal is to design two one-way systems instead of one closed-loop, therefore retaining options of more advanced features in both systems. Further, one needs to explicitly think about managing ontology: first drawing out explicitly the different schemas of the various systems, and then asking how to map subsystems to each other. The overall goal is that users can set up their own system without having to commit to one tool (as is usually the case with commercial software), but rather can access a variety of tools, drawing on the power of collaboration in open-source software. To move forward successfully, it is vital that the community develops a common vision for cross-platform interoperability.
+The workshop aimed to identify technical barriers and promote the development of cohesive learning platforms. This approach is vital for enabling educators to integrate various technologies, like STACK, WeBWorK and PreTeXt, and for this to be possible within a given context and with local ownership lead to the concepts of multi-layered ownership and evolvable (meta-)ontology. +The aim emerged to have an ecosystem based on content rather than the choice of tool. This could be based on open principles, with a distributed model for appropriate multi-layered ownership and version control, allowing for both public and private repositories. Making these ideas a reality asks for new approaches and new technological tools to be developed - exciting future directions.
+Both technological and human-related barriers may prevent the adoption of maths education technologies. From the discussions emerged a structured list of barriers to adoption in the African context, focusing on secondary and tertiary education. Further discussions focused on innovative uses of open online curriculum and assessment tools to address these barriers in terms of digital and human–driven innovations in training, education, and instruction. Participants discussed how to use these tools to tap into existing student intuition and skills, and how to build on the historical nature of exercising maths, which tends to be pen-and-paper based. Innovative uses in instruction include using technology as a tool to improve interaction between instructor and students by focusing on thought-revealing activities, on overcoming social impediments in teacher-student relationships and the learning process, on task improvements (eg via scaffolding as in faded worked examples), and on real-world problem based learning. Innovative uses may allow students to concentrate on the problem rather than the maths, and on student-to-student interaction (e.g. through team-based grading, dialogic interaction, de-emphasizing competition, collaboration and more able peer-to-peer learning). If done well, this enables students to participate in the development of tech not as testers, but as conceptualizers. In this, the educational context is shaping the innovation, e.g. access to different types of devices.
+A key focus of the workshop was inclusivity. Considerations for students with special needs in the development and implementation of educational technologies are still lacking behind. Participants discussed access barriers and how tools can be adapted to overcome them, needs assessment of existing technologies, identification of appropriate assistive technologies for various categories of special needs, and engagement with mathematics experts with special needs for identification of appropriate technologies and necessary adaptations. Concrete ideas for an exploratory study on assistive technology and learning of mathematics in Kenya emerged. Next steps include bringing together experts to start exploratory research, with a focus on identifying needs and potential technological solutions.
+Overall, the following questions took centre stage in the discussions on innovative uses: How can innovation introduce more empathy in the learning process? Can it create space for non-intuitive maths students, e.g. via a tool for capturing how non-intuitive maths learners have breakthroughs in learning? How can we use technology as specialised, personalised tools for students to learn in the way(s) they learn best? Can the tools reflect the cognitive empathy of a great teacher (e.g., anticipating student responses, responding to student reasoning)? How can we innovate maths communication, leveraging different ways that people arrive at understanding the same maths concepts? Participants recommended including research in the process of continuously improving STACK and other educational tools to answer these questions.
+Participants discussed the adoption of the Kenyan Competency Based Curriculum (CBC) and shortcomings/benefits thereof. The CBC has been introduced recently (2024) in Kenyan secondary schools, and it is expected that lecturers adapt their curriculum and delivery modes once these students arrive in Kenyan universities. These discussions are relevant more broadly as several countries across Africa are considering CBC implementation. Further, the group built a shared understanding on generic challenges in mathematics teacher education and challenges associated with the use of STACK and other tools for pre-service mathematics teachers. Moving forward, participants proposed collaborations on multinational curriculum design teacher training, including via synchronous curriculum discussions/meetings and joint funding proposals.
+There is a pressing need for training on emerging technologies to enhance the teaching of mathematics at all levels, including specifically for basic education and higher education facilitators in response to the CBC curriculum. Technological advancements in mathematics education have highlighted the gap in awareness and optimal usage among mathematics teachers. Participants worked on concrete next steps: (i) Developing and launching a postgraduate program in Master of Science in Mathematical Innovation; (ii) An EdD (Mathematics Education Professional Doctorate Program) designed for mathematics educators, leaders, and professionals who want to advance their knowledge and expertise in education; (iii) building partnerships and collaboration networks for Mathematics Educators and Researchers, with Kenya Mathematical Society taking a lead in the process.
+The Master of Science in Mathematical Innovation was identified as a key priority for the Kenyan context. The Open University of Kenya was proposed as an initial implementing partner, given the potential of a large-scale eLearning programme. International and Kenyan partners agreed to support the development and initial implementation, as it was recognised that no individual institution has the capability to develop this ambitious ground-breaking programme alone.
+The general discussion focused on opportunities within the community to develop research projects related to existing instructional contexts in Africa, noting that developing publishable results can be observational and would be excellent contributions to the literature. After specific research questions are identified, next steps could include applying quantitative and qualitative methods to more refined versions of these studies.
+Seeing challenges as opportunities can drive positive change; for instance, the large classes in many African universities provide the opportunity to rapidly innovate on teaching practices, for the use of technology and comprehensive data collection for impactful research. Participants discussed implications of large classes in terms of assessment, instructional methods, instructor training, and student learning experiences, and it was recognised as a driving force behind many of the issues discussed at this workshop.
+By the end of the workshop, participants produced plans for 7 concrete collaborative studies, including a study on "STACK Online Assessment in Large University Mathematics Courses" to be launched this month involving universities in several African countries and a growing list of international collaborators, focusing on the following questions:
+The workshop participants emphasised the hidden costs of open-source tools: whilst the software itself may be free, implementation may still rely on investment of appropriate resources for servers and training / authoring / integration. Whilst the long term goal includes sustainable partnerships, it is more advantageous to prioritise successful implementation in the short-term. The key question is how to transition from unsustainable to sustainable solutions for the use of EduTech tools in maths. The priorities may look different for different types of users (instructors, administrators and students). A model that has proven successful is for an enthusiastic faculty member to become the main local manager. Several business models for open-source software already exist in the literature. STACK seems to be mostly operating under Business Model 4 of FOSS (Free & Open Source Software) business strategies From Fitzgerald (2006) as it helps sell research and implementation. An alternative revenue opportunity that may be leveraged by the STACK team is course-specific material creation within STACK.
+In general, what is needed for broader adoption is the right combination of (1) a turnkey solution with excellent user experience, (2) reliable pricing and services, and (3) research-based evidence of outcomes. The community formed at this workshop can serve as a resource for developers and users trying to work out sustainable business models.
+Lecturers expressed concern how to adapt to the increasing use of generative AI by students. This sparked various conversations leading to a consensus that this shift presents an opportunity to reframe our assignments and exams, encouraging the development of higher-order assessment items for students. To support educators in this transition, resources like the Responsible AI for Lecturers' course developed by IDEMS were made available for use, offering guidance on effectively integrating these tools in their teaching. +This was identified as a highly topical issue which warranted deeper consideration. While some participants intend to collaborate on trialling initiatives in their context, it was felt that this topic could benefit from taking a more central role in a future event.
+The workshop concluded with several key recommendations and plans for future actions. Concrete next steps already on the way include:
+Collaborating towards launching a new masters program “Master of Science in Mathematical Innovation” at the Open University Kenya, in time to support the training of secondary school teachers for the upcoming switch to a Competency Based Curriculum, with the first student potentially to enrol in January 2025. +Seven concrete collaborative research studies on the timescale of 5+ years with a growing list of partners and with the first study to be launched in September 2024. +Conceptualising pathways for content sharing with multi-level ownership including first concrete ideas how to structure ownership levels and re-envision version control, bringing together developers, key partners from the Topos Institute and teams managing question banks.
+In addition to the above concrete outcomes, plans for research visits and joined grant proposals have been launched at the workshop, as well as concrete ideas for future AIM workshops to be held in Africa on related topics. More generally, participants agreed to establish a network of educators, researchers, and developers to address the challenges identified during the workshop. This network aims to drive the adoption of open-source tools like STACK and ensure they are effectively integrated into educational practices and a wider ecosystem of tools. Emphasis was put on the importance of community support, professional development, and iterative feedback loops to improve the integration of STACK and similar tools.
+The meeting ended with a sense of real momentum, and the participants have committed themselves to leading the way in enhancing mathematics education through innovative technologies. The workshop has not only laid the groundwork for future collaborations but also set a clear path for the continued development and improvement of tools like STACK, ultimately benefiting students and educators across diverse learning environments.
Juma Zevick, Michael Obiero, Ben Obiero, Danilo Lewanski, Christine Letitia, Mary Ochieng
+The second African STACK Conference for Undergraduate Mathematics was held at the Technical University of Kenya from August 12th to 16th, 2024. It was built upon the momentum from the first conference, further bridging the gap between experienced mathematics educators and the need for effective formative assessment in African universities through the use of STACK resources. With participation from institutions across Africa – including Ghana, Kenya, Rwanda, and Tanzania – and international collaborators from Germany, Italy, and the UK, the conference emphasised the exchange of experiences, methodologies, and the impact of STACK integration. Key discussions centred on developing a certification framework for novice STACK users, the expansion of IDEMS' Open Question Banks, and strategic planning for the institutionalisation of STACK across African universities. The conference highlighted Africa’s potential to lead in digital assessment innovations and provided a forum for peer review and strategic planning. The event concluded with clear next steps, including plans for broader adoption, integration into Technical and Vocational Education and Training (TVET) institutions, and future conferences, solidifying the foundation for continuous growth and collaboration in digital education across the continent.
+The second African STACK Conference for Undergraduate Mathematics was conceived as a continuation of the successful first conference, which set the stage for advancing the use of technology in teaching and assessing mathematics across Africa. The initial conference highlighted the pressing need for effective formative assessment and the potential of STACK to address this challenge — the second conference aimed to build on these discussions by deepening engagement with STACK resources and methodologies. +With the African STACK Community Leadership Team in place and access to IDEMS' Open Question Banks, the conference served as a strategic opportunity to drive broader adoption of STACK. It facilitated the exchange of best practices and experiences among educators who implemented STACK, enriching the community's collective knowledge and method. +Additionally, the conference sought to forge international partnerships to evaluate the impact of digital interventions in formative assessment, particularly in low-resourced environments. By leveraging the data and insights gained from STACK's growing adoption in undergraduate courses, the conference aimed to inform and refine the strategies for improving mathematics education. +In essence, the conference built on the foundation laid by the first event, strengthening discussions and collaborations to enhance the use of digital technologies in formative assessment and to promote innovative practices across African universities.
+This collection of proceedings captures the discussions and presentations that emerged from the conference, reflecting the ongoing efforts of educators in enhancing formative assessment and fostering innovation in STEM education in African universities.
+The conference kicked off with an introductory session led by the host, followed by a series of talks and presentations throughout the day. +David Stern from IDEMS International gave a keynote address, focusing on the importance of local leadership in driving educational change. He shared examples from Ethiopia (Bahir Dar University), Kenya, and Tanzania, highlighting how challenges can be turned into opportunities for innovative educational approaches, especially in resource-limited environments. The 5-quiz model was mentioned by David Stern as an ideal model for online instruction. Aspects of the 5-quiz model are diagnostic, content, mastery, test, and extension quizzes, all designed to enhance student experience with online learning of mathematics. +Prof. Levi Mbugua (Technical University of Kenya) spoke about the need for engaging, technology-driven teaching methods in mathematics, advocating for personalised, interactive learning environments supported by digital tools and AI. Idrissa (University of Dar es Salaam) presented his ongoing research, focused on improving maths education in Tanzania through Participatory Action Research. Nixon (a member of the INNODEMS organisation, which has been at the forefront of supporting STACK integration) shared his experience in supporting STACK integration across African universities, highlighting the collaboration between the African STACK Professionals' team, INNODEMS, and local universities in Africa and Italy (University of Trieste). +The presentations were followed by a Q&A session that brought up important discussions on licensing, ownership of STACK content, and the need for contributions to the Open Questions Bank on the IDEMS Moodle server. Content adaptation for diverse contexts and the challenges of implementing digital tools like STACK in different educational environments were also discussed. The day ended with a practical workshop led by the INNODEMS team, aimed at familiarising participants with STACK and the open question banks on the IDEMS' Moodle site.
+The second day of the conference began with Mike Obiero's opening remarks on the day’s events, followed by a series of presentations. +Ben Obiero (Technical University of Kenya, TUK) gave the first presentation on STACK's implementation at TUK. STACK has been integrated into various courses at TUK, significantly aiding lecturers and students with automated assessments and immediate feedback. This integration has improved student performance and engagement, though challenges such as formatting issues and misuse concerns were noted. Ben also highlighted ongoing efforts to collaborate with key stakeholders at TUK to streamline the integration. +Immaculate Odhiambo (INNODEMS) shared her experience in supporting STACK integration at Rongo University, emphasising the need for continuous capacity building and offering support. +Franca Hoffman (Caltech, USA) reflected on the broader motivations for teaching mathematics and outlined her efforts to integrate STACK into Caltech's curriculum, to support mathematics education. +The rest of the presentations were from Daudi (University of Dar es Salaam) on student experiences with STACK, Wastalas (University of Trieste) on integrating Adaptive Learning and Item Response Theory, and a second keynote by Mary Ochieng (Strathmore University) on enhancing pedagogical practices with STACK. Mary's keynote speech focused on using formative assessment to improve instruction and how formative assessment can guide our research agenda with the use of STACK. +Prof. Chris Sangwin's (The University of Edinburgh) presentation focused on improving online assessments and course design to enhance student learning and engagement. He emphasised that assessments should align with teaching objectives to genuinely support and measure learning, and proposed structuring courses around quizzes rather than traditional textbooks. This method creates an interactive feedback loop, improves knowledge retention, and allows real-time application and testing of knowledge. +After all the presentations concluded, there was a Q&A session and participants were encouraged to ask questions and share their reflections on the presentation. The discussions centred on providing detailed feedback for assessments, measuring question difficulty, and improving teaching through data. Participants debated methods for breaking down questions and delivering specific feedback to aid student understanding. They also discussed measuring question difficulty using various methods and how data science can capture and interpret student performance to refine teaching strategies. Chris’s insights into organising STACK questions in a sequence further contributed to the conversation on effective pedagogy. +In the afternoon, the workshop continued from the previous day, with lecturers selecting and reviewing STACK questions to ensure their relevance and mathematical suitability for their upcoming courses.
+On the third day, the morning session opened with remarks from Beth Kiratu (Institution is missing), setting the stage for a series of insightful presentations. Chris Sangwin (The University of Edinburgh) delivered the keynote lecture, focusing on recent developments in STACK, particularly around Parsons and Proofs. He demonstrated various types of proofs, such as reformulation and proof by contradiction and induction, emphasising their pedagogical and technological aspects. Chris highlighted the hierarchical structure of proofs and the challenges in teaching and assessing them using tools like STACK, advocating for continued innovation to bridge the gap between traditional proof-writing and technological assessment. +Christine Letitia (INNODEMS) followed with an overview of the organisation's initiatives. She discussed INNODEMS' role in educational innovation across Africa, including workshops, training, and the expansion of STACK usage. Letitia highlighted the growing interest in STACK integration and the development of an internship program to train students in STACK authoring. +After the tea break, Francis Torgbor from GHAIDEMS (a local branch of IDEMS in Ghana) shared insights from his teaching experiences at AIMS Ghana and AIMS Cameroon, highlighting the positive impact of STACK-based quizzes on statistics education. Wycliffe Rao (University of Nairobi) discussed how STACK has been adopted to handle large-scale student assessments and its advantages over traditional methods. Wastalas (in his second talk) presented his research on integrating adaptive learning features into STACK using machine learning algorithms. Juma Zevick (University of Trieste) detailed his analysis of STACK usage at the Department of Mathematics in Trieste and highlighted an ongoing initiative to foster collaborative research between Italian and African educators, including a currently available funding opportunity for a maths education research exchange program to Italy. Lastly, Godfrey Wabwire (INNODEMS) explained the process of developing and reviewing STACK questions, from conceptualization to implementation, and addressed the associated challenges and improvement measures.
+The morning session of the conference began with a keynote by Dr Beth Kiratu (Open University of Kenya) who emphasised the transformative role of digital tools in mathematics education. She shared how maths stereotypes during her childhood led to a decline in interest in the subject. Dr Kiratu stressed the need to shift these perceptions and foster a more supportive learning environment, particularly in resource-limited settings, through the use of technology. She also highlighted the challenges revealed by the COVID-19 pandemic, particularly the difficulties in adapting to online learning, and advocated for the integration of digital assessment tools like STACK to offer immediate feedback and support personalised learning. +Following Dr Kiratu's keynote, Dr Michael Obiero (Maseno University), presenting on behalf of Abdu (Bahir Dar University, Ethiopia), discussed the role of STACK in supporting students preparing for national exams in Ethiopia. This initiative is part of the Ethiopian Government's new system to standardise entrance and exit exams across universities, aimed at improving accountability and educational outcomes in higher education institutions. Dr Obiero explained that exit exams are intended to ensure all graduates have mastered core competencies, aligning educational achievement with labour market demands. His presentation highlighted the effectiveness of mastery-based learning through STACK and addressed challenges such as large student populations and resistance to new assessment methods. He also spoke about the ongoing efforts to develop content for national exit exams and remedial programs, despite obstacles such as the political instability in the region. +Lucian Talu (INNODEMS) focused on the importance of faculty training for the successful integration of STACK. She emphasised the need to invest in structured, peer-led training programs to ensure that educators can effectively use STACK, thereby enhancing content creation and improving student outcomes. In subsequent presentations, Owen Kizito (INNODEMS) discussed the potential of using STACK in mathematics competitions such as Math Kangaroo to improve assessment accuracy and enrich learning experiences. Mary Sayuni, also from INNODEMS, shared her recent work on course development involving real-time data and data visualisation. The session concluded with David from Mt. Kigali University (formerly a branch of Mt. Kenya University) expressing his intent to implement STACK in the upcoming semester to address assessment challenges. Birte Reich (Paderborn University, Germany) shared insights from her work on the positive impact of STACK on student performance in digital learning environments, as well as details of a current project at her university, focused on building an open question bank for STACK. +The afternoon session continued with a workshop where lecturers worked on organising STACK questions from the Open Question Banks into a structured course outline for the upcoming semester.
+On the final day of the conference, the morning session began with a presentation from Juma Zevick, who showcased his ongoing project of building an interactive Moodle Analytics web tool to support STACK users. Juma demonstrated the current progress, stemming from the 2024 Sage Foundation Hackathon Challenge, which aimed to support large research projects with technical expertise. The project is still in development, and the next steps involve collaboration with experts to finalise it into a STACK-type analytics plugin. +Following this, Prof. George Lawi from Masinde Muliro University of Science and Technology (MMUST) discussed the institutionalisation of STACK at African universities, focusing on the progress made at MMUST. He addressed challenges such as limited resources, large class sizes, and academic dishonesty, highlighting how STACK has been integrated into various mathematics courses since 2019. Prof. Lawi emphasised the importance of embedding innovations like STACK within institutions, with clear indicators of successful institutionalisation, including widespread use and continued implementation. +Dr Michael Obiero (Maseno University) then reflected on the integration of STACK across Africa, highlighting the need for technological innovations in teaching and assessment. He discussed the challenges that African universities face, such as large class sizes, insufficient resources, and resistance to change. Dr Obiero presented STACK as a crucial tool for addressing these issues but stressed that successful integration requires proper training and support. He advocated for local solutions, collaboration, and the development of resources and support structures to ensure the effective use of STACK. +The plenary session focused on key strategies for integrating STACK across the diverse educational contexts in Africa. Discussions covered certification and professional development, research capacity building, integration of STACK in TVET institutions, building a digital culture in education, and institutionalising STACK across universities. Participants emphasised the importance of collaboration, sharing success stories, and ensuring the sustainability and scalability of digital initiatives such as STACK. +The conference concluded with a commitment to collaboration, with participants agreeing on specific action points, including developing a strategic plan, establishing a certification framework, and creating a support network for educators using STACK. The discussions underscored the need for continuous improvement, monitoring, and strategic partnerships to sustain the momentum of STACK and its impact on mathematics education in Africa.
+The conference was marked by several notable successes, reflecting the growing impact and adoption of STACK across African universities besides other non-African countries. Key achievements included:
+Widespread Participation and Engagement. The conference brought together a diverse assembly of educators, researchers, and stakeholders from both Universities and TVETS. They represented institutions across Africa (such as Ghana, Kenya, Tanzania, and Rwanda) and internationally (from Germany, Italy, the UK, and the US). This broad participation enabled the exchange of ideas, best practices, and innovative strategies for integrating STACK into undergraduate mathematics education, as well as inspiring new collaborative research directions in maths education at the tertiary level.
+Enhanced Understanding and Application of STACK. Through a series of presentations, workshops, and discussions, participants gained profound insights into the diverse capabilities of STACK, especially in the realm of formative assessment. Chris Sangwin's updates on the Parsons' block and proof writing, along with David Stern's innovative 5-quiz model, demonstrated STACK's potential to significantly enhance student learning and engagement. Additionally, projects shared by other speakers such as Wastalas (from Benin, a PhD student at the University of Trieste, Italy), and Birte (another PhD student from the University of Paderborn, Germany) among others, highlighted STACK's broader applications, showcasing it as a tool not only for assessment but also for instruction. +The discussions on licensing, content ownership, and contributions to the question banks were particularly impactful, addressing key concerns within the African community and setting the stage for more effective and streamlined integration of STACK across institutions.
+Successful Implementation Stories, Strengthening New Networks and Collaborations. Several successful implementations of STACK were highlighted by institutions such as the Technical University of Kenya, the Open University of Kenya (a virtual Kenyan university), the University of Dar es Salaam, and Rongo University. These case studies demonstrated tangible improvements in student performance and engagement, highlighting both the successes and challenges of implementing STACK. They underscored the technology's potential to transform mathematics education, particularly in resource-limited environments. Additionally, the conference emphasised ongoing cross-continental research and development initiatives between African universities and the University of Trieste. Discussions also touched on plans to establish STACK at Caltech in the US, further driving the agenda of expanding STACK usage and fostering new networks and research opportunities beyond Africa.
+Strategic Planning for Future Growth. A significant outcome of the conference was the consensus on key strategic actions. These include developing a certification framework for novice STACK users, who will receive training on its usage and implementation, followed by certification. Additionally, the conference emphasised the institutionalisation of STACK in African universities, ensuring its sustainability with ongoing support from the INNODEMS team, who have been instrumental in the question authoring and providing technical assistance. The creation of support networks for educators, along with commitments from key stakeholders to drive integration efforts, were also highlighted. These initiatives are vital for maintaining the momentum generated by the conference and ensuring the continued growth and impact of STACK across the region.
+To build on the momentum generated by the 2nd African STACK Conference, the following next steps were outlined:
+Development of Strategic Plans: The conference outlined the creation of a certification framework to recognise educators who integrate STACK into their teaching, with structured training programs and clear timelines for adoption. Additionally, efforts to institutionalise STACK within university curricula will continue, focusing on measurable success indicators such as increased usage, sustained implementation, and improved student outcomes, alongside exploring new collaborative research initiatives.
+Expansion and Scaling Efforts: The next steps include expanding STACK's reach to more universities, particularly in low-resource settings, by scaling successful practices, sharing resources, and providing ongoing support. Additionally, efforts will be made to integrate STACK into Technical and Vocational Education and Training (TVET) institutions, which would stand to benefit greatly from its capabilities in formative assessment and personalised learning.
+Future Conferences and Events: Plans are in place to organise future conferences or related events, with the next potential conference in Ethiopia, to deepen discussions initiated at this conference. These events will focus on themes of innovation in maths education, collaboration, and capacity building, ensuring continued progress in the use of STACK across Africa.
+Resource Development and Support: the IDEMS' Open Question Banks will continue to be developed and expanded, incorporating feedback from the conference. This will involve creating new questions, reviewing existing ones, and ensuring their relevance to various educational contexts. Support structures for educators, including peer networks, online forums, and regular training sessions, will also be enhanced to keep educators updated on best practices in STACK integration.
+Continued Collaboration: The conference emphasised the importance of ongoing collaboration through regular meetings, updates, and collaborative projects to sustain momentum and foster innovation within the STACK community. Strengthening existing partnerships and exploring new ones with policymakers, funding bodies, and international organisations will also be key to supporting the expansion and scaling of STACK across the region.
+We extend our sincere gratitude to all individuals and organisations that contributed to the success of the 2nd African STACK Conference. Special thanks go to our keynote speakers, Prof. Chris Sangwin, Dr Michael Obiero Oyengo, Dr Beth Nyambura Kiratu, Dr David Stern, Prof. Franca Hoffmann, and Dr Mary Achieng Ochieng, for their insightful contributions. We appreciate the organising efforts of the Technical University of Kenya, alongside the support from IDEMS International, INNODEMS, and the Kenya Mathematical Society. We also thank the participating universities and research institutions from Africa, Europe, and the US for their valuable input. +Finally, our heartfelt thanks to all attendees, presenters, and volunteers for their active engagement and dedication, which made this conference a remarkable success.
+Luke Longworth
+I have been developing questions and quizzes with STACK for the University of Canterbury, New Zealand, since the end of 2020, and have seen several courses develop original quizzes from scratch in that time. When writing large amounts of questions in a short space of time, especially when multiple people are working on it, I find myself encountering two problems frequently:
+[1,2,3]
, a tuple (1,2,3)
, something more complicated like a matrix matrix([1],[2],[3])
, or use a matrix input type?These problems cannot be solved easily for existing large question banks, nor do I think that the former is inherently a "problem" anyway (it's more that I would like to have a set of rules/standards to break rather than no rubric at all). However, when constructing new quizzes, we can design with these considerations in mind to create a more internally-consistent product.
+At the end of 2023, the creation of quizzes for our 200-level engineering linear algebra course was sanctioned. This is a course I had been closely involved with for the past few years, and was the largest advance warning I'd ever received for a course wanting to implement new quizzes. I took this as an opportunity to design quizzes "properly" and have been attempting to avoid the above issues.
+My solution was to create a new Maxima file, linearalgebra.mac
that I could include in my new questions by running stack_include_contrib('linearalgebra.mac');
at the beginning of each question which would contain various helper functions and could be easily updated to service an entire question bank at once.
At time of writing (June 2024), this file is now mostly complete and the course question bank is being written. This case study serves to document the process I went through, briefly describe the contents of the file, and to reflect on the successes and failures encountered so far. The intention is to publish the file to STACK at the end of 2024 once it has been used (and thus stress-tested) in a real course, and then any user can use it with stack_include_contrib
by including within the CAS-logic.
The engineering linear algebra course, EMTH211, is a second-year compulsory course for our electronic and mechatronic engineering students at University of Canterbury, New Zealand. It has a greater focus on numerical linear algebra than the other 200-level course on offer, MATH203, and has a large Python-coding component in lectures and assessment. Expected prior knowledge includes basic matrix arithmetic, Gaussian elimination, determinants, and eigenproblems.
+Students enrolled in this course have been using STACK for at least 3 semesters prior and are familiar with the way that we use quizzes and STACK. The course currently focuses on tutorials that encourage hand-working and Python coding, and are marked for engagement rather than correctness. The first major piece of assessment is not returned until week 7, so students who do not self-assess well may not have had any feedback on their ability to correctly solve problems until the course is over halfway through. The hope is that the introduction of regular quizzes will keep students engaged, help them to identify misconceptions, and let them practice basic computations.
+EMTH211 is a good candidate for this type of project because of its status as a 200-level linear algebra course. There are lots of regularly used routines in linear algebra, so code-sharing is commonplace. The existing linear algebra support in Maxima is mostly concerned with manipulating matrices, with some other packages doing very specific things (such as diag
for computing Jordan normal forms, or lapack
for running numerical routines). This leaves plenty of room for functions that are widely applicable, simple, and useful. Additionally, these students have experience with STACK and will hopefully be comfortable with things like new input types.
My writing process is not particularly good practice and wouldn't scale well, but it has worked for me and is entirely in-browser. This suits me, as I use a remote desktop connection at work and don't have much control over my own installs and settings.
+I first began by planning the quiz structure on paper. This included logistical questions (when will they be due), content questions (which pieces of the curriculum work well in STACK) and pedagogical questions (how do I want/expect students to engage with these quizzes as learning activities?). I decided that my goal was to write questions that would differentiate themselves from tutorials and assignments (which are more focused on key concepts and coding) by focusing the quiz questions on producing correct numerical results. After settling on fortnightly quizzes of approximately 8 questions per quiz, I identified 4 key computations for each week, leading to approximately 48 individual types of computation (as well as first year revision for early in the semester).
+Once I knew the types of questions I was writing, I began to write functions that felt like they would be useful when developing one or more of the questions I had planned. Initially I began working in a single STACK question that included all of the functions in one place but this became very clunky when the test cases piled up and slowed down the compilation significantly. Next, I moved my work to GitHub (see below).
+I wrote some questions as I went, mostly to test out my new vector notation, but most of the questions have not yet been written. I will discuss this more below.
+If you're new to GitHub (I certainly am) here is a rundown on my process. +- From the STACK home on GitHub click the Fork button to create your own copy of STACK.
+linear-algebra-beta
.linearalgebra.mac
.stack_include("https://raw.githubusercontent.com/LukeLongworth/moodle-qtype_stack/linear-algebra-beta/stack/maxima/contrib/linearalgebra.mac");
at the beginning of my questions, and will eventually transition it to stack_include_contrib("linearalgebra.mac")
when this file is published to STACK properly.This is where my work was stored, but when writing individual functions it's nicer to get some more immediate responses to check that everything is working properly. Rather than going back to my single STACK question with horrible load times, I used the following workaround:
+This is twisting the intended use case of this function a bit, but it has worked excellently for me. I often use the "Send general feedback to the CAS" link for quick-and-dirty testing in a real STACK environment. You can access this link from the STACK question dashboard (I have an empty question bookmarked for this). You can edit the question variables and general feedback fields of a question temporarily, and even use stack_include
.
My main workflow was to include this header:
+simp: false;
+tests: [];
+s_test_case(sa,ta):= tests: append(tests,[[is(sa=ta),sa,ta]])
+
+followed by either stack_include
or simply a copy-paste of the code I'm wanting to check, and then I would put the following into the General Feedback field:
[[foreach test="tests"]]
+{@first(test)@} {#second(test)#} {#third(test)#}<br><br>
+[[/foreach]]
+
+As I added more to the code, I would use s_test_case(sa,ta)
with sa
being the function application and ta
being the expected output. This was a really useful way to build test cases into the development process. The three lines setting simp
and dealing with s_test_case
shouldn't be included in the final file, these are just for testing.
Test cases are important to ensure that small edits to this open source software don't have unintended effects. These are important for the main codebase of STACK, but in the event that a new version of STACK would break existing questions, users can anticipate this before the upgrade. The stack_include
feature copies the source code from the chosen file in the contrib
folder and pastes it directly into the question when compiling (usually this means when you click Save). This means that this code is independent of the version of STACK you are running, and changes made to these files can impact other users without them knowing*. Detailed test cases are therefore particularly important here!
*Note: Any edits to files like linearalgebra.mac
will only affect existing questions when they are compiled (saved), so you won't be immediately breaking existing questions/quizzes by editing these files. Caution should still be taken!
At present, test cases are kept separately from the main function. For example, I am currently maintaining both linearalgebra.mac
and linearalgebra_test.mac
, the latter of which hosts the test cases. The test cases should all be of the form s_test_case(test case, expected output)
. For example, one of my functions has the following set of test cases:
s_test_case(diagmatrix_like([1,1,1],3,3),ident(3));
+s_test_case(diagmatrix_like([1,2,3],3,4),matrix([1,0,0,0],[0,2,0,0],[0,0,3,0]));
+s_test_case(diagmatrix_like([1,2,3],4,3),matrix([1,0,0],[0,2,0],[0,0,3],[0,0,0]));
+s_test_case(diagmatrix_like([1,2,3],4,4),matrix([1,0,0,0],[0,2,0,0],[0,0,3,0],[0,0,0,0]));
+s_test_case(diagmatrix_like([1,2,3],2,3),matrix([1,0,0],[0,2,0]));
+s_test_case(diagmatrix_like([1,2,3],3,2),matrix([1,0],[0,2],[0,0]));
+
+diagmatrix_like(L,m,n)
constructs a mxn matrix with the entries of L on the diagonal. I chose those test cases to cover all of the possible cases I could think of: a square matrix with a full diagonal, tall and squat matrices with full diagonals, a too-short list of given diagonal entries, and over-full lists of diagonal entries with rectangular matrices.
Documentation should be in the main file. This is an ongoing project, so the format is not finalised. At present, I have been following the format of
+/**
+ * Description of function
+ *
+ * @param[data type] parameter_1_name Description of parameter
+ * @param[data type] parameter_2_name Description of parameter
+ * @return[data type] Description of returned value
+ */
+
+One specific example is shown below for a function that takes a matrix and vector and returns the general solution:
+/**
+ * Solve the matrix equation Ax = b given matrix A and column vector (or list) b.
+ * Optionally will find a least squares solution
+ * Always returns a general solution if one exists, even in the least squares case
+ * If a single solution is required, use pseudoinverse(A) . b instead.
+ *
+ * @param[matrix] A An mxn matrix
+ * @param[matrix] b A mx1 matrix (or a list with m entries)
+ * @param[boolean] lstsq Optional: if given true then a least squares solution will be obtained. If false or omitted, only exact solutions obtained.
+ * @return[matrix] The general solution to Ax = b. If no solution exists and lstsq is not true, then matrix([]) is returned.
+ */
+mat_solve(A,b,[lstsq]):= block([m,n,vars,eqns,sol],
+ ... function here ...
+);
+
+The first thing that appears is the c
and r
functions. These are inert functions that students and teachers can use to represent column and row vectors respectively, and the validation box will correctly display them. This allows students to write expressions containing vectors without needing to use Maxima's matrix notation.
The LaTeX formatting is done using texput
to extract the arguments, format them individually using tex1
, and then put them all together within appropriate LaTeX matrix wrapping. The two functions are also declared nonscalar, so that expressions like c(1,2) + matrix([3],[4])
will simplify to matrix([4],[6])
rather than matrix([3 + c(1,2)],[4 + c(1,2)])
.
As of STACK v4.6.0, only direct texput
commands in the Question Variables are picked up by the validation to change the LaTeX display. This means that you cannot put the texput
inside a larger block of code (notably, including if statements) or refer to other existing variables, functions or flags within the texput
code itself. Ideally, teachers would be able to choose whether to include this part of the file with a flag variable (in case they are wanting to use the variable c
for a constant of integration, or r
to refer to the vector [x,y,z]
etc.), but this is not possible. Even nicer would be the ability to choose what type of matrix bracket is used (questions that ask for lists of vectors are a bit confusing to read at present), or to match the question-wide matrix bracket choice. Perhaps future versions will give more control over texput
and validation, but for now these suffice.
There are also a number of functions that convert statements between various "standard forms". It is much easier to write this library of routines knowing that everything is either a list of lists or a matrix. Convenience functions are provided that will convert groups (i.e. lists, sets, ntuples, spans, and others) of vectors (matrices, c
, r
, lists, ntuples) into a list of lists, and then to convert a list of lists into a matrix.
Predicates are exceptionally useful little tools to use in PRTs or inside larger routines to avoid unwanted error messages. There are not many Maxima-native predicates to do with linear algebra outside of matrixp
, zeromatrixp
and blockmatrixp
, and there are many properties of matrices we may want to check. There are many predicates here that teachers can use to quickly check properties of student answers, or that they can use to filter out answers of unexpected form.
There are also a handful of comparison functions that test some sort of equivalence between a teacher's answer and a student's answer. These are mostly to do with equivalence of subspaces and linear independence, as algebraic equivalence is too restrictive to test these properties.
+This is too varied to explain in detail here. Interested parties should look at the file itself to see what sort of capabilities it has. In future I hope to write a detailed set of documentation including examples of use.
+This one is a little self-indulgent, as it usually makes little sense to find exact representations of a matrix QR factorisation, SVD, or diagonalisation unless the matrix has been set up explicitly to produce nice answers in this case. In EMTH211, we use Python to find solutions to these generic cases. I can't think of any cases where these functions are likely to be that useful, but it felt wrong to write a linear algebra package and not include these common factorisations.
+The inclusion should work for simp:false
and simp:true
and return appropriate results in either case. This caused me a few headaches, as I forgot to work with simp:false
for a large chunk of the original writing process. I only noticed when I went back through to add test cases the first time and realised that everything broke. Some things to watch out for:
makelist
doesn't work in expected ways with simp:false
. Rather than expr: makelist(func(i),i,1,n)
you might prefer expr: map(lambda([ex], func(ex)), ev(makelist(i,i,1,n),simp))
i = i+1
instead of the expected i = 2
. To avoid this, you can simply include i: ev(i, simp)
at the beginning of the loop.zeromatrixp(apply(matrix,[[0,0,0]]))
was returning false
even though apply(matrix,[[0,0,0]])
will correctly return a matrix of zeros. I fixed this with zeromatrixp(ev(apply(matrix,[[0,0,0]]),simp))
.s_test_case
didn't quite work as intended in all instances. I don't want things to be forcibly simplified before comparing, and this led to what I suspect is a similar issue to the above problem, where sa
and ta
would print identically, but is(sa=ta)
would return false. This typically happened when comparing lists or matrices, or when variables like %r1
or %r2
were generated in an answer. In these instances, I ignored the false flag if sa
and ta
seemed the same.There were also design consequences I didn't initially expect. For example, as mentioned above, there are many matrix factorisations included in the file simply for completeness' sake, but I don't think they are very useful for designing STACK questions. A good question that concerns itself with matrix factorisations would begin by generating the factors and multiplying them out to get the desired matrix. This ensures that the matrix has the properties you expect, and that students can easily compute and type in their answer without needing to deal with an absurd amount of simplification. If you are getting students to practice finding a SVD for any given randomly generated matrix, then I personally don't believe you should be using STACK (try CodeRunner!).
+A related problem appears when trying to make functions that are too all-encompassing. For example, I originally wanted to make a function that would check whether two given vector parametric equations were equivalent to one another. For example, is c(1,2) + t*(3,4)
equivalent to c(-2,-2) + t*(-6,-8)
? I was halfway through writing the function when I realised that in a real question I would rather individually check all of the components; has the student given me an answer in a sensible form, is the student's direction vector pointing in the right direction, and is their fixed point actually on the line? Then I can give targeted feedback to the student such as "your line is parallel to the correct answer". Therefore, I simply provided a function to extract each of these components and will leave the teacher to mark them appropriately.
I initially wanted to include some randomisation functions such as rand_orth
to generate a random orthogonal matrix, or rand_integer_invertible
to generate a random integer matrix whose inverse is also populated by only integers, both of which would be very useful to me. However, it became very clear very quickly that I want much finer control than what a general function like this could produce. For example, I tried writing a question that asks students to find all eigenvalues and eigenvectors of a 3 by 3 matrix from scratch. This is quite a lot of work, especially when we don't include factorising a cubic as a learning objective in our course, so my list of desired properties for the matrix became: An integer matrix with small integer eigenvalues, simple eigenvectors, and two off-diagonal zeros in the same row or column. Another example was writing a factorisation question; it is not enough to generate a random orthogonal matrix, optionally remove some columns, and multiply it by a random upper triangular matrix. The resulting product will have horrible entries and be unusuable in most cases. Ultimately I concluded that this was a futile** task outside the scope of this project, but I may revisit it in the future.
**Note: I have since worked on this a bit more and decided that there are sensible ways to do this when assuming that teachers will correctly use deployed variants, and am considering publishing rand_matrix.mac
as a separate file.
If I were to begin this project again, I think I would start with writing the quizzes. That way, my motivation to add something to linearalgebra.mac
would be noticing myself doing something that felt both annoying to code and broadly applicable. It would mean that I would inevitably need to do a second pass over the question bank at the end, but I think it would keep the project more focused. Feature creep was a big issue for me to deal with, and I'm not convinced that everything I wrote will turn out to be useful. Perhaps I should split this into multiple smaller files, but there is so much inter-dependency that only a few functions could reasonably be extracted without flow-on consequences.
In fact, it has become increasingly clear that what I have done with linearalgebra.mac
is somewhat outside the intended scope of of stack_include_contrib
. This function simply pastes the included code into the question variables as if the user typed them in. Users cannot pick and choose which features they want, and contributions cannot use banned functions like error
.
I think that stack_include_contrib
is working as intended here and doesn't need large-scale changes. In fact, the knowledge that anyone who uses this file in the future will be going out of their way to include it gave me some comfort: the intended users are already comfortable using STACK. It gave me freedom to just make something that works and then tidy it up later. Furthermore, this process of (1) identifying a limitation, (2) writing some code that fixes the problem, (3) generalising the code for wider use, and (4) adding it to the contrib
folder leads very naturally to a step (5): pick out key features that are particularly broadly applicable and work with the lead developers to add them to core STACK. I hope that in future some of these features, particularly the vector input, augmented matrix, and predicate functions could be modified and added to the basic STACK install for long-term support. I would recommend this process to anyone who has some experience working with Maxima code but is not confident working directly on an open-source project like this.
Daniel Doz, Zevick Otieno Juma, Motognon Wastalas D. Dogbalou, Danilo Lewański
+The University of Trieste (UniTS) is a prominent player in Italy’s academic landscape, recognised for its efforts to align with contemporary educational trends in education. This is despite the largely traditional nature of Italy’s education system ( for more details click here). +In Italy, university assessments are traditionally summative, relying heavily on high-stakes, end-of-semester exams to measure student understanding. This model places emphasis on theoretical knowledge assessed at the end of the semester, often lacking interim feedback that could aid in bridging knowledge gaps as they arise. Although effective in certain respects, this system has significant limitations, particularly in its lack of continuous feedback mechanisms that could help students identify and address learning gaps throughout the semester. By regulation, lecturers are not obligated to administer continuous assessments during the semester. +In response to this, UniTS launched a pilot project to integrate STACK, an open-source assessment tool designed for STEM subjects, starting with the Department of Mathematics, Informatics, and Geological Sciences (MIGe). Developed through a partnership with IDEMS International, UK-based social enterprise focused on supporting education and sustainable development projects worldwide. +This was done during the 2022/23 academic year, starting with a Probability and Statistics course and later scaling it up to Linear Algebra in the subsequent academic year. STACK was selected due to its automated, real-time feedback, allowing students to learn from their mistakes and progress through iterative practice.
+The Italian Government, through its National Recovery and Resilience Plan PNRR - Piano Nazionale di Ripresa e Resilienza, which calls for innovation and digitalisation in education, provided financial support accross various learning institutions in Italy, which allowed UniTS to sponsor two PhD research positions focused on STACK integration and expand collaborative opportunities with institutions abroad, mostly located in Africa.
+This case study outlines the progress and insights gained from implementing STACK at UniTS to guide similar educational advancements in Italy and beyond. The project’s primary goal is to modernise assessment methods in STEM, providing students with more frequent, individualised feedback and aligning assessment practices with UniTS’s commitment to sustainable educational improvement.
+In the Italian higher education system, continuous assessment practices are generally absent in STEM subjects. Instead, courses rely on multiple written exam sessions, with each course offering 6 or 7 sessions per year. Students have considerable flexibility: they can attend as many sessions as they choose, reject passing grades, and retake exams in subsequent years without any time constraints.
+The assessments typically includes the following key phases:
+1. Written Exam – Students begin with a written exam, which requires them to apply their knowledge to solve practical problems. Only those who achieve a sufficient score in this initial phase are eligible to proceed to the final assessment stage.
+2. Oral Exam – In this phase, students participate in an oral examination, where they articulate and defend their understanding of the subject through discussion with lecturers. This stage assesses not only the students' depth of knowledge but also their ability to communicate complex ideas effectively.
The grading system in Italy ranges from 18/30, the minimum passing score, to 30/30, with “30 e lode” (cum laude, occasionally noted as 31/30) for exceptional performance. Continuous assessment, while increasingly valued, remains optional and is typically up to individual instructors.
+In the 2023/2024 academic year, STACK was integrated into three courses at the University of Trieste. The first subject, Probability and Statistics, was taught in a course that had used STACK in the previous year. The second subject, Linear Algebra, was divided into Course A and Course B, which followed identical curricula but were taught by different lecturers. STACK was used to support with continous assessment, give feedback to students, and even running exams in the Probability and Statistics course. +Only mastery quizzes were implemented—these are specialised quizzes designed to help students achieve a deep understanding of key concepts, with only the highest grade among all attempts taken into account. The quizzes remained open throughout the semester until the first exam, allowing students ample time to practice and master the material at their own pace.
+Content development. The initial introduction of STACK at UniTS was supported by IDEMS International, which focused on providing STACK content to instructors.
+Customising Questions. The next step was to tailor the questions to fit the specific learning objectives of each course. Instructors, with IDEMS support, reviewed resources from the IDEMS open question bank and created new questions where necessary. IDEMS also provided access to open-source resources like the African STACK Open Question Bank to enrich the question pool.
+Engaging Students. To help students get comfortable with STACK, instructors provided informal onboarding sessions within each course. These sessions included tutorials on navigating quizzes, using feedback effectively, and retaking quizzes to reinforce learning. Initially, students were uncertain about the option for multiple quiz attempts, but they quickly adapted and came to value the immediate, constructive feedback provided.
+The integration of STACK at UniTS came with a few key challenges:
+Technical Issues. There were initial technical difficulties with STACK’s server integration, but UniTS supported by IDEMS resolved these issues. Additionally, the lead instructor addressed technical issues as they arose throughout the course, ensuring smooth ongoing functionality.
+Question Development. Crafting effective questions that met STACK’s technical requirements was initially time-consuming. However, by using pre-existing materials from the African Question Bank and the HELM Project, instructors were able to expedite the question development process, tailoring questions to fit course objectives more efficiently.
+To evaluate STACK’s impact on student performance, we analysed engagement data from three courses at UniTS: two Linear Algebra courses (Course A and Course B) and one Probability and Statistics course. This analysis aimed to understand how varying levels of STACK engagement influenced student outcomes. +In Course A of Linear Algebra, students had access to all STACK quizzes and completed each one throughout the semester, showing consistent engagement. In contrast, Course B students attempted only 6 out of the 10 available quizzes, despite following an identical curriculum and sitting for the same final exams as Course A. This variance in engagement levels between the two otherwise identical courses provided a unique opportunity to explore how consistent quiz participation might affect exam performance and mastery of course content.
+In the Probability and Statistics course, STACK had been used previously, and this time the new student cohort also engaged with all available quizzes, completing at least one attempt per quiz. Therefore, we did not analyse pre- and post-STACK implementation for this course as we did for the two Linear Algebra courses. Since this was the second cohort to use STACK, there was no baseline data without STACK for comparison. Instead, our focus here was on evaluating overall performance trends with STACK already in place.
+
In comparing student performance in the two Linear Algebra courses (A and B), there were notable differences post-STACK integration. For Course A, the analysis showed a significant improvement in student performance after integrating STACK. The Mann-Whitney U test indicated a significant difference in grade distributions between the pre- and post-STACK groups, suggesting that STACK’s continuous assessment and feedback likely contributed to improved outcomes. For Course B, however, the results did not show a statistically significant change in performance post-STACK integration, as the Mann-Whitney U test returned a non-significant result.
+
+
+
+While the results suggest a positive impact of STACK, we recognise that correlation does not equal causation. The improvement in performance could partly stem from factors like individual motivation or external influences. We discuss these considerations and potential limitations in detail in a separate research paper. That paper also presents insights from a K-means classification analysis. This analysis showed that students with steady engagement tend to achieve moderate performance and are more often female. In contrast, students with irregular study habits span all performance levels and are more commonly male.
To gather insights into student experiences with STACK, we conducted a qualitative feedback analysis based on a survey administered to students in the Probability and Statistics course. Since this was the second time STACK was used in this course, students were able to provide well-informed feedback on its advantages and challenges. Due to time constraints, similar qualitative feedback was not collected from the two Linear Algebra courses. For more in-depth findings, please refer to our published paper click here.
+Here are some key themes that emerged from the feedback:
+Instant Feedback and Clarity
+Many students highlighted the benefit of immediate feedback and clear instructions provided by STACK, which allowed them to quickly assess their understanding.
++“...Comodo e si ha subito il risultato… mi sono trovato molto bene, soprattutto per studiare. L'esame era chiaro e non necessitava di dover dimostrare i vari passaggi...”
+
+Translation: "...Convenient and you get the result quickly... I found it very good, especially for studying. The exam was clear and didn’t require demonstrating each step."
Concerns about Evaluating the Thought Process
+Some students expressed concerns that STACK emphasised final answers rather than the problem-solving process. They felt this approach might limit their ability to demonstrate comprehensive understanding during exams.
++“Rimango dell’opinione che gli esami online siano molto più scomodi rispetto a quelli cartacei. Credo che con carta e penna sia più facile valutare il processo risolutivo, mentre su Stack viene preso in considerazione solo il risultato finale.”
+
+Translation: "I remain of the opinion that online exams are less convenient than paper-based ones. With pen and paper, it's easier to assess the solution process, whereas STACK only considers the final answer."
Reduction of Exam Anxiety
+For some students, practicing with STACK before exams helped reduce anxiety, making them more comfortable with digital assessments.
++“Fare l'esame su STACK non crea ansia e pressione. Avendo fatto molta pratica sulla piattaforma, mi sembra normale fare gli esercizi attraverso lo schermo.”
+
+Translation: "Doing exams on STACK doesn’t create anxiety or pressure. Having practiced a lot on the platform, it feels normal to do exercises on the screen."
Concerns over Technical Issues
+Despite improvements, some students continued to report technical difficulties, including issues with devices and internet access, which occasionally disrupted their assessment experience.
+These insights provide meaningful perspectives on students' experiences with STACK in the Probability and Statistics course, and we believe they likely reflect broader patterns that could apply to students in the two Linear Algebra courses (A & B) as well.
The integration of STACK in our Linear Algebra and Statistics courses has yielded promising outcomes, particularly when students fully engage with the quizzes. Moving forward, our efforts will concentrate on refining and expanding our STACK content. We have already developed new STACK questions, including Parsons problems, which will be implemented in the 2024/2025 Linear Algebra course. This will enable us to further assess how we can leverage STACK teaching and learning approaches.
+Refinement and Expansion: We will continue to refine existing STACK questions, particularly those used in Linear Algebra. The newly authored STACK questions will be rolled out in the 2024/2025 semester, allowing us to monitor their impact on student learning outcomes.
+Collaboration and Research: We are constantly looking out to establish collaborations through STACK development and research ideas, since we are still new at this, both within Italy and beyond. This will play a vital role in expanding our understanding and ability to carryout research experiments that explore the broader application of STACK in mathematics education.
+Integration into More Courses: We plan to extend STACK to more first-year courses, particularly in disciplines like Physics, where it can significantly aid in students’ comprehension of complex concepts. Expanding STACK's application will not only provide students with a more interactive learning experience but also generate additional data to evaluate its overall effectiveness across different fields.
+Salvatore Mercuri
+Proof is the cornerstone of mathematics and as students advance through an undergraduate degree, proof becomes the principal mode through which they learn and practise mathematics. +In this case study we will describe recent developments in STACK that enable new ways of automating the assessment of proof. +These features not only allow us to move more content of an undergraduate mathematics degree onto STACK, but they also present opportunities for students to interact with and understand proof in new and interesting ways.
+A typical mathematical proof might look something like the following.
+Theorem. An integer is odd if and only if is odd.
+Proof. If is odd then, for some , we can write , which is odd.
+On the other hand, if is even then, for some , we can write , which is even.
+
Free-flowing text-based arguments like the above are the norm in mathematics education and research. +However underlying each proof is a formal logical structure. +For example, the above proof can be written in a more structured way.
+Theorem. An integer is odd if and only if is odd.
+Proof.
+() If is odd then is odd.
Assume that \(n\) is odd. | +
Then we can write \(n = 2m + 1\) for some \(m \in \mathbb{Z}\). | +
Therefore \(n^2 = (2m + 1)^2 = 2(2m^2 + 2m) + 1\), which is odd. | +
(\(\Leftarrow\)) If \(n^2\) is odd then \(n\) is odd.
++
We reformulate "If \(n^2\) is odd then \(n\) is odd" as the contrapositive. | +
If \(n\) is not odd then it is even, so \(n = 2m\) for some \(m \in \mathbb{Z}\). | +
Then \(n^2 = (2m)^2 = 2(2m^2)\) is even, and therefore not odd. | +
Structured proof writing can be helpful in clarifying the logic of a proof. +It can allow us to signal concepts such as scope of variables; for example, the that appears in the block is different to the that appears in the block. +Variable scope is a core part of any programming language where it is typically signalled through the use of structural elements, usually a mix of indentation and/or braces. +Anyone familiar with programming may already have noticed that the above structured presentation of a mathematical proof resembles code to some degree.
+In computer science, Parson's problems were introduced in 2006 [3] and have since been a popular method for automating the assessment of programming ability. +These problems involve dragging and dropping lines of pre-written code into the correct order and structure so as to provide a functioning program which performs the requested task. +Parson's problems have been shown to be highly effective for early-stage students in programming because they reduce the cognitive load required to remember correct syntax and generate steps [4]. +Thinking about proof as a formally structured logical argument, as in the second example above, can help us to apply Parson's problems to proof.
+In the next two sections, we will describe more details of the representation of mathematical proof as formal structures and assessment algorithms that can be leveraged to deliver automatic line-by-line feedback. +After that, we will talk about how Parson's problems were implemented in STACK before finishing with a look ahead to future possibilities in automating proof assessment.
+A suitable representation of mathematical proof that can allow us to understand when an ordering of pre-written proof steps is considered a correct proof. +Sometimes, a given proof consists of a set of statements that can only be given in a single correct order. +Consider the following example:
+Theorem. If an integer is odd then is odd.
+Proof.
As written, the above proof can only ever appear in that order, because step 2 depends on the assumption in step 1, and the calculation in 3 makes use of the variable which is introduced in step 2. +However, this is often not the case. +In fact, because the order of the () and () blocks does not matter, the following is an alternatively ordered correct proof of the example in the previous section.
+Theorem. An integer is odd if and only if is odd.
+Proof.
(\(\Leftarrow\)) If \(n^2\) is odd then \(n\) is odd.
++
We reformulate "If \(n^2\) is odd then \(n\) is odd" as the contrapositive. | +
If \(n\) is not odd then it is even, so \(n = 2m\) for some \(m \in \mathbb{Z}\). | +
Then \(n^2 = (2m)^2 = 2(2m^2)\) is even, and therefore not odd. | +
(\(\Rightarrow\)) If \(n\) is odd then \(n^2\) is odd.
+Assume that \(n\) is odd. | +
Then we can write \(n = 2m + 1\) for some \(m \in \mathbb{Z}\). | +
Therefore \(n^2 = (2m + 1)^2 = 2(2m^2 + 2m) + 1\), which is odd. | +
Our representation of proof should capture such non-unique orderings of a proof. +This begins by recognising that a proof may contain a number of structural logical arguments, which may decompose a proof into a number of sub-proofs whose orderings usually do not matter. +For example, an "if and only if" proof decomposes into two sub-proofs, one for the "if" direction and another for the "only if" direction, as in the above example. +The most common forms of structural logical reasoning in mathematics fall under one of the following:
+This decomposition of proofs into sub-proofs allows us to think of proofs as trees, where nodes consist of proof steps and branches consist of direct logical deduction. +If the logical deduction involves decomposition into sub-proofs then this branching will be multiple, else it is singular. +For example, the above if-and-only-if proof looks like the following.
+In the above diagram, we represent structural logical reasoning by diamonds and singular deductive proof steps by rectangles. +Each structural diamond has its own specific properties; the diamond, for example, has the property that it is a binary node whose children commute.
+To implement proof trees in STACK, we developed a Maxima proof library in STACK called prooflib.mac
which contains a number of functions which represent various structural diamond nodes.
+For example, the function proof_iff
takes two arguments and satisfies proof_iff(A, B) = proof_iff(B, A)
, whereas the function proof
takes an arbitrary number of arguments which must appear in order.
+See the documentation for more information on these proof functions.
A Parson's problem involves dragging and dropping pre-written lines into order, so that the student's answer is a list. +However, the teacher's model answer for a Parson's problem in STACK takes the form of a proof tree. +This is so that the system is able to capture all the non-unique correct proofs for the given set of proof steps. +To understand whether the student's flat list of steps is a correct ordering we first use the proof-tree representation to generate all possible correct orderings of the steps as a list. +Then, we iterate over all correct list and compare the student's list using the Damerau-Levenshtein distance, which we will describe in more detail below. +If, for some correct list, this metric returns zero then the student's answer is a match and they have a correct proof. +If not, then the student has an incorrect ordering of steps and, moreover, the Damerau-Levenshtein distance allows us to track exactly which modifications the student needs to make to change their incorrect proof into a correct proof. +This algorithm enables us to not only assess correctness, but also automatically generate line-by-line feedback on the student's answer.
+The Damerau-Levenshtein distance ([1], [2]) is used to measure the minimum number of edits needed to go from one string to another. Here, the permissible edits include:
+This can easily be applied to any arbitrary sequence, rather than strings. +In particular, we can look at two lists of proof steps and apply the Damerau-Levenshtein algorithm between them to compute the specific insertions, deletions, substitutions and transpositions of proof steps required to go from one to the other. +The diagram below shows the overall assessment algorithm used within a Parson's question in STACK.
+Within the Maxima proof library prooflib.mac
provided in STACK, there are functions that allow users to compute each link in the above diagram.
+The function proof_alternatives
will take an author's model answer proof tree and compute all possible alternatives.
+The function proof_assessment
will compute the smallest Damerau-Levenshtein distance between a student's input and all the correct alternatives, and for that minimum pair it will give the edits required to get from the student's answer to a correct answer (the green box in the above diagram).
+Finally, proof_assessment_display
will generate a visual representation of the edits the student needs to make to correct their answer.
+Some examples of how this looks on the question page in STACK for non-unique correct answers and an incorrect answer with automatic line-by-line feedback are given in the following figures.
To write a Parson's problem in STACK, the library prooflib.mac
should be loaded using stack_include_contrib
within Question variables.
+This makes available all the various proof functions needed to construct proof trees and assess answers.
+All of the pre-written steps and the model proof-tree answer can be written in the Question variables field, see the documentation for details on the syntax required.
+The main drag-and-drop functionality is achieved through the use of the Sortable JavaScript library, which is a widely used and supported library for drag-and-drop features.
+In STACK, this functionality is wrapped by a parsons
block so there is no need for the author to write any JavaScript themselves.
+However, authors are able to customise the drag-and-drop functionality to a certain degree, either by block header parameters in the parsons
block or through the options field in the block contents.
+For further details on how to get started writing a Parson's problem, follow the documentation.
As we have seen, writing proofs in highly structured ways can help us capture important properties of proof and enable the application of Parson's problems to this area. +However, structured proofs have other benefits such as clarity of logic and scope which can be helpful to early-stage learners. +A future priority for proof assessment in STACK will be developing ways that enable students to interact with proof in such structured ways. +This could be through allowing indented blocks within the current style of Parson's problems, for example, allowing the student's answer to also represent a tree rather than just a flat list.
+Within recent years, there has been a rise in research in formal programming languages such as Lean. +Mathematical proofs written in Lean, for example, can be automatically verified as correct or incorrect. +This growing research has even led to recent successful applications of Lean in artificial intelligence at the International Mathematical Olympiad in 2024. +While it seems natural to harness the power of languages such as Lean for automating proof assessment in STACK, these languages do not form a standard part of a mathematics curriculum nor have they been shown to increase a student's level of understanding of a proof. +So we cannot currently assume students or question writers are able to directly interact with such formal programming languages. +It may be the case that future research within AI and automated theorem proving may help to bridge the gap between formal programming languages and mathematical education, unlocking a raft of benefits for automated proof assessment.
+[1] F. J. Damerau, "A technique for computer detection and correction of spelling errors", Communications of the ACM, 7 (3): 171-176 (1964)
+[2] Vladimir I. Levenshtein, "Binary codes capable of correcting deletions, insertions, and reversals", Soviet Physics Doklady, 10 (8): 707-710, (1966)
+[3] D. Parsons and P. Haden, "Parson’s Programming Puzzles: A Fun and Effective Learning Tool for First Programming Courses", Proceedings of the 8th Australasian Conference on Computing Education, 52: 157-163, (2006)
+[4] D. Paul, A. Luxton-Reilly, and B. Simons, "Evaluating a new exam question: Parson's problems", Proceedings of the Fourth International Workshop on Computing Education Research", ICER: 113-124, (2008)
+This work is part of the Knowledge Transfer Partnership between the University of Edinburgh and IDEMS International, funded by Innovate UK.
+Idrissa S. Amour, Daudi A. Zacharia, Abdi T. Abdalla, Fatuma Simba, Septimi Kitta
+STACK was first implemented at the University of Dar es Salaam for a big class (of about 950 students) in the 2021/2022 academic year. It was considered as a cost-effective approach to mathematics teaching when compared to the classical tutorial sessions that required more staff, and had timetable constraints. In that academic year, it was used in two Engineering courses: Matrices and Basic Calculus for non-majors, and One-Variable Calculus and Differential Equations for non-majors.
+At the beginning of the semester, students are given a quiz which is intended for them to familiarize with the STACK syntax. Students can only proceed to other quizzes when they pass the syntax quiz with a score of 100%. After that, each week students have to undertake one mastery quiz which they have to pass by at least 50% in order to attempt a test quiz. In the coursework contribution to the course, only the score for test quizzes are counted.
+After its second year of implementation, a survey was done aiming to analyse the students' experience with this online practice. A total of 725 responses were received.
+One common claim is that Mathematics assessment being done online destroys the students' mathematical thinking, and has a negative effect on their performance in the course. We asked the students to rate these claims and the results showed that the majority of them disagreed.
+One aspect that we really wanted to improve with the use of STACK was to encourage students to work independently. This was lacking in the classical tutorial sessions, where just a few bright students were active. In the survey results, many students strongly agreed that STACK increased their ability to work independently, and also recommended that STACK can replace classical tutorials. The students also agreed that most of the solutions to the questions they did were accurate.
+Our students have to pay for internet access, typically via mobile phone. We wanted to explore, whether or not the students could afford these internet costs. The results show that most students could afford the cost, and that the level of learning they received through STACK was of good value for money. Additionally, many of these students recommended STACK to be used in the next academic year.
+"Tutorials are included in timetables but most instructors don't attend or they get substituted with lectures. May be you are the only instructor who is serious about tutorials." This comment affirms that STACK has been helpful for the students learning as compared to the classical tutorials.
+"The passmarks in tutorials should be removed as people fail to conduct quizzes just because they failed to achieve the required passmark in tutorials and at the end people become fatigue." This student's recommendation was worked on in the 2023/2024 academic year, where mastery quizzes were split into smaller quizzes.
+STACK syntax seems to be an issue for many students (about 54%), because most of them lack background in programming. We will try adding one or two syntax quizzes so that the students may become more comfortable before attempting the mastery and test quizzes.
+There are also technical issues that we faced, like it being impossible for live quizzes because of the overload it causes for the Moodle server, so the time span for the quiz had to be extended to 1 week in order for the attempts to spread out.
+There is a shortage of staff to support the students with the challenges they face using STACK, an effort is ongoing to put other members on board.
+Students engagement with STACK means that a lot of data is available to be used in Mathematics education research. Many aspects can be studied in order to improve the learning process. So far our experience with the use of STACK in the University of Dar es salaam has been good, and we hope to leverage it to more courses in the coming years.
+The project has been funded by the University of Dar es Salaam through its competitive research grant. We appreciate the support we have received from IDEMS by allowing us to have unlimited access to its server.
+STACK has users all over the world. To highlight some of the ways STACK is used and developed around the world, we have written a number of case studies. Explore the map, or see the full list of case studies below.
+A report of an AIM-sponsored meeting on Open source mathematics curriculum and assessment tools
+This is the African STACK Conference, advancing digital assessment and formative education across African universities.
+Considerations when writing a collection of quiz questions in a specialised topic, and using stack_include to achieve this.
+A reflection on the analysis of STACK usage in continous assessment, exam results and student attitudes on the first STACK Implementation in the Italian Higher Education.
+A study on the implementation of Parson's problems within STACK to automate proof assessment
+A description of the use of STACK at the University of Dar es Salaam (UDSM), Tanzania
+The first African STACK Conference for Undergraduate Mathematics took place at MMUST, Kenya, from 19 to 23 June 2023. This case study includes details and conclusions arising at the conference.
+Responding to national educational upheaval with student-centred support for Ethiopia.
+This case study details the work and insights gained from a STACK-based internship during the summer of 2023 at the University of Edinburgh.
+The INNODEMS STACK Internship Programme empowers African universities to revolutionize math education through efficient assessment solutions and comprehensive support. Join us in transforming learning outcomes and shaping a brighter future for students across the continent.
+A STACK question on integration by partial fractions.
+This case study describes using STACK at The University of Edinburgh in, Honours Complex Variables, a Year 3 pure mathematics module.
+This is a report on a 5-day STACK workshop held at Maseno in July 2022.
+Using the Damerau-Levenshtein distance between strings to develop assessment of short free-text answers.
+Converting the "Helping Engineers Learn Mathematics" workbooks into STACK quizzes for self-study.
+Using STACK in Real Analysis at the University of Warwick.
+The MINTFIT Math Test is an online diagnostics test that uses STACK.
+OTH Amberg-Weiden uses STACK in extra-occupational maths bridging courses for non-traditional students.
+Aalto University uses STACK for their Engineering Mathematics courses, and they have also developed the material bank Abacus.
+The Ruhr-Universität Bochum uses STACK in online courses designed for self-study. Some questions are adaptive, and guide students through complex tasks.
+The University of Edinburgh has in-house support for online assessment, mostly with STACK, for most year one and two mathematics modules, and many more.
+"Fundamentals of Algebra and Calculus" is a fully online course that uses STACK.
+Developers at Nagoya University are building a "flick interface" for STACK, similar to the popular Japanese keyboard mode.
+STACK was integrated into the ILIAS learning management system to support projects for learning content.
+At Loughborough University, STACK has been implemented across many disciplines.
+IDEMS international is working with Maseno University to implement online assessment for their mathematics courses.
+Physics Curriculum & Instruction have developed STACK questions to accompany a Physics textbook.
+The Open University uses STACK for large-scale online courses.
+The optes project uses STACK in their pre-course, designed to help students improve their self-studying skills.
+The African STACK Community Leadership Team was created in response to the need for a concerted effort to drive STACK integration in African institution forwards during The First African STACK Conference for Undergraduate Mathematics.
+ +
+ Technical University of Kenya and Kenya Mathematics Society, Kenya
+ beth.kiratu@tukenya.ac.ke
+ Kenya Mathematics Society Representative
+
+ Strathmore University, Kenya
+ machieng@strathmore.edu
+ African STACK Conferences Lead
+
+ University of Namibia, Namibia
+ martinmugochi@gmail.com
+ Namibia STACK Representative
+
+ University of Dar es Salaam, Tanzania
+ idrissaamour@gmail.com
+ Tanzania STACK Representative
+
+ Bahir Dar University, Ethiopia
+ abdum442@yahoo.com
+ Ethiopia STACK Representative
+
+ Masinde Muliro University of Science and Technology, Kenya
+ glawi@mmust.ac.ke
+ Institutionalization of STACK at African Universities
+
+ IDEMS International, Rwanda
+ herine.otieno@idems.international
+ STACK Research agenda in African Institutions
+
+ Maseno University, Kenya
+ jkmusyoka@maseno.ac.ke
+ STACK for Statistics and Data Science
+
+ Masinde Muliro University of Science and Technology, Kenya
+ aokoth@mmust.ac.ke
+ STACK for Statistics Integration
+
+ University of Nairobi, Kenya
+ ongaro@uonbi.ac.ke
+ Policy and Implementation Agenda
+
The STACK Professionals Network was established during the summer of 2022 to bring together early career professionals (not necessarily academics), across various institutions, who spend a significant amount of their time authoring STACK questions. Authoring STACK questions requires specialist skills, and the people who do this job somtimes work in relative isolation in their institutions. They may not have much help regarding various issues that they encounter authoring questions. The Network will help the STACK professionals to solve various authoring difficulties and also make it easier to share existing questions/quizzes. Furthermore, the Network can provide training to colleagues who are new into STACK and define the pedagogy around creating STACK questions.
+ +To join the Network come in contact with Chris Sangwin: c.j.sangwin@ed.ac.uk or Konstantina Zerva: k.zerva@ed.ac.uk.
+ +To contact the Network: network_stack_prof@mlist.is.ed.ac.uk.
+ +Hochschule Ruhr West and Ruhr-Universität Bochum, Germany
jonas.lache@hs-ruhrwest.de
Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Germany
wigand.rathmann@fau.de
Universität Kassel, Departement of Mathematics, Diskrete Mathematics and Algebra, Germany
hendrikje.schmidtpott@mathematik.uni-kassel.de
University of Trieste, Italy
motognonwastalasd'assise.dogbalou@phd.units.it
h² - Hochschule Magdeburg-Stendal, Fachbereich Wasser, Umwelt, Bau und Sicherheit, Germany
oleg-boruch.ioffe@h2.de
The STACK Researchers Network brings together education researchers with an interest in computer-aided assessment tools such as STACK.
+The range of topics for research involving STACK is very broad. A recent collaborative project to develop a research agenda for e-assessment in undergraduate mathematics has identified 55 research questions, many of which are relevant to STACK:
+Kinnear, G., Jones, I., Sangwin, C., Alarfaj, M., Davies, B., Fearn, S., Foster, C., Heck, A., Henderson, K., Hunt, T., Iannone, P., Kontorovich, I., Larson, N., Lowe, T., Meyer, J. C., O’Shea, A., Rowlett, P., Sikurajapathi, I., & Wong, T. (2024). A collaboratively-derived research agenda for E-assessment in undergraduate mathematics. International Journal of Research in Undergraduate Mathematics Education, 10(1), 201–231. https://doi.org/10.1007/s40753-022-00189-6
+The STACK Researchers Network aims to:
+If you would like to join the Network, please contact George Kinnear (G.Kinnear@ed.ac.uk).
+The University of Edinburgh, UK
+ G.Kinnear@ed.ac.uk
+ Network Organiser
+
Interests: + Course design involving STACK + Example-generation tasks +
+Boise State University, USA
+ JoeChampion@boisestate.edu
+
Interests: + Effects of implementation +
+EduHubAfric, Rwanda / Education Development Trust, UK
+ herine.otieno@idems.international
+
Interests: + Adaptive learning + Equity + STACK-supported flipped teaching +
+The University of Edinburgh, UK
+ C.J.Sangwin@ed.ac.uk
+
Interests: + Educational Technology + Task Design +
+Members of the STACK Researchers Network have contributed to the following studies related to STACK:
+Kinnear, G. (2024). Comparing example generation with classification in the learning of new mathematics concepts. Research in Mathematics Education, 26(1), 109–132. https://doi.org/10.1080/14794802.2022.2152086
+Kinnear, G., Jones, I., Sangwin, C., Alarfaj, M., Davies, B., Fearn, S., Foster, C., Heck, A., Henderson, K., Hunt, T., Iannone, P., Kontorovich, I., Larson, N., Lowe, T., Meyer, J. C., O’Shea, A., Rowlett, P., Sikurajapathi, I., & Wong, T. (2024). A collaboratively-derived research agenda for E-assessment in undergraduate mathematics. International Journal of Research in Undergraduate Mathematics Education, 10(1), 201–231. https://doi.org/10.1007/s40753-022-00189-6
+Kinnear, G., Wood, A. K., & Gratwick, R. (2022). Designing and evaluating an online course to support transition to university mathematics. International Journal of Mathematical Education in Science and Technology, 53(1), 11–34. https://doi.org/10.1080/0020739X.2021.1962554
+Sangwin, C. J. (2013). Computer Aided Assessment of Mathematics, Oxford University Press (ISBN 978-0-19-966035-3)
+STACK has a very active user base, with a wide variety of colleagues in different roles. Four particular groups are
+The easiest way to engage with the community is through our free online chat platform. All kinds of questions, suggestions and comments are welcome there.
+Continuously developed and used since 2004.
+
+ The University of Edinburgh, UK
+ C.J.Sangwin@ed.ac.uk
+ Chairperson of the International Advisory Board
+
The Open University, UK
+Aalto, Finland
+
+ SURLABS
+ jcopado@surlabs.es
+ STACK for ILIAS developer and Service Provider
+
The Open University, UK
+The University of Edinburgh, UK
+The University of Edinburgh, UK
+ IDEMS International, UK
The International Advisory Board encourages engagement from the community of users of the STACK +online assessment system and helps to oversee strategic developments of +the system. The Advisory Board promotes the long-term stainability of the software and invites +contributions from diverse stakeholders.
+ +Guangdong Technion-Israel Institute of Technology
antti.rasila@gtiit.edu.cn
Founder of the Abacus Consortium
Ostbayerische Technische Hochschule (OTH) Amberg-Weiden
m.weinmann@oth-aw.de
+ Chairperson of the 2024 conference
Ostbayerische Technische Hochschule (OTH) Amberg-Weiden
k.dechant@oth-aw.de
TTK University of Applied Sciences
oksana.labanova@tktk.ee
Chairperson of the 2023 conference
Durham University, UK
m.t.matuszewski@durham.ac.uk
+ Chairperson of the 2025 conference
The annual meetings of the STACK Community are for all STACK users to exchange experiences, ideas and research topics. +From 2024 it is likely we will have international community meetings in Europe and Africa.
+2024 JSXGraph conference 2024 will take place on 8th-10th October 2024 at University of Bayreuth, Germany. +The conference will bring together developers and teachers, instructors and designers who are interested or already experienced in using JSXGraph to enhance digital learning of STEM topics. As in the years before, the conference will be an entirely online conference. All participants are required to register, registration is free.
+We invite all participants to contribute a talk or workshop and
+The 5. International JSXGraph Conference 2024 will be organized by the "Center of Mobile Learning with Digital Technology"
+2025: The International Meeting of the STACK Community 2025 will take place on 7 - 9 of April 2025 at Durham University, UK.
+Title | +Date | +Location | +Description | +
---|---|---|---|
The 2nd African STACK Conference for Undergraduate Mathematics | +12-16 August 2024 | +The Technical University of Kenya (TUK), Kenya | +The conference aims to bridge the gap between educators with experience and access to STACK resources and the pressing need for support in teaching and formative assessment in the continent. It also aims to open avenues for collaboration with international partners in research and evaluation, particularly on the effectiveness of digital interventions in assessment in low-resource environments. | +
International Meeting of the STACK Community 2024 | +11-13 March 2024 | +OTH Amberg-Weiden, Amberg, Germany | +The annual International Meeting of the STACK Community is a forum for all STACK users to exchange experiences, ideas and research topics. | +
African STACK Conference for Undergraduate Mathematics | +19-23 June 2023 | +Masinde Muliro University of Science and Technology, Kakamega, Kenya | +A conference to enable African educators with experience on STACK to define a roadmap for transformation of African Undergraduate Maths Education | +
International Meeting of the STACK Community 2023 | +24-26 April 2023 | +TTK University of Applied Sciences, Tallinn, Estonia | +The annual International Meeting of the STACK Community is a forum for all STACK users to exchange experiences, ideas and research topics. | +
International Meeting of the STACK Community 2022 | +24-28 April 2022 | +Leoben, Austria | +A forum for all STACK users to exchange experiences, ideas and research topics | +
The 4th International STACK Conference | +26-29 April 2021 | +TTK University of Applied Sciences, Tallinn, Estonia Proceedings |
+This conference aims to act as a forum for the exchange of experience, ideas and research associated with implementing STACK. The target group is academics who teach undergraduate and postgraduate STEM courses in higher education institutions. | +
The 3rd International STACK Conference | +27 April 2020 | +Online | +The aim of the conference is to provide a platform for academics, researchers, and scholars, to address common challenges, share knowledge and ideas as well as recent trends and brainstorm creative solutions in the field of STACK. | +
The 2nd International STACK Conference | +30 April 2019 | +The University of Edinburgh, Edinburgh, UK | +This conference was day two of a wider meeting. | +
The 1st International STACK Conference | +15-16 November 2018 | +Friedrich-Alexander-Universität Erlangen-Nürnberg, Fürth, Germany | +The mission of the Conference is to offer a forum for the STACK Community, to exchange ideas about possibilities and challenges in creating questions, to get closer to solving technical questions and to shape the future of STACK. | +
This event is associated with the Eurasmus+ IDIAM project.
+We have been using, and evaluating, an organising principle for online learning materials which is increasingly guiding our thinking in the development of university mathematics courses. Essentially we have taken the book and put it inside automatically assessed online quizzes. In doing this, we embrace the potential provided by new technology to implement evidence-based practices such as spaced retrieval practice. This workshop discusses details of this innovation, and demonstrates examples of courses in which we have implemented it.
+When: Wednesday 16th November 2022.
+Where: The University of Edinburgh, James Clerk Maxwell building (for in-person attendees) and online.
+9:30 am - 12:30 pm, JCMB - Room 5323
+13.00 pm - 17.00pm, JCMB - Room 5326
9:30-10:00 Opening
+10:00-11:00 Designing materials with STACK & "putting the book inside the quiz", George Kinnear
+11:00-11:15 Coffee break
+11:15-12:15 Discussion of case studies of use:
12:15-13:15 Lunch
+13:00-15:00 STACK authoring workshops: +- Getting your first question working (no STACK experience required). +- Testing and maintaining STACK questions.
+You are welcome to attend the event in person. To register please email Konstantina Zerva. Lunch will be provided for every participant attending in-person. Please specify any special dietry requirements you have.
+Demonstration materials associated with this event will be online here: https://stack-demo.maths.ed.ac.uk/demo/course/view.php?id=176
+Morning talks: https://ed-ac-uk.zoom.us/rec/share/7qRa6xCbEAZr4iFKMD_fBEgp-7LfaGQPxGYNbmY8F53aa3r2TMen4bAvHWPqq8ch._16wt4GzJNRH0hx7
+Afternoon workshop: https://ed-ac-uk.zoom.us/rec/share/8endrAjD4C95N3pW-tmN5ZySNI19zvfrreE-Nk2aSazIToC-1IeR59RsFayOHGH6.LtKOedIaaORNUe-D?startTime=1668605212000
This event is associated with the Eurasmus+ IDIAM project.
+JSXGraph is a cross-browser library for displaying interactive geometry, function plotting, graphs, and data visualization in a web browser https://jsxgraph.org/wp/index.html. Mathematical and physical phenomena can be better understood and explored via visualization. This workshop will provide hands-on practise on how to create JSXGraphs, which you can embend in your educational resources (e.g. in markdown editor) and how to use JSXGraph in STACK assessments.
+When: Thursday 8th December 2022.
+Where: The University of Edinburgh, James Clerk Maxwell building, Room 5323 (for in-person attendees) and online. Times below are GMT.
+Zoom link for online participation:
+Join Zoom Meeting
+https://ed-ac-uk.zoom.us/j/86838815060
+Meeting ID: 868 3881 5060
+Passcode: iWRxP3Lk
9:30-10:00 Pre-meeting coffee
+10:00-10:10 Opening
+10:10-11:00 Presentation: Math visualization with JSXGraph - an overview, Alfred Wassermann
+11:00-11:20 Coffee break
+11:20-13:00 Programming Workshop: "Create your first JSXgraph", Alfred Wassermann
13:00-14:00 Lunch
+14:00-14:40 Presentation: Using JSXgraph in STACK questions, Jonas Alexander Lache
+14:40-15:00 Coffee break
+15:00-16:30 Workshop: How to use JSXGraph in STACK, Wigand Rathmann and Jonas Julius Harang
+The workshop will cover:
+- Create a graph which follows the randomisation.
+- Create a graph with binding of variables (the graph is part of the answer).
You are welcome to attend the event in person. To register please email Konstantina Zerva. Lunch will be provided for every participant attending in-person. Please specify any special dietary requirements you have.
+Demonstration materials associated with this event will be online here: https://stack-demo.maths.ed.ac.uk/demo/course/view.php?id=176
+Morning session: https://ed-ac-uk.zoom.us/rec/share/LCcw1po5f7gpRk1qkm0cLX4m8R0QLL5olBWsGW75FavnnF5Yx_DZhyNLV-BnUtus.7Q6FsDBSWC0PzLOZ?startTime=1670493588000
+Afternoon session: https://ed-ac-uk.zoom.us/rec/share/LCcw1po5f7gpRk1qkm0cLX4m8R0QLL5olBWsGW75FavnnF5Yx_DZhyNLV-BnUtus.7Q6FsDBSWC0PzLOZ?startTime=1670508198000
This event is associated with the Eurasmus+ IDIAM project.
+When: April 2023 - TBC.
+Where: The University of Edinburgh, James Clerk Maxwell building.
The first African STACK Conference for Undergraduate Mathematics will take place at Masinde Muliro University of Science and Technology (MMUST) from 19 to 23 June 2023.
+ +The conference aims to enable African educators with experience on STACK to define a roadmap for transformation of African maths education. Since 2019, African universities started incorporating STACK assessments to their undergraduate courses, particularly to deal with issues of large class sizes (some classes are over 1,200 students) with a single lecturer and no additional human resources. A concerted effort was started by groups of universities, with the support of IDEMS International, to promote STACK in the region through open resources and question banks, which are slowly spreading throughout the continent, with the objective of full curriculum coverage.
+ +The conference will provide a platform for lecturers to present their experiences incorporating STACK to undergraduate courses, their methodologies and impact found. Key stakeholders will have an opportunity to extend their collaboration and accelerate the integration of STACK.
+ +The conference will include a 2-day STACK workshop, 2 days of paper and poster presentations, and a final day roundtable discussion with key stakeholders or an additional workshop.
+ +Please fill in the registration form below to attend the conference. We will write to you shortly after receiving your application with payment details.
+ +Limited funding may be available for partial or full waivers and financial support for additional expenses. Due to limited funding this will only be possible in exceptional circumstances and places at the conference are not guaranteed if support is requested. Decisions will be made on a case-by-case basis. If you require financial support to attend the conference, please submit your registration form no later than 8th May 2023.
+ +The conference will include a combination of 30-minutes Paper Presentations and 15-minutes Poster Presentations. Please submit an abstract (maximum 4000 characters) in the form below. The final deadline for submissions has been extended to 14th May 2023.
+ +INTERNATIONAL ONLINE CONFERENCE “LEARNING MATHEMATICS : DIGITAL AND INTERACTIVE” ON 3 FEBRUARY 2024
+The Erasmus Projct AuthOMath (www.authomath.org) has achieved its objectives!
+On 3 February 2024, the project will present its results:
+The presentation will be part of a one-day online conference with speakers
+In addition, Tim Lutz (Landau) and Patrick Galm (Heidelberg) will offer workshops on technical and didactic aspects of the combination of GeoGebra and STACK.
+Details on the programme and registration can be found here:
+https://www.authomath.org/?page_id=1448
+We look forward to seeing many familiar and as yet unfamiliar faces from the STACK community!
+Guido Pinkernell and the Teams from AuthOMath and the Heidelberg MathBrücke
The Erasmus Project AuthOMath https://www.authomath.org/ has achieved its objectives!
+On 3 February 2024, the project will present its results:
+The presentation will be part of a one-day online conference with speakers
+In addition, Tim Lutz (Landau) and Patrick Galm (Heidelberg) will offer workshops on technical and didactic aspects of the combination of GeoGebra and STACK.
+Details on the programme and registration can be found here:
+https://www.authomath.org/?page_id=1448
+We look forward to seeing many familiar and as yet unfamiliar faces from the STACK community!
+Guido Pinkernell and the Teams from AuthOMath and the Heidelberg MathBrücke.
The Erasmus Project AuthOMath https://www.authomath.org/ has enabled us to integrate STACK with GeoGebra. In particular, we can now include GeoGebra worksheets as an input enabling students to give an answer which is a geometric diagram. Mathematical and physical phenomena can be better understood and explored via visualization. This is a practical workshop for users interested in getting started with this new feature.
+Examples of materials developed by the AuthOMath project are given in the AuthOMath project page.
+Documentation for this new feature is available in the STACK documentation
+When: Wednesday 27th March 2024.
+Where: The University of Edinburgh, James Clerk Maxwell building, Room 5323 (for in-person attendees) and online. Times below are GMT.
+Zoom link for online participation: please register to receive the link
+12:00-13:00 Lunch (for in-person participants)
+13:00-14:00 Presentation: Using GeoGebra in STACK questions
+14:00-14:30 Coffee break
+14:30-16:30 Workshop: How to use GeoGebra in STACK
Demonstration materials associated with this event will be online here: https://stack-demo.maths.ed.ac.uk/demo/
+Please register using this form by Wednesday 20th March 2024: https://www.authomath.org/?page_id=1876&lang=en
+You are welcome to attend the event in person. A vegetarian sandwich lunch will be provided for every participant attending in-person. If you have any special dietary requirements, please email Konstantina Zerva after you have completed the registration form.
The AuthOMath project https://www.authomath.org/, supported by the European Union’s Erasmus+ programme, has dedicated two and a half years to integrating GeoGebra and STACK. Its primary aim is to provide teaching guidelines for designing dynamic mathematical tasks with tailored feedback for students' errors.
+To present the achievements achieved by researchers from Heidelberg University of Education (Germany), the University of Edinburgh (Scotland), Johannes Kepler University (Austria), and the University of Cantabria, the Discovering AuthOMath (Descubriendo AuthOMath) event is organized at the Faculty of Sciences of the University of Cantabria. This event will offer insights into innovative teaching approaches and practical applications for secondary and tertiary education mathematics teachers.
+When: 8 May 2024
+Where: Faculty of Sciences, University of Cantabria, Santander, Cantabria, Spain.
+Further information about the program can be found at the following link: https://www.authomath.org/?p=2065.
+Please register using this form by May 3, 2024: https://www.authomath.org/
++ | The project is co-funded by the EU under the Erasmus+ Programme No. 2021-1-DE01-KA220-HED-000032031 |
++ |
The 2nd African STACK Conference for Undergraduate Mathematics will be held at the Technical University of Kenya (TUK) from 12th to 16th August 2024.
+ +The conference aims to bridge the gap between Mathematics educators with experience and access to STACK resources and the pressing need for support in teaching and formative assessment with immediate feedback in mathematics within African universities. With the establishment of the African STACK Community Leadership Team and access to IDEMS’ Open Question Banks (OQB) of STACK questions, there is the potential for widespread adoption and institutionalization of STACK for continuous assessment with timely feedback across African institutions, which has been a universal challenge in these contexts.
+ +The conference also aims to open avenues for collaboration with international partners in conducting research and evaluation, particularly on the effectiveness of digital interventions in assessment in low-resource environments. Large class sizes and growing adoption of STACK in undergraduate courses in the region has proven to be a rich source of data for analysis.
+ +Africa is uniquely positioned to lead in the large-scale adoption of digital technologies for formative assessment of undergraduate mathematics. The conference will serve as a platform for lecturers to share their experiences incorporating STACK into undergraduate courses, including their methodologies and observed impacts. Lecturers who have utilized STACK in their teaching will also have the opportunity to present their courses for peer review and engage in discussions on their work. Additionally, other participants will have the opportunity to share their STACK work, which will help define strategies for improved STACK integration in African institutions.
+ +The conference will include 2-days of STACK workshop, 2 days of paper presentations, and a final day roundtable discussion with key stakeholders or an additional workshop, and a panel discussion with key stakeholders. Workshops will offer opportunities to develop, improve and analyze courses delivered with STACK integration with the objective of lecturers leaving the conference with courses they can directly deliver.
+ +Paper presentations at the conference will serve as a platform to share experience of STACK integration in African universities and educational institutions, share key STACK features that could serve the community, promote the STACK research agenda, within several other purposes. We welcome presentations from STACK users off all type, from developers, through lecturers using STACK, to STACK professionals. + +Please submit an abstract (maximum 4000 characters) in the form below. The deadline for submissions is Sunday 7th July at 23:59 East Africa Time.
+ +Please register to participate in the conference by filling in the registration form below. The organizing committee will write to you shortly after receiving your application with payment details.
+ +Limited funding may be available for partial or full waivers and financial support for additional expenses. Due to limited funding this will only be possible in exceptional circumstances and places at the conference are not guaranteed if support is requested. Decisions will be made on a case-by-case basis. If you require financial support to attend the conference, please submit your registration form no later than Monday 1st July 2024.
+ +Information on how to submit an abstract for presentations will be published soon, together with relevant deadlines.
+ + + +Explore STACKs features by visiting our Demo site. If you have STACK installed and want to write your first question, see our question authoring quick start guide.
++ Read the information for students on our documentation, including accessibility and FAQ, or get used to STACK syntax with our syntax quiz.
+STACK can be used via a hosted partner, installed within Moodle or ILIAS, or integrated into other Learning Management Systems via LTI.
+STACK is open-source and welcomes contributions: added features, integrations, translations, bug reports and more.
+We believe online assessment should be available and accessible to all.
+We strive to comply with the Web Content Accessibility Guidelines (WCAG) 2.1 on www.stack-assessment.org ("the website") at most of level AA requirements.
+This statement covers the STACK website, and does not cover the STACK interface for teachers and students.
+The main website www.stack-assessment.org,
+For comments or suggestions regarding to accessibility, please email Chris Sangwin at C.J.Sangwin@ed.ac.uk
The contents of stack-assessment.org and docs.stack-assessment.org is Licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.
+The following are related licenses for STACK, stack-assessment.org and docs.stack-assessment.org ("the website").
+STACK is licensed under the GNU General Public License Version 3.
+The STACK documentation, contained in the subdirectories of doc/ and on docs.stack-assessment.org, is Licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.
+The website is hosted on GitHub pages. Please see the GitHub terms of service.
+The website is built with MkDocs, licensed under a BSD license.
+The website uses the mdx_math markdown compiler. python-markdown-math license.
+stack-assessment.org uses Bootstrap, licensed under a MIT License.
+Bootstrap is integrated into MkDocs using the mkdocs-bootstrap theme, licensed under a BSD 2-Clause "Simplified" License.
+docs.stack-assessment.org uses the Material theme, licensed under a MIT License.
This statement was last revised on 13/5/2022.
+This website is hosted on GitHub Pages. GitHub may collect User Personal Information from visitors to this website, including logs of visitor IP addresses, to comply with legal obligations, and to maintain the security and integrity of the Website and the Service. Please see GitHub's privacy statement for information on how your data is used and stored by GitHub.
+We do not otherwise collect any personal information about you when you visit this website.
+This statement refers only to this website (www.stack-assessment.org), and not to any other STACK websites, or any third party sites linked to on this website. You are encouraged to review privacy statements on other websites when you visit them.
+The date of the latest revision is shown at the top of the page. We may change this privacy statement at any time and for any reason. You are encouraged to regularly review this statement.
+If you have questions about our privacy policy, please email Professor Chris Sangwin at C.J.Sangwin@ed.ac.uk.
The STACK Professionals Network was established during the summer of 2022 to bring together professionals, across various institutions, who spend a significant amount of their time authoring STACK questions.
+Between Dec 2019 and Dec 2022, the 3 pandemic years, the need to create more online resourses expanded significantly and many universities created staff posts for people whose main role is to author STACK questions. These posts also support a large and growing community of STACK users in their institutions. The Network will bring these people together and contribute to the knowledge sharing among them.
+The 1st STACK Network meeting took place on the 9th of December 2022 in the School of Mathematics at the University of Edinburgh. The purpose of the meeting was to bring together the people of the Network and set some initial goals for the Network. The topics that were discussed during the meeting can be divided into two categories: (i) how the members can contribute to the STACK project and (ii) how the Network will support its members.
+Using STACK to automate mathematics and STEM assessments requires specialist skills, and the people who do this job typically work in isolation in their institutions. In many cases there is only one person in the department/university doing this job, so they don't have much local help for STACK-related issues that they may encounter. The Network will help the STACK professionals to solve various authoring difficulties and also make it easier to share existing questions/quizzes amongst themselves. Mathematics curriculum, especially topics like algebra, calculus, or differential equations, are very similar around the globe, so many existing STACK questions can be easily shared and reused with minor edits. Example questions can help a question author get started in a new topic area. While some colleagues are reluctant to freely post whole courses, most are very willing to share examples especially with colleagues with whom they already have a professional relationship.
+The Network can provide training to colleagues who are new to STACK and support them with authoring, testing and maintaining their questions. Members of the Network can also support the development of the pedagogy associated with using STACK questions effectively.
+Another goal of the Network is to provide visibility to this specialist, technical work, so that the work of the STACK professionals will acquire better recognition. Most of the content created by the STACK professionals is behind passwords, and intellectual property belongs to institutions rather than individuals, so the authors of the questions don't have a long-term track record of their work. Not all professionals contribute towards research, they don't necessarily publish papers, so all this work can easily be unnoticed.
+The members of the Network are highly encouraged to contribute by improving the documentation, with a particular focus on sharing expertise and "know-how" of system features for specific subjects. Other ways that the members can contribute are:
+Since STACK has become widely used in many countries the Network can help with translating the system and various sample materials to different languages. Members can certainly help translators understand how the various language strings are used within the software, something not immediately obvious to some translators.
+The technical notes of the meeting can be found here (end of the page): https://github.com/maths/moodle-qtype_stack/blob/dev/doc/en/Developer/Future_plans.md.
+In-person:
+Online:
+Anyone who spends a significant proportion of their professional work writing STACK questions, and who is interested in joining the Network should contact Chris Sangwin or Konstantina Zerva at the University of Edinburgh.
+The full list of STACK professionals can be found here: https://stack-assessment.org/Community/
The 2nd STACK Network meeting took place on the 9th of June 2023 online. The purpose of the meeting was to discuss that issues that raised during the recent International meeting of the STACK Community (April 2023). It was also a good opportunity to welcome new members.
+New STACK Release: +Chris presented the new STACK release, STACK 4.4.3, and demonstrated its new features. The key features of this release included improved cleaning of Unicode from students' input strings, descriptions to the question (castext), the PRT nodes, and the question tests. Additionally, an input extra option validator was introduced to allow user-defined validation functions. A new reveal block in questions was also added. It was emphasized that users should check their existing questions after updating to STACK 4.4.3, especially in relation to the changes in JSX graphs. Further details about the new features can be found in the documentation provided at https://docs.stack-assessment.org/en/Developer/Development_history/.
+Bulk Testing for STACK Update:
+It was recommended that everyone who is updating to the new STACK version should perform bulk testing of their questions. However, not everyone has admin privileges on their local servers, making it necessary to seek assistance from the local IT team. To support colleagues who lacked admin privileges, Maciej volunteered to write guidelines for testing quizzes, which will be shared with the team.
+Note, for moodle users there is a capability qtype/stack:usediagnostictools
. If you don't have admin rights at your university you can make a case for a group of STACK users to be granted this capability, which is needed for the bulk test, and other STACK maintenance functions.
API, Managing Questions Outside Moodle: +The meeting included a discussion on exploring API options for managing questions outside Moodle. While the topic proved to be highly technical, it generated interest among some participants. Santiago expressed willingness to assist in putting together a funding application if there is genuine interest from the team. It was proposed to organize a dedicated meeting specifically focused on this topic to ensure a detailed discussion with interested members. +Update on 2023-07-07: Tim and Andreas made a start with this new Moodle tool: https://github.com/maths/moodle-qbank_importasversion which allows you to import a question from a Moodle XML file as a new version of an existing question.
+Github training +All of the STACK sourcecode, documentation and the community website can be found in the Github repository https://github.com/maths. However, since not everyone is familiar with using Github, this can serve as a barrier for contributing to the documentation. To address this issue, we decided to organize a Github training. The aim of this training is to assist people in becoming more familiar with Github and to boost their confidence in contributing to the STACK website https://stack-assessment.org/ (Github repository https://github.com/maths/stack-web) by adding case studies, advertising events, and updating documents.
+Funding Opportunities, Grant Opportunities: +The possibility of securing funding for professional networks was discussed. A stream was created in Zulipchat to coordinate efforts in writing grant applications.
+Communications: +Zulipchat was identified as the preferred communication platform for the organization, providing a centralized space for discussions and issue reporting. Github was emphasized as the primary platform for reporting code-related issues. https://stack-assessment.zulipchat.com/
+Despite the approaching summer season, it was suggested to schedule another meeting in July and another in early September. This would ensure continued progress and provide opportunities for further discussions and updates.
+The 3rd STACK Professionals Network meeting took place on the 21st of July 2023 online.
+Santiago Borio (SB), Chris Sangwin (CS), Konstantina Zerva (KZ), Maciej Matuszewski (MM), George Ionita (GI), Steffi Zegowitz (SZ), Aviva Lisann Lenth (ALL), George Ionita (GI), Luke Longworth (LL), Ruth Reynolds (RR), Juma Zevick (JZ), Sam Fearn (SF), Edmund Farrow (EF).
+Apologies: Andreas Steiger, Tim Lowe, Jonas Lache.
+Acknowledgements: Thanks to MM for taking notes.
+CS – a new minor STACK version has been released. The testing form (STACK Question Dashboard) has been improved, there is a new question description field.
+SB outlined the IDEMS questions writing process. We might want to introduce a similar process more widely. More discussion on people’s workflow preferences (and how this might influence changes we want to make to the question type) is required.
+Comments and discussion points: +* There is an existing authoring guide in the STACK docs from CS (https://docs.stack-assessment.org/en/Authoring/Workflow/) +* There are many different workflow choices depending on the question/institutional needs (e.g. should there be just one question with lots of randomisation, or many separate questions which are randomly selected at the quiz level). +* Non-expert users may not be certain about the distinction between Moodle/ILIAS and STACK so our documentation and workflow should include information about this former element as well (e.g. how to create quizzes) +* We need to investigate if there is an easy way of seeing how many times students take a quiz / how long it takes them to do it.
+Monthly meetings were previously proposed and this works for most people. The exact time will be confirmed by email.
+A new guide for updating the website is being prepared (see https://github.com/maths/stack-web; https://github.com/maths/stack-web/pull/26/). Live guide can be accessed here: https://stack-assessment.org/WebsiteDocs/WebsiteUpdates/
+It would be useful to have a server with the network members having the “manager” level of permission. This can already be done on https://sandbox.moodle.idems.international/ for anyone who wants – contact SB to be made an admin on that site. KZ will create an area in the Demo Site with manager rights for network members to share materials easily.
+Updates to Moodle 4 and the latest version of STACK is ongoing at various institutions. A recommendation was given to be careful about direct comparisons between floating points – this might break with the latest update. Some people are having issues with bulk tests not testing all the questions.
+Individual meetings with the new member of the STACK development team, Edmund Farrow, are encouraged. This will help EF understand different user cases for his first project.
+German speakers and African networks are being set up.
+There are likely to be annual European meetings, and annual African meetings (perhaps an Australasian and South American one as well in the future?). Then one of these could be designated as the International conference venue.
+ICME 15 is in Australia in July next year – it might be a good opportunity to build connections there.
+A new projects page could be added to the STACK website; this has been drafted in a branch in the website repo. Currently it contains only have 2 but if people have ongoing projects they can submit them to the branch so we have enough content to justify having a projects page.
The 4th STACK Professionals Network meeting took place on the 21st of September 2023 online.
+Chris Sangwin (CJS), Konstantina Zerva (KZ), Edmund Farrow (EF), Santiago Borio (SB), Georg Osang (GO), George Ionita (GI), Juma Zevick (JZ), Matti Harjula (MH), Ruth Reynolds (RR), Stephen Nulty (SN), Tim Lowe (TL), Wigand Rathmann (WR), Alex Fowler (AF), Sam Fearn (SF), Marc Peterfi (MP), Ian Jones (IJ), Jonas Lache (JL), Kim Kneher (KK), Andreas Steiger (AS)
+Apologies: Maciej Matuszewski, Chris Nelson, Curtis Hooper.
+Acknowledgements: Thanks to KZ for hosting and to SB for taking notes.
+All present introduced themselves briefly, outlining their institution and key role. +New members were welcome.
+SB proposed a team to manage and implement, review PRs, etc. More structural changes would be discussed with the Development team. EF is happy to join the team. SF is happy to contribute here and there but is busy this term so will do what he can.
+KZ is considering applying for funding to complete the HELM workbooks migration to STACK. She is writing a proposal. It would possibly include translations. WR can share his use and experience. Any research on translating paper based questions to electronic versions would be beneficial, IJ may be able to find literature from OFQUAL and may have small bits of funding (TBC). The project would start next British summer.
+SB mentioned the INNODEMS Internships and African STACK Conference case studies. SF asked if a project on a possible new question type could lead to a case study. SB suggested it would be and gave examples of various styles of case studies.
+CJS encouraged members to provide case studies, it has a big impact for funding to show that the project is alive. SB offered to upload case studies for others.
+New section in the website to report on ongoing, upcoming and completed projects. This could be to report on larger scale interventions than case studies or provide partial reports on achievements. Currently there is a branch in the website repo that would be published once more projects are included.
+SB outlined the new community published in the website: the African STACK Community Leadership Team. SB encouraged members of other groups to reach out to him for support to create their own page if needed.
+It wasn’t particularly clear for JL that the upgrade of STACK to 4.4.3 could break some questions, in particular JSXGraphs questions. A request was made for future releases to include warnings for possible issues in the release notes. CJS acknowledged and agreed we need to be more careful and provide context.
+The idea of a dev machine for testing and highlighting issues pre release would be useful was raised again. Some members don’t have admin access to their Moodle setups and struggle to test things. SB suggested that IDEMS has a sandbox server where he can provide admin rights to anyone who would like to test, which should be kept updated with the latest Moodle and STACK versions.
+MH suggested that some testing tools are available to non-admin users under qtype/stack:usediagnostictools. +CJS suggested he and his team would draft documentation on how to use require_capability('qtype/stack:usediagnostictools', $context);
+TL reminded the group that we had a ‘release notes page’ when we moved to Moodle/Stack 4 and suggested that perhaps we need one for every release that people can contribute to. CJS and others explained that this was due to significant changes in the release but agreed that more thorough release notes would be a sensible idea and reminded all that contributions to the documentation are welcome.
+TL asked the network if it there is a way to map the ‘20230605’ style version number in the XML to the ‘4.4.3’ friendly version number?
+MH replied that this is not easy, but one can search through the github repo. GO mentioned a goemaxima table (middle 2 columns are the relevant ones). SF requested to have that in the STACK docs and TL requested for this could be included in the user docs, maybe linked to the release notes. MH mentioned the need to link to Ilias.
+The 5th STACK Professionals Network meeting took place on Thursday 30 of November 2023 online.
+Marc Peterfi (MP), Jonas Lache, Konstantina Zerva (KZ), Matti Harjula (MH), Wigand Rathmann, Kinga Sipos, Marie-Pauline Wiechmann, Tim Hunt, Juma Zevick, Stephen Nulty, Maciej Matuszewski (MM), George Ionita.
+Apologies: Tim Lowe, Chris Sangwin, Santiago Borio, Steffi Zegowitz.
+Acknowledgements: Thanks to KZ for hosting and to MM for taking notes.
+All present introduced themselves briefly, outlining their institution and key role. +New members were welcome.
+KZ asked a question about LTI connection between Learn Ultra and Moodle (4.1). The marks that come back to Learn are not the marks for each quiz. The LTI creates only one column in the Learn Ultra gradebook, where it gives the total mark for the course (the same issue existed with the previous LTI connection between old Learn and Moodle 3.9). +TH mentioned the LTI Moodle forum (questions about LTI can be asked here): Moodle LTI forum
+The next meeting will take place on Thursday 25th January. A poll will be send out for the exact time.
The 6th STACK Professionals Network meeting took place on Tuesday 12th March 2024 in person during the International STACK Conference. There was some remote participation but unfortunately due to technical issues not all online participants were able to fully join.
+Acknowledgements: Thanks to OTH Amberg-Weiden for allowing the network to join during the conference. Thanks to all the participants who attended who are not official members of the network and for their contributions. Thanks to Sam Fearne for taking notes.
+The nature of the meeting was different than usual and it was an open discussion on new development, priorities and challenges that STACK users are facing. The meeting was co-facilitated by Santiago Borio and Chris Sangwin. Below is a list of the key items that were discussed.
+Chris outlined why the network is useful in order to demonstrate the continued value of the project.
+Santiago introduced the network and how it works, what it aims to achieve both for the project and for its members.
+Corné pointed out the STACK Professionals Network is useful for forming consortia to apply to European grants, which usually require an internation consortium, and help people fund their existing positions.
+A question from the audience was posed about a drag-drop editor for STACK questions. Chris created dragmath some time ago, but the project died. If this is of value, perhaps some other audience member could take up this project. Chris also mentioned that MathPix might be a better future for natural input, and Sam mentioned having used STACK-JS in similar contexts.
+Stephen requested more regular workshops (git basics, other special interests). Santiago mentioned that the conference included an introductory STACK workshop that was offered in multiple languages, and there would be scope for further events for targeted groups.
+A member of the audience suggester struggling with custom settings and useful scripts due to Moodle permissions. There was agreement that it would be good to have a list of Moodle capabilities required, listed on the main webpage, to lend authenticity and help convince institutions to grant these capabilities to STACK authors. Santiago offered an IDEMS server as a playground for new users to use to learn more about STACK and Moodle admin.
+An audience member learned from video tutorials online, specifically learned about question tests. Videos exist in the quick start guide. The docs website also gives specific advice by topic. Particular examples are good.
+What open and semi-open question banks exist? Some were mentioned.
+A question from audience was raised about being able to edit questions direct in text format. There was an existing project for writing questions in a YAML format. Chris noted that using such a format introduces the possibility of typos, etc. This project stalled a while ago and never added support for rich media, though YAML is capable of this. Sam added that being able to import PRTs would be useful. The audience thought that the YAML format was a good item to be high priority.
+Georg suggested trying to keep track of the various projects people were working on, such as projects analysing databases of questions. Chris noted that such projects were in a better place to succeed now that the gitsync plugin exists. Santiago noted that focus on multilingual development is necessary for open databases to thrive. ChatGPT and other tools can help do auto-translating, though it is possible for context to be lost between the different strings; It would be useful to be able to filter by language in a database, and see screenshots of the questions for users to decide whether the question is worth translating further.
+Question tests are currently basic and test score, penalty and final tree destination. There were requests for more advanced question testing: external libraries, timeouts for large sets of PRTs. Chris asked the audience whether they would rather have hard limits set in STACK, or soft limits created by timeouts. Audience unanimous they would rather not have hard limits. Chris introduced the s_assert function which can enable testing in feedback variables in Maxima.
+Edmund asked for thoughts about whether a change to monospace font would be a problem for people. Asked for contributions via Zulip.
+A feature allowing authors to write drag-and-drop matching problems was developed for release in STACK v4.6.0. +A preview and feedback session was held on Monday, 13th May 2024 with members of the STACK Professionals Network +to establish any desired changes or additions to the current state of development prior to release.
+Present at the meeting: Sal Mercuri, Matti Harjula, Luke Longworth, Sam Fearn, Andreas Steiger, Stephen Nulty, +Juma Zevick, Wigand Rathmann, Edmund Farrow, Danilo Lewanski.
+A drag-and-drop proof assessment feature based on Parson's problems was released in STACK v4.5.0 in Dec. 2023
+through a parsons
block.
+In these questions, students are presented with two lists: an available list populated with pre-written strings
+corresponding to steps in the proof; an empty answer list they must drag proof steps to in the correct order.
+The priority for STACK v4.6.0 is to extend this drag-and-drop functionality beyond proof into more general
+"grouping" (dragging items into the correct category) and "matching" (dragging multiple corresponding items
+to the same row) questions.
matchlib.mac
STACK library was also covered.\\(...\\)
numerous times. It was raised that this would require further development on how CASText is handled by STACK.A growing number of commercial partners are using STACK as part of their professional services. Partners are listed here in no particular order.
+Physics Curriculum & Instruction publish textbooks supported by STACK exercises.
+++STACK has played a central role for Physics Curriculum & Instruction in creating our extensive library of algebra-based and calculus-based physics problems. STACK’s features allow the creation of advanced multi-part physics problems that support our educational pedagogy. These important features include: numerical student answers with scientific units, student input of mathematical expressions, unit conversion, proper handling of significant figures, use of randomized variables, and the ability to provide specific student feedback. STACK's sophisticated and carefully thought-out programming enables our team of developers to design physics problems that allow higher-level learning to take place, leading to greater student success.
+
IDEMS International offer a range of services, including running meetings and bespoke STACK question authoring.
+Catalyst IT who provide hosted moodle services, including STACK.
++Catalyst IT is a strong advocate for STACK. As the largest Moodle provider in higher education, we're committed to helping institutions get the most out of this innovative technology. We recognise that STACK can be complex, requiring not only access to additional infrastructure but also a solid understanding of Maxima CAS system syntax. Both of these factors can be a significant hurdle for many educators. Our team at Catalyst IT is here to help you overcome these challenges. Whether you host your Moodle site with us or elsewhere, we can assist you by hosting your STACK infrastructure and provide expert guidance on creating engaging math questions. +By partnering with Catalyst, you'll gain seamless integration between your Moodle site and STACK, ensuring a smooth user experience for you and your students. Catalyst STACK experts are committed to helping you unlock the full potential of this powerful tool. For furthermore information please contact STACK@catalyst-eu.net
+
STACK offers the possibility of Combining STACK and GeoGebra for better feedback. When placed in feedback, interactive applets allow for what could be called "activating feedback", i.e. adaptive learning material that students to work out the necessary knowledge themselves.
+The object of this task is the well-known translation of an algebraic expression into its geometric representation.
+Here, GeoGebra provides an interactive applet in both STACK's task and feedback area. The latter comes in three steps, each after some delay:
+Example-generation tasks ask students to produce examples of mathematical objects. Many mathematics education researchers are interested in how students approach this type of task, and how they can help students to learn about mathematical concepts (Kinnear et al., 2022). +E-assessment tools like STACK enable teachers to use example-generation tasks even with very large classes of students, so that each students get personal feedback.
+Thanks to the AuthOMath project, GeoGebra applets can now be used as a way for students to input a graphical answer to a STACK question. +This opens up a wide range of topics where teachers are now able to set example-generation tasks.
+For example, this question is taken from a first-year course at the University of Edinburgh, where students are learning about the properties of functions:
+To answer the question, the student can drag the four points and draw the graph of a piecewise linear function with the required properties. Behind the scenes, STACK can check the properties of the student's graph.
+If there are any errors, the student can be given customised feedback that refers to their particular answer:
+For this example, it turns out that students found the e-assessment task harder than the same one on paper (Kinnear et al., 2023). This was likely because the applet constrained students to using examples that are piecewise-linear functions, which are not the most familiar examples for students.
+There is great potential for further work to develop tasks (and sequences of tasks) like this, making full use of the power of GeoGebra so that students can produce graphical examples.
+Kinnear, G., Jones, I., Sangwin, C., Alarfaj, M., Davies, B., Fearn, S., Foster, C., Heck, A., Henderson, K., Hunt, T., Iannone, P., Kontorovich, I., Larson, N., Lowe, T., Meyer, J. C., O’Shea, A., Rowlett, P., Sikurajapathi, I., & Wong, T. (2022). A Collaboratively-Derived Research Agenda for E-assessment in Undergraduate Mathematics. International Journal of Research in Undergraduate Mathematics Education. https://doi.org/10.1007/s40753-022-00189-6
+Kinnear, G., Iannone, P., & Davies, B. (2023). Insights about functions from example-generation tasks: combining e-assessment and written responses. In P. Drijvers, C. Csapodi, H. Palmér, K. Gosztonyi, & E. Kónya (Eds.), Proceedings of the Thirteenth Congress of the European Society for Research in Mathematics Education (CERME13) (pp. 2399–2406). Alfréd Rényi Institute of Mathematics and ERME.
+ +STACK offers the possibility to give adaptive feedback. STACK tasks could therefore be an enrichment for maths lessons and a relief for teachers, especially in schools where self-directed learning plays an important role.
+This example task was created by teacher-education students in a didactics seminar at the Heidelberg University of Education. In this task children test their understanding fractions as part of one whole. Different types of feedback can be shown, depending on the input of the students.
+If the sliders are set so that the dynamic illustration correctly represents the required fraction, there is a detailed confirmation of correctness in mathematically correct language, as well as a request to repeat the task five times in a row, with the required fraction changing each time.
+If the input is incorrect, this is fed back and the student can either repeat the task directly or wait 15 seconds for the sample solution.
+The programme also identifies if an equivalent fraction is entered; the feedback can be different here, in the case presented in the video it refers to a learner who has not yet formally learned to expand and reduce fractions.
+STACK offers the possibility to randomly generate questions and give feedback, which is relevant to each specific random variable. The task presented in the video is a very simple one, and it was deliberately chosen to be simple, because the main focus here is the randomisation and the things that question authors need to consider when creating random variables in a question.
+The task asks students to drag the point so that the slope of the line is the same as the given value; in this case . The values of the slope are randomised.
+When the student load the question they all see the same graph. The initial slope of the line is set to in the GeoGebra app and the point is placed at . The students can move the point along the line and drag it around so that the line changes slopes.
+In this example the slop needs to be . The slope is defined as so the students can drag the point and place it at and (point ). Then click the check button and see if their answer is correct or not. Another possible solution here and .
+A common mistake that students do is believe that the slope is defined as so if they put at the point they'll receive specific feedback about this mistake.
+Let's look at the background of the question and how to deal with the randomisation. All the slopes need to be ratios of two integers (e.g. ) and also they shouldn't simplify to an integer (avoid cases like ).
+The variable defines the denominator of the slope. We pick the denominator to be an even number, in this case a power of . We could also predefine a list of even numbers and randomly pick from the list. +The variable defines the numerator of the slope. For the numerator we pick the values from a predefined list and these values are all odd numbers. +We define the slope as and in our case all the slopes will be fractions.
+ +STACK offers the possibility of immediate formative feedback. It's great to have everyone in a class of 100 students know they would be able to get feedback even if you are away. The teacher's presence is vital, but it is impossible to be there all the time, and AuthOmath is the perfect teaching assistant combining STACK and GeoGebra.
+This example is a task sequence to help the student relate factored equations to the area of a rectangle. The first task asks the student to factorise a square of binomial.
+If students write an incorrect answer, they are get the following feedback:
+Now, a student can drag point to observe the area of square and try to find a relation between the expression, the area and the length of the sides. When they drag point , they get a square composed of rectangles. The area of each rectangle is shown in the applet. The idea is for the student to add these areas, and obtain the expression needed as the answer to the task. STACK and GeoGebra help the student during the feedback.
+Then, the second part of the sequence is related to expressions. Where "" and "" are different natural numbers. In this second part we don't have a square.
+Again, if students write a wrong answer, they get feedback with a GeoGebra applet. This applet lets the student try out the relation between the length of the sides of the rectangle and the area of it. In that case, they can drag points and to get a rectangle composed of four rectangles. The area of each one is shown.
+This feedback allows the students to interact with a geometric polynomial representation. At the same time, they can find out a strategy to find a factorized expression. In this way, the geometry and the algebra reinforce each other.
+ +The original purpose of STACK was to accept answers which are algebraic expressions, moving away from reliance on multiple choice questions. Through AuthOMath we have included GeoGebra diagrams within questions, allowing a student's answer to include the configuration of a diagram. This question illustrates how dynamic diagrams can be used to support more advanced topics in university mathematics.
+In the problem below students need to relate the eigenvectors of a matrix to the transformation represented by the matrix. Eigenvectors is a technical term for vectors which are scaled by a transformation, but remain in the same direction (or reverse direction). Understanding the effect of transformations through calculating eigenvectors and the corresponding eigenvalues is an important topic in vector spaces. Students need to find a matrix and perform some routine calculations. In addition, four points in the diagram are ready to be dragged by the user to define the endpoints of two eigenvectors.
+Students get very adept at calculating eigenvectors through a mechanical procedure, but their geometric understanding can remain fragile. Excessive calculation can reinforce this problem.
+In the second figure a student has attempted to illustrate the positions of the eigenvectors. The vector is correct, but is not an eigenvector. The GeoGebra diagram returns the endpoints of the vectors, the computer algebra system establishes the relevant properties (is an eigenvector?) and then generates the feedback shown in the yellow feedback box.
+GeoGebra is used to provide a visual representation of the vectors. In principal this problem could be extended to a wider range of 2D transformations, and other situations where students can demonstrate their understanding of linear algebra by positioning vectors.
+ +The AuthOMath Project combines GeoGebra and STACK. It allows us to present multiple representations of the same mathematical object. STACK is especially useful to create different approaches to feedback.
+This task is about solving linear equations step by step. Whenever the students make a mistake, STACK is able to identify it and provide specific feedback to it. Below is an empty question ready for a student.
+The image below shows feedback generated automatically by the system. Notice some lines are correct, but the student has made one mistake.
+These formative hints can be delivered in many ways. Some students are more receptive to symbolic feedback, while others appreciate written comments. It is also possible to combine both.
+This is a useful way to distinguish the characteristics of the different feedbacks and determining which are more helpful and effective in fostering mathematical abilities and learning.
+ +The AuthoMath Project allows us to use GeoGebra applets on STACK. This offers +a wide range of tasks that we can create and provide feedback to. Students +have the chance to work in an interactive environment and use different +tecniques to solve the same task.
+There are many ways to compute the area and the perimeter of a figure. In +this task, we go through a few of them using Tangram pieces that our +pre-service teachers can move around and rotate. They can use this applet to +make the calculations using different units of measurement.
+In this task we ask them to compute and compare the area and the perimeter of +two figures, but also to reflect on what they are doing as a way to teach +these concepts to primary school students. Mathematical concepts go hand in +hand with didactical notions.
+Official project website: https://www.authomath.org/
+This project aimed to foster the development of innovative approaches to online and distance +learning especially in primary and secondary education. In particular, the project aims at enabling teachers to create digital online learning material with multimodal, +dynamic, and interactive elements, (GeoGebra) and with adaptive automatic feedback for use in hybrid settings.
+The project provides an authoring tool (AuTo) and a didactic concept (DiCo) that teachers can use for creating digital online material in mathematics with +multi-modal, dynamic, interactive elements in task formulation and automated, adaptive +feedback. We combined two existing authoring tools that are well-established +in the mathematics educational community. GeoGebra is an OER authoring +tool for creating multimodal, dynamic and interactive mathematical applets, with an easy +accessible user interface of widespread use in the mathematical educational community. +STACK is an OER authoring tool +for randomized mathematical tasks with automatic, answer-based feedback.
+Combining GeoGebra and STACK is a promising approach to developing an authoring tool for letting teachers create high-quality content. This outcome will be achieved by providing the following project results:
+Participating Organisations
+This site contains sample materials which illustrate the outcomes from the project.
++ | +The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant No. 2021-1-DE01-KA220-HED-000032031. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources. + | +
Official project website: https://www.oth-aw.de/ideal
+The IdeaL project set up an innovation network in which flexible learning modules for adaptive digital teaching are developed and produced. Among other project objectives, such as the development of innovative learning spaces, IdeaL also includes two fields where STACK plays an important role: first, interactive self-learning modules for engineering mathematics, and second, the service center for digital exercises.
+The goal of the service center for digital exercises is to facilitate and promote the use of e-assessment across faculties. Therefore, a team of STACK developers works together with lecturers to create STACK questions for online quizzes in moodle. One of the main goals is to convince lecturers to include online assessment in their courses without resulting in an increased workload or the need for intermediate programming skills. Positive feedback from lecturers in the early stages of the service center reinforced the team to focus primarily on STACK due to the possibilities for providing detailed feedback and also to include interactive applets. The work of the service center resulted in numerous STACK questions used for various subjects. Especially for lectures in electrical engineering, many STACK questions including interactive applets have been developed.
+The IdeaL project team is also developing interactive self-learning modules for different topics of engineering mathematics such as linear algebra, complex numbers, or differential calculus. These modules are implemented as moodle courses and provide students with an additional possibility to learn and practice. This is especially helpful for students who begin their studies in summer term and thus have to catch up on topics from the first semester without having a traditional lecture. But the self-learning modules also aim at students who just need additional practice or repetition.
+STACK plays an integral role in these self-learning modules. They consist of interactive chapters, in which STACK questions are not only used for practice but also to engage with new content. The chapters are accompanied by a set of practice tasks, which are all implemented using STACK. STACK is thus used both for classical assessment in the practice tasks as well as for interactive elaboration of mathematical topics in the chapters. To illustrate these two aspects, one STACK question each from the interactive chapters and the practice quiz is introduced below. The questions are part of the self-learning module on complex numbers.
+The first example is a question from the elaboration phase on the multiplication rule, i.e. from an interactive chapter. It is divided into three parts. First, the formula for the multiplication of complex numbers is given. Second, students are asked to complete statements on the geometric interpretation of multiplications involving complex numbers using dropdown menus. And third, students have the opportunity to explore the geometric representation in an interactive JSXGraph applet. As stated above, STACK is not primarily used for classical online assessment here, but to enhance engagement with new topics. The possibility to provide detailed feedback based on students' answers supports this goal, as students can answer the questions multiple times and use the feedback to revise their answers.
+STACK questions are also used in practice quizzes where the main goal is to provide a range of tasks with different difficulty levels for students to practise what they have learned in the interactive chapters. Here, students also have unlimited tries and can check their answers. In the example question shown below, student are asked to perform arithmetic operations such as subtraction or division on complex numbers. STACK provides the possibility to give dedicated feedback for partially correct or incorrect answers (for instance if the resulting complex number is correct but not simplified, students receive feedback that the answer is algebraically equivalent but not in the correct form). In some cases, additional feedback is given for correct answers explaining important concepts or pointing out connections between topics. After submitting the practice quiz, worked solutions are shown using the general feedback.
+Overall, STACK plays an important role in the IdeaL project because of the possibility to provide interactive and adaptive online learning materials for students.
+The project IdeaL is funded by the German Foundation for Innovation in Higher Education. For more information visit https://www.oth-aw.de/ideal or contact us via email: Prof. Mike Altieri.
+The project responded to challenges faced during COVID-19 by helping partner institutions use STACK to develop a) digital tools that assess competencies within science, technology, engineering and mathematics (STEM), particularly mathematical sciences +and b) digital assignments within several specific mathematical subjects at the partner institutions.
+Based on the STACK system, the project aimed at development of a) digital tools that assess competencies within science, technology, engineering and mathematics (STEM), particularly mathematical sciences +and b) digital assignments within several specific mathematical subjects at the partner institutions.
+Outcomes from this project include
+Project partners
+HELM_Sept_23.mbz
Moodle backup file for the collection of HELM materials released through this project.Example Given a plot of a function in two variables, ask the student to select all local maxima/minima by clicking on the plot itself. The algebraic expression for +is not given to the student.
+ +The motivation behind this question is to help the student to get familiar with surfaceplots, multivariable functions and points of interest in such plots and functions. Currently we only ask the student to identify local maxima and minima, but the question could be adapted to also ask for saddlepoints, global and local extremas and so forth.
++ | + | + |
This two-year project, funded by Innovate UK, is a collaboration between the School of Mathematics at the University of Edinburgh and IDEMS International. The project focuses on two main workstreams which both aim to enhance STACK as follows.
+Alongside the above two core aims, the project also involves the continuous addition of smaller-scale features and general code-base maintenance, which aims to improve the user experience in STACK.
+Mathematical proof forms the core of an undergraduate degree in mathematics. While STACK was originally designed for reasoning within algebraic equivalences, it has grown to encompass numerous aspects of mathematics. +However it remains difficult to assess text-based proof questions. +One of the main aims of the project is to address this, by adding features that can not only allow STACK to automate assessment of proof questions but also facilitate new avenues in which both students and authors can interact with and understand mathematical proof.
+In recent years STACK has seen broader applications, with it being used by members of the STACK Professionals Network and IDEMS International to write questions for statistics and data science. +Questions in these fields typically require the student to interact with and analyse datasets. +One approach for writing such questions involves randomly generating a dataset which the student can then download. +As datasets grow, such processes understandably lead to system timeouts. +Optimising the STACK codebase so that larger datasets can be processed is key to broadening its usage within statistics and data science.
+The currently released project outcomes of the project include:
+parsons
question block in STACK v4.5.0 enabling author's to write Parson's problems for proof. See the case study for details on the development of Parson's problems within STACK. Parson's problems require the student to drag and drop pre-written steps into the correct order in order to prove a given statement. General tools for generating specific line-by-line feedback and assessing non-unique answers are included. An example of the user interface and automated line-by-line feedback are displayed below. parsons
question block in STACK v4.6.0 (Documentation). This block now supports more general layouts involving multiple columns and grids, which allow the student to drag items to specific groups or specific points on a grid. An example of a question in the grid layout is given below. Official project website: https://www.oth-aw.de/stack-net
+STACK Net is an initiative aimed at enhancing collaboration and communication in the educational sector, particularly among the German-speaking STACK community. The project brings together sixteen participants from six different higher education institutions in Germany. This diverse group includes vice-presidents, an associate dean, a teaching mentor, professors, directors, a lecturer, and educational media developers, underscoring the project's broad approach and commitment to advancing educational practices across various academic fields.
+The STACK Net initiative is led by Prof. Dr. paed. Dipl.-Math. Mike Altieri and aims to foster innovative teaching practices. The main objectives of STACK Net include +- the regular issuance of a STACK newsletter, +- the development and implementation of a community-oriented website designed for networking, +- and active participation in various events where STACK is relevant.
+These initiatives are designed to promote greater engagement and dissemination of knowledge within the STACK community.
+Since its inception, STACK Net has achieved noteworthy milestones such as the consistent distribution of newsletters, with a growing subscriber base within the first year standing at 124 individuals (Figure 1). Additionally, the networking website is set to be completed in 2024 and will serve as a central hub for community interactions.
+ + +Moving forward, the next steps for STACK-Net involve rolling out further newsletter articles and finalizing the implementation of the community website. This website, known as "STACK Connect", serves as a hub for the German-speaking STACK community. It aims to foster collaboration, facilitate knowledge sharing, and provide support for both seasoned members and newcomers alike. The platform features discussion forums tailored to address common challenges within the community and specific interest groups for more focused conversations. Additionally, it encourages active participation, knowledge exchange, and engagement with STACK newsletters.
+Future milestones include internal quality enhancement through the creation and deployment of screencasts, as well as the automation of the STACK contact network establishment on the homepage, which will further streamline operations. Additionally, routine distribution of the STACK newsletter, including regular data collection from the global STACK community, will be operational.
+The project STACK Net is funded by the German Foundation for Innovation in Higher Education. For more information visit https://stack-connect.oth-aw.de or contact us via email: Prof. Mike Altieri.
+This page contains details of projects involving, or based around, STACK.
+STACK Net is an initiative aimed at enhancing collaboration and communication in the educational sector, particularly among the German-speaking STACK community.
+The IdeaL project set up an innovation network in which flexible learning modules for adaptive digital teaching are developed and produced.
+The project aims to develop proof-assessment features for STACK as well broaden its applicability within statistics and data science.
+The project aims at enabling teachers to create digital online learning material with multimodal, dynamic, and interactive elements, (GeoGebra) and with adaptive automatic feedback for use in hybrid settings.
+The project responded to challenges faced during COVID-19 by helping partner institutions use STACK to develop a) digital tools that assess competencies within science, technology, engineering and mathematics (STEM), particularly mathematical sciences +and b) digital assignments within several specific mathematical subjects at the partner institutions.
+If you have projects involving STACK, and would like section of this website, please contact the STACK development team.
We regularly host conferences and training workshops for STACK. You can find videos from previous events on our YouTube channel.
+ + +Here is a list of upcoming events:
+ + +Title | +Date | +Location | +Description | +
---|---|---|---|
STACK library and other v4.8.1 features | +3 Dec 2024 | +Online (Zoom) | +This online talk is to demonstrate the new features in STACK v4.8.1, including the STACK library and "todo" blocks. | +
OER Design and Use in Mathematics Education | +Autumn 2024 | +TTK, Tallinn | +This is a 5 day, on-site, training in developing OER as part of the Gate2Math Erasmus+ project. | +
During 2020 we ran a series of online workshops, with support from the London Mathematical Society, Universitas 21, and the School of Mathematics at the University of Edinburgh. Details, and recordings, are online..
+A list of past STACK workshops and conferences:
+ + +Title | +Date | +Location | +Description | +
---|---|---|---|
Discovering AuthOMath | +8 May 2024 | +University of Cantabria, Santander, Cantabria, Spain | +This event will offer insights into innovative teaching approaches and practical applications for secondary and tertiary education mathematics teachers, developed through the AuthOMath project. | +
AuthOMath multiplier event: using GeoGebra in STACK | +27 March 2024 | +School of Mathematics, University of Edinburgh. Online | +AuthOMath has enabled us to integrate GeoGebra into STACK. This is a practical workshop for users interested in getting started with this new feature. | +
International Meeting of the STACK Community 2024 | +11-13 March 2024 | +Ostbayerische Technische Hochschule (OTH), Amberg-Weiden, Germany | +A forum for all STACK users to exchange experiences, ideas and research topics. | +
Learning Mathematics : digital and interactive | +03 February 2024 | +Heidelberg University of Education, Germany | +This one day digital conference will highlight the outcomes of of the AuthOMath Erasmus+ project. | +
African STACK Conference for Undergraduate Mathematics | +19 - 23 June 2023 | +Masinde Muliro University of Science and Technology, Kakamega, Kenya | +A conference to enable African educators with experience on STACK to define a roadmap for transformation of African Undergraduate Maths Education | +
MoodleMoot DACH 2023 | +13-16 June 2023 | +ETH Zürich, Switzerland | +A DevCamp and a BarCamp for German-speaking Moodle users, with a special track on STACK. English speakers welcome! | +
International Meeting of the STACK Community 2023 | +24 - 26 April 2023 | +Tallinn, Estonia stack2023.com |
+A forum for all STACK users to exchange experiences, ideas and research topics | +
The 1st Northern e-Assessment Meeting | +31 May - 2 June 2023 | +Trondheim, Norway uia.no |
+An informal conference for people interested in e-Assessment in Mathematical Sciences at higher education level | +
Using JSX graphs in STACK | +8 Dec 2022 | +Hybrid event: The University of Edinburgh, James Clerk Maxwell building and online | +This workshop will provide hands-on practise on how to create JSXGraphs and how to use JSXGraph in STACK assessments. | +
Using STACK to put the "book inside the quiz" | +16 Nov 2022 | +Hybrid event: The University of Edinburgh, James Clerk Maxwell building and online | +This workshop discusses how we have used a metaphore of "putting the book inside automatically assessed online quizzes" in a variety of university mathematics courses. | +
Mathematics STACK Workshop | +4th-8th Jul 2022 | +Maseno University, Kenya | +The goal of this workshop is to help lecturers prepare the courses they will be teaching at their institutions using STACK. There will be a parallel session on authoring STACK questions. | +
EAMS 2022 | +13 - 24 Jun 2022 | +Online. eams.ncl.ac.uk |
+There will be various sessions of interest to STACK users at this conference. | +
International Meeting of the STACK Community 2022 | +24-28 Apr 2022 |
+Leoben, Austria unileoben.ac.at/stack22 |
+A forum for all STACK users to exchange experiences, ideas and research topics | +
EAMS 2021 | +21 Jun - 2 July 2021 | +Online. eams.ncl.ac.uk |
+There will be various sessions of interest to STACK users at this conference. | +
The 4th International STACK Conference | +26-29 Apr 2021 | +TTK University of Applied Sciences, Tallinn, Estonia stack21.edu.ee Proceedings |
+This conference aims to act as a forum for the exchange of experience, ideas and research associated with implementing STACK. The target group is academics who teach undergraduate and postgraduate STEM courses in higher education institutions. | +
EAMS 2020 | +Week of 22 Jun 2020 | +Online. eams.ncl.ac.uk |
+There will be various sessions of interest to STACK users at this conference. | +
The 3rd International STACK Conference | +27 Apr 2020 | +Online. 3rd International STACK Conference Website |
+The aim of the conference is to provide a platform for academics, researchers, and scholars, to address common challenges, share knowledge and ideas as well as recent trends and brainstorm creative solutions in the field of STACK. | +
Putting educational research into practice in HE Mathematics and Statistics teaching | +29-30 April 2019 |
+The University of Edinburgh, Edinburgh, UK Putting research into practice conference website |
+Day 1: Putting educational research into practice in HE Mathematics and Statistics teaching. Day 2: The 2nd International STACK Conference. |
+
The 1st International STACK Conference | +15-16 Nov 2018 | +Friedrich-Alexander-Universität Erlangen-Nürnberg, Fürth, Germany www.stack-konferenz.de |
+The mission of the Conference is to offer a forum for the STACK Community, to exchange ideas about possibilities and challenges in creating questions, to get closer to solving technical questions and to shape the future of STACK. | +
This guide outlines the process for contributing to the ongoing development of the STACK website. It is not necessarily complete and in some cases it may be useful to find additional tutorials.
+If you are already confident with git and markdown then the technical details are given in the git README.md file.
+This website is stored in the git version control system. To update the website you need to
+You can review your proposed changed before you commit them, and a full history of all your commits will be available to everyone. (Drafts which you do not commit remain private to your machine.)
+If you only need to make a minor update to a single page, e.g. adding confirmed dates to a listed event, then you can edit the files directly on github. This is the simplest way to contribute. All you need is a github account (step 1 below). Any proposed changes you commit will be reviewed by a colleague who has permissions to "push to the master branch", so you will not affect the live website immediately.
+Git and version control processes are widely documented online. Feel free to search for guides if you'd like a better understanding of version control. Two recommended sources are Roger Dudler's git - the simple guide and GitHub's About Git page.
+In order to update the website you will need the following:
+This guide is based on using VSCode for code editing and GitHub management. However, other combinations of code editors and GitHub GUIs can be used in similar ways. To set up VSCode you can see this guide or follow the steps below.
+The STACK website content is stored in, and built from, the GitHub repository maths/stack-web
. Git is used to manage files, and the process for managing files with git is as follows (full details on these are provided below).
maths/stack-web
to accept your changes by sending them a "pull request". The pull request is when you ask them to bring their version up to date with a particular commit from your fork of the repository. (Git will merge any changes, and colleagues with relevant permissions will be responsible for resolving any conflicts caused (rarely) when two people edit the same file!)This process appears complex! GitHub is a public space where you can store a repository and share it with others. You need an account on a public space when you ask someone else to accept your changes, i.e. when you later send them a "pull request".
+(More experienced users, who have permission to write direct to the STACK project, might not use their own fork. They might push directly to the original repository. Git is "distributed", so advanced users can share code directly between git repositories without going via public sites like github.)
+The practical process of creating a copy of a repository in your own github account is known as "forking" and is done as follows:
+maths/stack-web
repository.master
branch only by checking the relevant box (you can choose to include branches by unchecking the box and all branches will be copied). Click "Create fork": .A new repository your-github-username/stack-web
will be created and you will be redirected to it. This is your own personal copy of the website files. You have permission to write to this copy and (normally) anyone can read it. This repository is stored online in your personal account.
The practical process of creating a copy of a repository in your own local computer is known as "cloning". It is recommended to sign in to Github in VSCode at this stage if you haven't yet done so.
+To sign in to GitHub in VSCode follow these steps:
+git config --global user.email "your-email-address".
git config --global user.name "your GitHub username"
.To clone the repository follow the steps below:
+your-github-username/stack-web
.https://github.com/your-github-username/stack-web.git
, by clicking on the copy button next to it: . A "Copied!" message will temporarily appear.File --> New window
.Modify the website files to make the updates required. Ensure you follow the guidelines outlined in the git README.md file. If you would like to preview your changes you will need to follow the instructions in the Testing the website locally section of the guidelines.
+The process of getting the changes saved in your local computer into your online repository (on github) is composed of two stages:
+A commit saves the state of one or more files in the git repository in an indentifiable way. It is good practice to commit a small number of changes, preferably on a single topic, at a time and provide a short description. This helps reviewers understand the purpose of your changes. You can push commits as you create them or all together at the end. Each commit will be outlined separately in GitHub.
+When you save changes to a file or create a new file, a number will appear in the Source Control icon on the side bar: . The number represents the number of files changed. Click on the Source Control icon to commit and push your changes as follows:
+The process to update the website with your changes is done through a "Pull Request". A pull request asks colleagues (with the relevant permissions) to review your changes and "pull" your changes into the main repository. If they accept your request they will "Merge" your changes into the maths/stack-web
repository. When changes are merged into the repository, GitHub automatically updates and re-builds the website to include the changes. Creating a pull request is done as follows:
your-github-username/stack-web
repository.base repository: maths/stack-web
and base:master
. The source (the right hand side of the arrow) should be head repository: your-github-username/stack-web
and compare: master
. It should look similar to the following figure, with your username on the greyed-our area instead:
+ The following diagram summarises the steps to update the website:
+maths/stack-web
repository to create a copy of it in your own GitHub account.your-github-username/stack-web
repository to get a copy in your local computer.If others update the website while you are working on your Fork, it is recommended to sync these changes into your repository. This process consists of the following steps (full details will be added to this guide soon):
+maths/stack-web
repository into your your-github-username/stack-web
repository.Coming soon...
STACK is the world-leading open-source online assessment system for mathematics and STEM. It is available for Moodle, ILIAS and as an integration through LTI.
+Students can answer with algebraic expressions, like \(x^2+y\), and answers are graded based on mathematical properties.
+Answers are validated before they are marked, so students are not penalised for poor programming skills.
+Students are given feedback that refers to their specific answer and mistake, as if marked by hand.
+STACK can generate random questions so students are shown different variants of questions, and can repeat quizzes with new variants.
+STACK is available in most European languages and many more, including Hebrew and Japanese.
+STACK is free to use and is developed by educators like yourself. Contributions are welcomed and encouraged.
+STACK is used by schools, universities, commercial partners, developers and more, in over 25 countries. Please visit our Case Studies page to find out more. +
+We host regular training workshops and a yearly international STACK conference. To sign up for one of our future events, or see materials from our past events, go to our Training and Events page.
+ +STACK has extensive documentation available locally through GitHub and online. To get started, see the Authoring quick start or the self-contained "Getting started with STACK" guide.
+ +