tag:blogger.com,1999:blog-79325462941207011412024-02-19T12:04:23.899+02:00Learning outside the wireA blog on education and technologyCarlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.comBlogger23125tag:blogger.com,1999:blog-7932546294120701141.post-54661911560678790672013-10-10T15:28:00.001+02:002018-06-11T10:37:42.373+02:00The groundhog problem<div dir="ltr" style="text-align: left;" trbidi="on">
We had a bit of a problem in Intelligent Practice, where users would
sometimes see the same type of exercise (that is, the same template) repeatedly. This made it look
like there weren't many exercises available and made for a frustrating
experience.<br />
<br />
The good news of the day is that this, the groundhog problem, has been resolved. Here is how.<br />
<br />
<a name='more'></a><br />
<h3 style="text-align: left;">
The old way</h3>
The learning algorithm behind Intelligent Practice keeps track of the level of mastery of each user. A user's mastery score is used to determine how likely it is that she will get any given exercise right. I won't go into the details of how this is done, but it gives us the starting point for the groundhog problem.<br />
<br />
Let's say our user — call her Rita — is practising the Chemistry chapter on Acids and Bases. Let's say this chapter has 20 templates. Each template represents many different exercises since each exercise is generated programmatically from a template, with different chemical compounds, scenarios, etc. Still, all level-of-mastery calculations are done on a per-template basis since a template represents a <i>type</i> of exercise.<br />
<br />
The template sequencing algorithm presented templates in a way that would challenge the user by just the right amount. Too many difficult exercises leave the user demoralised. Too many easy exercises mean that the user won't be stretched enough to learn anything.<br />
<br />
Let's say the figure below represents the probabilities that Rita will successfully answer exercises from each of the 20 available templates (represented as dots). Some templates are quite difficult, relative to Rita's level of mastery, and have low probability of being answered correctly. Some templates are easy and these have a high probability of being answered correctly.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF2kWtHTvaAuMaoMRHMZAWVpWkhlbrPBarvxGTRThCQdRUxVlAqR99qIE8-oRLBALda9mLWO7gzmXd_3rN1zHaY9wZT79ERq2Ybem8XTanmKgJcTbJfvG-XI-H1UusFMcBF_X3Z9mNTeA/s1600/probabilities.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF2kWtHTvaAuMaoMRHMZAWVpWkhlbrPBarvxGTRThCQdRUxVlAqR99qIE8-oRLBALda9mLWO7gzmXd_3rN1zHaY9wZT79ERq2Ybem8XTanmKgJcTbJfvG-XI-H1UusFMcBF_X3Z9mNTeA/s400/probabilities.png" width="400" /></a></div>
The sequencing algorithm tried to balance the difficulty of the exercises shown to Rita by picking out templates that have a probability of success, \(P(\text{success})\), between 0.5 and 0.7. This means that she'll have a reasonable chance of getting it right, but will have to do some thinking — exactly what we want for optimal learning from practice.<br />
<br />
<h3 style="text-align: left;">
<b>The problem</b> </h3>
Every now and again it happened that there were only 1 or 2 templates in the [0.5, 0.7] range, with other exercises of a similar difficulty just barely outside this range.<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9atybgzDWfmf0RRZPzRa3RgOF02eAhCh6zrEcAaeZlJ7C7_5dmnGr0Znc9h5Sqa7fpHwRDQS2Sti7Op2ElRnw-aK8F_aCVnbOG8_QY6I1kjE-JAenPAteR8qYk9nplUFnVahZGpvXxI8/s1600/bracket.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9atybgzDWfmf0RRZPzRa3RgOF02eAhCh6zrEcAaeZlJ7C7_5dmnGr0Znc9h5Sqa7fpHwRDQS2Sti7Op2ElRnw-aK8F_aCVnbOG8_QY6I1kjE-JAenPAteR8qYk9nplUFnVahZGpvXxI8/s400/bracket.png" width="400" /></a></div>
In this case Rita would see the same type of exercises over and over, even though there are actually other slightly easier or slightly harder exercises available, which should also really be shown to her.<br />
<br />
<h3 style="text-align: left;">
The new way</h3>
The problem above is really that the system has a <i>hard decision boundary</i>. If an exercise is just a little to the left of the 0.5 cut-off probability, it would not be selected, even though it could be. The solution is to soften or blur the decision boundary. We do this by mapping \(P(\text{success})\) to a new value, indicating the likelihood of showing a template next, relative to all the other templates.<br />
<br />
Here's the mapping. The height of a dot indicates the relatively probability, \(p\) of a template being selected next.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhihEF9bNY1TlU0nlthGu1tbU96Hjdrf7cedxppIXCswPLTS7lyE-A8c_J3XmVqRug2I3TDcWeP7_sugmlSiXfZlFIIRk7nxSJhAwYSmLl91JrE6rLMcGWLsQEgVSeE1sgwZhayJlbjs84/s1600/transfer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="302" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhihEF9bNY1TlU0nlthGu1tbU96Hjdrf7cedxppIXCswPLTS7lyE-A8c_J3XmVqRug2I3TDcWeP7_sugmlSiXfZlFIIRk7nxSJhAwYSmLl91JrE6rLMcGWLsQEgVSeE1sgwZhayJlbjs84/s400/transfer.png" width="400" /></a></div>
As the red dots above show, exercises that used to be in the [0.5, 0.7] range still have a good chance of being selected. Meanwhile the templates that were just to the left of the 0.5 threshold now also have a chance of being picked — a smaller chance than the red dots.<br />
<br />
This already solves the groundhog problem, since there is no hard decision boundary and all templates have some chance of being picked (although that chance is small for templates to the left and the right of the plot above). But while we're at it, why not add some more new features.<br />
<br />
<b>Recency</b> We really don't want to show the same template twice in a row and generally want to make it less likely that a template that was shown recently gets shown again. Define the recency, \(r\), of a template to be 1 if it was the last one shown, 2 if it was the one before that, etc. Multiply the relative probability of seeing a template by <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJ_t0U66TvkQNo6FKB-ukMV5qZMJUdVCITMPK9suNP2ErJ6tnYf8Ro4nd9KHLM3tL9lx1Sb7mMc-w4B9w4IVdvTba3DI-dlVq6q0cpj89Sk7QiGVusxODdMj3QBZZAe8M9NC0ONWkIG1g/s1600/recency.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhJ_t0U66TvkQNo6FKB-ukMV5qZMJUdVCITMPK9suNP2ErJ6tnYf8Ro4nd9KHLM3tL9lx1Sb7mMc-w4B9w4IVdvTba3DI-dlVq6q0cpj89Sk7QiGVusxODdMj3QBZZAe8M9NC0ONWkIG1g/s400/recency.png" width="400" /></a></div>
\[1 - \frac{1}{r}\]<br />
The template with a recency of 1 (the template that was just seen) will now have a probability of 0 of being seen again.<br />
<br />
<b>Frequency</b> We also want to penalise templates that have been seen a lot over the past while. Let \(n\) be the number of times that a template has been seen during the past week. Multiply the relative probability of seeing that template again by <br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSCg0csnY3TyyQx4ybbsnjNuqqmnZvnjrfIgmL5hjaaK1-xTvvWQUeEpPC_loreZ0O_zC3OmsR5lYzEhyGJ7kGrYtADwYbt2bQZyYuCQdYA-QOLKLmUiTQAd0aV3qayzB-LLkiNzekkvQ/s1600/frequency.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSCg0csnY3TyyQx4ybbsnjNuqqmnZvnjrfIgmL5hjaaK1-xTvvWQUeEpPC_loreZ0O_zC3OmsR5lYzEhyGJ7kGrYtADwYbt2bQZyYuCQdYA-QOLKLmUiTQAd0aV3qayzB-LLkiNzekkvQ/s400/frequency.png" width="400" /></a></div>
\[\frac{n+4}{2n+4}\]<br />
This does not penalise templates that have never been seen, and reduces the relative probability by a factor of at most 2 for templates that have been seen a lot.<br />
<br />
So, the final formula for the relative probability of showing a given template next is<br />
\[ p \times \left(1 - \frac{1}{r}\right) \times \left(\frac{n+4}{2n+4}\right) \]<br />
<br />
P.S. If, after reading this post, the name of the problem still doesn't make sense, you should watch the classic film <a href="http://www.imdb.com/title/tt0107048/" target="_blank"><i>Groundhog Day</i></a>.<br />
<br />
P.P.S. The plots in this post were generated using the wonderful <a href="http://xkcd.com/" target="_blank">xkcd</a>-style plot generator for <a href="http://matplotlib.org/" target="_blank">matplotlib</a> <a href="http://jakevdp.github.io/blog/2012/10/07/xkcd-style-plots-in-matplotlib/" target="_blank">by Jake Vanderplas</a>. </div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com1tag:blogger.com,1999:blog-7932546294120701141.post-6897993789485938732013-07-16T16:11:00.003+02:002013-07-16T16:11:44.153+02:00Monassis release 1.4 (major overhaul)<div dir="ltr" style="text-align: left;" trbidi="on">
This is the first new release in about 3 months, but it's a major one that <a href="http://projects.siyavula.com/team/" target="_blank">we're</a> really excited about. In short,<br />
<br />
<ul style="text-align: left;">
<li>the machine learning engine, which keeps track of how well you're doing and decides which exercises to show you next, was overhauled;</li>
<li>the dashboard shows much more information and in a more visual way than ever before; and</li>
<li>many more exercises got added.</li>
</ul>
<br />
This release of the practice service software coincides with the release of a new look on the <a href="http://www.everythingmaths.co.za/" target="_blank">Everything Maths</a> and <a href="http://www.everythingscience.co.za/" target="_blank">Everything Science</a> websites. These sites do really look better than ever before, so head over there to have a look.<br />
<br />
<h4 style="text-align: left;">
Major Improvement: Level of Mastery Calculation</h4>
The level of mastery is our new method for indicating how well a user is doing. Mastery is measured on a 4-star scale and getting all 4 stars in a section or chapter means that you have successfully completed it.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKqrxOWpF1RLCpxY6InCOBoesLMdsRPKUKaHwICWIo3DwYwfk_6IrxANPkMUf6PjWgWsKFdJxvutBTC6PYBUmJpUrMPICfJbjpXts6AV23YkgLTC3jykB05poNoHCSQodARFX6Fe9i4Xk/s1600/progress-bar.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKqrxOWpF1RLCpxY6InCOBoesLMdsRPKUKaHwICWIo3DwYwfk_6IrxANPkMUf6PjWgWsKFdJxvutBTC6PYBUmJpUrMPICfJbjpXts6AV23YkgLTC3jykB05poNoHCSQodARFX6Fe9i4Xk/s1600/progress-bar.png" /></a></div>
We had complaints from users that the system was taking too long to give them more difficult exercises. It is now more aggressive in that it will give users at least a few difficult exercises earlier on and increase their level of mastery accordingly if they get those questions right.<br />
<br />
<h4 style="text-align: left;">
Major Improvement: New Dashboard</h4>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjefOM4reSOY8veIcdhikOUVcaCRuyA0x77dfLXhW5uU6Obo6DAnP0d2Rebow6BfIqVOyRSvVDyb9pgeTTjDrPdVfwBXcZ50M1oWqKyZb_KECXD-ElOwUseZRcYbjrVc9ui44J6b7s28gA/s1600/new-dashboard.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Screenshot of new Intelligent Practice dashboard" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjefOM4reSOY8veIcdhikOUVcaCRuyA0x77dfLXhW5uU6Obo6DAnP0d2Rebow6BfIqVOyRSvVDyb9pgeTTjDrPdVfwBXcZ50M1oWqKyZb_KECXD-ElOwUseZRcYbjrVc9ui44J6b7s28gA/s1600/new-dashboard.png" height="190" title="" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">(click to zoom)</td></tr>
</tbody></table>
New plots and stats on the dashboard show you how well you are progressing. All progress is measured on the 4-star level of mastery scale. Users can set a goal in terms of stars and monitor their progress through the book over time.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw254nrW49_pycDjBPn_TqH1u6CAaRy6BeaLpZm5XyXtaWejO98zLVrQOjCIZdjhmUTbkYOWbs_91CJ3_ZF34oa-qWJSMYQPLhiaMudHyWb3ECy7XDOYIK22vVsRszVbOqXAynjO0GtAs/s1600/mini-dashboard.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw254nrW49_pycDjBPn_TqH1u6CAaRy6BeaLpZm5XyXtaWejO98zLVrQOjCIZdjhmUTbkYOWbs_91CJ3_ZF34oa-qWJSMYQPLhiaMudHyWb3ECy7XDOYIK22vVsRszVbOqXAynjO0GtAs/s1600/mini-dashboard.png" /></a></div>
A mini-dashboard is shown while working on exercises. The mini-dashboard shows a user's progress in the current section or chapter and also shows how many other users are currently active on the service.<br />
<br />
Other, smaller changes include the following<br />
<br />
<ul style="text-align: left;">
<li>Small question mark icons show where online help is available, to explain new features to users.</li>
<li>The dashboard now has 3-state checkboxes next to chapters, to indicate when a chapter is partially selected (i.e. some but not all of its subsections).</li>
</ul>
<br />
<br />
<h4 style="text-align: left;">
Minor improvements to teacher dashboard</h4>
<br />
<ul style="text-align: left;">
<li>Question mark icons show where online help is available, to explain what the plots show.</li>
<li>Level of mastery is shown, rather than the percentage of points correct.</li>
<li>Teachers can download PDFs of their class lists.</li>
<li>Teachers can edit their class names.</li>
</ul>
<br />
<br />
<h4 style="text-align: left;">
Minor improvements to browser support</h4>
We addressed some issues with browsers on older mobile phones as well as older versions of Internet Explorer (now supported: IE8+).<br />
<br />
Happy practising!<br />
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-72324672089003103922013-04-26T16:29:00.000+02:002013-04-26T16:29:03.434+02:00Monassis release 1.3<h2>Teacher dashboard changes</h2>
By popular demand, we implemented 3 new features for teachers. You can now<br />
<ul>
<li>download class lists as spreadsheets;</li>
<li>view a list of questions available in each chapter and section; and</li>
<li>view a summary of learners' activity in any subset of chapters or sections.</li>
</ul>
<b>Downloading class lists</b> The class summary is now available as a CSV file. You should be able to open this using any spreadsheet software. The button for downloading your spreadsheet is in the top right corner of the class summary.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBbWjd4zmnab30Eb3dFmAnlyQ-xZvSzb7Ai6eQBb5LkZXZYESe9sLsm0kYNXfEcNFuViAF-Vx7aqcp6FshLx42oboFuyLfESh-a223fBQURDtM-wcBhSHfA1p7XAyszjOXPVkOzkw7xww/s1600/download-classlist.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBbWjd4zmnab30Eb3dFmAnlyQ-xZvSzb7Ai6eQBb5LkZXZYESe9sLsm0kYNXfEcNFuViAF-Vx7aqcp6FshLx42oboFuyLfESh-a223fBQURDtM-wcBhSHfA1p7XAyszjOXPVkOzkw7xww/s1600/download-classlist.png" /> </a></div>
<b>Viewing a list of available questions</b> This was a tough one, since we don't really have questions, but templates. However teachers motivated that they would like to be see a representative list of questions available in each chapter or section so that they know how much homework to assign to learners. Such a list of questions is now available at <a href="http://everythingmaths.co.za/@@practice/question-list" target="_blank">http://everythingmaths.co.za/@@practice/question-list</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJEUjKSVt4-hrbzOxqxiPOSZA87CTQLOq1QFK6eTD19qdK2-zXawNRrO7qrdKH8UrBk5E22TXx4EcLWZ8C2GrYaKDcDxw3sf7Jfpor49dLe9r16GS18aH1CVOIvWUhsN5JGl-P36MqJ3A/s1600/question-list.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJEUjKSVt4-hrbzOxqxiPOSZA87CTQLOq1QFK6eTD19qdK2-zXawNRrO7qrdKH8UrBk5E22TXx4EcLWZ8C2GrYaKDcDxw3sf7Jfpor49dLe9r16GS18aH1CVOIvWUhsN5JGl-P36MqJ3A/s1600/question-list.png" height="210" width="320" /></a></div>
You can show or hide different sections and chapters by clicking on them. Note that multiple, different versions of each template will be shown, to provide an overview of the spread of questions that learners will encounter in a particular section. The points awarded for each questions are also shown, so that teachers can, for example, determine how many points to assign for homework.<br />
<br />
<b>Filtering by chapter or section</b> This is new functionality on the teacher dashboard, allowing you to view learners' activity in a custom selection of sections, rather than all of them. This may be useful when reviewing learners' homework. If you asked them to practice Algebraic Expressions only, you can filter on that chapter to exclude the points they attained on other sections.<br />
<br />
Initially, you will get an indication of the number of sections or chapters that are active, along with the grade or grades within which they were selected.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6ag2iBYiSSjV47UJIw3rZtZOwV7RuCJA4TOq7nO2oLtsHxD9kApnnJlUine79N9GnC6rREMEVFXUHbBlcf8CjihoEYu0-K-PhaTy0VLKJM1DppiePYOgjSeR-f2TAk-1wbUNDkhgu49g/s1600/filter-section-0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6ag2iBYiSSjV47UJIw3rZtZOwV7RuCJA4TOq7nO2oLtsHxD9kApnnJlUine79N9GnC6rREMEVFXUHbBlcf8CjihoEYu0-K-PhaTy0VLKJM1DppiePYOgjSeR-f2TAk-1wbUNDkhgu49g/s1600/filter-section-0.png" /></a></div>
Clicking on the <i>Change this</i> button will show the available filters, namely for sections and the date.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSorUS3WNYiACMil_qI9G8qdnelwNxAe6Zz7M9QQ6VxzPbeNGvicQuenNePL22d2d-pVPZNNwHaA2Qr8R0EWfkYhsPOk6Pb_veTk4S-xWS0LjL4ry7AhyupRKjYu4yW84SWbSqyCbGqIE/s1600/filter-section-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSorUS3WNYiACMil_qI9G8qdnelwNxAe6Zz7M9QQ6VxzPbeNGvicQuenNePL22d2d-pVPZNNwHaA2Qr8R0EWfkYhsPOk6Pb_veTk4S-xWS0LjL4ry7AhyupRKjYu4yW84SWbSqyCbGqIE/s1600/filter-section-1.png" /></a></div>
Each of the 3 grades has its own selection box, from which you can activate or deactivate chapters or sections.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuFMhLgYSps09HsA75GSe1AFk6cX76RGGROXfRuffC2KtMbbKbG72nIKGz3upyaU3YoYTFOnCSrzeiaqrZrWui30u0gStRV_Y_NwlzSYhNXqaA3fRqPA5vaC-WmWncVO21CDfvOLA5k4c/s1600/filter-section-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuFMhLgYSps09HsA75GSe1AFk6cX76RGGROXfRuffC2KtMbbKbG72nIKGz3upyaU3YoYTFOnCSrzeiaqrZrWui30u0gStRV_Y_NwlzSYhNXqaA3fRqPA5vaC-WmWncVO21CDfvOLA5k4c/s1600/filter-section-2.png" /></a></div>
<br />
<h2>Mobile phone interface</h2>
The table of contents and question views on the mobile phone interface (also the table interface) have been cleaned up. The table of contents now also shows the number of points that you have attained in each chapter. We will be improving the dashboard significantly before the start of the third term, but this is a small step in that direction.<br />
<br />
For improved navigation, we added extra links to the practice service from each grade and chapter when reading a book on your mobile phone. We also removed some logos and links from the top of the page, to make it quicker to get to the question itself.<br />
<br />
<h2>More questions</h2>
Our templating team has expanded! Two teachers — one in Mathematics and one in Physical Sciences — joined us around the start of April, meaning that we now have a total of 5 people working on producing more questions for the practice service. Since the release of the previous version of the practice service, we've added 40 new templates, equating to roughly 400 new questions.<br />
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-44468182582429459692013-03-12T16:56:00.000+02:002013-03-12T16:56:08.279+02:00Monassis release 1.2<div dir="ltr" style="text-align: left;" trbidi="on">
<b>Based on teacher feedback</b> during our first user group meeting, we made improvements to the Teacher Dashboard. Some teachers reported that it can be difficult to locate a learner on the different plots, especially when classes are large.<br />
<ul style="text-align: left;">
<li>You can now choose to sort your learners alphabetically rather than by performance. This makes it easy to locate a learner by name.</li>
<li>When clicking on a learner, his or her information is now highlighted in blue on all plots. This is especially helpful on the time–performance plot where it can be difficult to know which dot belongs to which learner. </li>
</ul>
There is high demand for <b>Geometry questions</b>, of which we don't have many yet. Monassis now supports TikZ and PStricks images. These are two very powerful LaTeX-based languages for specifying vector graphics. Our template team is hard at work, learning these languages and making new questions. Expect to see the first batch soon!<br />
<br />
Since generating questions and especially images from templates can be computationally intensive, we now have even more <b>aggressive caching</b> than before. Users might notice an increase in the responsiveness of the practice service, but the main gain is that we can handle more simultaneous users. At the moment we're handling about <b>1,000 questions per hour during peak times</b> and about <b>40,000 questions per week</b>, without breaking a sweat. A thumb-suck estimate is that we need to be able to process about 20 times that, to handle expected traffic later this year. On the software roadmap is some serious performance testing to check whether we need to do more work to support the required 20,000 questions per hour.<br />
<br />
We're in the process of cleaning up and improving the <b>mobile phone interface</b> to the practice service. There is still a lot of work to be done, but you might notice a few small changes already. The main changes that will show up before the next Monassis release are<br />
<ul style="text-align: left;">
<li>less cruft: we're removing or moving to the bottom of the page a lot of logos and navigation elements, to make the practice questions visible more quickly;</li>
<li>progress indicators, showing the number of points attained per chapter.</li>
</ul>
We've also <b>fixed a number of bugs</b>.<br />
<ul style="text-align: left;">
<li>Rounding of decimal numbers.<br />Due to a numerical precision issue, some numbers were rounded down when they should have been rounded up.</li>
<li>Distinguishing between sqrt(x) and x^(1/2).<br />Some questions specifically ask the learner to distinguish between square root notation and exponential notation. The system did not support this well.</li>
<li>Stricter enforcement of standard form.<br />In some questions a learner would be marked correct when writing, for example, y = 3 + x^2 rather than y = x^2 + 3 as standard the standard form of a quadratic equation.</li>
</ul>
A huge thank you to all our users and for the error reports and suggestions that you send in. They really help us to improve the practice service!<br />
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-76263629537233385612013-02-15T09:49:00.000+02:002013-02-15T09:49:07.466+02:00Monassis release 1.1.1<div dir="ltr" style="text-align: left;" trbidi="on">
As mentioned in the previous post, it's consolidation time and this release is mostly about scaling up to handle more users, streamlining a few processes and fixing bugs and user interface problems.<br />
<br />
Highlights:<br />
<ul style="text-align: left;">
<li>We're now running four copies rather than one copy of the Monassis server, to handle increased traffic. A load balancer monitors and distributes incoming requests to the different servers.</li>
<li>Streamlined handling of users' error reports, meaning that you'll get a response from our team much more quickly than before.</li>
<li>Simple profiling of different types of requests, to find ones that are taking too long to compute, so that we can optimize them.</li>
<li>Automatic processing of school spreadsheets for setting up class lists.</li>
</ul>
<br />
User interface tweaks:<br />
<ul style="text-align: left;">
<li>We've been having trouble with some images not showing up on lower end mobile phones. We're now using a different technical mechanism for displaying images, so this problem should disappear. </li>
<li>Made teacher dashboard plots scrollable for teachers with large numbers of learners in their classes.</li>
<li>Changed the default time filter range on the teacher dashboard to include today, even though it's only a partial day. It seems that teachers like to monitor their learners as they are practising in real time.</li>
</ul>
</div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-26538435796226389512013-01-24T23:51:00.000+02:002013-01-24T23:51:04.038+02:00Monassis release 1.1 (MVP)<div dir="ltr" style="text-align: left;" trbidi="on">
This release constitutes what <a href="http://siyavula.com/" target="_blank">we</a> consider the minimum viable product for use by individual learners or by teachers in the classroom. The practice service has been live since April 2012 (with a total of about 4,000 users completing about 64,000 questions since then!), but the following new features make it a must-have for the discerning teacher and the keen student.<br />
<ul style="text-align: left;">
<li><b>A teachers' dashboard</b> that shows the engagement and performance of the learners in their classes. If you're interested in how we use analytics to inform teachers where their students are succeeding or struggling, have a look at the <a href="http://everythingmaths.co.za/tour/teachers-dashboard/" target="_blank">teacher dashboard tour</a>.</li>
<li><b>A probabilistic model</b> of users' mastery of questions, sections, chapters
and subjects allows the practice service to <b>automatically adjust the
difficulty</b> of questions being presented to the user, based on their level of mastery.<br />(If you're a Bayesian inference geek: The generative model is a hierarchical Gaussian prior over mastery scores with a binomial likelihood function and a logistic link function between the prior and likelihood. Inference is done with a fully factored variational approximation.)</li>
<li>We now organise exercises using a section hierarchy rather than tags,
allowing for better navigation and filtering from the teachers' and
learners' dashboards. This also allows teachers and learners to drill down and select questions at the section rather than the chapter level.</li>
<li><b>Navigation on mobile phones </b>is functional. The www interface is still much prettier, but we're working on making mobi better every day. Making the practice service as accessible as possible over mobile phones is a huge priority for us since <b>more than two thirds of our traffic comes from mobile devices</b>. See <a href="http://projects.siyavula.com/reports/analytics/everything-series-online-readership-report-2012/" target="_blank">our 2012 readership</a> report for more information.</li>
<li>Better Postgresql database migration for altering table structure.</li>
<li>And, as always, some bugfixes.</li>
</ul>
As you can see, it's been a busy few weeks! There is still lots of cool new functionality coming this year, so watch this space. For the next few weeks, I'm planning on consolidating the code base a bit and on working on caching strategies so that we can scale to tens of thousands of users.<br />
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-53792578879821593452012-12-03T16:48:00.000+02:002012-12-03T16:48:32.674+02:00Monassis release 1.0.9<div dir="ltr" style="text-align: left;" trbidi="on">
The first release for a while, mostly because we've madly been <a href="http://projects.siyavula.com/blog/2012/11/30/creation-of-natural-sciences-and-technology-workbooks/" target="_blank">writing free textbooks over at Siyavula</a>. Now that the dust has settled, I'm back onto Monassis development and the first order of business was to switch to a new database server. We've been using SQLite until now, which has been holding up really well with about 3700 users, but it won't scale for the future, so we're moving to PostgreSQL.<br />
<br />
Changes:<br />
<ul style="text-align: left;">
<li>Migrated to PostgreSQL.</li>
<li>Much improved ASCIIsvg to PNG pipeline. The latest figures are beautiful!</li>
<li>Mathematics and Physical Sciences, Grades 10 and 12 are complete.</li>
<li>Bugfixes and unit tests.</li>
</ul>
<br />
Over the course of the next month we're getting ready for the start of the 2013 school year. This means completing alignment of Grade 11 exercises to the new curriculum and finalising a few additional features for learners and teachers. Expect the next release in about 4 weeks!<br />
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-57968677912172368132012-09-28T15:34:00.002+02:002012-09-28T15:34:19.320+02:00Monassis release 1.0.8 (codename: nostalgia)<div dir="ltr" style="text-align: left;" trbidi="on">
I just noticed that it's 1 year ago to the day that the first version
of Monassis went live. It was version 0.1, which had 6 templates and supported only numeric and multiple choice responses.
We now have more than 650 templates and can interpret chemical compounds, equations and spectroscopic notation, multiple choice, coordinates, mathematical expressions, equations and inequalities, factorised and expanded polynomials, proper fractions, decimal numbers, intervals and lists!<br />
<br />
The main purpose of the v0.1 release was to have
Grant Hillebrand's class try out the system and give some feedback on
how to improve it. The practice service has come a long way since then
thanks to a great templating team and some very strong support and great feedback from
school teachers and learners.<br />
<br />
To celebrate the occasion there's a small release going out today. (Ok, that's not entirely causal. I was actually preparing this release and then noticed that v0.1 went out on 28 September 2011.)<br />
<br />
In this release: <br />
<ul style="text-align: left;">
<li>Implemented equation responses </li>
<li>Better error reporting.</li>
<li>Support for logarithms with non-default base using underscore notation.<br />Some more prep for releasing on mobile and Mxit.</li>
</ul>
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-19741500908173036062012-09-11T11:15:00.001+02:002012-09-11T11:15:29.168+02:00Monassis release 1.0.7<div dir="ltr" style="text-align: left;" trbidi="on">
This is a minor release and mostly a collection of fixes (listed below). The exciting bit is that some of the modifications are in preparation for releasing the practice service over mobile phones... but, you'll have to wait a few more weeks for detailed news on that front. <br />
<br />
<ul style="text-align: left;">
<li>New features:</li>
<ul style="text-align: left;">
<li>Each chapter in dashboard is now a link, filtering on that chapter only</li>
<li>Implemented coordinate response type</li>
</ul>
<li>Fixes:</li>
<ul style="text-align: left;">
<li>Minor fixes to ASCIIsvg to PNG converter (Thanks Leen!)</li>
<li>Do not show "future chapters" section on dashboard when there are no more future chapters. (We finally need this since the Physical Sciences Grade 12 book has been fully templated!)</li>
<li>Fixed list response type so that it can have sub-types that use semi-colons</li>
<li>Implemented separate sessions for each book. You can now practice maths and science simultaneously without confusing the practice service.</li>
</ul>
<li>Mobi prep:</li>
<ul style="text-align: left;">
<li>Check for Mxit in request headers</li>
<li>Added spans and divs around dynamic elements in dashboard and question views</li>
<li>Implemented per-book access control</li>
</ul>
</ul>
</div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-84615035120884913182012-08-30T13:28:00.001+02:002012-08-30T13:28:14.510+02:00Monassis release 1.0.6<div dir="ltr" style="text-align: left;" trbidi="on">
Version 1.0.6 of the intelligent practice service went live late yesterday! This release is mostly a collection of fixes.<br />
<br />
The major fix is that <a href="http://www1.chapman.edu/~jipsen/svg/asciisvg.html" target="_blank">ASCIIsvg images</a> are now being converted to PNG images on the server side. Some browsers (notably IE and Chrome under Windows) do not support SVG images natively. Since most of our clients are Windows users, this was a bit of a problem. The cool thing is that images are still created using the ASCIIsvg format, making it possible to template and randomize them.<br />
<br />
The ASCIIsvg to SVG code was created by Leen Remmelzwaal here at Siyavula and is available under the GPL <a href="http://github.com/Siyavula/asciisvg-sandbox" target="_blank">here</a>. The new code base has an improved version of the original Javascript library (cleaned up and with some additional functionality) for client-side conversion and a new Python library for server-side conversion.<br />
<br />
A whole bunch of chapters have also been released in the mean time. We are very close to finishing up all of Grade 12 Mathematics and Physical Sciences and not too far off from finishing Grade 10 too.<br />
<br /></div>
Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-58791160996979883762012-06-26T17:46:00.003+02:002012-06-26T17:57:41.380+02:00Monassis release 1.0.5<div dir="ltr" style="text-align: left;" trbidi="on">
The next release (version 1.0.5) of the practice server went live today.<br />
<br />
The most notable new features are<br />
<br />
<ul>
<li>a "Try this question again" button, for when you got a question
wrong, read the feedback in the worked solution and want to give it
another try with a new random set of numbers.</li>
<li>8 new chapters to practise including</li>
<ul>
<li>Mathematics Grade 10, Finance and growth</li>
<li>Mathematics Grade 10, Trigonometry</li>
<li>Mathematics Grade 11, Hyperbolic functions and graphs</li>
<li>Physical Sciences Grade 10, Electric circuits</li>
<li>Physical Sciences Grade 10, Quantitative aspects of chemical change</li>
<li>Physical Sciences Grade 11, Quantitative aspects of chemical change</li>
<li>Physical Sciences Grade 12, The chemical industry</li>
<li>Physical Sciences Grade 12, Electromagnetic radiation</li>
</ul>
<li>Corrections to existing templates, based on comments from our
users. The "Report a problem" button is really working and we're able to
iron out any remaining problems within 24 hours of a report. Whoever
reported the problem gets a personal thank-you note and the other users never even know that there was a problem!<br />
</li>
</ul>
Thanks again to the team of technical content editors for pushing out
templates at an alarming rate and welcome to Eric Gulbis who joined us on the
18th!<br /><br />
(P.S. For those who are paying attention: version 1.0.4 was a minor bug-fix release.)<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-76017754052065004382012-06-12T09:12:00.000+02:002012-06-12T09:12:36.260+02:00Monassis release 1.0.3<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
Another development cycle completed! New user-facing features:<br />
<ul style="text-align: left;">
<li>A user feedback form for reporting problems with the service or any particular problem. The issues get logged in <a href="http://github.com/">GitHub</a> and our <a href="http://projects.siyavula.com/team/">crack team</a> of technical content editors process them.<br /><center style="margin-top: 1em;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwJyeZpoAes9Hcs9hYKOy3wUwC9KSPDa8iKCe_APIquDMHG5ge2jzdfp1IhSlZ5EbL5BRmGzz65VQ4acFuQc9d5Bwjma-kxnKMH_ZP8S9iMXD0Rba3jBLoxfGlKq-kgSHVygX4NAYwCrQ/s1600/monassis-report-a-problem.png"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwJyeZpoAes9Hcs9hYKOy3wUwC9KSPDa8iKCe_APIquDMHG5ge2jzdfp1IhSlZ5EbL5BRmGzz65VQ4acFuQc9d5Bwjma-kxnKMH_ZP8S9iMXD0Rba3jBLoxfGlKq-kgSHVygX4NAYwCrQ/s320/monassis-report-a-problem.png" width="173" /></a></center></li>
<li>A "How should I write this?" button that shows the user a short tutorial on the notation for different types of inputs. It is context sensitive, so if you are answering a chemistry problem, it will show you how to write chemical notation. Here's an example for fractions:<br /><center style="margin-top: 1em;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig9SBR2_dJaqlV6REZ_4DsFY53zuUdT5FDQpeFwD5-3mhBzErxsivdnbePydRBG5k7dfSHKXZmamvzyE7le63m0dC_TRNWp47VBJMV6oZXIFf1PyoOlflOGFU3MRi-L3pdlKuHH0Qaox8/s1600/monassis-how-to-write.png"><img border="0" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig9SBR2_dJaqlV6REZ_4DsFY53zuUdT5FDQpeFwD5-3mhBzErxsivdnbePydRBG5k7dfSHKXZmamvzyE7le63m0dC_TRNWp47VBJMV6oZXIFf1PyoOlflOGFU3MRi-L3pdlKuHH0Qaox8/s400/monassis-how-to-write.png" width="400" /></a></center></li>
<li>12 new chapters available between the 6 books! There are now 438 templates in the database.</li>
</ul>
New back-end features:<br />
<ul style="text-align: left;">
<li>Implemented the <rawvalueof> tag, so that XML can be generated directly using Python.</li>
<li>Consolidated the type library and added the chemical equation and string set types.</li>
<li>Bugfixes: better scientific notation rendering, changed default numeric error function to relative (rather than absolute), interval parsing failed on + operator.</li>
</ul>
The next release will focus more on back-end development, although the following user-facing features are high on the list:<br />
<ul style="text-align: left;">
<li>links from practice problems to the textbook, so that you can review content related to the problem before attempting it;</li>
<li>a "Try this question again" button if you want to repeat the same problem, but with a different set of random values.</li>
</ul>
</div>
As always, you can sign up for a free trial at <a href="http://everythingmaths.co.za/">everythingmaths.co.za</a> or <a href="http://everythingscience.co.za/">everythingscience.co.za</a>.</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-36992602629861603642012-05-28T15:59:00.002+02:002012-05-28T15:59:21.478+02:00Monassis release 1.0.2<div dir="ltr" style="text-align: left;" trbidi="on">
I'll be trying out a micro-release schedule over the next few weeks. The schedule consists of a 2-week cycle with feature development, testing and bug-fixing in-house after a feature freeze, and finally roll-out to the live site. The first cycle just completed!<br />
<br />
This is the first non-bugfix release since v1.0 with the following new developments (roughly in order of importance):<br />
<ul style="text-align: left;">
<li>lots of new chapters available (398 templates total)</li>
<li>more robust response type library</li>
</ul>
<ul style="margin-left: 40px; text-align: left;">
<li>spectroscopic notation for electron configurations</li>
<li>chemical compound notation</li>
<li>arbitrary precision floating point numbers</li>
</ul>
<ul style="text-align: left;">
<li>better unicode support</li>
<li>display subject and grade on dashboard</li>
<li>file and email logging of server errors</li>
<li>rudimentary usage analytics</li>
</ul>
Many thanks to the <a href="http://projects.siyavula.com/team/">Siyavula team</a> of technical content editors for churning out templates at an alarming rate :) The practice service would be nothing without its exercises.<br />
<br />
The next release will focus on new UI and active users should see a few cool new features on 11 June 2012!</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-804340976572644332012-04-16T17:52:00.000+02:002012-04-16T17:52:34.369+02:00We're live!<div dir="ltr" style="text-align: left;" trbidi="on">Good news everyone! The Monassis practice service is now at version 1.0 and running as part of <a href="http://siyavula.com/">Siyavula</a>'s <a href="http://everythingmaths.co.za/">Everything Maths</a> and <a href="http://everythingscience.co.za/">Everything Science</a> textbook sites. This actually happened 3 days ago, but I was too exhausted to even think about writing a post.<br />
<br />
If you'd like to try out the software, you can register as a user on <a href="http://everythingmaths.co.za/">everythingmaths.co.za</a> or <a href="http://everythingscience.co.za/">everythingscience.co.za</a>, which will get you a free 30-day trial subscription to the service. Note that this is currently a paid-for service that supplements Siyavula's free, <a href="http://creativecommons.org/licenses/by/2.0/">CC BY</a> textbooks, so if you want to use it for more than 30 days, you will have to sign up for a paid account.<br />
<br />
<h3 style="text-align: left;">What's under the hood?</h3><b>Templated questions:</b> Each question that you see is actually generated from a Python and XML-based template in a <a href="http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller">MVC framework</a>. Numbers, names, variables and a little bit of problem structure are generated randomly each time that you reload a template. We currently have about 350 templates, which might not sound like a lot but each templates equates to a bunch of individual exercises.<br />
<br />
<b>Cool maths parser:</b> Maths problems requiring symbolic responses -- like factorised or expanded polynomials, inequalities, intervals on the real line, proper fractions, sequences and sets -- are all parsed from a simple calculator-like notation; and compared to a model solution.<br />
<br />
<b>Instant feedback:</b> This is a practice service, which means that you get a detailed solution to each exercise once you've submitted your response.<br />
<br />
<b>Dashboard:</b> Simple feedback on your progress with different types of exercises. Exercises are organised by chapter in the Everything Maths and Science textbooks and users see a breakdown of their progress per chapter.<br />
<br />
<h3 style="text-align: left;">What's next?</h3><b>Gathering data: </b>You can't make an omelette without eggs and you can't do inference without data. Now that the service is live, we'll be encouraging people to sign up with the aim of getting as broad coverage as possible of school-age students in the country.<br />
<b> </b><br />
<b>Diagnostics:</b> We're already gathering usage data from the textbook websites, giving insight into how our books get used around the country. This will be extended to mining user data from the practice service for a couple of purposes:<br />
<ul style="text-align: left;"><li>helping individual users identify weaknesses and helping them find educational resources to remedy those weaknesses;</li>
<li>discovering correlations between exercises across grades and subjects, to better understand how students can learn difficult concepts;</li>
<li>reporting on the state of education in South Africa, to inform decision about educational programmes or interventions.</li>
</ul><b>Scalability: </b>We're aiming to get a lot of people to sign up and we'll need a bit more work on the hardware and software to handle the load.<br />
<br />
<b>More templates:</b> Work continues to get good coverage of all the chapters in all our current and future books. Making templates takes time, effort, technical skill and good exercise design. Our crack team of technical content editors is on the job.<br />
<br />
<b>Mobile interface:</b> Most of our target audience use mobile phones to access online content and the service will soon be re-styled to accommodate them.<br />
<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-77532726812169031142011-11-28T14:30:00.000+02:002011-11-28T14:30:21.668+02:00Funded<div dir="ltr" style="text-align: left;" trbidi="on">As you can see from the logo to the right, the <a href="http://www.shuttleworthfoundation.org/">Shuttleworth Foundation</a> has very generously provided some funding towards the development of Monassis! The funding is part of their mini-grant programme.<br />
<br />
This is great news for since I'll be able to spend more time on development and perhaps even a bit of money on hardware. Since I'm developing the platform for both browsers and fairly low-end mobile devices, it's important to be able to test on these.<br />
<br />
Thanks SF and thanks <a href="http://kefletcher.blogspot.com/">Kathi</a> for the nomination!<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-45920295658616274062011-11-02T18:41:00.000+02:002011-11-02T18:41:53.132+02:00Keen teachers create Monassis templates<div dir="ltr" style="text-align: left;" trbidi="on">I have to admit that I was skeptical when <a href="http://www.markhorner.net/">Mark</a> suggested that we let teachers try their hand at creating templated questions for Monassis. To give you an idea: this requires writing at least a <a href="http://www.python.org/">Python</a> file (for the question logic) and a XML file (for the question layout and scripting the user responses and feedback). There is also the option of writing some <a href="http://www1.chapman.edu/%7Ejipsen/svg/asciisvg.html">ASCIIsvg</a> to generate graphs client-side, at run time, based on the values generated in the Python script. Oh, and I nearly forgot, you have to write LaTeX to get the mathematics right.<br />
<br />
All in all, no easy task.<br />
<br />
But, a few hours later, all of them had successfully created their own templated questions. With graphs and everything!<br />
<br />
Looks like I'm going to have to rethink my stance on distributed volunteer creation of templated questions.<br />
<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-84738807053790103882011-10-18T17:36:00.000+02:002011-10-18T17:36:03.227+02:00Version 0.2<div dir="ltr" style="text-align: left;" trbidi="on">By and a large, the past two weeks have been a success. Admittedly, I tested the boundaries of sanity along the way.<br />
<br />
The highlights are that<br />
<ul style="text-align: left;"><li>XML templates with linked Python for logic and PNG or ASCIIsvg files for images are now stored in SQL database and can be interpreted by Monassis;</li>
<li>templates have tags (based on the chapters and sections of the FHSST books) which can be used to filter the questions the user would like to practice;</li>
<li>student, teacher and author accounts exist, with students potentially belonging to class lists, each associated with a teacher.</li>
</ul><br />
The only bits that didn't make it in during the sprint are the reporting bits.<br />
<br />
I'm about to show a demo to a group of teachers (in about 1 hour) and again some time tomorrow or Thursday to another group. Hoping to get some feedback from that.<br />
<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-3672662655515807242011-10-04T10:53:00.002+02:002011-10-04T14:23:03.589+02:00Sprint: Structure and User Management<div dir="ltr" style="text-align: left;" trbidi="on">The time has come for the next round of all-out coding to push the exam practice system to the next stage of usefulness. The goal is to make it possible for a small number of teachers to have their classes practice for the upcoming exams, and to monitor their students' progress. The target date for showing this v0.2 prototype to a group of teachers is 18 October — exactly 2 weeks from now.<br />
<br />
List of features to implement:<br />
<ul style="text-align: left;"><li> <b>Structured format for questions.</b> The question templating is still too haphazard and not all question metadata is stored in a way that's easy to extract and process. The logic of questions will still be specified using Python. The question presentation and metadata will go into XML.</li>
<li><b>Hierarchical labels for questions,</b> to make filtering manageable. The tags will be along the following dimensions: topic (e.g. Physics::Mechanics::2d motion::Projectile motion), concepts (e.g. interpreting linear graphs, quadratic equations, rearranging equations), grade level.</li>
<li><b>Many more questions.</b> The structure format should make it easier for other people to help add templated questions. I should focus on whatever will be covered in the grade 10 final exams for mathematics and physics.</li>
<li><b>Teacher and student accounts.</b> Each teacher can set up a class list of students.</li>
<li><b>Reporting to students:</b> total problems completed, percentage answered correctly overall and filtered by topic, ranked list of topics that need practice.</li>
<li><b>Reporting to teachers:</b> same as students, but at both individual student level and average class level.</li>
</ul><br />
P.S. The project now has a name: Monassis.<br />
</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-14541245664354186302011-09-28T10:20:00.000+02:002011-09-28T10:20:16.368+02:00System goes live!<div dir="ltr" style="text-align: left;" trbidi="on">Version 0.1 of the software is ready for testing by the pilot user group. This version implements:<br />
<ul style="text-align: left;"><li>A web UI running as a Python Pyramid application.</li>
<li>6 templated questions that showcase the following functionality:</li>
<ul><li>Random generation of values for the question.</li>
<li>Graphs and plots generated with those values.</li>
<li>Scaffolded questions, where users answer subquestions and get feedback in stages.</li>
</ul><li>User tracking: the system records all responses that users gave to questions. No reporting is currently done, but this is on the to-do list.</li>
</ul><div style="text-align: left;"><br />
High on the to-do list are:</div><ul style="text-align: left;"><li>Adding more templated questions.</li>
<li>Categorizing questions based on a curriculum-aligned topic tree. This is critical for the future of the project, where analysis of users' responses to questions and how it relates to users' understanding of the concepts underlying said questions will play a leading role.</li>
<li>Providing some kind of reporting to teachers and students. It's not clear yet what this should look like, but a first draft will hopefully emerge from the pilot. </li>
<li>Switching to structured XML storage of templates. Templates are currently a bit ad hoc and with an eye on the future, it makes sense to XML the presentation of questions asap. The internal logic will stay in Python for now as I'm not yet convinced that MathML is capable of expressing it.</li>
</ul><div style="text-align: left;"><br />
</div></div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-53334288144526827682011-09-19T13:41:00.000+02:002011-09-19T13:41:32.336+02:00Project description<div dir="ltr" style="text-align: left;" trbidi="on">I'm developing software that will allow learners to practice exam questions through a browser or their mobile phone. This is only the outside layer of the project though. What it is <i>really</i> about is mapping learners' success at answering questions to their understanding of the concepts underlying those questions. By knowing with which concepts learners are struggling, the system can then do useful things like<br />
<ul style="text-align: left;"><li>providing more practice on the types of questions with which the learner is struggling;</li>
<li>recommending revision material to the learner from freely available educational resources (for example, see the <a href="http://www.fhsst.org/">Free High School Science Texts</a>);</li>
<li>providing feedback/reports to learners (and possibly teachers and parents) about their progress and about the specific concepts to which they should pay more attention.</li>
</ul>All of the above is done per individual learner. In this sense the system is delivering a customised practice and revision schedule for each learner.<br />
<br />
<span style="font-size: large;">How will it work?</span><br />
<b>Templated questions</b> Each test item (question) is described as a <i>template</i>. A template is a generalisation of a particular question, where the details of the question -- such as numbers and some of the structure -- can be different in each <i>instance</i> of the template. For example, a template for solving quadratic equations with integer roots might produce the following set of questions.<br />
<blockquote class="">Solve for \(x\). Write the two solutions, separated by a comma. Example: 2,-3<br />
<ol style="text-align: left;"><li>\(x^2 - 5x - 6 = 0\)</li>
<li>\((1-x)(10-x) = -8\)</li>
<li>\(x^2 - 21 = -4x\)</li>
</ol></blockquote>Note that the numbers and factorisation of each question are different, while the overall structure (quadratic with integer roots) is the same.<br />
<br />
<b>Concept map</b> Test questions are designed to test subject-specific concepts. Furthermore, concepts typically require prior understanding of other concepts — covered earlier in the curriculum — for proper understanding. By incorporating the question-to-concept and concept-to-concept dependencies into the system, it can infer with which concepts learners are struggling based on their ability to answer test questions.<br />
<br />
<b>Inference</b> This is where all the clever stuff happens. Based on the mapping of questions to concepts, the data collected of learners' responses to questions, and some mathematical modelling, the system estimates each learner's proficiency at each of the concepts in the curriculum. This information can be reported / visualised to give learners and teachers an indication of mastery and problem areas. More importantly, knowledge of a learner's mastery of concepts allows the system to provide targeted review material or additional practice for his or her problem areas.</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com1tag:blogger.com,1999:blog-7932546294120701141.post-57161272917752226802011-09-16T10:17:00.000+02:002011-09-16T10:17:14.950+02:00Update: Templates and web framework<div dir="ltr" style="text-align: left;" trbidi="on">Templating has turned out to take more time than I had expected. It is not just a matter of replacing one set of numbers in a question with another. For example,<br />
\[2x^2 - 3x - 5 = 0\]<br />
is mathematically equivalent to<br />
\[(1 - 2x)(1 - x) = 6\]<br />
but requires more understanding/work in terms of manipulating polynomial equations to solve. Perhaps I am trying to make them too general, but I think templates should be able to encode and generate these structurally different, but mathematically equivalent questions.<br />
So far I have 3 templated questions, implemented as <a href="http://python.org/">Python</a> classes, each with a <code>generate_question</code> and a <code>test_response</code> method.<br />
<br />
I've also been learning the <a href="https://pylonsproject.org/projects/pyramid/dev/">Pyramid web framework</a> for Python. I'll use this to build the front-end for the online assessment pilot. The project is very well documented. I wouldn't recommend it to someone with no web app development experience, but the barrier to entry is pretty low.</div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com5tag:blogger.com,1999:blog-7932546294120701141.post-76513305223580513832011-09-14T13:27:00.000+02:002011-09-14T13:27:42.046+02:00Frameworks vs libraries<div dir="ltr" style="text-align: left;" trbidi="on">Wow, this is the most friendly and concise description that I've ever read. In less than a minute it cleared up almost every point of confusion I had about web frameworks. Sourced from the <a href="https://docs.pylonsproject.org/projects/pyramid/1.1/narr/introduction.html">Pyramid documentation</a>:<br />
<blockquote>A <em>framework</em> differs from a <em>library</em> in one very important way: library code is always <em>called</em> by code that you write, while a framework always <em>calls</em> code that you write. Using a set of libraries to create an application is usually easier than using a framework initially, because you can choose to cede control to library code you have not authored very selectively. But when you use a framework, you are required to cede a greater portion of control to code you have not authored: code that resides in the framework itself. You needn’t use a framework at all to create a web application using Python. A rich set of libraries already exists for the platform. In practice, however, using a framework to create an application is often more practical than rolling your own via a set of libraries if the framework provides a set of facilities that fits your application requirements.</blockquote></div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0tag:blogger.com,1999:blog-7932546294120701141.post-43440757761043739042011-09-13T17:02:00.000+02:002011-09-13T17:02:48.432+02:00Sprint: Creating an exam drill system in 2 weeks<div dir="ltr" style="text-align: left;" trbidi="on"><br />
In 45 days the first paper for the national senior certificate exam will be written by grade 12 learners across South Africa. That gives me 2 weeks to create a web-based system that will allow learners to practice their exam skills so that they can actually use the system for the last month.<br />
<br />
There are, of course, larger goals and I'll write more about these later. In short, this is a really good opportunity to get a bunch of grade 12 learners to provide feedback on an automated assessment and progress tracking system that we can roll out on a larger scale in future.<br />
<br />
Components:<br />
<ul style="text-align: left;"><li>Templated questions</li>
<ul style="text-align: left;"><li>process past papers</li>
<li> develop templating language (Python + LaTeX/AsciiMath/MathML + AsciiSvg)</li>
<li>build concept map from FHSST TOC and link questions to concepts</li>
</ul><li>UI: web-based, using Python + Pyramid + MathJax + AsciiSvg</li>
<li>Back-end:</li>
<ul style="text-align: left;"><li>Templated question database + API, eventually morphing into the FullMarks API</li>
<li>User account management, user history tracking. The user history will be vital in future, for finding correlations between user performance on various concepts and for suggesting revision material based on a user's performance.</li>
</ul><li>Community feedback</li>
<ul style="text-align: left;"><li>The purpose of the pilot is to build a better system for the following school year.</li>
<li>Q&A forum where I can address their issues, and they can help one another. Use www.question2answer.org</li>
<li>Notify users of new features or questions on the system.</li>
<li>Explicitly solicit post-exam user feedback on how to improve the system.</li>
</ul><li>Hosting (Siyavula, Rackspace?), branding, domain. Maybe the last two are not so important for now.</li>
</ul></div>Carlhttp://www.blogger.com/profile/04787599184352311587noreply@blogger.com0