The groundhog problem

We had a bit of a problem in Intelligent Practice, where users would sometimes see the same type of exercise (that is, the same template) repeatedly. This made it look like there weren't many exercises available and made for a frustrating experience.

The good news of the day is that this, the groundhog problem, has been resolved. Here is how.

Monassis release 1.4 (major overhaul)

This is the first new release in about 3 months, but it's a major one that we're really excited about. In short,

• the machine learning engine, which keeps track of how well you're doing and decides which exercises to show you next, was overhauled;
• the dashboard shows much more information and in a more visual way than ever before; and
• many more exercises got added.

This release of the practice service software coincides with the release of a new look on the Everything Maths and Everything Science websites. These sites do really look better than ever before, so head over there to have a look.

Major Improvement: Level of Mastery Calculation

The level of mastery is our new method for indicating how well a user is doing. Mastery is measured on a 4-star scale and getting all 4 stars in a section or chapter means that you have successfully completed it.

We had complaints from users that the system was taking too long to give them more difficult exercises. It is now more aggressive in that it will give users at least a few difficult exercises earlier on and increase their level of mastery accordingly if they get those questions right.

Major Improvement: New Dashboard

(click to zoom)

New plots and stats on the dashboard show you how well you are progressing. All progress is measured on the 4-star level of mastery scale. Users can set a goal in terms of stars and monitor their progress through the book over time.

A mini-dashboard is shown while working on exercises. The mini-dashboard shows a user's progress in the current section or chapter and also shows how many other users are currently active on the service.

Other, smaller changes include the following

• Small question mark icons show where online help is available, to explain new features to users.
• The dashboard now has 3-state checkboxes next to chapters, to indicate when a chapter is partially selected (i.e. some but not all of its subsections).

Minor improvements to teacher dashboard

• Question mark icons show where online help is available, to explain what the plots show.
• Level of mastery is shown, rather than the percentage of points correct.
• Teachers can download PDFs of their class lists.
• Teachers can edit their class names.

Minor improvements to browser support

We addressed some issues with browsers on older mobile phones as well as older versions of Internet Explorer (now supported: IE8+).

Happy practising!

Monassis release 1.3

Teacher dashboard changes

By popular demand, we implemented 3 new features for teachers. You can now

• download class lists as spreadsheets;
• view a list of questions available in each chapter and section; and
• view a summary of learners' activity in any subset of chapters or sections.

Downloading class lists  The class summary is now available as a CSV file. You should be able to open this using any spreadsheet software. The button for downloading your spreadsheet is in the top right corner of the class summary.

 

Viewing a list of available questions  This was a tough one, since we don't really have questions, but templates. However teachers motivated that they would like to be see a representative list of questions available in each chapter or section so that they know how much homework to assign to learners. Such a list of questions is now available at http://everythingmaths.co.za/@@practice/question-list

You can show or hide different sections and chapters by clicking on them. Note that multiple, different versions of each template will be shown, to provide an overview of the spread of questions that learners will encounter in a particular section. The points awarded for each questions are also shown, so that teachers can, for example, determine how many points to assign for homework.

Filtering by chapter or section  This is new functionality on the teacher dashboard, allowing you to view learners' activity in a custom selection of sections, rather than all of them. This may be useful when reviewing learners' homework. If you asked them to practice Algebraic Expressions only, you can filter on that chapter to exclude the points they attained on other sections.

Initially, you will get an indication of the number of sections or chapters that are active, along with the grade or grades within which they were selected.

Clicking on the Change this button will show the available filters, namely for sections and the date.

Each of the 3 grades has its own selection box, from which you can activate or deactivate chapters or sections.

Mobile phone interface

The table of contents and question views on the mobile phone interface (also the table interface) have been cleaned up. The table of contents now also shows the number of points that you have attained in each chapter. We will be improving the dashboard significantly before the start of the third term, but this is a small step in that direction.

For improved navigation, we added extra links to the practice service from each grade and chapter when reading a book on your mobile phone. We also removed some logos and links from the top of the page, to make it quicker to get to the question itself.

More questions

Our templating team has expanded! Two teachers — one in Mathematics and one in Physical Sciences — joined us around the start of April, meaning that we now have a total of 5 people working on producing more questions for the practice service. Since the release of the previous version of the practice service, we've added 40 new templates, equating to roughly 400 new questions.

Monassis release 1.2

Based on teacher feedback during our first user group meeting, we made improvements to the Teacher Dashboard. Some teachers reported that it can be difficult to locate a learner on the different plots, especially when classes are large.

• You can now choose to sort your learners alphabetically rather than by performance. This makes it easy to locate a learner by name.
• When clicking on a learner, his or her information is now highlighted in blue on all plots. This is especially helpful on the time–performance plot where it can be difficult to know which dot belongs to which learner.

There is high demand for Geometry questions, of which we don't have many yet. Monassis now supports TikZ and PStricks images. These are two very powerful LaTeX-based languages for specifying vector graphics. Our template team is hard at work, learning these languages and making new questions. Expect to see the first batch soon!

Since generating questions and especially images from templates can be computationally intensive, we now have even more aggressive caching than before. Users might notice an increase in the responsiveness of the practice service, but the main gain is that we can handle more simultaneous users. At the moment we're handling about 1,000 questions per hour during peak times and about 40,000 questions per week, without breaking a sweat. A thumb-suck estimate is that we need to be able to process about 20 times that, to handle expected traffic later this year. On the software roadmap is some serious performance testing to check whether we need to do more work to support the required 20,000 questions per hour.

We're in the process of cleaning up and improving the mobile phone interface  to the practice service. There is still a lot of work to be done, but you might notice a few small changes already. The main changes that will show up before the next Monassis release are

• less cruft: we're removing or moving to the bottom of the page a lot of logos and navigation elements, to make the practice questions visible more quickly;
• progress indicators, showing the number of points attained per chapter.

We've also fixed a number of bugs.

• Rounding of decimal numbers.
  Due to a numerical precision issue, some numbers were rounded down when they should have been rounded up.
• Distinguishing between sqrt(x) and x^(1/2).
  Some questions specifically ask the learner to distinguish between square root notation and exponential notation. The system did not support this well.
• Stricter enforcement of standard form.
  In some questions a learner would be marked correct when writing, for example, y = 3 + x^2 rather than y = x^2 + 3 as standard the standard form of a quadratic equation.

A huge thank you to all our users and for the error reports and suggestions that you send in. They really help us to improve the practice service!

Monassis release 1.1.1

As mentioned in the previous post, it's consolidation time and this release is mostly about scaling up to handle more users, streamlining a few processes and fixing bugs and user interface problems.

Highlights:

• We're now running four copies rather than one copy of the Monassis server, to handle increased traffic. A load balancer monitors and distributes incoming requests to the different servers.
• Streamlined handling of users' error reports, meaning that you'll get a response from our team much more quickly than before.
• Simple profiling of different types of requests, to find ones that are taking too long to compute, so that we can optimize them.
• Automatic processing of school spreadsheets for setting up class lists.

User interface tweaks:

• We've been having trouble with some images not showing up on lower end mobile phones. We're now using a different technical mechanism for displaying images, so this problem should disappear.
• Made teacher dashboard plots scrollable for teachers with large numbers of learners in their classes.
• Changed the default time filter range on the teacher dashboard to include today, even though it's only a partial day. It seems that teachers like to monitor their learners as they are practising in real time.

 

Monassis release 1.1 (MVP)

This release constitutes what we consider the minimum viable product for use by individual learners or by teachers in the classroom. The practice service has been live since April 2012 (with a total of about 4,000 users completing about 64,000 questions since then!), but the following new features make it a must-have for the discerning teacher and the keen student.

• A teachers' dashboard that shows the engagement and performance of the learners in their classes. If you're interested in how we use analytics to inform teachers where their students are succeeding or struggling, have a look at the teacher dashboard tour.
• A probabilistic model of users' mastery of questions, sections, chapters and subjects allows the practice service to automatically adjust the difficulty of questions being presented to the user, based on their level of mastery.
  (If you're a Bayesian inference geek: The generative model is a hierarchical Gaussian prior over mastery scores with a binomial likelihood function and a logistic link function between the prior and likelihood. Inference is done with a fully factored variational approximation.)
• We now organise exercises using a section hierarchy rather than tags, allowing for better navigation and filtering from the teachers' and learners' dashboards. This also allows teachers and learners to drill down and select questions at the section rather than the chapter level.
• Navigation on mobile phones is functional. The www interface is still much prettier, but we're working on making mobi better every day. Making the practice service as accessible as possible over mobile phones is a huge priority for us since more than two thirds of our traffic comes from mobile devices. See our 2012 readership report for more information.
• Better Postgresql database migration for altering table structure.
• And, as always, some bugfixes.

As you can see, it's been a busy few weeks! There is still lots of cool new functionality coming this year, so watch this space. For the next few weeks, I'm planning on consolidating the code base a bit and on working on caching strategies so that we can scale to tens of thousands of users.