16 September 2011

Update: Templates and web framework

Templating has turned out to take more time than I had expected. It is not just a matter of replacing one set of numbers in a question with another. For example,
\[2x^2 - 3x - 5 = 0\]
is mathematically equivalent to
\[(1 - 2x)(1 - x) = 6\]
but requires more understanding/work in terms of manipulating polynomial equations to solve. Perhaps I am trying to make them too general, but I think templates should be able to encode and generate these structurally different, but mathematically equivalent questions.
So far I have 3 templated questions, implemented as Python classes, each with a generate_question and a test_response method.

I've also been learning the Pyramid web framework for Python. I'll use this to build the front-end for the online assessment pilot. The project is very well documented. I wouldn't recommend it to someone with no web app development experience, but the barrier to entry is pretty low.


  1. Are you still convinced that a template needs to encode equations that are structurally different. Does this not add unnecessary complexity? Can't one achieve the same thing by creating a template for each structure and maybe tagging certain templates (that differs in structure) as related? It still feels to me that annotating content mathml is all you have to do to create and persist templated questions. This has the added benefit of encoding templated questions in a language neutral way, not tied to a specific programming language. Additionally, it has more potential to become an API.

  2. I guess you're right that questions that are mathematically equivalent but structurally different could just be generated from different templates. This is a little wasteful since you might need to encode a couple of constraints into the template (and then have to repeat those if you have another template with a different structure).

    I do however have other issues with content MathML as a template specification language.

    Here's an example with some comments: Solve for \(x\) in
    \[x^2-x-42 = 0\]
    \[(x+2)(x-3) = 36.\]

    Note 1   These two problems are mathematically equivalent and have exactly the same solutions: \(x=7\) or \(x=-6\).

    Note 2   I don't think you can combine them into 1 template using content MathML. However I agree that having 2 templates for the 2 forms is not so bad — and a lot easier for teachers to specify.

    Note 3   The biggest problem I see with using content MathML templates is that I don't think you can even specify the form of the second part of the example question above.

    In my Python template, it goes something like this:

    1. Randomly generate integer roots \(r_0, r_1 \in \{-9,-8,\ldots,9\}\) such that \(r_0,r_1\ne 0\) and \(r_0\ne r_1\). The constraints are to prevent the problem from being too easy. A zero root means that \(x\) just factors out on its own, and equal roots have their own special way of being spotted).

    2. Randomly generate 2 other integers, call them \(p_0\) and \(p_1\), with the same constraints as in Step 1 and the additional constraints that \(p_0+p_1=r_0+r_1\) and \((p_0,p_1) \ne (r_0,r_1)\). So we want two integers that sum to the sum of the roots, but are not equal to the roots.

    3. The problem is to solve for \(x\) in \((x-p_0)(x-p_1) = p_0p_1-r_0r_1\).

    Can we write this specification in content MathML? (That's a real question, by the way. I don't yet know enough about the ins and outs of MathML to answer it.)

    This might seem a bit contrived, but keep in mind that it's a fairly simple real-world quadratic equation. Some other problems are significantly more complex, with more complex constraints on what the numbers or structure in the problem should be.

  3. If we assume we have a single template for a given form then I don't think it is that hard to invent some attributes to annotate the template with. Come to think of it, we do need to annotate presentation MathML since the form is important, not content MathML as I originally thought.

    Another important thing to realise is that the template does not have to be intelligent, you only need to annotate it with enough information so that you can do the clever bits in Python (or the language of implementation). In principal you would have your own namespace, eg "gen" for generator and you would have a set of attributes that indicate whether an integer or variable must be generated and what the rules are that govern it. I can imagine one can take all the rules you specified above and annotate the presentation MathML with it but let me go try it out first.

  4. I have to ask, is it your intent to have users make templated questions? If so, would you expect them to write the rules?

    If not, how feasible is it to develop all the templated questions in Python? Aren't there too many possible variations?

  5. I do intend to have users write templates. Exactly how this will pan out is still an open question. I need to solve one problem at a time though.

    1. Prototype this thing so that learners can actually start using the system and provide feedback. Templates need to be written by semi tech savvy people. [Currently in progress]

    2. Prototype the templating UI so that anybody (but probably teachers) can write them. The templating language and UI used by teachers will absolutely be different from the internal storage. Internal storage plan at the moment: problem logic in Python; problem template (which substitutes in variables from the logic part) in XML.

    Reasons. XML is obvious. Python: I'm not convinced that we won't have to revert to a Turing-complete programming language to specify the logic eventually anyway. At this stage I don't have enough time to try and design or test an existing language that is more constrained and perhaps easier to use. I'd rather get the first prototype out asap (currently aiming for Monday 26 September) and worry about how non-programmers will write templates a little further down the line.