About a Computer Algebra based online Mathtest

Thomas Wolf Department of Mathematics, Brock University St.Catharines, Ontario, Canada L2S 3A1 email: twolf@brocku.ca

Winfried Neun Zuse Institute Berlin, D-14195 Berlin-Dahlem, Germany email: neun@zib.de

November 29, 2007

Abstract

The article describes the online mathematics test http://lie.math.brocku.ca/mathtest, its typical applications and experiences gathered.

1 Introduction

This article reports on a test of (mostly) computational skills which is accessible online under http://lie.math.brocku.ca/mathtest. The idea for such an automated mathematics test emerged during an earlier employment of TW at a university with about 130 students starting in mathematics each year. Many of them had a weak mathematical background and even fewer computational skills. The intention was to create a tool that identifies weaknesses, which is neither embarrassing to the student nor to the lecturer/TA when checking repeatedly how to do very simple operations, like adding rational numbers. The test can be repeated endlessly without requiring much supervision. The result was an online test that was used during the last 6 years (now at a different university for currently different purposes). It was slowly improved and extended, typically at the end of each academic year.

2 The Test

The test web page has the following characteristics.

• It is accessible online with a Java enabled web browser.

• The test web page allows composing a test within a few minutes by selecting question types with a mouse click out of more than 150 question templates covering 12 subjects.

• Parameters, numbers, formula expressions and variable names in questions are randomly varied, allowing students to take a test an unlimited number of times with questions being varied slightly each time.

• Tests taken by students are automatically marked, the mark is stored in a database and an email is sent to the student including the questions, the answers and a remark whether solved correctly or not.

• The user/teacher/lecturer who created a test can easily inspect all test results.

• Students can inspect all outcomes of any test they took. Their data are password protected and accessible only by themselves and by the creator of the test.

• The generation of the question posed to the student and the marking of the student’s answer are based on the computer algebra system REDUCE [1] and thus mathematical input is required for most of the questions; only few are of multiple choice type. To prevent typing mistakes the students can inspect and correct their input to every question before submitting it.

• It is free of charge to create or to take a test.

• The server hosting the test web page allows about 20 students to run the test at the same time. Currently, for each student taking a test a computer algebra session is running, therefore the students should not start the test all at the same time.

3 About the Content

Over 150 question types, grouped into 12 sections (Rational numbers, Algebraic equations, Polynomials, Power laws, Trigonometry, Logarithms, Differentiation, Integration, Substitutions, Inequalities, Basic Applications, Complex Variables), cover a great deal of high school mathematics and beyond.

Apart from typical computational questions, like polynomial multiplication or division, the use of power laws and rules about logarithms to simplify expressions, there are plenty of questions which would not be considered elsewhere, for example:

Which of the following weights is the best estimate for the weight of a cup of coffee? Type in a list, like {2,4} or {3} with one or more numbers 1,2,3,4 and use

1 for 3.010^{−6} (metric) tonnes,

2 for 3.010^{−5} kilogram,

3 for 300.0 gram,

4 for 3.010^{−7} milligram.

Wrong estimates are off by a big factor, so you do not really need to know the correct weight to answer this question.

Another specialty is that a computerized math test is not embarrassed to ask dead easy common sense math questions which are normally not asked and thus related weaknesses that would otherwise have a good chance to stay unnoticed forever.

Examples are: sorting numbers by size (including negative integers, rational numbers, square roots), and

• If x is any non-zero number then what is x^{0} ?

Type 1 for 0, 2 for x, 3 for 1, 4 for |x| , or 5 for 1/x.

• Compute the indefinite integral of f(x) with respect to x where, for all x, f(x) = 0. Use c as the constant of integration.

• Assume A, B to be two non-zero real numbers with A < B. Type in a list with one or more numbers, like 2,4 or 3, one number for each of the following statements that is TRUE. Use 1 for 1/A < 1/B 2 for 1/A < 1/B if in addition A > 0 3 for 1/A < 1/B if in addition A < 0 4 for 1/A < 1/B if in addition B > 0 5 for 1/A < 1/B if in addition B < 0 6 for 1/A > 1/B 7 for 1/A > 1/B if in addition A > 0 8 for 1/A > 1/B if in addition A < 0 9 for 1/A > 1/B if in addition B > 0 10 for 1/A > 1/B if in addition B < 0

Some questions are born out of daily experience, for example, that students can differentiate some given function y = y(x) wrt. x but not the same function c = c(α) wrt. α. This is taken care of by randomly changing variable names.

Another addressed typical weakness concerns the ability to do simple substitutions. A section on pattern matching is planned as a future extension.

4 Formula Input

A problem that comes with asking for formula input (to avoid multiple choice questions as far as possible) is that there must be some freedom in entering solutions, for example, y + x should be correct if x + y is correct. On the other hand, the answer should not involve a quotient if the question is to simplify (x^{2} − y^{2})/(x − y). Therefore, when each question is generated automatically during the test, not only is a solution generated and attached to it but also a specification which (equality) test is to be used to check the solution, i.e., how much simplification of the input shall be done by the computer.

To expect formula input as the answer has another consequence. For multiple choice tests the process of clicking a solution tag takes less than a second. To type in a formula correctly may take 20 sec or 40 sec. For some students this is already a challenge. For them the test is as much a concentration test as a mathematics test. The risk of mistyping a formula is reduced by displaying it and asking the student to confirm it.

5 Typical Applications

The online test can be used for two purposes: as a pre-requisite test at the start of a course or as a progress test during a course.

With new regulations in the Ontario mathematics curriculum, differentiation and integration are taught first time at university so these sections and the section on complex variables are suitable to test computational skills acquired during a course.

The main application though is to use the online test as a pre-requisite test at the start of a course. What makes the test suitable for this purpose is its diversity and its richness in many simple questions exposing holes in elementary skills.

The following are typical situations where the test was successfully used:

• pre-requisite test for students starting a new course (some courses, like numerical solution of partial differential equations need many pre-requisites: differentiation, integration, linear algebra, trigonometry, complex numbers (for stability analysis)).

• identifying missing basic skills in a high school math club that is run at university and aims to prepare students for math contests,

• identifying holes in the knowledge and technical skills of home schooled students (tests of home schooling packages can only check some aspects, not absolutely everything; missing basic skills stay undetected because of the missing contact with a human teacher who would otherwise notice them),

Areas where the test has not been used but should be suitable are:

• for mass testing of incoming students starting university (due to its automatic marking support),

• as a basic skills test for distant learning students (due to its online accessibility), Because the test automatically varies parameters and formulas in questions it can be run repeat-

edly. Therefore it has been used as homework to pass some online test with as many tries as necessary or as a pre-requisite for being admitted to mid-term tests also with as many tries as necessary. There are occasionally students who try an online test up to 20..30 times! Although this is not often the case, the opportunity to try a varying test 20 times would not exist if each test would have to be hand made and hand marked.

6 Planned Extensions

The following improvements and extensions should be worked on next.

Because the test has currently a text interface, it can not display symbols like √ which would be useful, for example, in the section on powers. A graphical interface would also make the test more appealing to some students.

The 12 subjects that are currently covered shall be extended gradually, for example, to include questions concerning differential equations.

The number of concurrent users could be increased considerably by having one computer algebra session serving all online users instead of having one computer algebra system being started for each user which leads to a memory shortage for more than 30 users.

A procedure that is able not only to check whether two expressions are equal but to establish a grade of similarity between two expressions can be used to give partial credits if the solution contains only minor errors. A program to establish similarities is available but the web interface is not extended yet to allow its use.

7 Technical Pre-Requisites and Portability

For composing a test one only needs a web browser and for running a test either a web browser with Java installed and enabled or a terminal window, either ssh under any Unix, or running an ssh terminal application (e.g., putty) under Windows.

The test server is running under Linux with the REDUCE computer algebra system. All test software is freely available, although there is currently no installation script as it is installed on only one Linux PC.

References

[1] Anthony C. Hearn and John P. Fitch: REDUCE User’s Manual 3.7, Konrad–Zuse–Zentrum, 1999.