Creating Randomized Tests in Lectora and Lectora Online
Designing effective assessments is a key principle of instructional design. You want to have confidence that successful completion (a.k.a. passing) of the test(s) in your course is a true and reliable indicator that the learner has mastered the material. For your reference we suggest reading our eLearning 101: A Practical Guide for information on development, from phase 1 to completion.
One way to improve the reliability of tests is to present different questions to each learner every time the test is taken. Randomizing your questions reduces the likelihood that the learner has, at best, memorized answers, is using the process of elimination on subsequent attempts to improve his or her score or, at worst, has been given the answers from other learners. Lectora® and Lectora® Online support this instructional tenet by allowing you to randomize tests or portions of tests.
Before Randomizing
To set up a randomized test, the first thing you’ll need to do is add a test with questions. If you plan on incorporating test sections you should also include these upfront. All of these options are on the Test and Survey ribbon. For more information on adding a test within Lectora or Lectora Online please reference our Test and Questions video.
By default Lectora and Lectora Online create a “standard” test. This refers to a test without randomization and does not include the elements of your course located at your title level i.e. your theme or template. In standard tests you need to include the Process Test/Survey action, typically on the last page attached to a “Done” or “Submit” button. This is not necessary for a fully randomized test or if the last test section is randomized. Since Lectora is selecting the desired number of pages randomly from within the test, there is no way of knowing which page will be selected last. Therefore, Lectora will automatically process the test after the desired number of pages has been selected; e.g., when the learner leaves the 10th page in the example below.
Standard tests in Lectora may include as many questions on a page as you’d like, including having your entire test contained on one long, scrolling page. However, when creating a randomized test, you will probably want to place each question on its own, separate page. The reason for this is that the randomization function doesn’t actually select questions. It randomly selects pages. For example, if your 100-question test contains 50 pages and you set the randomization number to 10, Lectora will select ten pages from the set—not 10 questions:
- Pages in Test: 50
- Questions per page: 2
- Total Number of Questions in “pool”: 50 x 2 = 100
- Random Pages: 10
- Questions student will see: 10 x 2 = 20
Therefore, if your goal is to randomly select individual questions, you’ll want to put only one question on each page:
- Pages in Test: 100
- Questions per page: 1
- Total Number of Questions in “pool”: 100 x 1 = 100
- Random Pages: 10
- Questions student will see: 10 x 1 = 10
Randomizing a Full Test
If you elect to randomize the entire test, Lectora will choose pages from among all of the pages in the test, regardless of the section containing them. In other words, the sections are treated as organizational only; the 10 pages for the test are pulled from ANY section.
You can only randomize a test at the test level or test section(s) level. The use of both will cause errors.
1. To randomize the entire test, select the checkbox next to Randomly Select Pages on the Test’s Behavior ribbon.
2. Then you’ll need to indicate how many pages the test should include in the Pages From Test. You can either select to randomize the test (the total number of questions) or a specific number of questions (a part of the whole) commonly known as a question pool.
Randomizing Your Test in Sections
If you’d like to section off your questions into “pool” you can use test sections to separate and organize questions, perhaps by learning objective, topic, or difficulty. A pool typically refers to taking a specific number of questions from the total. For example:
Pages in Test: 30
Sections: 4
Randomly selected from each section: 5
Questions students will see: 20
You can only randomize a test at the test level or test section(s) level. The use of both will cause errors.
1. To randomize a test section, select the checkbox next to Randomly Select Pages on the test section’s Behavior ribbon.
2. Then you’ll need to indicate how many pages the test should include in the Pages From Test. You can either select to randomize the test (the total number of questions) or a specific number of questions (a part of the whole) commonly known as a question pool.
Randomizing Only a Section of a Test
You can also choose instead to not randomize the entire test, but rather select the specific section(s) from which to randomly choose pages mixed with a non-randomized section(s). In the example below:
Pages in Test: 15
Sections: 2
Randomly selected from first section: 5
Required questions: 5
Questions students will see: 10
Important: If the last section of the test is NOT randomized you will need to include the Process Test/Survey action (for an example see the screenshot above). Alternatively, if the last section of the test is randomized you’ll need to make sure the action has been removed. Otherwise, Lectora will not know that the test is finished and calculate the score. For example, you may have a page or a section at the beginning of the test which is not randomized (i.e., required to be seen by all learners), followed by one or more randomized test sections and concluding with required (i.e., non-randomized) instructions to submit the test:
You can see this example in the Let’s Cook title which is available for preview and download.
Scoring
Remember that in all of these cases the test score is based on the point values of the questions presented. Therefore, if the point values vary by the question, the possible number of points totaling 100% will vary for each learner every time the test is taken. You will want to keep this in mind as you determine the Mastery Score (%) (i.e., the required minimum passing score) for the test.
Also important to note is that multiple tests, randomized or not, will result in an average test score of all of the tests within an Assignable Unit (AU).
Things to take into consideration
When using randomization in any of the above circumstances it’s important to note that some course elements may no longer be appropriate. For example, if your page names include a numbering system and you’re using a page title auto text, the numbers will most likely appear out of order. Also, the name of the page in your browser will default to what is in your Title Explorer. So while you may have removed the page title from the page, it may still appear numbered in your browser tab. You may also want to reconsider any numbering systems you have within the test. It’s recommended that instead of using Page x of y that you change the naming convention to Question x of y. You can also adjust the Y value to include a fixed value which represents the total questions you intend on having the learner answer. As always, make sure your navigation is set up to properly support all functionality in place within your test.
Automated Test Navigation
This article last reviewed April 2021. The software may have changed since the last review. Please visit our Release Notes to learn more about version updates.