With COVID-19 shifting us all online, and as someone who has taught online (fully and partially), I have had a number of colleagues approach me with questions and ask advice. Though I don’t consider myself an “expert”, on online teaching, I can (and want to) share what I do know, and my own experiences, and I’m going to try to do that here.

There are a lot of things to think about when running an online course, and ideally, we would have adequate time and resources to plan ahead. That, obviously, was not an option for most of us in March! The term “Emergency Remote Teaching” was suggested as a better descriptor of what we had to do (and are doing), and it’s worth remembering that under our current circumstances, we should be striving for survival and kindness, rather than perfection (which is impossible even under ideal conditions). That being said, we all want to do the best job we can under these circumstances and be fair to our students.

The biggest concern among the colleagues I talked to? Assessment. This can be a challenge even under normal circumstances, particularly in large courses and/or courses without a lot of grading support. With the pandemic, we had the added challenges of trying to maintain academic integrity and suddenly having all tests/exams necessarily becoming “open book” (really, open EVERYTHING). (The instructors who already planned open book exams were a few steps ahead!) Meanwhile, we also knew that our students were also facing unprecedented stresses, some of whom had to suddenly move back home, lost jobs, were concerned about their health and those of their family, and so on.

I plan to share my thoughts/reflections on some of the larger issues to do with online assessments soon, but here, I’m sharing some of tips with some examples of question types I’ve used in my online tests. (Too late for W20, I know.) I don’t claim that these tips are original – I’ve been teaching for almost 17 years, and have been influenced by numerous wonderful educators (in person, in literature, online, etc.). I’m hoping, though, that they may be of some use to others – likely folks in Biology, or other fields that would lend themselves to the types of questions I put together.  If you have questions/suggestions to add, please comment below!

Download PDF version of these tips

Considerations and suggestions for making open-book, online test questions:

Note: For open-book (open-internet!) tests, it makes sense to use questions that rely on application and analysis (higher order cognitive skills), rather than basic knowledge/recall (lower level skills). Questions should assess course learning objectives/outcomes, and students should have had some opportunity to practise the skill (as well as the knowledge) being tested (in your course or prerequisites).

It’s often challenging to figure out appropriate timing (for timed tests), and it’s worth remembering that questions that require higher order thinking skills in non-experts can take much more time than we (experts in the domain) would need. Even simpler questions involving scenarios/cases will involve time for reading and comprehension of what is being asked before the student can even start working through possible answers. If you have a colleague in a slightly different area, or graduate students who can try to answer the questions, this may give you some idea of how long your students will need (which will be longer than your colleague/grad students).

Avoid easily Google-able questions:

There are academic honesty aspects to this, but also, you’re not really testing a student’s understanding if they can quickly copy and paste something from the web to answer a question.

1. Avoid making questions asking for simple definitions.
2. Be aware that some publisher test banks have been posted on the web, and their questions can be found quite easily.
3. Use non-technical language where possible, adjusting/reordering the wording used in text/your slides.
4. Develop (or update) scenarios or cases that use unique names, different organism examples, etc.
5. Google your own questions before finalizing them.

Creating questions that use higher order cognitive processes (e.g., application, analysis, integration):

This is not an exhaustive list, and obviously not every suggestion would be appropriate in all courses – it is here to help give you some initial ideas. Some of these could be used to make multiple choice and written answer questions, while others would mostly be feasible only with written answer questions. I’ve indicated whether each suggestion below could be used for the different question types.  I’m including some examples at the end of the document, sharing more MC questions as I find they are often more challenging to make than written answer versions of a similar question. Note that students should have had previous opportunity to learn and practise the skills we are asking them to demonstrate in a high-stakes test.

1. Show data/figures that students should be able to interpret if they understand/can apply key concepts. [MC/WA]

Caution: Be sure that students are at a stage where they are expected to have the ability to understand the data you are providing. Some graphs or other visualizations require additional skills (beyond what we’ve been teaching) to be able to interpret them. Some tables/figures might need simplification to use in a test situation.

2. Provide short scenarios/cases and questions based on them. [MC/WA]

AH note: Slightly different versions can often be made (for random distribution in a test or if you want to use similar question in later tests).

3. Have students identify key pieces of information in a short scenario/case. [MC/WA]
4. Give questions on a recent scientific finding (provide paragraph from news release, or a scientific abstract) that require students to consider/identify important, relevant concepts in a course. [WA]
5. Make questions that require understanding of concepts for prediction, trouble-shooting, etc.
6. Have students identify errors in a statement or short paragraph. (With WA, can also ask for corrections.) [MC/WA]
7. Write (or identify) a (or best) description of an important concept, or identify the best example of a concept. [MC/WA]
8. Ask students to suggest/identify a relevant hypothesis or plan an experiment in a particular situation, or identify the most logical next step in an experiment in a particular situation. [MC/WA]
9. Ask students to compare and contrast with examples, or relate relevant aspects of examples. [WA]
10. Ask students to choose an example of a later topic/concept covered in the course, and relate that to earlier material in the course (or, possibly, covered in prerequisite courses). [WA]
11. Ask students to relate course topics/concepts to everyday decisions, and/or elements of their own lives. A reflection question can work well for this. [WA]
12. If there are some ongoing debates/controversies in your course, ask students to make logical arguments for one side (the one they support, or you could assign a particular side) or the evidence for/against both sides. [WA]

Some (additional) academic honesty considerations:

While we assume students have access to their books, notes, Wikipedia, etc., if it’s feasible and equitable (i.e., all students have access to computer, good internet, quiet location for test), you can adjust the timing so that there won’t be a lot of extra time to look things up. Be cautious, though, not to put students in an unnecessarily stressful time crunch.*

If possible, prepare slightly altered versions of some questions (and/or scenarios) that can be given to different students (e.g., randomly in LMS test, or by preparing a couple of versions of the test that each can be sent to half the class).

*COVI9-19 caveats

As mentioned, students will not have ideal test-taking environment or circumstances at the moment. Even though it is often recommended that time limits be constrained for academic integrity reasons, we currently need to think about the additional cognitive load and stress our students are dealing with and adjust expectations accordingly. Also keep in mind that many online systems and internet providers are dealing with higher than normal usage, which could lead to more technical problems than usual.

Some examples (sorry – all are microbiology!):

Show data/figures that students should be able to interpret if they understand/can apply key concepts. [MC/WA]

Table 1.  Initial antibiotic-containing broth cultures of a test bacterium were incubated 24 hours and observed. Samples from each were used to inoculate subcultures in regular broth medium (without the drug), which were observed for growth after another 24 hours incubation.

Concentration of antibiotic in initial broth	Growth with antibiotic (initial broth)	Growth with no antibiotic (subculture)
0	+	+
3.125 μg/ml	+	+
6.25  μg/ml	+	+
12.5  μg/ml	–	+
25.0 μg/ml	–	–
50.0 μg/ml	–	–

Based on the results in the table above, what is the minimum inhibitory concentration (MIC) of this antibiotic against this bacterium?

A. 3.125 μg/ml.
B. 6.25 μg/ml.
C. 12.5 μg/ml.
D. 25 μg/ml.
E. 50 μg/ml.

Provide short scenarios/cases and questions based on them. [MC/WA]

Answer the following two questions based on the information below.

Occasionally, a few (0-17/year)  isolated cases of bubonic plague (caused by Yersinia pestis) occur in humans in the USA. Patients develop fever, headache, chills, and weakness and one or more swollen, tender and painful lymph nodes (buboes). This illness can be fatal without prompt antibacterial treatment. In most cases, the pathogen is transferred to humans by flea bites, after fleas have fed on prairie dogs or rats. The bacterium is commonly found in prairie dogs, rats, and other wild rodents.  There is no evidence of direct rodent-to-human transmission in recent decades.

Q1: Which of the following reflects the role of rodents (rats, prairie dogs)?

1. Rodents are the reservoir of the pathogen.
2. Rodents are biological vectors of the pathogen in transmission to human hosts.
3. Rodents are mechanical vectors of the pathogen in transmission to human hosts.
4. Rodents are vehicles of the pathogen in transmission to human hosts.

Q2: Which of the following describes bubonic plague in humans?

1. It is nosocomial.
2. It is zoonotic.
3. It is endemic.
4. It is epidemic.

Have students identify key pieces of information in a short scenario/case. [MC/WA]

Consider the case below:

During and after an environmental disaster, a number of people suffered deep wounds that developed into massive, life-threatening infections. The infections involved spreading necrotizing wounds with serious tissue damage. All patients had fever. Surgical removal of affected tissues and amputations had to be performed in some patients. Antibiotic treatment was administered to all patients. Bacteria isolated from the wounds were found to be endospore-forming Gram positive rod-shaped obligate anaerobes.

Q3: If you are using the above information to identify whether the likely pathogen is Bacillus anthracis, Clostridium perfringens, or Streptococcus pyogenes, which of the following lists key facts about the case that help narrow down the pathogen, and which possible pathogen(s) could be responsible?

1. The “deep”, “spreading necrotizing wounds”, and “endospore-forming Gram positive rod-shaped obligate anaerobes” are key, indicating the pathogen can only be Clostridium perfringens.
2. The “deep”, “spreading necrotizing wounds”, “fever” and “antibiotic treatment” are key, indicating the pathogen could be Clostridium perfringens, or Streptococcus pyogenes (and NOT Bacillus anthracis).
3. The “spreading necrotizing wounds” , and “endospore-forming Gram positive rod-shaped” are key, indicating the pathogen could be Bacillus anthracis or Clostridium perfringens (and NOT Streptococcus pyogenes).
4. The “deep”, “spreading necrotizing wounds” , “fever” and “antibiotic treatment” are key, indicating the pathogen could only be Streptococcus pyogenes.

Make questions that require understanding of concepts for prediction, trouble-shooting, etc.

Q4: You’re doing a Gram stain on an unknown organism, and after Step 3, you realize you have no more safranin. There are other stains in the lab, such as basic fuchsin and Congo red (reddish/pink), as well as dark blue and black stains. Could you substitute a different stain for the final step?

1. Yes, any counterstain could be used.
2. Yes, if the replacement stain does not mask crystal violet.
3. Counterstaining using safranin is necessary to be able to differentiate between Gram positive and Gram negative cells.
4. Decolourization using safranin is necessary to be able to differentiate between Gram positive and Gram negative cells.

Write (or identify) a (or best) description of an important concept, or identify the best example of a concept. [MC/WA]

Q5: Which of the following is an example of an iatrogenic infection?

1. Antoni is recovering from surgery to remove his appendix and becomes infected after inhaling Pseudomonas aeruginosa in aerosols from the sink of his hospital room.
2. Ming is a geriatric patient who contracts influenza from the patient who is sharing her hospital room.
3. Davide develops a hepatitis infection following an endoscopy examination with an endoscope that had not been sterilized.
4. Nasrin gets a Staphylococcus aureus skin infection after getting her ears pierced with a contaminated piercing gun.

Ask students to compare and contrast with examples or relate relevant aspects of examples. [WA]

Q6: Considering basic growth requirements, and using appropriate terms for categorizing organisms based on nutrition (nutritional patterns), how do bacteria compare to plants and animals? Identify similarities and differences.

Ask students to relate course topics/concepts to everyday decisions, and/or elements of their own lives.

Q7: Cruise vacations have been popular, providing rooms, meals, entertainment, etc. on a cruise ship for thousands of people at a time, typically stopping at a few tourist destinations for short visits on shore. Given what you’ve learned in this course, would you recommend that older relatives (e.g., grandparents) take a cruise? Explain your answer from an infectious disease/health standpoint, including at least a couple of points/examples.