Designing open book exams
This Toolkit provides evidence-based recommendations for designing open book exams and preparing students to take them.
17 February 2021
ThisÌýguidanceÌýis informedÌýbyÌýUCL’sÌý°ä°¿³Õ±õ¶ÙÌý³¾¾±³Ù¾±²µ²¹³Ù¾±´Ç²ÔÌýfor assessment, ourÌýoperating modelÌýforÌý2020-2021 centrally managedÌýexaminationsÌýandÌýthe introduction of theÌýnewÌýdigital assessment platform,Ìý´¡²õ²õ±ð²õ²õ³¾±ð²Ô³Ù±«°ä³¢.Ìý
Online open book exams provide a number of opportunities to promote high-quality studentÌýassessment.ÌýIf they are well designed, they invite students to explore and demonstrate their insights in more complex ways, replicating access to sources of knowledge which they will have in the world of work.Ìý
However,Ìýas studentsÌýwill have unfettered access to textbooks and web-based resources, there is an increasedÌýrisk, in the absence of invigilation,Ìýof:
- copying and pasting information;
- collusion with peers or friends; or
- seeking help from contract cheating companies.Ìý
The points belowÌýwillÌýhelp you to think through how you design your open book examÌýand support your students:
When designing the exam, consider which module/programme learning outcomes you are testing and how your questionsÌýreflect these.ÌýStructure questions that test students’ ability toÌýapplyÌýanalyse, evaluate, create, synthesise, interpret etcÌý(table 1).Ìý Ìý
Structure your open book exam questions around particular problem-based scenarios or real-world cases. Provide background information on a given topic or area of study.ÌýThis gives students an opportunity toÌýapply their skills and knowledge.ÌýÌý
Link to relevant qualitative or quantitative data and ask students to interpret and apply thisÌýin particular ways,Ìýe.g.:Ìý
What does the data show?
What relevance does thisÌýdataÌýorÌýthe scenario have?
What other factors could potentially affect this data?
How would you test for these?ÌýWhen devising questions,ÌýSocratic questions and questions reflective of levels and stages of learning may be useful (see table 2).Ìý
Devise coursework-type questions that probe student understanding, skills and knowledge rather than recall ().ÌýÌý
RemoveÌý‘bookwork’Ìýparts of questions: definitions; standard proofs etc,Ìýas answers can easily be found in books or online.Ìý
Adapt your assessment criteriaÌýto reflect the changed emphasis from recalled knowledge to applied knowledge and skills, andÌýensure that students are fully informedÌýof them.ÌýCheck assessment criteria with colleagues so they know how they have been adapted to the open book context.ÌýÌý
If possible, before the exam,ÌýgiveÌýstudentsÌýthe opportunity to apply the criteria to their work in small groups to promote discussion and deepen understanding.ÌýEnsure they practise this newÌýassessment format.Ìý
Create question papers with colleagues. Look upon this process as a supportive peer review.ÌýÌý
Ensure you haveÌýgood communicationÌýchannelsÌýwith students so they can get answers to their queries before and after the exams. Act on their suggestions if you can. You couldÌýset upÌýa Moodle Forum for shared communicationÌýand create a set of FAQs based on students’ questions.Ìý
ReduceÌýunnecessary stress through the judicious use of tone –Ìýaddress students directly using ‘you’ rather than the more impersonal ‘students’ orÌý‘they’.Ìý
Ensure that students have access to clear, detailed instructions on the open book exam(s) (see student guidance).Ìý
State a word countÌýrange andÌýprovideÌýclearÌýrecommendationsÌýfor how long a student shouldÌýtake to complete an exam. There is a risk that some students willÌýspend 24 hours writing andÌýover-produce.ÌýExplain in general terms what key qualities you are looking for in answers.ÌýThe QAA notes that students require clear instructions and communications about how long they are supposed to spend on this form of exam and how their workÌýwill be marked (Ìý5 February 2021).Ìý
Encourage students to use the practice area inÌý´¡²õ²õ±ð²õ²õ³¾±ð²Ô³Ù±«°ä³¢. They will be able to access questions, do a mock testÌýandÌýpractiseÌýuploading their answers.ÌýÌý
TableÌý1:ÌýExample questions relating to recall and applicationÌý
Type or level of questionÌýÌý | Students are asked to...ÌýÌý | Example questions and startersÌýÌý |
---|---|---|
Knowing and rememberingÌýÌý | Recall knowledge of subject matter relevant to the discussion.ÌýÌý |
|
Understanding | Demonstrate understanding by constructing meaning from informationÌýÌý |
|
Applying | Apply knowledge and understanding of a particular task or problem.ÌýÌý |
|
AnalysingÌýÌý | Examine different concepts and make distinctions between them. |
|
EvaluatingÌýÌý | Make judgements about concepts or ideas.ÌýÌý |
|
CreatingÌýÌý | Develop new ideas from what they know and understand.ÌýÌý |
|
Table 2: Examples of question typesÌý
Type of Socratic question | Example questions and startersÌýÌý |
---|---|
Clarification questionsÌýÌý |
|
Assumption questions |
|
Reason and evidence questionsÌýÌý |
|
Origin or source questionsÌýÌý |
|
Implications and consequence questionsÌýÌý |
|
Viewpoint questionsÌýÌý |
|
Reducing academic misconductÌý
Students will need to agree to a declaration of integrityÌýbefore they submit their paper to AssessmentUCL.ÌýÌýÌý
Discuss the use of Turnitin similarity reports with your students – many students will be new to this as well as the dangers of using contract cheating sites, even in timed exams. See our student article advising against contract cheating.Ìý
ReduceÌýthe chances of cheating by making it very clear what kind of behaviourÌýis unacceptable and detail what forms of material will be permitted in the context of the open book examÌý(Table 3).Ìý
Consider ways to reduce cheating (Table 3) and share UCL's guidance on academic integrity.Ìý
TableÌý3:ÌýStrategies to minimise chances of specific types of cheatingÌý
Cheating activityÌýÌý | Strategies to minimise cheatingÌý |
---|---|
Collusion: students help other students via text message or phone during examsÌý | Randomise the order of questions and answer options in multiple-choice question (MCQ) and short answer tests.Ìý |
Impersonation: someone else sits the exam for you | Include assessment tasks which follow on from earlier work which the student has done as part of online discussions which preclude impersonation.Ìý |
Advice/support from parents and familyÌý | This is difficult to provide in higher education as exams are likely to focus on relatively new research. Integration with previous work also helps here.Ìý |
Sending questions to contract cheating firms for help during examsÌý | Watermark exam questions to make it difficult to share them.ÌýAlert students to the implications for careers.Ìý |
Searching online for answers to questionsÌý | Ensure that MCQ/short answer exams are fast-paced and do not re-use questions which could be found on the web.Ìý |
Copying and pasting in pre-prepared or earlier textsÌý | Turnitin will detect texts which have been submitted before but cannot guard against copying and pasting ofÌýnewly-writtenÌýtexts. Questions built on new cases or scenarios or asking for fresh perspectives on an issue/problem can make it more difficult to use pre-prepared texts.Ìý |
Marking,ÌýmoderatingÌýand feedbackÌý
- Work with yourÌýfacultyÌýExam Liaison Officer (ELO) and decide howÌýmarkers/moderators and examinersÌýwillÌýhaveÌýaccess to scripts and how the marking process will work.Ìý´¡²õ²õ±ð²õ²õ³¾±ð²Ô³Ù±«°ä³¢ÌýwillÌýautomatically captureÌýcandidate numbers on submissionÌýand in the filename itself. SubmissionsÌýare anonymised andÌýcan beÌýdistributed as PDFs.ÌýÌý
- Marking within the new platform is optional but there are benefits including usingÌýin-line comments, highlighting, stamps,ÌýdrawingÌýtoolsÌýand easily accessed Turnitin similarity reports.ÌýTheÌýdrawbackÌýof marking in the new platform isÌýthe time spent becoming familiar with it.Ìý
- Provide informationÌýto studentsÌýon when results can be expected and how they can be accessed.Ìý
Return to theÌýtop of the page
What is an online exam?Ìý
- Explain what a remote or online open book exam is (see for example our guidance on Supporting your students to complete an online open book exam or , 2021, p.3).Ìý
How to complete an online examÌý
- Highlight the importance of identifying and organising any materials you plan to refer to during the exam.Ìý
- Advise students to revise thoroughly because time constraints limit the extent of the open book resources you can refer to during an exam.Ìý
How to use the systemÌý
- Refer students to Examinations at 911±¬ÁÏÍøÌý2021 pageÌýconcerningÌýthe technicalities of joining the exam, downloading question papers, starting and ending the exam.Ìý
- Encourage students to practise uploading their answers. If they’re using handwritten responses (calculations, graphs, etc) they will have to scan or take a photograph of their responses, convert these to a pdf documentÌýand upload these.ÌýCheckÌýthe quality of these ‘mock’ documents is acceptable and provide advice if needed.Ìý
- Inform students fully of what to do if they encounter technical glitches during the exam; the Examinations at 911±¬ÁÏÍøÌý2021Ìýexplains the purpose of the Exam Query Form and how toÌýreport on any issues they have encountered.Ìý
Time allowance
- Timeframes may vary for different examsÌýespecially forÌýtimed examsÌýwithin a 24-hour window. Your guidance should set out if there is an additionalÌýupload window allowedÌýand studentsÌýshouldÌýbeÌýmadeÌýaware of theÌýreason for this time.Ìý
- Additional time allowances and rest times: make sure that students with disabilities know exactly how theirÌýadditional requirementsÌýwill be met during online exams.Ìý
Time zones
- International times:Ìýprovide guidance toÌýstudents who are not based in the UKÌýonÌýwhen to start their exams and how long they have.Ìý
Supporting students' wellbeingÌý
- Signpost to UCL's Student Support and WellbeingÌý– a team of expert wellbeing, disability and mental health staff providingÌýa safe, confidential and non-judgemental space, in which students can discuss any issues that may be affecting their ability to study.
- Provide tips on creating ‘your own exam conditions’: quiet space, comfortable chair, suitable refreshments (Ìýadvises safeguarding against spilling drinks), access to dependable time-keeping etc. See the 'Setting up for success - what you'll need on the day'Ìýsection of our page onÌýSupporting your students to complete an online open book exam.Ìý
Return to theÌýtop of the page
Aston, P., Grove, M., Houston, K., & Sangwin, C. (2020).Ìý
Clay, J. (2020).
Elsalem, L., Al-Azzam, N., Jum'ah, A. A., Obeidat, N., Sindiani, A. M., & Kheirallah, K. A. (2020). , 60, 271-279.Ìý
Emerge Education and JISC. (2020).
Gopalan, A., & Chatley, R. (2020).
Harrison, D. (2020).
Imperial College London. (2021).
Lancaster, T., & Cotarlan, C. (2021). Contract cheating by STEM students through a file-sharing website: a Covid-19 pandemic perspective.ÌýInternational Journal for Educational Integrity, 17(1), 3. doi:10.1007/s40979-021-00070-0Ìý
Oxford Brookes University. (2020).
Quality Assurance Agency. (2021).ÌýÌý
Redden, E. (2021, 5 February).
Rees, A. (2020). ÌýTimes Higher Education.
University of Newcastle Australia. (n.d.).
University of Oxford. (2021, 14 January 2021).
University of St Andrews. (2020). Ìý
This guide has been produced by UCL Arena Centre for Research-based Education.ÌýTables 1 and 2 are kindlyÌýReproduced from University of Newcastle Australia.ÌýÌýYou are welcome to use this guide if you are from another educational facility, but you must credit the project.