Thoughts on setting class assessment boundaries

Knowing Where They Are

As we move towards linear assessment, in year marking and the awarding of grades is going to become increasing more important to guide students and have a gauge on progress. Here is an excel sheet that I have developed this year to help staff gain an overview of student progress within their department that can then be used within discussions with their line managers to address emerging themes across cohorts, departments and faculties.

I use this within the context of a large sixth form college where many courses have >150 students on and there are few with <40 students. This sheet comes with the following health warnings:

  1. This should not be used to make students grades match three year trends or experience extend grades.
  2. With small cohorts (<50) trends and patterns are less reliable.
  3. No-one knows what the profile of grade boundaries will look like under the linear system yet.

It can be a useful tool to review where a cohort is mid-year following assessments. I encourage departments to use this with each assessment to get any overview of the spread of results that they have given in comparison to expected outcomes and three-year trends to give a pulse measure. Further to this, I expect them to complete a cumulative version at each assessment point to a running measure of cumulative/average student achievement across the year.

How to Use the Tool 

On the first sheet you need to enter some information about your cohort and previous grade profiles. On the first sheet enter:

  • your departments historical grade profiles for the last three years.
  • the expected grade profile of your current students (referred to as MEG – minimum expected grade. You may use one of many measures to set this within your institution).
  • the maximum score for your assessment (this can be raw or percentage as long as it matches the scores that you enter in the left hand ‘student scores’ column).
  • the student scores for the current assessment / cumulative assessments.

Paste or enter your student assessment scores into the ‘student scores’ column – don’t worry about blanks if you’re pasting from a spreadsheet – these will be ignored.

Screen Shot 2016-01-17 at 10.05.45

Initial page to enter assessment grades on and grade profiles to match current assessment up against.

As soon as you have entered your trend profile, you can immediately get an visual overview of your historical data on the ‘Three Year Trend %‘ and ‘Three Year Trend (raw)‘ sheets. It will also show you summary statistics, in both raw and percentage form, of the current assessment such as maximum score, minimum score, average score (mean), as well as measures of dispersion such as range and SD. This is a useful summary sheet and I encourage staff to print this out (along with the boundaries sheet) for each standardised assessment and have this available in their departmental folders.

 

At this point, you can go to the ‘% boundaries’ and ‘Raw boundaries’ sheets and see how the scores in your current assessment compare to your three year trend and expected profile of grades. You are able to change the green cells to alter the boundaries – either by percent or by raw score depending on the sheet. As you change the grade boundaries you will see the graph update to show the new profile.

Spread of grades against three year trends and expected grade profile of cohort.

Spread of grades against three year trends and expected grade profile of cohort.

This can be used to see how the students on your current assessment, or cumulative assessments, are achieving compared to three year trends and against their expected outcome as a cohort. It can also be used to ‘award‘ grade boundaries so that they produce a spread of results that you would be comfortable with – if you are doing this though, it should not just be a case of making the current assessment ‘fit’ the shape of the expected or expected profiles.

You can download the spreadsheet below – within the zip file is a blank sheet and one with some sample data so that you can see what it can do straight away. I’d be interested in any comments, changes or additions that people make to the sheet – so if you do, please share and let me know about it.

ghani-tricks-download-button

No more highlighting – Improving learning with effective techniques.

Teaching is not just about giving the students knowledge but also providing the learner with signposts to help develop their studentship skills and become a better learner in general. A recent monograph has considered the relative benefits of a variety of revision and learning strategies that students utilise and reflected on the impact they have on both learning and retention of content.

Screen Shot 2013-07-31 at 11.32.48Some of the findings will not come as a surprise to you (highlighting and rereading are not effective strategies) but there is probably more to be gained by focusing on the top performing techniques that both us as teachers and also students themselves should be fostering. I know when students are ‘revising‘ many think that revisiting the course reader armed with a handful of multicoloured highlighters is an effective way — well at least there is something visible to show for the work!

In table 1 you can see a summary of the findings from Dunlosky’s review of research into ten different techniques often practiced by teachers and learners and their relative effectiveness.

Screen Shot 2013-07-31 at 11.25.52

Elements that seem to be key to improving retention are techniques that encourage the student to think about what they are reviewing and distributing their efforts over time.

The full article is quite a read at over 50 pages but it is possible to drop into it and review each of the ten techniques individually or just read the discussion of the article. I am planning on using the findings from this research in several ways from developing more elaborative questioning techniques to altering worksheets to encourage students to explain their answers to demonstrate their understanding (more on this here).

In class I am going to develop this as a critical thinking exercise. I am going to give a summary of this paper and the table above to students and ask them to develop their own revision strategies using a variety of the discussed techniques. Then I will ask them for a rationale for why it will be an effective method.

A technique called ‘Spaced Learning’ builds on many of the concepts that Dunlosky identify as ‘highly’ effective. This involved students completing their revision in stages, which each stage (about an hour) having four key components:

  1. Review of topic (about ~20 minutes) – this can take any form and is not prescriptive – it is about allowing the student to revisit their notes and build their knowledge base before completing the next tasks. Make sure that the topic is small – anything too large and it will be too much – this is about distributing topics over shorter, more effective revision periods.
  2. Transformation exercise (about ~20 minutes) – here students put their notes away and transform the knowledge into something else – this could be a mind map, a drawing, a song, a poem, flash cards etc. (There are some excellent ideas in this resource). The idea is that by transforming their knowledge they have to keep asking ‘why’ – why does that item link with another item on a mind map, why should that part be in the drawing …
  3. Practice testing (about 10 minutes) – with a friend, family member, study buddy – or just using the cover, write check method – quickly test yourself on the topic.
  4. Exam question (about 10 minutes) – complete an exam question on the topic / sub-topic you have completed and check this with a mark scheme.

Example of Spaced Learning I designed for my students mock revision.

Example of Spaced Learning I designed for my students mock revision.

This process takes place on a rolling timescale. On day 1 of revision students complete part 1 & 2. Then the next day return to the topic and complete part 3 & 4. On that second day the student does part 1 & 2 for their second topic as well … onwards over many days and weeks.

Spaced Learning

Are there any surprises in the articles? How might you change your practice, resources or classroom in light of this? Are the techniques ranked as you would imagine them?


References

Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J., and Willingham, D.T. (2013) Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58.

DOI: 10.1177/1529100612453266 (Full text article currently available here)

Some interesting thoughts on teaching psychology

Week 1 of my MSc in Teaching Psychology and we were asked to reflect on why students take psychology and if it’s a good preparation for university. Here are my thoughts…

Why Psychology?

To investigate the reasons for my students choosing to study psychology I set up a surveymonkey.com questionnaire that I emailed around to my current classes and placed on a Facebook page for previous students to complete. Over the duration of 5 days 54 students responded with the majority of those being current student AS 38%, A2 22%, 4% both) and 33% of the respondents being former students.  See the full results of the survey here: http://dl.dropbox.com/u/61514/msc/PsychologyPerceptions.docx

Reflecting on the results to some of the closed questions I was initially surprised to find that 95% of the students had chosen psychology within their top three subjects with 40% of students choosing it as their 1st choice subject at enrolment. Also, it was interesting to see that the vast majority of students had some prior experience of the expectations of us as a psychology department and of the content of the course through year 10 taster days (20%), open evenings (52%) and Wyke Start (a three day taster in July for prospective students)(93%).

Reviewing the answers to the question ‘why did you choose to study psychology’ the overwhelming nature of the responses fell into the ‘sexy’ or intrinsic interest category suggested by Jarvis.  Of the responses I categorised 35 of the responses at falling into this category.  Further, when looking at these 35 there were four sub-categories that were evident: career or university progression, sounded interesting (both recognised by Jarvis) but two others were specifically stating that they wanted to know about people and why they behave the way they do (this could be merged with ‘interesting’ but I felt that it was worth noting) and those that stated it was a 4th option and didn’t know what else to take.

In relation to the ‘rigor hypothesis’ put forward by Jarvis no students referred to taking the course as they felt it was an easy option or that it would be more accessible than other courses in their program.  Actually, the responses to the ‘what advice would you give to a friend who was thinking of studying psychology’ suggest quite the opposite feeling from students who are half-a-term into studying the course. Analysing the responses I categorised them into the following:

Comments about the psychology course being:

Hard work, challenging etc. Fun, exciting etc. Interesting, eye opening etc.
18 5 16

The results from the question on comparative difficulty gave slightly different results to those found by Hirschler and Banyard (2003, Jarvis pg. 3) with only 24% of students finding the course slightly or much more difficult than their other courses and 50% stating it was ‘about the same’.

Although not in the remit of this task the comments to the ‘how could psychology be better’ often referred to course content, the specification content of the boards that we deliver and student reflection on teaching methods and resources in the department.  All things that I am currently reflecting on and will hopefully get to revisit over the duration of this course.  As touched on by Joanna a key point that is often noted by the students’ perceptions of psychology is that of interest in the subject.  A challenge is for psychology teachers to keep this interest as we deliver specifications that do not always allow the students to explore areas of their interest or more recent developments in psychology.

Preparation for University

Working at a sixth form where last year 325 of the 399 students applied for a higher education course and of these 254 took places in September 2011 it is vital that we are preparing those students for university. Almost 20% of our A2 psychology cohort (111 students) took placement on a psychology related course this year. Looking at the results from my questionnaire this could increase over the coming years with 53% of respondents stating that they would like to study psychology further following their A Levels.

More so than other subjects I feel that psychology develops many transferable key skills in research, analysis, data manipulation and evaluation and this may put the students at an advantage during their first year at undergraduate level (which is supported by the findings of Linnell, 2003). It seems we are preparing students at some level with some (maybe not enough) skills but the subject knowledge that we deliver at A Level is so far removed from that of many university psychology courses that it cannot prepare them for the demands, awareness of current research, and more specific disciplines within these courses.

I make the following comments with the full awareness it could open a massive debate on the pro-vs-anti coursework issue … but … now that coursework, assessed practical research skills (APRS), or whatever term you wish to attach to such work, has been removed from A Level specifications I do fear that we are not preparing our students with the level of ‘hands on’ awareness of conducting psychological experiments. Research and research methodology is central to any psychology degree and it could be argued that all the specifications do currently assess research methods through examination but I don’t feel that this is the same thing, not by a long shot. There is quite an interesting discussion about APRS in the October 2011 ATP Today magazine.

Currently I am delivering the Edexcel specification, which involves students conducting a variety of ‘studies’, that they then could be asked questions on in the examination. I do get the students to conduct a study, or at the very least, collect some data that can be collectively analysed as a class, but I don’t feel that they are getting the same experience as they used to pre-2008.

In a recent article in The Psychologist (The journey to undergraduate psychology) Phil Banyard is quoted as saying: “…the ideal psychology undergraduate is someone who engages, who has an interest, has their own ideas, and the ability to look beyond the question… we’re looking for people who ask questions, to have a sense of wonder and to be inquisitive” (http://bit.ly/o2kb9U). Not too much to ask then! How are we as psychology teachers expected to foster and encourage all of these skills as well as covering the increasing content and skills required to be successful in any of the current A Level specifications?