Archive

Archive for the ‘Assessment’ Category

Multicultural groups – additional thoughts

December 15, 2013 Leave a comment

Two weeks ago I posted my reflections on using compulsory multicultural groups in an essay assignment.  In that post I discussed the issues that arose from the assignment.  Unfortunately I was wearing blinkers at the time: I had tunnel vision.  I missed what is probably the most important benefit from using multicultural groups.

Happy Business TeamThe assignment essay was due three weeks before the end of the semester.  In the last four weeks of the semester I noticed far greater engagement in tutorials from international students generally and from students who spoke English as a second language in particular.  The timing of this improved engagement and that I had not detected similar changes in engagement in other units convinces me that the improved engagement was a result of the international students improving their cross-cultural communication skills.  The two elements I noticed in particular were that international students were more comfortable in conversing in English and that they had, somewhat, overcome their fear of being wrong.

This outcome makes all the effort worthwhile.

Literature research by students … yet again

December 12, 2013 2 comments

Six months ago I wrote of my frustration at students inability to perform literature research.  My frustrations remain.  This post details my experiences this semester, looks at what worked and questions what still can be done to improve the quality of literature research undertaken by students.

I implemented a few changes this semester to the way I taught literature research skills.  These changes have been partially successful.  Further modifications are needed.  This semester I amended my lecture notes to stress:

  • The importance of background reading before starting the literature search
  • Using reference lists from the background reading to find the first articles
  • Using the reference lists from the first articles to find other articles
  • Using the “cited by” link in Google Scholar to find more recent articles which cite the articles you have already found
  • Finally, using specialist research search engines (Google Scholar, Ebsco, Proquest, Jstor, etc) to find articles not found using other methods

I had foreshadowed making these amendments in my post six months ago.  I added an additional change which I had not written about; requiring students to submit a 300 word reflection on what they had learned about research in undertaking the assignment.  These reflections helped me understand the issues from students’ perspectives better.

No googleThe first point that surprised me is that a number of students wrote a reflection on what they had learned from their research, not a reflection on what they had learned about research.  I don’t think the requirement was ambiguous but it indicates that students’ focus is on acquiring technical skills rather than generic skills.

A second point that disturbed me was that the library was subverting my actions.  They were teaching students that literature research was using a search engine to find articles.  The library training taught students how to use search engines more effectively and how to choose the appropriate search engine but it did not discuss the importance of background reading for both choosing search terms and for assessing relevance of search results.  Also, the library did not discuss how to follow the reference chain using the article’s reference list and the “cited by” link in Google Scholar.

The third issue that surprised me was that the material I presented on how to judge quality of publications (ie identifying scholarly articles and books) and the importance of primary sources over secondary sources was novel to most students.  My students were, generally, second year undergraduate students.  I would have expected this to be existing knowledge by second year but apparently it is not.

However, the most important thing I learned from the reflections is that most students did not know what literature research was, let alone how to conduct it.  Many of the reflections described the activities undertaken and many discussed how they had used Google (or similar search engines) and how they were unable to find many articles on the topic.  I still need to find a way to address this problem.  I don’t think this is problem of the students being too lazy to do research properly.  What the problem is, I think, is that students do not understand what is wrong with their method and they think their method is the correct way of doing literature research.  I don’t think they are hearing the message that their are better methods because they do not get that their method is wrong.  The first step in training them how to do literature research is to convince students that their existing research method is faulty.  Only when they have accepted that message will they become open to learning about better methods.

I am going to experiment on methods of how to communicate this message by developing some multimedia tools.  I will play with creating a video or use Powtoons to create an animation.  The multimedia tools may help deliver the message more effectively than I can achieve just by putting the message in my lectures.

To get that message through to students I need to undo years of unthinking inductivism; hopefully I can achieve this without students suffering the same tragic demise as Russell’s inductive turkey.

Multicultural groups

December 6, 2013 1 comment

This semester, for the first time, I forced students to form multicultural groups for their major assignment: an essay on the usefulness of accounting rules.  The idea for this came from one of my colleagues who has championed this approach to group formation.  Our purpose in forcing students into multicultural groups was to improve their generic skills:  communication and teamwork skills in particular and the issues which arise concerning those skills when dealing with people from different cultures.  We consider this to be an important element of generic skill development for both domestic and international students.

multiculturalismThe Australian workplace, in general, and the accounting industry’s workplace, in particular, are multicultural environments.  Students who intend to work in the accounting industry in Australia need to be able to work effectively in that environment.  That includes communicating with and forming teams with colleagues from different cultures.  In addition, clients may come from a range of cultural backgrounds so accountants need to be able to communicate effectively with clients from cultural backgrounds different to their own.  Also, many accountants spend part of their career working overseas.  Exposure to cross-cultural communication issues while at university will help those graduates who will work overseas.  We believe that the University will fail to equip graduates properly for their experiences after graduation if our students do not know how to communicate and collaborate with people from cultural backgrounds different from their own.

The issues facing international students are different; it is more their current studies rather than their future career which will benefit from cross-cultural communication skills.  Many of our international students live in mono-cultural enclaves either in University residences or in shared accommodation off campus.  Within these enclaves they communicate using their home language, watch television and movies from their home country over the Internet, and communicate with family and friends back home using social media.  They have negligible interaction with people from other cultures and do not use English to converse except in classrooms. It is easy to understand why international students would choose to live within these mono-cultural enclaves; it is easier.  Dealing with people from other cultures and communicating in your second language is hard.  Living in these mono-cultural enclaves diminishes students’ experiences from studying in Australia, often leads to a deterioration in their English language skills, and limits their understanding of Australian educational styles.  Forcing students to work outside of these enclaves will help address these problems and help them with their studies at university.

International students who seek to remain in Australia after graduation get double benefit from being forced to work in multicultural groups.  They get the benefits that all international students get plus they get the benefit that domestic students get.

We also have anecdotal evidence of additional barriers to cross-cultural educational experiences that impinge on international students.  It appears that there is a social pressure within the mono-cultural enclaves for students to remain within the enclaves.  Students who voluntarily go beyond the enclave face ostracism.  Being compelled to go beyond the enclave prevents that ostracism.

Multicultural groups have come to be called “rainbow groups” at the University.  I am not comfortable with this label.  First, it suggests that skin colour or race is the reason for cultural differences.  This is not the case.  The wealth of literature on national culture clearly distinguishes the concept of culture from race.  Second, it suggests that culture and  ancestral origin are the same thing which they are not.  Immigrant countries like Australia and the United States provide ample evidence that ancestral origin and culture are different.  Third, the label “rainbow” and the rainbow image have been appropriated by the LGBT community and use of the term “rainbow groups” may suggest groups constructed on sexual orientation or identity rather than groups constructed on culture.  While the term “rainbow groups” is a convenient shorthand, I believe it is a term we should avoid and that we should use the more accurate but more clumsy terms of “cross-cultural groups” or my preferred term of “multicultural groups”.

Students were required to form their multicultural groups themselves.  The parameters they had to work within were that groups were to comprise three or four members and that no more than 50% of group members could be from the same culture.  As culture is difficult to measure, I used citizenship as a proxy with the ability for individual students to make a case for reclassification if they believed that citizenship did not properly represent their culture.  The example I gave for a case where reclassification might be appropriate was where a student had recently been awarded Australian citizenship but had lived for most of their life outside of Australia.  Where students held dual citizenship, I advised them to classify themselves based on the country in which they went to primary school.  If that did not resolve what country they were from then I would consider each case individually.  There were no cases which required individual consideration.

monopoly chance jailStudents who did not comply with the requirements were penalised 4 marks (out of 30) if they formed a mono-cultural group and 2 marks if they formed a multicultural group but with more than 50% of the group members coming from one country.  These penalties were waived if students could demonstrate that they had taken all reasonable steps to form a group that complied with the requirements but had been unable to do so.  Evidence of taking all reasonable steps included posts to the forum on the learning management system, email logs and mobile phone logs.  Penalties were also waived if the group disintegrated but it was not the fault of some of the group members.  An example of this was where a member withdrew from the unit shortly before the due date of the assignment.

Using citizenship as a proxy for culture does create some anomalies.  Students from Australia and New Zealand were treated as from different cultures even though cross-cultural research suggests there are few cultural differences between Aussies and Kiwis.  Conversely, students from north and south China were treated as coming from the same culture even though there is evidence of distinct cultural differences between northern and southern Chinese.  In my opinion the pragmatic advantages of using citizenship as a proxy for culture outweighed the damage from potential misclassification.

Some students objected to the requirement to form multicultural groups when the assignment requirements were released.  One student objected because she already worked in a multicultural workplace and she would not benefit from the multicultural group requirement.  This student had a valid point.  However, as with all assessment items, I design them for the average student; I do not customise assessment to meet the needs of individual students.  My reason for this is purely pragmatic, I do not have the resources to customise the assessment to meet the individual needs of every student.  Therefore, I did not modify the assignment requirements to address valid objections to the requirement.

students and globeIn addition to the main assignment requirement of a group submission of an essay on the usefulness of selected accounting rules, the assignment requirements included an individual submission of a 300 word reflection on cross-cultural communication.  This reflection was worth 3 marks and was graded solely on whether it showed reflection on what had been learned rather than mere description of the processes undertaken.  My main reason for requiring the reflective submission was to force students to think about cross-cultural communication issues as they related to their assignment.  It had a secondary benefit of telling me what cross-cultural issues were important to students.

The most common issue raised in the reflections was language.  Many students equated cross-cultural communication issues with language issues.  While there are many more issues with cross-cultural communication than just language, if language was a problem it tended to swamp any other problems there may have been and it is understandable that in a 300 word reflection that students would only address language difficulties.

Google-TranslateStrategies to deal with language issues varied among participants.  Many English speakers described how they found it necessary to slow their speaking and to avoid the use of slang and idiomatic expressions.  Speakers of other languages described how they had to ask English speakers to repeat things and often expressed surprise that the English speakers were willing to accommodate their weaker English skills.  Many groups described using online translation services such as Google translate to facilitate understanding.  Some groups addressed the difficulties of face-to-face communication by using written communication methods only.

Other cross-cultural communication issues that were raised in the reflections included punctuality for meetings, lack of commitment to the task, unreliability of group members, failure to understand the assignment task, and failure to apply the referencing and plagiarism standards of the university.  One concern I have about the issues raised is that students may be labeling individual traits as cultural traits.  For example, I have not, in my years of experience, been able to identify any cultural factor in students propensity to incorrectly reference or to plagiarise.  I believe that referencing failures and plagiarism are personal traits rather than cultural traits.  It is possible that the other perceived issues were problems with the person, not problems flowing from the culture.  I am not sure how to detect if an issue is personal or cultural which means I can’t expect my students to be able to do this either.

There were some problems with the multicultural group component of the assignment that need to be addressed in future semesters.  These include:

  1. Some students did not formalise the construction of their groups.  They had no written agreement and no agreed upon plan established.  Some of these poorly-organised groups fell apart.  To minimise the risk of this happening again I will recommend that groups sign a formal group contract.  I will post a pro-forma contract on the LMS which students can use or adapt.
  2. Groups were unable or unwilling to discipline members of the group who did not contribute.  To address this, the pro-forma group contract will include default performance standards and disciplinary measures.
  3. The largest cultural grouping in the unit was Chinese.  The Chinese were somewhat disadvantaged as embryonic groups were usually looking for a non-Chinese person to complete the group.  To address this in the future I will present the cultural statistics for the unit in lectures and advise students that members of the most populous national groups need to pro-active in forming groups or they will be in the same position that some of this semester’s groups found themselves when they discovered that they were in breach of the assignment requirements and there were no non-Chinese students left who did not belong to a group.

Overall, I was pleasantly surprised by how successful the multicultural group assignment was.  It achieved most of its desired outcomes and the benefits outweighed any of the problems that arose.  I will repeat this multicultural group assignment task, with modifications, in future semesters.

Literature research by students

I know others have this problem as I have read about it elsewhere but it does not stop if from being frustrating.  Why do students think that a literature search requires putting the first search terms they can think of into Google (not Google Scholar) and summarising the first ten hits?  I spend time in lectures explaining how to do a literature search and how to use search engines properly, and I send students to the library training courses on literature research yet it does not change their behaviour.

I will go back and amend my lecture notes and other teaching materials to improve the scaffolding to address the problems that arose this semester.  In particular, I will stress:

  • The importance of background reading before starting the literature search
  • Using reference lists from the background reading to find the first articles
  • Using the reference lists from the first articles to find other articles
  • Using the “cited by” link in Google Scholar to find more recent articles which cite the articles you have already found
  • Finally, use specialist research search engines (Google Scholar, Ebsco, Proquest, Jstor, etc) to find articles not found using other methods

Maybe this time it will have an impact on students.  However in a recent article by a Wendy Fleet (2013), she stated that when students prepare assignments they do what they have done in the past.  So, in addition, I have added an explanation that the standard of work expected increases as students progress through their degrees.

Fleet, W. (2013). Why do students choose not to follow all instructions when completing assessment tasks? Accounting Education: An International Journal, 22(3), 299-301. doi: 10.1080/09639284.2013.793919

Using the Research Literature

September 27, 2011 Leave a comment

As part of an assessment task in Business Risk Management, I ask my postgraduate students to find 6-10 peer-reviewed journal articles and incorporate them into their assignment.  Their assignment involves a case study of their choice with their analysis of how well their chosen organisation has handled the risk management issue that they have selected as the focus of their analysis.  Most students find that the hardest part of the assignment is finding the research literature.

Students tell me that they often have classes where they are expected to read an assigned research article but they do not really know how to select an appropriate article for their assignment  and when they do find articles, many students tend to simply summarise them rather than using the research literature to inform their thinking for their assignment.

Research articles can pertain to the methodology, the method or the particular organisation and its management of the chosen risk area.

Research can tell students how others have thought about:

  1. The theoretical lens through which a problem is viewed.  There are generally many ways to look at an issue, such as a feminist perspective or a Marxist perspective, as an example of monopoly power, or cultural hegemony … or whatever theoretical lens provides a way of approaching the problem.  This is the methodological approach to the assignment.
  2. The method used to tackle the problem.  Will students use a survey, interviews, observation, logic or some other method?  Undoubtedly there will be literature that provides guidance on that method.
  3. The actual problem at hand.  Maybe others have looked at the specific organisation, or for risk management students, the specific risk and perhaps how that risk has been handled in a similar context to the one being analysed.

I suggest to my students that brief do a brief concept map of their assignment outline and work out how the research literature sits within it.

It is important that students see the relevance of engaging with the research literature and that they do not see research as something you read at university but which has no real place in the business world.  In areas such as accounting, finance and business risk management, it is very easy for students to dismiss research in favour of the more technical or practical aspects of the discipline.

I feel that part of our role as educators is not simply to make sure students know how to use the library but to make sure they know why they should use the library.  We look at research from the perspective of researchers but most of our students will not become career researchers.  We have to be able to explain to our students not only why research matters while they are students but also, that it is has a role in their workplace.  It can be a challenge but unless we can explain why research in accounting is important to a practicing accountant, for instance, then we need to question the relevance of that aspect of our work.

 

Categories: Assessment, Research

Preparing students for answering exam questions

September 19, 2011 Leave a comment

I used to allow students to bring in one A4 page of summarised notes because I thought that it would increase their confidence by allowing them to write down key memory joggers, thus freeing them to focus in the exam room on demonstrating their understanding.

I was wrong. The A4 pages were crammed with microscopic print as students tried to cram a semester’s notes onto them. Exam answers looked like a competition: the one who writes the most facts in the least amount of space wins.

The problem was that my exam questions are not written to elicit a shopping list of facts. In exams, I want my students to show me that they can think about what they have learnt, critique it, then select and apply relevant information to answering a question.

Every year I would tell students that the one exam question I can guarantee they will not get is “Regurgitate everything you know about ….” (fill in a topic). Yet every year I continued to see exam answers to this question, irrespective of the actual questions set. And every year, students would ask me why they had done poorly in their exam “because I studied all the facts”.

I find that students who appear in classes to understand course content but whose exam performance is below the quality expected fall down in three main areas:

  1. Too much time is spent giving background information. Students then think they have written a lot so must have answered the question. This is the student who feels every term must be described at length … with an example … but only gives a passing nod to answering the set exam question.
  2. A key word in a question triggers an outpouring of every fact connected with that word which the student can recall. The actual exam question is largely ignored.
  3. Students write without planning their answer. Freely flowing writing is wonderful and cathartic in personal journals but can be hit and miss when used as a style for answering exam questions.

So I started giving my students “mini exams” with no prior warning. I would ask them to spend 5 to 10 minutes outlining an answer to a question using dot points or a mind map. They compared answers in groups of 2-3 students then we discussed their responses as a group. Students were then asked to give their answer a mark out of 10 and reflect on why they selected that mark. Initially, that was the part they found hardest until they realised, to quote one student, “it’s not just what I write … it’s what I don’t write that really counts”.

I also provide examples of “good” and “poor” answers I have prepared and ask my students to explain why one answer is better than another.

I find it takes about 2 weeks for my undergraduate students to become adept at critiquing an answer I have given them and 4-6 weeks to become adept at critiquing their own answers. Since introducing this approach, student writing both in assignments and exams has become more structured and focused. An unanticipated bonus is that many students tell me that in learning to write more focused answers, they have become more discriminating readers.

Seeing is believing – the value of thinking before writing

September 15, 2011 Leave a comment

I can tell my students to plan an exam answer before commencing to write the answer and I can even provide them with statistics to demonstrate that students who plan an answer tend to perform better in exams but I find that many students need to see the difference planning makes before they internalise my message.

Dividing a class and asking half to immediately start writing an answer to a “mini exam question” while the other half spend half the allotted time planning the answer and the other half of their time writing up their answer, discussing the correct answer then asking them to self-assess their answer always shocks about one-third of my students.  I have done this many times with many different groups of students and those who prepare the answer before writing average a score of 2 to 4 marks more out of 10 than those who start writing without any forethought.

Exams can constitute a substantial percentage towards a student’s overall grade yet most students only face exams at the end of a semester when they are so concerned with revising everything covered during a semester that they are unlikely to spend time on developing the skills that allow them to perform at their best in their exams.

Both learning how to plan an answer that is succinct, well-structured and addresses the question asked, and being able to critically reflect on and evaluate the quality of one’s own writing are important generic skills transferable both to the workplace and other areas of study.

Categories: Assessment, Essays, Exams Tags: