Posts Tagged ‘assessment’

Multicultural groups

December 6, 2013 1 comment

This semester, for the first time, I forced students to form multicultural groups for their major assignment: an essay on the usefulness of accounting rules.  The idea for this came from one of my colleagues who has championed this approach to group formation.  Our purpose in forcing students into multicultural groups was to improve their generic skills:  communication and teamwork skills in particular and the issues which arise concerning those skills when dealing with people from different cultures.  We consider this to be an important element of generic skill development for both domestic and international students.

multiculturalismThe Australian workplace, in general, and the accounting industry’s workplace, in particular, are multicultural environments.  Students who intend to work in the accounting industry in Australia need to be able to work effectively in that environment.  That includes communicating with and forming teams with colleagues from different cultures.  In addition, clients may come from a range of cultural backgrounds so accountants need to be able to communicate effectively with clients from cultural backgrounds different to their own.  Also, many accountants spend part of their career working overseas.  Exposure to cross-cultural communication issues while at university will help those graduates who will work overseas.  We believe that the University will fail to equip graduates properly for their experiences after graduation if our students do not know how to communicate and collaborate with people from cultural backgrounds different from their own.

The issues facing international students are different; it is more their current studies rather than their future career which will benefit from cross-cultural communication skills.  Many of our international students live in mono-cultural enclaves either in University residences or in shared accommodation off campus.  Within these enclaves they communicate using their home language, watch television and movies from their home country over the Internet, and communicate with family and friends back home using social media.  They have negligible interaction with people from other cultures and do not use English to converse except in classrooms. It is easy to understand why international students would choose to live within these mono-cultural enclaves; it is easier.  Dealing with people from other cultures and communicating in your second language is hard.  Living in these mono-cultural enclaves diminishes students’ experiences from studying in Australia, often leads to a deterioration in their English language skills, and limits their understanding of Australian educational styles.  Forcing students to work outside of these enclaves will help address these problems and help them with their studies at university.

International students who seek to remain in Australia after graduation get double benefit from being forced to work in multicultural groups.  They get the benefits that all international students get plus they get the benefit that domestic students get.

We also have anecdotal evidence of additional barriers to cross-cultural educational experiences that impinge on international students.  It appears that there is a social pressure within the mono-cultural enclaves for students to remain within the enclaves.  Students who voluntarily go beyond the enclave face ostracism.  Being compelled to go beyond the enclave prevents that ostracism.

Multicultural groups have come to be called “rainbow groups” at the University.  I am not comfortable with this label.  First, it suggests that skin colour or race is the reason for cultural differences.  This is not the case.  The wealth of literature on national culture clearly distinguishes the concept of culture from race.  Second, it suggests that culture and  ancestral origin are the same thing which they are not.  Immigrant countries like Australia and the United States provide ample evidence that ancestral origin and culture are different.  Third, the label “rainbow” and the rainbow image have been appropriated by the LGBT community and use of the term “rainbow groups” may suggest groups constructed on sexual orientation or identity rather than groups constructed on culture.  While the term “rainbow groups” is a convenient shorthand, I believe it is a term we should avoid and that we should use the more accurate but more clumsy terms of “cross-cultural groups” or my preferred term of “multicultural groups”.

Students were required to form their multicultural groups themselves.  The parameters they had to work within were that groups were to comprise three or four members and that no more than 50% of group members could be from the same culture.  As culture is difficult to measure, I used citizenship as a proxy with the ability for individual students to make a case for reclassification if they believed that citizenship did not properly represent their culture.  The example I gave for a case where reclassification might be appropriate was where a student had recently been awarded Australian citizenship but had lived for most of their life outside of Australia.  Where students held dual citizenship, I advised them to classify themselves based on the country in which they went to primary school.  If that did not resolve what country they were from then I would consider each case individually.  There were no cases which required individual consideration.

monopoly chance jailStudents who did not comply with the requirements were penalised 4 marks (out of 30) if they formed a mono-cultural group and 2 marks if they formed a multicultural group but with more than 50% of the group members coming from one country.  These penalties were waived if students could demonstrate that they had taken all reasonable steps to form a group that complied with the requirements but had been unable to do so.  Evidence of taking all reasonable steps included posts to the forum on the learning management system, email logs and mobile phone logs.  Penalties were also waived if the group disintegrated but it was not the fault of some of the group members.  An example of this was where a member withdrew from the unit shortly before the due date of the assignment.

Using citizenship as a proxy for culture does create some anomalies.  Students from Australia and New Zealand were treated as from different cultures even though cross-cultural research suggests there are few cultural differences between Aussies and Kiwis.  Conversely, students from north and south China were treated as coming from the same culture even though there is evidence of distinct cultural differences between northern and southern Chinese.  In my opinion the pragmatic advantages of using citizenship as a proxy for culture outweighed the damage from potential misclassification.

Some students objected to the requirement to form multicultural groups when the assignment requirements were released.  One student objected because she already worked in a multicultural workplace and she would not benefit from the multicultural group requirement.  This student had a valid point.  However, as with all assessment items, I design them for the average student; I do not customise assessment to meet the needs of individual students.  My reason for this is purely pragmatic, I do not have the resources to customise the assessment to meet the individual needs of every student.  Therefore, I did not modify the assignment requirements to address valid objections to the requirement.

students and globeIn addition to the main assignment requirement of a group submission of an essay on the usefulness of selected accounting rules, the assignment requirements included an individual submission of a 300 word reflection on cross-cultural communication.  This reflection was worth 3 marks and was graded solely on whether it showed reflection on what had been learned rather than mere description of the processes undertaken.  My main reason for requiring the reflective submission was to force students to think about cross-cultural communication issues as they related to their assignment.  It had a secondary benefit of telling me what cross-cultural issues were important to students.

The most common issue raised in the reflections was language.  Many students equated cross-cultural communication issues with language issues.  While there are many more issues with cross-cultural communication than just language, if language was a problem it tended to swamp any other problems there may have been and it is understandable that in a 300 word reflection that students would only address language difficulties.

Google-TranslateStrategies to deal with language issues varied among participants.  Many English speakers described how they found it necessary to slow their speaking and to avoid the use of slang and idiomatic expressions.  Speakers of other languages described how they had to ask English speakers to repeat things and often expressed surprise that the English speakers were willing to accommodate their weaker English skills.  Many groups described using online translation services such as Google translate to facilitate understanding.  Some groups addressed the difficulties of face-to-face communication by using written communication methods only.

Other cross-cultural communication issues that were raised in the reflections included punctuality for meetings, lack of commitment to the task, unreliability of group members, failure to understand the assignment task, and failure to apply the referencing and plagiarism standards of the university.  One concern I have about the issues raised is that students may be labeling individual traits as cultural traits.  For example, I have not, in my years of experience, been able to identify any cultural factor in students propensity to incorrectly reference or to plagiarise.  I believe that referencing failures and plagiarism are personal traits rather than cultural traits.  It is possible that the other perceived issues were problems with the person, not problems flowing from the culture.  I am not sure how to detect if an issue is personal or cultural which means I can’t expect my students to be able to do this either.

There were some problems with the multicultural group component of the assignment that need to be addressed in future semesters.  These include:

  1. Some students did not formalise the construction of their groups.  They had no written agreement and no agreed upon plan established.  Some of these poorly-organised groups fell apart.  To minimise the risk of this happening again I will recommend that groups sign a formal group contract.  I will post a pro-forma contract on the LMS which students can use or adapt.
  2. Groups were unable or unwilling to discipline members of the group who did not contribute.  To address this, the pro-forma group contract will include default performance standards and disciplinary measures.
  3. The largest cultural grouping in the unit was Chinese.  The Chinese were somewhat disadvantaged as embryonic groups were usually looking for a non-Chinese person to complete the group.  To address this in the future I will present the cultural statistics for the unit in lectures and advise students that members of the most populous national groups need to pro-active in forming groups or they will be in the same position that some of this semester’s groups found themselves when they discovered that they were in breach of the assignment requirements and there were no non-Chinese students left who did not belong to a group.

Overall, I was pleasantly surprised by how successful the multicultural group assignment was.  It achieved most of its desired outcomes and the benefits outweighed any of the problems that arose.  I will repeat this multicultural group assignment task, with modifications, in future semesters.


Literature research by students

I know others have this problem as I have read about it elsewhere but it does not stop if from being frustrating.  Why do students think that a literature search requires putting the first search terms they can think of into Google (not Google Scholar) and summarising the first ten hits?  I spend time in lectures explaining how to do a literature search and how to use search engines properly, and I send students to the library training courses on literature research yet it does not change their behaviour.

I will go back and amend my lecture notes and other teaching materials to improve the scaffolding to address the problems that arose this semester.  In particular, I will stress:

  • The importance of background reading before starting the literature search
  • Using reference lists from the background reading to find the first articles
  • Using the reference lists from the first articles to find other articles
  • Using the “cited by” link in Google Scholar to find more recent articles which cite the articles you have already found
  • Finally, use specialist research search engines (Google Scholar, Ebsco, Proquest, Jstor, etc) to find articles not found using other methods

Maybe this time it will have an impact on students.  However in a recent article by a Wendy Fleet (2013), she stated that when students prepare assignments they do what they have done in the past.  So, in addition, I have added an explanation that the standard of work expected increases as students progress through their degrees.

Fleet, W. (2013). Why do students choose not to follow all instructions when completing assessment tasks? Accounting Education: An International Journal, 22(3), 299-301. doi: 10.1080/09639284.2013.793919

Writing Wikipedia

August 15, 2012 Leave a comment

Last semester I had my third year undergraduate students in the advanced financial reporting unit undertake an assignment on Wikipedia.  They were required to choose a Wikipedia article associated with financial reporting and:

  1. Evaluate the article using Wikipedia’s own criteria
  2. Write an improved article on the topic
  3. Explain why their improved article was superior to the original using Wikipedia’s own criteria.

My rationale behind the assignment task was that students would use Wikipedia despite exhortations from academics never to use it.  I believe that our role as educators is to ensure that students understand the inherent strengths and weaknesses of Wikipedia and other encyclopaedia and are able to judge the quality of a Wikipedia article.  With this knowledge students will be less likely to rely on Wikipedia when it is not appropriate to do so.  Banning students from using Wikipedia is doing them a disservice because they don’t learn how to use it properly.

It is also a disservice to students to ban the use of Wikipedia as we are encouraging students to behave dishonestly.  When we design assessment tasks we should avoid putting temptation in the path of students.  This, however, is a different topic to the one I wish to consider in this post.

I reviewed many of the Wikipedia articles associated with financial reporting before setting the assignment.  The articles I reviewed were all terrible.  I felt confident that there was more than enough material for my students to examine.

The first part of the assignment (evaluate an existing article) was done reasonably well.  The only one of Wikipedia’s criteria that students struggled with was the “notability” criterion and this difficulty is somewhat understandable as Wikipedia’s definition of “notable” is geared towards articles on individuals and historical events.

The second part of the assignment was done poorly (writing an improved article).  While students could demonstrate that they had the necessary technical knowledge to write an article, they could not write a good article.  The most common problem was that they could not imagine what information a Wikipedia reader would require on the topic they chose.  Most students wrote it from their perspective – what information an accounting student would need – and, hence, tended to write it as if it was a textbook.  This cost them marks as Wikipedia states clearly in its criteria that it is an encyclopaedia, not a textbook.  In addition, students could not switch out of jargon and into plain English.  Too often I read something like I should “debit accounts receivable” instead of being told that “the amount of money owed to the firm by its customers had increased”.

Another problem that appeared in some of the assignments was the quality of the references used to support the articles the students had written.  Wikipedia states that its preferred references are academic journals or university textbooks.  Very few assignments referenced that material.  When students referenced accounting standards they tended to reference Australian accounting standards.  Australia comprises less than 1% of the world’s population and less than 2% of the population of English speakers in the world – references unique to Australia are not relevant to most Wikipedia readers when the pronouncements of the International Accounting Standards Board could have been used to tell the same story.

Part 3 of the assignment (critiquing their revised article) was also done poorly.  While students showed they could critique others’ work, they could not do the same to their own.  I don’t know whether this comes from a misplaced arrogance or from an inability to see flaws in their own work.  Either way, it is a concern.

My overall impression of this assignment is that achieved many of the things I hoped it would achieve.  I will use this basic structure again but I will improve the scaffolding information I provide rather than leaving it for students to discover through their own research.

Preparing students for answering exam questions

September 19, 2011 Leave a comment

I used to allow students to bring in one A4 page of summarised notes because I thought that it would increase their confidence by allowing them to write down key memory joggers, thus freeing them to focus in the exam room on demonstrating their understanding.

I was wrong. The A4 pages were crammed with microscopic print as students tried to cram a semester’s notes onto them. Exam answers looked like a competition: the one who writes the most facts in the least amount of space wins.

The problem was that my exam questions are not written to elicit a shopping list of facts. In exams, I want my students to show me that they can think about what they have learnt, critique it, then select and apply relevant information to answering a question.

Every year I would tell students that the one exam question I can guarantee they will not get is “Regurgitate everything you know about ….” (fill in a topic). Yet every year I continued to see exam answers to this question, irrespective of the actual questions set. And every year, students would ask me why they had done poorly in their exam “because I studied all the facts”.

I find that students who appear in classes to understand course content but whose exam performance is below the quality expected fall down in three main areas:

  1. Too much time is spent giving background information. Students then think they have written a lot so must have answered the question. This is the student who feels every term must be described at length … with an example … but only gives a passing nod to answering the set exam question.
  2. A key word in a question triggers an outpouring of every fact connected with that word which the student can recall. The actual exam question is largely ignored.
  3. Students write without planning their answer. Freely flowing writing is wonderful and cathartic in personal journals but can be hit and miss when used as a style for answering exam questions.

So I started giving my students “mini exams” with no prior warning. I would ask them to spend 5 to 10 minutes outlining an answer to a question using dot points or a mind map. They compared answers in groups of 2-3 students then we discussed their responses as a group. Students were then asked to give their answer a mark out of 10 and reflect on why they selected that mark. Initially, that was the part they found hardest until they realised, to quote one student, “it’s not just what I write … it’s what I don’t write that really counts”.

I also provide examples of “good” and “poor” answers I have prepared and ask my students to explain why one answer is better than another.

I find it takes about 2 weeks for my undergraduate students to become adept at critiquing an answer I have given them and 4-6 weeks to become adept at critiquing their own answers. Since introducing this approach, student writing both in assignments and exams has become more structured and focused. An unanticipated bonus is that many students tell me that in learning to write more focused answers, they have become more discriminating readers.