Assessment by Portfolio – Some Definitions & Academic Perspectives

A Traditional Definition of Portfolionoun (plural portfolios)

1 – a large, thin, flat case for loose sheets of paper such as drawings or maps: under his arm he carried a large portfolio of drawings

  • a set of pieces of creative work intended to demonstrate a person’s ability to a potential employer.
  • a varied set of photographs of a model or actor intended to be shown to a potential employer: entrants must have done professional modelling or have an existing portfolio.

 2 – a range of investments held by a person or organization:a portfolio of insured municipal securities.

  • a range of products or services offered by an organization: an unrivalled portfolio of quality brands.

 3 – [as modifier] denoting or engaged in an employment pattern which involves a succession of short-term contracts and part-time work, rather than the more traditional model of a single job for life: portfolio careers allow women to balance work with family.

4 – the position and duties of a Minister or Secretary of State: he took on the Foreign Affairs portfolio.

 

(Alt) Portfolio as a term may describe the Student’s Evidence of achieving given Learning Outcomes:

This should (according to our agreed taxonomy – HSAD 2013), project by project, module by module, evidence the “management, planning, monitoring and execution of an extensive and complex body of work, sustained over an extended period” by the student.

Portfolio Based Work may include: all art/design/creative artefacts; all 2D, 3D, traditional, digital, video, sound, or photography based or alternative media recordings of the same; including the recording of design processes, ideation and manufacture (screen capture, video etc.); all written assignment work, tutorial discussion, verbal and physical contributions to presentations and exhibition (both internal and external to the college); all reflective and development blog posts; all evidence of client working and facing methodology and all vocational experience (e-mails, letters); and all other methods of communicating the working practice of the individual.

This may be evidenced as a combination of: physical evidence – artifacts and hard copy imagery; digital record – blogs, digital workbooks, online portfolio sites, project management software records, email etc; record of discussion – written tutorial documentation, video/sound recording; and active participation* – verbal discussion, group working, presentaion of work and ideas, peer-to-peer teaching skills etc.

Assessment by Portfolio, particularly in Art and Design is a tried and tested method that if assessed by “experts” (i.e. people who have trained and have specialist subject knowledge) IS NOT wholly subjective, and DOES NOT have to be compromised by detailed checklists, flawed micro-compartmentalisation and in-flexible structuring.

See academic reinforcement below:

*The self-assessment process, [is] discussed heavily in arts education theory… as is oral communication by the students to peers and tutors…  all of this is part of our standard assessment process via group tutorial and discussion as well as more formal project presentation.

…appraisal based on arts talk (BLOGS/SELF ASSESSMENT VIA PORTFOLIO ON THESES BLOGS) will be holistic and qualitative rather than mechanistic or quantitative. The advantage we see in the use of conversational talk to explore and elicit aesthetic understanding is that it furnishes the appraisers (both teacher and pupil) with relevant, subjective yet public evidence upon which to base their judgements.

 – Malcolm Ross, School of Education, University of Exeter, England.

On Assessing Acheivement In The ArtsVague Assessment – or Experienced, Expert Objectivity?

Child and Siroto (1965), for instance, showed that, when American art experts’ judgements of Bakwele ceremonial masks were compared with choices by BaKwele judges, the least degree of agreement was found with tribesmen, most agreement with those who actually carved the masks, and an intermediate degree of agreement with the ceremonial leaders who used the masks. Similar results were found in other experiments, and it seems that, when competent judges of art are used, there is more cross-cultural agreement than is often supposed. Differences of material, style, subject matter, or intention of the works of art do not necessarily imply basic differences of aesthetic evaluation…

– (Pickford – ed, 1972, 1976) – Oxford Companion to the Mind

Portfolios are valuable largely because of the richness of the information they supply. They enable students to present documentation of their personal, authentic, educational experiences and experiences in real practice. Standardising those experiences would inevitably detract from the portfolio’s educational value.

E Driessen, et al 2005, MEDICAL EDUCATION Blackwell Publishing Ltd

So, can portfolio raters/assessors be trained regardless of specialist background?

Training of raters and shared rater experiences would improve interrater reliability, but studies on the objective structured clinical examination have warned against unrealistically high expectations of rater training, even with well defined instruments.

The lesson that detailed checklists can easily trivialise assessment has also been learned in other assessment.

Increasing the number of raters would be an effective strategy, were it not for practical constraints, such as the time-consuming nature of portfolio judgement.

In summary, portfolio assessment appears to be caught between the 2 classic evils of poor reliability and poor validity. This begs the question of how to achieve sufficient reliability of qualitative and subjective judgements for summative purposes without falling into the trap of corruption of portfolios for testing purposesCommon misconceptions pervading this discussion are that subjectivity equals unreliability and that objectivity equals reliability. This is not universally true.

Objective examinations may be unreliable (cf. a single-item, multiple-choice examination) and – more importantly – subjective judgements can be reliable provided an adequate number of different judgements are collected and collated.

∞ Norman GR, van der Vleuten CPM, De Graaff E. Pit- falls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ 1991;25:119–26.

a qualitative approach to portfolio assessment that can enhance reliability without taking recourse to large samples and rigid structuring.

…the feasibility of a qualitative approach to achieve reliable summative judgement using an inherently complex and non- standardised assessment instrument, which relies on holistic professional judgement.

So the student and the quality managers within the institution remain safe in the knowledge that:

after a careful assessment procedure a committee of experts had reached consensus on the final decision on a student’s portfolio.

The use of qualitative research criteria for portfolio assessment as an alternative to reliability evaluation: a case study – E Driessen, Et al

_________________________

This is a work in progress repository support Assessment by Portfolio. – last updated 26 March 2013

Advertisements

~ by hesir on March 21, 2013.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: