Australasian Journal of Educational Technology
2010, 26(5), 607-625.

Secondary students' online use and creation of knowledge: Refocusing priorities for quality assessment and learning

Kay Kimber and Claire Wyatt-Smith
Griffith University

In this increasingly convergent and digital world, young people are reportedly using new media with high engagement outside school, yet disengaged in those schools where technology access is low or restricted. Such an apparent disconnection is magnified when predictions of their futures are tied to requisites including technological expertise, adaptability to change, innovative capacities and complex problem-solving abilities. Such future oriented capacities challenge traditional views that basic literate and numerate proficiency is sufficient for academic success. They also raise questions about the sufficiency of digital engagement for developing higher order critical and creative skills. Collectively, these future oriented capacities heighten educational imperatives for improving the quality of young people's learning outcomes in this rapidly changing online world. This article addresses these issues. It draws on diverse literature sources and an Australian research study (2003-2008) into secondary students' curricular digital literacies (Appendix A) to present conceptual advances in understandings about how to recognise, talk about and value signs of quality learning in student-created multimodal products. Finally, the article offers an assessment framework with potential for assisting students and teachers to access core concepts and mobilise those essential capacities for enhancing performance when using and creating knowledge online.


Education, political and business sectors currently share a synergy of expectations of schooling for the 21st century. All acknowledge the fundamental difference between the educational needs of today's young people and those of earlier generations, as future workforces inevitably become even more technologised than at present. As consequence, educators face competing demands along several continua of learning needs: from basic literate and numerate proficiency to higher order thinking and problem solving; from strong disciplinary knowledge bases to interdisciplinary perspectives and innovations; from individual capability to collaborative teamwork; and, from technological proficiency to more creative and critical technology usage. As outlined here, educators are charged with preparing young people for life in a world where survival amid unimaginable change will require far more than basic literate and numerate proficiency. Rather, accessing and selecting information sources, discerning authorship, and assembling and communicating knowledge with digital technologies will assume critical importance for personal, academic and workplace success.

With the daily escalation of Internet usage, issues of (a) information access, (b) source authentication and (c) global connectedness compound the challenge for educators in meeting 21st century learning needs. Fisch, McLeod and Bronman (2006, 2008), for instance, have estimated the explosion of unique new information items generated across the world, via the Internet, to have already eclipsed that of the previous 5,000 years in just two years. So being able to locate, access and select the most relevant of that information for specific knowledge creation purposes immediately calls into question the user's ability to operate strategically in that digital environment. As revealed in a range of emergent research, writing and reading in the digital age can be more complex and involved than in print (see, for example, Coiro, Knobel, Lankshear & Leu, 2008; Leu, Kinzer, Coiro & Cammack, 2004). Other researchers cite source authentication as the most challenging issue for Internet users, whether adult or child. At the heart of source authentication are notions of credibility (Metzger & Flanagin, 2008) and believability (Lankes, 2008), as "one chilling reality in this brave new digital epoch is the blurring, obfuscation, and even disappearance of truth" (Keen, 2008, p. 16). Hence, for informed, discerning source authentication, constant vigilance and evaluative practices are crucial online knowledge-seeking behaviours. Other research projects involving young people and their technology usage (see Appendix A; Buckingham, 2007; Cheong & Cheung, 2008; Levy & Kimber, 2009; MCEETYA, 2007) have recommended users' more critical engagement with online texts and new media.

Another priority gaining global support for 21st century learning and workforces is creativity in terms of innovation, complex problem solving and design (Economic Review Committee, 2002; McWilliam & Haukka, 2008; Robinson, 2006; Robinson et al., 1999). A creativity perspective foregrounds the highly visualised, screen based practices that characterise online activity, particularly how knowledge is represented, understood and shared. For example, in multimodal texts, the knowledge meaning is constituted in the author's combination of verbal, visual, auditory and/or kinaesthetic modes and the viewer's decoding of the complex mix. While multimodal texts are easily created, published and shared via new technologies, this is no guarantee of high quality, credible content, ethical representation or aesthetic digital design. Even the ease of dissemination raises matters of responsible and ethical behaviours, given the expansive Internet connections. Here, the Internet's networked potential invites educators to generate tasks and activities that promote cross-disciplinary perspectives or investigations, yet also foster awareness of global issues and responsible citizenship (MCEETYA, 2008; Partnerships for 21st Century Skills, 2009). Closely allied to these relational type uses of the Internet are promotions of empathy (Gorry, 2009) and ethical usage (Jenkins, 2006; Kimber & Wyatt-Smith, 2009). From all these perspectives, there would appear to be an urgent need for educators to assist learners to become more digitally proficient, critical evaluators and creative, ethical producers of knowledge products using new technologies. This all rests on the assumption that educators have augmented their print legacy perspectives on learning and assessment with understandings of the dynamic inter-relatedness of words, images, sound and movement in multimodal text production, and the nature of the online world.

In what follows, this article is structured in two parts to address the key question: How do we recognise, talk about and value signs of quality learning in student-created multimodal texts? The first, Towards critical engagement, presents a range of views on the place of engagement, creativity and critical thinking in contemporary online learning environments. The second, Lenses on quality learning in multimodal texts, concerns how signs of quality in student-created multimodal texts can be talked about through two different lenses developed for, and from, a diverse literature field and the longitudinal digital literacies study outlined in Appendix A. These lenses represent conceptual advances in understanding about the demands of working online and culminate in the proposal of an assessment framework that details core concepts and principles with potential for informing 21st century learning.

Part 1: Towards critical engagement

As introduced in the previous section, the complexity of learning needs for networked, digital worlds requires more than basic literacy and numeracy. Higher order thinking is more desirable for attaining quality learning in these spheres, and is evident in evaluative practice and creativity, as will be explained in this section. This section first argues that young people's engagement with technology, whilst widely reported, is not sufficient to guarantee critical thought about online sources, interactions or actions. Discussion then turns to the primacy of creativity on so many agendas for 21st century learning and as a means to move towards more critical engagement with new technologies.


The term "engagement" is typically used in educational discourse in terms of attracting and sustaining student attention, the appeal of new technologies, and as precursor to learning. Various studies have investigated the extent of young people's engagement with new media (Ito, Horst, Bittanti, et al., 2008; Jenkins, 2006; Lenhart, Madden, Rankin Macgill & Smith, 2007) in terms of intensity of new media activity. McGonigal (2008) reported on online game players' sustained concentration and energetic participation as they deciphered clues and formulated often collaborative solutions to the initial problem. Similar levels of engagement are evident in young people's involvement in specialised online communities like art, music and writing (Ito et al., 2008; Jenkins, 2006) where a variety of skills is honed by peers and community experts. Here, engagement is synonymous with interest-driven, often self-directed, informal learning. This prompts consideration of the nature of young people's engagement in school and out of school online activities.

Researchers investigating young people's screen based literacy practices have noted a dissonance between their "engagement" at home and at school (see, for example, Buckingham, 2008; Ito, et al., 2008; Jenkins, 2006). In Equipping every learner for the 21st century: A white paper (CISCO, 2007), classrooms are labelled as "the only place where learners disconnect" (p. 5), inferring that disconnection is akin to lack of engagement or attentiveness to studies. Yet "engagement" with out of school screen based activities is not sufficient of itself to guarantee strong personal capacities for online meaning-making - whether at home or at school. Even when young people are frequent users of new technologies and teachers have incorporated activities like online discussion forums (Cheong & Cheung, 2008), there has been minimal evidence that much higher order thinking had been exercised. In research with first year university students, Kennedy, Judd, Churchward, Gray and Krause (2008) noted little uniformity in technology experience across the cohort, identifying instead a "digital divide" (p. 117). These researchers posited that widespread, off campus, everyday technology usage did not automatically transfer to more meaningful use of technology for learning. With this no doubt still being the case with secondary students, the challenge for educators extends from capturing and building upon their students' levels of engagement within classrooms, to promoting a raft of robust skills that might withstand technological change and generate creative solutions to complex problems.


Creativity has emerged as a new priority for schooling, flowing from the contemporary working environment where it is claimed as a critical workforce capacity (Fleming, 2008; McWilliam & Haukka, 2008; Robinson, 2006) across the arts, sciences and society in general. From All our futures: Creativity, culture and education report (Robinson, et al., 1999), creativity has been valorised across curriculum areas and precipitated a significant financial commitment to developing creativity in British children. Many countries have taken up the creativity challenge, including Singapore (Economic Review Committee, 2002), Australia (Prime Minister's Science, Engineering and Innovation Council, 2005) and Canada (The Conference Board of Canada, 2008) where schools are expected to build the creative capacities of their students and hence, the innovative capacities of economies. In terms of innovation, creativity means inventiveness and high levels of ingenuity, as creativity entails higher order thinking and the ability to relate unexpected juxtapositions of information or concepts. Such a breadth of creative possibilities is as core to schooling as it is to modern working environments.

For creativity advocates like McWilliam & Haukka (2008), the building of young people's creative capacities should be additional to their basic literacies, as creativity is the value-adding component to an individual's capabilities and the economy more generally. Here, creativity in learning is equated with high level processing of thought, ideas and information into some innovative transformation. Interestingly, "create" has superseded "analyse" as the most complex knowledge and cognitive process in the revised Bloom's Taxonomy (Anderson & Krathwohl, 2001) and was defined as "putting elements together to form a novel, coherent whole or make an original product", with three associated skills of "generating, planning and producing" (Krathwohl, 2002, p. 215). These views support the value of creativity in classroom lesson design and expectations of student performance. They also indicate the potential for students to enhance their understanding of how the creation of multimodal texts, whether their own or others, can be considered from a critical perspective. If so, it would be feasible to assume that mere engagement with online activities could be productively extended towards more critical engagement and creativity.

From creativity to critical engagement

One positive step towards unleashing creativity and critical engagement could be that of "unlearning" (Becker, 2006; McWilliam, 2005) as it involves challenging accepted ways of doing something and opening the mind to other possibilities. With "unlearning", teachers and students alike could find new approaches to technology mediated learning, fresh views on assessment, and possibilities for co-creation of knowledge. This is important if we are to move beyond print bound ways of thinking about how knowledge is used, created, shared and assessed.

Two further steps towards critical engagement involve students having two additional sets of knowledge, beyond their subject disciplines. The first is "curriculum literacies" (Cumming & Wyatt-Smith, 2001; Wyatt-Smith & Cumming, 2003), defined as those literate capabilities needed to learn in the curriculum, like how to access and use meaning systems in using and producing knowledge. These researchers found that to achieve academic success, students needed to understand not just curriculum requirements, but also the literacy demands of their subject areas. The second is "criterial knowledge" (Wyatt-Smith, 2001) or the explicitly articulated elements that constitute quality in a piece of work. According to Sadler (1989), when learners can identify the specific dimensions of criteria and engage with evidence (or lack thereof) in their own and others' work, criteria can play a role in self monitoring and improvement. With regular practice and constructive feedback, the learner is supported in developing expertise over time, an important aspect of the assessment for learning literature (Assessment Reform Group, 2002; Black & Wiliam, 1989; Gibbs & Simpson, 2004; Wyatt-Smith & Cumming, 2003), although no specific mention is made of how learners might think about and use new technologies, or improve the quality of their digital work. Criterial knowledge, when specifically linked to current (and future) uses of technologies, offers potential for enriching the quality of young people's learning.

In summary, the intent in this article is to rethink how quality in 21st century learning can be talked about, with particular focus on the multimodal products that young people create for school purposes. Core to this discussion is the premise that students' creative and critical capacities for the online use, creation and sharing of knowledge can be nurtured and extended. Given the heightened awareness of the complexity of online worlds, believability issues, the creativity priorities and need for critical engagement, then attending to how classroom teachers can nurture and extend these related online learning capacities in their students is warranted, especially their capacity for critical thinking about their actions and choices, whether in reading or online posting. These are explored in the following section.

Part 2: Lenses on quality learning in multimodal texts

This section is introduced with an overview of the digital literacies research study (Appendix A). Next follows two different lenses for thinking about what counts as quality in multimodal texts. These lenses represent the evolution of the researchers' thinking about quality, from a print centred yet emergent, digital consciousness (2004) to one more attuned to the digital challenges facing young people (2009). The reader is subsequently invited to consider the potential of an assessment framework specifically developed to tease out the desirable ways of using and creating knowledge online. Further, the reader is invited to consider this framework as a tool for nurturing and extending young people's 21st century capacities, given their borderless learning spaces and their need to exercise critical perspectives and creative actions, both in and out of school.

Overview of the study

The purpose of the research study (2003-2008), Using and creating knowledge in the high school years: Performance, production, process and value-adding in electronic curricular literacy, was to obtain a 'point in time' capture of secondary schools' students digital capabilities in completing a curricular-related online task involving online research and the creation of a multimodal text. It focused on how and how well secondary school students (a) used information and communication technologies to search for and read online texts, and (b) created and communicated new knowledge in "new" multimodal texts they generated in the absence of prior instruction. While students were supported to do the tasks insofar as sample websites were built into task design, there was no expectation that teachers would "teach" how to "do" the task. The intent was to track students' progress from 2004 to 2006.

Discussion in this section is confined to the way that notions of quality in the multimodal texts were identified and talked about, where "quality" referred to the level of learning and aesthetic design evident in the student digital creation. The evaluative criteria and standards for assessing quality in these student creations were developed and adapted through collaborative discussions with a teacher advisory group. Further details about the study are available in Appendix A and on the study website (see

Lens 1: Emergent digital consciousness

In first considering how to evaluate the quality of the 2004 student-created multimodal texts, the research team drew on Sadler's (1985) seminal work on assessment criteria and standards. Sadler argued that stated performance standards help to clarify and communicate expected features of quality, and informed the process of making judgments about the quality of work. In these ways, "a value claim is made easier to establish" (p. 289). The research team identified those features that could assist in talking about and determining quality in multimodal texts as e-proficiency, cohesion, content and design (Wyatt-Smith & Kimber, 2005). E-proficiency at that time was defined as basic technological operation and online activity, including use of software and various media: "the capabilities and repertoires of practice that students exercise in online environments, often on a daily basis" (Kimber & Wyatt-Smith, 2008, p. 335). In this framing, the notion of "e-credibility difficulties" (Haas & Wearden, 2003, p. 169) was raised as important for determining "qualities of trustworthiness, accuracy, completeness and timeliness" (p. 170). E-proficiency was considered to be (a) foundational in underpinning each of the other criteria and (b) reflected in the overall design of the texts students generated. It was subsumed into the other three criteria, ultimately shaped and developed into the Evaluative Criteria and Standards for Online Multimodal Texts, viewable on the study website and broadly explained below.

Cohesion was defined as "unifying the structure, representation, organisation of ideas, links" (Wyatt-Smith & Kimber, 2005, p. 28), acknowledging the potential of interactive links to give structure, explanation, and contrasting points of view. From this perspective, the integration and mobilisation of colours, images, language choices and movement via the affordances of the software all contributed to the effectiveness of the student's multimodal design in engaging the audience and representing knowledge. Content concerned the quality of the selection and organisation of the research information - the effectiveness of the students' ability to locate, use and create new knowledge online that went beyond cutting and pasting. Consideration was given to the thoughtfulness of resource usage, as well as the framing and structuring of information. The standards devised for Design, or "creating an aesthetic, artful design" (Wyatt-Smith & Kimber, 2005, p. 28), unpacked characteristics of quality at different levels, and focused in particular on how the linguistic, visual and technological choices were managed to create the multimodal text. Table 1 presents a summary of these descriptors for each criterion.

It is worth emphasising two main points, however. First, throughout the process of formulating, trialling and finally applying the assessment criteria and standards, they were taken to be provisional (that is, not fixed). This stance recognised that students might present "surprises" in their creations which could well call forth additional, previously unspecified criteria. For this reason, the rubric of criteria and standards had a space for what was referred to as the "X factor", recognising that assessors could take account of and reward innovation in the features of the actual work that went beyond or differed from the pre-set criteria.

Second, in the process of applying the criteria to a sample of student products, a necessary and new concept emerged, transmodal operation (Wyatt-Smith & Kimber, 2005, p. 31). This term was intended to capture the dynamic involved in crossing among the visual, verbal and kinaesthetic modes of representation, as well as different software applications, as the students negotiated and constructed their digital representation of knowledge. Essentially, the concept served to describe how the nine performance features presented in Table 1 intermingled holistically.

Table 1: Criteria for evaluating student-created multimodal texts

Criteria for evaluating student-created multimodal texts
Cohesion - Overall cohesion
  • Designing multimodally to engage audience and facilitate meaning-making
  • Cohesion of ideas within the text
  • Linking - technical proficiency
Content - What was said
  • Quality of information
  • Justification of solution
  • Sequencing and organisation of information within a node
Design - How it was said
  • Managing written language features
  • Managing visual and spatial elements of written text
  • Managing graphics and other web/screen elements

In determining the quality of each multimodal text, based on the above criteria, the researchers developed descriptors on a four-point scale: Outstanding performance; Accomplished; Developing; and Limited. To account for the number of incomplete task submissions, a fifth point was added: Lack of evidence of performance. Proficiency level was determined as midpoint in the scale (2.5) or the boundary between Developing and Accomplished. The Evaluative Criteria and Standards for Online Multimodal Texts can be viewed on the study website. Evaluator training sessions provided opportunities for the six researchers to judge the quality of exemplars using the above criteria and to calibrate judgments to achieve inter-rater consistency. All student-created multimodal texts (620 in 2004; 221 in 2006) were evaluated independently after the training. Validation checking exercises were undertaken throughout the evaluation phase to identify anomalies in judgment. Where anomalies occurred, individual student samples were re-evaluated to finalise grading.

Evaluations revealed some interesting patterns. First, Outstanding performances were minimal. Nearly 38 per cent of the cohort attained the proficiency level in 2004, and 25 per cent in 2006. The majority of the cohort scored below the proficiency level (62 per cent in 2004; 75 per cent in 2006). Second, Accomplished performers scored either higher Content than Design, or similarly high in both. Developing or Limited performances scored higher Design than Content. These findings suggested that effective transmodal operation (i) tended to be associated with Accomplished performances rather than with Developing or Limited performances, and (ii) was reflected in a seeming balance between Design and Content. Overall, these results tended to indicate that across the two year period, even though new technologies had become more ubiquitous, this cohort of young people had not demonstrated high levels of critical reflection, creative design or transmodal facility with this school-like curricular, multimodal knowledge creation.

These findings resonated with a 2005 Australian national study to determine the levels of technological proficiency of Years 6 and 10 students (MCEETYA, 2007) in relation to a six-level literacy scale and proficiency standards developed in consultation with teachers and IT experts in all states. Proficiency was determined as the boundary between levels 3 and 4 for Years 10, and between levels 2 and 3 for Year 6 students. It was found that 61 per cent of the Year 10 sample and 49 per cent of the Year 6 (i) attained their proficiency level and (ii) were using technology in limited ways. In other research, Buckingham (2007) noted that banality and superficiality rather than "spectacular forms of innovation or creativity" (p. 92) characterised much of young British and American people's everyday technological usage. In research with Singaporean thirteen and fourteen year old students using an online discussion forum, Cheong and Cheung (2008) found that only 57 per cent of the cohort contributed comments that indicated higher level information processing or their defence of a position. While not extensive, these findings seemingly challenge notions of young people as techno-savvy or discriminating users of new technologies, where high level thought and academic integrity are expected. They point towards the need for more systematic approaches to pedagogy and assessment to increase critical and creative usage. They also point to classroom use of software packages predominantly like PowerPoint, Inspiration or Photostory, rather than re-purposing content for an online audience using more recently developed Web 2.0 tools. Such limitations could perhaps be traced back to the demands on school budgets, although the advent of cloud computing opens opportunities for schools to expand creative possibilities without taxing their own infrastructure. Curriculum and task design, however, still require close attention if more complex thought processes are to be cultivated.

When the important notions of creativity and critical engagement as discussed in the previous section are considered in relation to the above findings, several key factors gain salience. First, any consideration of how young people connect, communicate, collaborate and create in actual and virtual locations must address the quality and manner of their activity. This is reflected in the individual's capabilities in that environment, in school and at home. Classroom teachers are well placed to be the arbiters of quality, negotiators of learning spaces, orchestrators of local and global connectedness, and supporters of young people in their acquisition of those desired capacities to the point of autonomous use out of school. This could well involve shared negotiation and explicit articulation of task, criteria and standards between teacher and students (Kimber & Wyatt-Smith, 2009).

Second, teaching for creativity warrants a higher priority in classrooms: "If we cannot 'transmit' creativity, we can certainly teach for creativity" (McWilliam & Haukka, 2008, p. 654, emphasis in original). Adlington and Hansford (2008) argued that despite young people's seeming proficiency with digital technologies, certain aspects of multimodal designs, especially of more subtle design elements, warrant careful scrutiny, explicit teaching and further research. Teaching for creativity means that whatever the subject area, teachers are challenged to design innovative approaches to curriculum delivery and find ways to foster those skills in individuals and collaborative teams. With creative problem solving encouraged amongst students, and the opportunity to find team solutions, young people are not just engaged in the activity but also stimulated to find innovative solutions. This will require young people to operate with insight, at any time. Hence critical thinking and informed action need to shape their online decisions as much as ethical and empathetic decision-making and inter-cultural consideration. This suggests an urgent need to shift the goal for learners to become more digitally proficient, critical evaluators, creative producers and ethical, empathetic users of new technologies.

From this platform, the second lens for talking about quality will be considered in the next section. This lens specifically targets those elements identified for more effective online use, creation and sharing of knowledge that will elevate the quality of students' thinking, evaluative practice and ethical actions.

Lens 2: Attuning learning and assessment to digital worlds

This section begins with a reflection on the criteria presented earlier in Table 1 as primarily print dominant in perspective. That is, while attention was given to multimodality in terms of transmodal operation, or how the student operated across visual, verbal and even kinaesthetic modes to create meaning, the focus in Content related purely to the information gathered and presented as evidence of learning. The major shift between the 2004 and 2006 criteria in this regard concerned the different task focus, from providing a solution to a problem (2004) to reconciling different viewpoints on an issue (2006). While the Design section focused on visual display, from aesthetic colours to spatial layout and even movement, the first element concerned linguistic accuracy. Further, Design was placed fourth on the criteria list, suggesting lower hierarchical value, and e-proficiency was not accorded any specific criteria in its own right. Finally, language choices focused primarily on actions like "managing", at a basic, operational level; "justifying" indicated the only expectation of higher level processing.

All of these points reveal the print-influenced perspectives of the researchers and teacher advisory group. Clearly, with teaching priorities aligned more closely to digital curricular literacies and assessment for learning practices that prioritise learner capacities as outlined, student performance could be significantly enhanced. Hence, a second lens for considering and talking about quality in student-created multimodal texts was developed by the researchers. This lens was a significant outcome of study, as we reflected on the nature of demonstrated quality in how students worked online. In conceptualising the indicators of quality, we were challenged to address the interconnectivity of task design, student learning activity and associated assessment. Our interest was in progressing discussion about expectations of quality in how students work online, and more specifically, how they generate new knowledge. This required that we articulate provisional indicators of quality and confront the demands of online environments where effective transmodal operation has become essential. This issue of what counts as quality is pressing on educators, given the changing possibilities afforded by Web 2.0 technologies.

This second lens takes as its starting point the premise that creative thinking, design and critical engagement are (a) essential for this century, (b) complex challenges, and (c) teachable. These rest on acknowledgment of the complexities involved in reading the Internet, locating and retrieving information (Leu, et al., 2004), and that ideally, sites should be mined to address issues of credibility, reliability and even ideology. From this basis, this section builds on the concepts of transmodal operation and e-proficiency, but reframes the e-proficiency, content and design criteria with the new concepts of e-credibility and e-designing as online practices that could be improved through explicit intervention. Further, in working towards building young people's more critical engagement with, and greater discrimination in, their learning and online actions, the concept of metalearning, or metacognitive reflection on actions/decisions as they occur, is proposed as the pinnacle towards which students and teachers can aim (Kimber & Wyatt-Smith, 2009). In this reframing, the learner would exercise evaluative practices in making informed decisions along the way and operate with transmodal facility or the fine-tuned ability to work with and across source texts, technology platforms and modes of representations to create new digital texts, and where critical thinking about content and concepts would be balanced with the aesthetics of design (Wyatt-Smith & Kimber, 2010). A fuller discussion of many of these ideas is available in Kimber and Wyatt-Smith (2009), but a summary follows.

Firstly, it must be recognised that using, creating and sharing online requires technology, the Internet and particular software, and their separate needs require specific skills and strategies, many of which are far removed from print based ways of reading and communication. The blurring of boundaries between them is as much a defining characteristic as the speed with which the actions can occur and connections made. For the purposes of this explanation, they are separated into two strands: (i) using existing knowledge, texts or materials; and (ii) creating and sharing new knowledge, texts and materials. Secondly, learner agency will be instrumental as the learner needs to operate with transmodal facility across various platforms, modes and activities, and exercise evaluative practices metacognitively, if a quality outcome is to be achieved.

Through this second lens, the concept of e-proficiency is taken to extend beyond basic technological competencies to more critical and applied usage. For example, being net-savvy might begin with the ability to search for and locate relevant information on the Internet, but being e-proficient will ensure that the user knows and can select from a variety of search engines and data bases to suit different purposes and contexts, rather than automatic selection of one favoured search engine. As well, the e-proficient user will have more advanced working knowledge of a range of software protocols and fine functions. From this perspective, an accomplished user has a heightened knowingness about a wider choice of options from which to create a quality digital knowledge product and in understanding how others' digital texts have been created. All these skills enable production as distinct from consumption of digital products and are foundational to any creative design possibilities using digital media. In these ways, e-proficiency can extend the learner's digital capabilities towards more purposeful, critical and ethical use and production of knowledge in online environments. With e-proficiency as a digital learning goal, teachers can assist their students to improve the quality of their knowledge use and production by digital means.

The concept of e-credibility assumes critical importance through the invisibility of the Internet and the need for constant credibility and trustworthiness checks. This involves being able to accept or reject indicators of reputed expertise at times and places where informed corroboration may be difficult to ascertain. For example, many young people seek instant corroboration from their networked friends (Flanagin & Metzger, 2008), rather than informed "experts". With so much erroneous and misleading information a keystroke away, young people need to be able to apply discriminating evaluations for themselves, so educators need to play a vital role in this area with explicit instruction and scaffolding (Cheong & Cheung, 2008). With careful evaluation to inform their courses of action, young people will be able to make their own, independent, more discriminating selection of sources, with corroborating evidence and accommodation of different viewpoints.

E-credibility is also significant when young people's growing propensity for digital text creations is considered (Lenhart, et al., 2007). With speedy communication to wide audiences, issues of plagiarism and intellectual copyright are raised, as well as just how to copy, paste, remix or morph others' work into their own creations. Keen (2008) noted the impact of remixing on authorship and creativity: the "culture of the ubiquitous remix is not only destroying the sanctity of authorship but also undermining our traditional safeguards of individual creativity" (p. 25). In these ways, ethical use and appropriate acknowledgment will inform transmodal facility and the level of e-credibility of the user. Those who create with e-credibility at the forefront of their consciousness could well be "architects of credibility" (Flanagin & Metzger, 2008, p. 18) whose quality of academic performance and social interactions are significantly enhanced.

E-designing is the visible process and instantiation of creativity. It initially requires active engagement with source material, and "unlearning" (Becker, 2006) is a useful way to allow the imagination free rein to explore solutions, innovations, transformations or original creations. Synthesis of other ideas and accommodation of different viewpoints are required, as well as the technological e-proficiency to exploit the fine functions of software or technology tools. Several researchers have found that academic progress and improved student performances can result from students as designers of multimodal texts (Facer & Williamson, 2004; Kimber, Pillay & Richards, 2007; Walsh, 2007). Further, The New London Group's (2000) notions of 'Designing and the Redesigned' endorsed the proactive reshaping of available designs in imaginative ways, attracting widespread support in many education systems and classrooms. While the evaluative practices at the core of e-proficiency and e-credibility also permeate e-designing, here, desirably, their critical and ethical dimensions are balanced by creativity and a sense of the aesthetic. Consistent efforts to embed e-designing as both process and goal for digital learners could help young people to develop rigorous thinking, sensitivity to aesthetic spatial arrangements, and a desire for achieving elegance of design. Success in these areas could help cement e-designing as a value-adding incentive for digital learners. The accomplished e-designer can apply, transform and represent critiqued knowledge from various sources into their own digital, multimodal creation - but ideally, with a strong measure of ethical responsibility and personal pride to ensure that plagiarism does not misrepresent themselves or others' work.

Table 2 presents the assessment framework for (a) using and (b) creating and sharing knowledge online developed by the researchers. The two columns allow focus on their distinctions in terms of each of the learning priorities - e-proficiency, e-credibility and e-designing. While each is presented in its own row, the arrangement is not hierarchical and is considered as a dynamic, mutually informing and overlapping set of learning priorities. Hence, the dotted lines denote both the boundlessness and the opportunity for the coalescence of the several components. The first row foregrounds Transmodal facility as the synthesising, connecting element that marks the successful integration of all other elements. It must also be mentioned that the use of "ability" refers not to an innate intelligence but rather a capacity that can be taught, nurtured and developed over time. Based on the many aspects raised in the preceding discussion, "ability" is predicated on the need for critical thinking and evaluative decisions at several points in the learning and creating process.

Table 2: Assessment framework for using, creating and sharing knowledge online

Table 2 image version

The potential of the framework

In considering the potential of this framework for talking about and assessing quality in student-created multimodal texts, several points of note emerge. With the first lens on quality in multimodal text production being print oriented and offering no explicit statements about what might constitute a quality production or opportunities for teacher or peer feedback at any time during the task completion, it is not surprising that there were so few performances deemed proficient. As well, when Tables 1 and 2 are compared, it is clear that the second offers many more opportunities for clear task setting, points for prior teaching, guided instruction or intervention, detailed feedback and goal setting for desirable online performances than in the first. This notion resonates with the emphasis placed on incorporating criterial knowledge and curriculum literacies into foundational knowledges (Kimber & Wyatt-Smith, 2009) and with the research based principles for assessment for learning (Assessment Reform Group, 2002), but drawing those principles more closely into digital learning contexts.

When this second lens is considered, opportunities are presented in that both an evaluative and a creative stance are taken up, both in the use of knowledge and in the production of new material. Those close moments for transition between location, selection, copying and transforming material require evaluative consideration on a constant basis, and especially in the creation of new texts, in line with the discussion on ethical decision making given earlier. All this mirrors the connectivity of networks, the speed of accessing and transforming digital texts, and the complex interplay between both activities as characterised by screen based activity where users can be, simultaneously, users, consumers and producers of digital texts.

An area for further development and closer consideration lies in the notion of sharing knowledge, particularly the collaborative way in which young people work online. For example, in the digital literacies research study, students were frequently observed initiating digital interactions, in the same classroom, even when oral communication was possible. In addition, when the nature of community knowledges is considered in the context of online, informal learning, wider opportunities for collaboration and sharing of feedback, with community experts as well as peers, are possible.

As well, through this second lens and the way that the framework has been presented, the profiling of e-credibility requires a critical stance to be taken up and e-designing and e-proficiency encourage reflection on quality in a much more focused way than in the earlier version. These are understood to be dynamic elements for a holistic view of what counts as quality with the transmodal being the synthesising feature in terms of working within and across modes of representation. So the act of creation is now anchored back to the informed use of texts and platforms and modes which can then be understood relative to the working of these other three. In short, it is looking in new ways for quality whereby there is potential for the cognitive, creative and the aesthetic to come into view and be focal considerations in how teachers and students think about qualities in learning and qualities in performance. Such elements are essential if learners are to develop their capacities for self monitoring and improvement. Thus the framework offers portability for the capacities that it recognises as essential for achieving a quality digital performance and experience.


This article has considered the importance of creativity in strengthening learning in digital worlds, where creativity has been aligned with higher order thinking, empathy-building aptitudes and innovative challenges. Of particular note have been the conceptual advances in understanding of indicators of quality in multimodal texts, from a print dominant, emergent digital consciousness to a view more attuned to online learning. This evolution, informed by empirical data and a diverse literature field, demonstrated that what we have traditionally come to know about criteria and standards in assessment does not carry forward into the digital world of today. Given the anticipated changes in future digital technologies, current criteria and standards have little guarantee of longevity in future schooling scenarios. What we know about achievement in former eras of schooling in defined curriculum areas does not extend to these new ways of working online. The current synergy of thinking between business, education and research suggests that today's students require a different, more complex skill set than in the past, and that their teachers have particular responsibilities in elevating seemingly superficial levels of online activity to more critical, creative, empathetic and ethical activity. Just as we can no longer think of knowledge as a fixed entity, we must find ways to carry forward those capabilities that can adapt to, critique and create newer notions of co-created knowledge. This assessment framework opens for discussion the portability of desired capabilities for using, creating and sharing knowledge online.


The authors wish to acknowledge the Australian Research Council for funding to enable the longitudinal research study hosted at Griffith University, Australia, 2003-2008 (chief investigators: Professors Claire Wyatt-Smith and Mike Levy); Dr Sandy Muspratt for statistical analyses; and the Principal, Brisbane Girls Grammar School, for support of Dr Kay Kimber's research secondment to Griffith University, 2007-2009.


Adlington, R. & Hansford, D. (2008). Digital spaces and young people's online authoring: Challenges for teachers. Paper presented at 2008 National Conference for Teachers of English and Literacy, Adelaide, Australia, 6-9 July. [viewed 30 May 2010]

Anderson, L. & Krathwohl, D. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom's taxonomy of educational objectives. New York: Allyn-Bacon Longman.

Assessment Reform Group (2002). Assessment for learning: 10 principles. Research-based principles to guide classroom practice. [viewed 30 May 2010]

Becker, K. (2006). Unlearning: A people development issue for sustainable change and innovation. In 7th International CINet (Continuous Innovation Network) Conference, 8-12 September, Lucca, Italy. [viewed 30 May 2010; access denied 20 Jun 2010]

Black, P. and Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80. [verified 20 Jun 2010]

Buckingham, D. (2007). Beyond technology: Children's learning in the age of digital culture. Cambridge, UK: Polity Press.

Buckingham, D. (Ed.) (2008). Youth, identity and digital media. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press. [verified 20 Jun 2010]

Cheong, C. M. & Cheung, W. S. (2008). Online discussion and critical thinking skills: A case study in a Singapore secondary school. Australasian Journal of Educational Technology, 24(5), 556-573.

CISCO (2007). Equipping every learner for the 21st century: A white paper. San Jose, CA: Cisco Systems, Inc. [viewed 30 May 2010]

Coiro, J., Knobel, M., Lankshear, C. & Leu, D. (Eds.) (2008). Central issues in new literacies and new literacies research. Handbook of Research on New Literacies, 1-21. New York, NY: Lawrence Erlbaum Associates.

Cumming, J. & Wyatt-Smith, C. (2001). Literacy and curriculum: Success in senior schooling. Melbourne: ACER.

Economic Review Committee (ERC) (2002). Creative industries development strategy. [viewed 30 May 2010]

Facer, B. & Williamson, K. (2004). Designing technologies to support creativity and collaboration. Bristol: Futurelab. [viewed 30 May 2010]

Fisch, K., McLeod, S. & Bronman, J. (2006). Shift happens. Did you know? [video] In The Digital Blur [blog] [viewed 13 Jun 2009, verified 20 Jun 2010]

Fisch, K., McLeod, S. & Bronman, J. (2008). Did you know? 3.0. [video, in YouTube; viewed 13 Jun 2009, not found 20 Jun 2010]

Flanagin, A. J. & Metzger, M. J. (2008). Digital media and youth: Unparalleled opportunity and unprecedented responsibility. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth and credibility, The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, pp. 5-28. Cambridge, MA: The MIT Press.

Fleming, D. (2008). Imagination, design and innovation: The drivers of 21st century success. Lead paper, Australian Council for Educational Leaders National Conference, New Metaphors for Leadership in Schools, 30 September - 2 October. [verified 20 Jun 2010; 1.8 MB, PowerPoint]

Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education, 1, 3-31.

Gorry, G. (2009). Empathy in a virtual world. Jesse H. Jones Graduate School of Management Working Papers Series. [viewed 30 May 2010, verified 20 Jun 2010]

Haas, C. & Wearden, S. T. (2003). E-credibility: Building common ground in web environments. L1-Educational Studies in Language and Literature, 3(1/2), 169-184.

Ito, M., Horst, H., Bittanti, M., Boyd, D., Herr-Stephenson, B., Lang, P., Pascoe, C., Robinson, L. et al. (2008). Living and learning with new media: Summary of findings from the Digital Youth Project. The John D. and Catherine T. Macarthur Foundation. [viewed 30 May 2010; 2.6 MB]

Jenkins, H. (2006). Confronting the challenges of participatory culture: Media education for the 21st Century. [viewed 30 May 2010, verified 20 Jun 2010]

Keen, A. (2008). The cult of the amateur. How blogs, MySpace, YouTube and the rest of today's user-generated media are killing our culture and economy. Clerkenwell, London: Nicholas Brealey Publishing.

Kennedy, G. E., Judd, T. S., Churchward, A., Gray, K. & Krause, K-L. (2008). First year students' experiences with technology: Are they really digital natives? Australasian Journal of Educational Technology, 24(1), 108-122.

Kimber, K., Pillay, H. & Richards, C. (2007). Technoliteracy and learning: An analysis of the quality of knowledge in electronic representations of understanding. Computers & Education, 48(1), 59-79.

Kimber, K. & Wyatt-Smith, C. (2008). Assessing digital literacies: Can assessment ever be the same? In L. Unsworth (Ed.), New literacies and the English curriculum, 328-354. London, UK: Continuum.

Kimber, K. & Wyatt-Smith, C. (2009). Valued knowledges and core capacities for digital learners: Claiming spaces for quality assessment. In A. Burke & R. Hammett (Eds.), Assessing new literacies: Perspectives from the classroom, 133-156. New York, NY: Peter Lang Publishing.

Krathwohl, D. (2002). A revision of Bloom's Taxonomy: An overview. Theory into Practice, Autumn, 212-218.

Lankes. R. D. (2008). Trusting the Internet: New approaches to credibility tools. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth and credibility, The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, pp.101-121. Cambridge, MA: The MIT Press. [viewed 30 May 2010, verified 20 Jun 2010]

Lenhart, A., Madden, M., Rankin Macgill, A. & Smith, A. (2007). Teens and social media. Washington, DC: Pew Internet and American Life Project. [viewed 30 May 2010]

Leu, D. J., Kinzer, C., Coiro, J. & Cammack, D. (2004). Towards a theory of new literacies emerging from the Internet and other information and communication technologies. In R. Ruddell & N. Unrau (Eds.), Theoretical models and processes of reading, pp. 1570-1613. Newark, DE: International Reading Association. [verified 20 Jun 2010]

Levy, M. & Kimber, K. (2009). Developing an approach for comparing students' multimodal text creations. Australasian Journal of Educational Technology, 25(4), 489-508.

McGonigal, J. (2008). Why I love bees: A case study in collective intelligence gaming. In K. Salen (Ed.), The ecology of games: Connecting youth, games, and learning, pp. 199-228. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press. [viewed 30 May 2010, verified 20 Jun 2010]

McWilliam, E. (2005). Unlearning pedagogy. Journal of Learning Design, 1(1), 1-11.

McWilliam, E. & Haukka, S. (2008). Educating the creative workforce: New directions for twenty-first schooling. British Educational Research Journal, 34(5), 651-666.

Metzger, M. J. & Flanagin, A. J. (Eds.) (2008). Digital media, youth and credibility. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press. [viewed 30 May 2010, verified 20 Jun 2010]

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2007). National Assessment Program - ICT literacy Years 6 and 10 Report, 2005. ACER. [viewed 30 May 2010; 1.8 MB]

Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) (2008). Melbourne Declaration on educational goals for young Australians. Melbourne, Australia: Curriculum Corporation.

Partnerships for 21st Century Skills (2009). P21 Framework Definitions explained: White paper. [viewed 19 Jun 2009 at; verified 20 Jun 2010 at]

Prime Minister's Science, Engineering and Innovation Council's Working Group (2005). Imagine Australia: The role of creativity in the innovation economy. [viewed 30 May 2010]

Robinson, K. (2006). Do schools kill creativity? TED Conference, USA. [viewed 30 May 2010; verified 20 Jun 2010]

Robinson, K., Minkin, L., Bolton, E., French, F., Fryer, l., Greenfield, S., et al. (1999). All our futures: Creativity, culture and education. National Advisory Committee on Creative and Cultural Education Report. [viewed 23 Feb 2008, verified 20 Jun 2010]

Sadler, D. R. (1985). The origins and functions of evaluative criteria. Educational Theory, 35(3), 285-297.

Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144.

The Conference Board of Canada (2008). Compendium of research papers: The international forum on the creative economy. Canada: The Conference Board of Canada. [verified 20 Jun 2010]

The New London Group (2000). A pedagogy of multiliteracies: Designing social futures. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures, 9-37. London: Routledge.

Walsh, C. (2007). Literacy in the new media age: Creativity as multimodal design. Paper presented at Critical Capital: Teaching and Learning, AATE and ALEA National Conference, July 8-11. [verified 20 Jun 2010; 2.2 MB]

Wyatt-Smith, C. M. (2001). The function of evaluative talk to structure instructional sequences. In J. Cumming & C. M. Wyatt-Smith (Eds.), Literacy and the curriculum: Success in senior secondary schooling (pp.117-131). Melbourne: ACER.

Wyatt-Smith, C. & Cumming, J. (2003). Curriculum literacies: Expanding domains of assessment. Assessment in Education: Principles, Policy & Practice, 10(1), 47-59.

Wyatt-Smith, C. & Kimber, K. (2005). Valuing and evaluating student-generated online multimodal texts: Rethinking what counts. English in Education, 39(2), 22-43.

Wyatt-Smith, C. & Kimber, K. (2010). How shall we know them? Student assessment and digital futures. In M. Lee & G. Finger (Eds.), Developing a networked school community: A guide to realising the vision, pp. 259-275. Camberwell, Victoria: ACER Press.

Appendix A: Overview of Using and creating online knowledge study

Using and creating knowledge in the high school years: Performance, production, process and value-adding in electronic curricular literacy (2003-2008) was funded by an Australian Research Council Discovery Grant and hosted by Griffith University. The intent was to obtain a 'point in time' capture of secondary schools' students digital capabilities in completing a curricular-related online task involving online research and the creation of a multimodal text, and to track students' progress across a two-year period.

Key questions

  1. How and how well do high school students combine symbolic systems in their use and production of curricular knowledge in high school curricular activities?
  2. How and how well do high school students work individually and collaboratively in these multi-modal curricular activities?
  3. What features of classrooms (e.g., pedagogies, assessments, materials) and schools (e.g., policies, support systems) relate to value-adding to performance and process?


Sixteen government and independent secondary schools across a range of socio-economic areas in Queensland participated in this study. Participants included 736 students from Years 8 and 10 in 2004, and 248 from Years 10 and 12 in 2006. 138 students were common to both data collection rounds.

The tasks

Two separate online tasks were devised in consultation with a teacher advisory group, following a pilot study in 2003. The aim in devising both tasks was to embed them in curriculum requirements for Years 8, 10 and 12 in national priority areas (English, science, mathematics), while also taking account of the set curriculum in history, studies of society and the environment, and technology.

The 2004 task was designed as a cross-curricular, inquiry-based activity that focused on the environmental threats posed by plastic bags. The 2006 online task retained its inquiry-based framing but had a greater emphasis on web site evaluations. It focused on biometrics, global warming, or the participating school's own curricular focus. Unlike the 2004 task where students were required to present a solution to a problem, the 2006 task required students to investigate alternative views on the topic and represent findings.


A range of data types was collected in 2004 and 2006. This data included
  1. surveys (918 student and 272 adults) - on out-of-school technology use
  2. product data (841 student-created multimodal texts) - mostly PowerPoint, some Word documents, a few web sites
  3. process data (concept maps, decision-making matrix, web site evaluations - completed as students were using online knowledge and creating their own multimodal text - and their reflections of the process, their product and the experience) plus
  4. screen capture recordings of students' real time working online, searching the Internet, selecting relevant resources and constructing their texts), and
  5. interaction data - recordings of talk as a sub-set of students as they worked in pairs to collaborate on the 2004 task.
All data were created, collected and archived electronically.

Further details about the study and samples of student creations are available on the website,

Authors: Dr Kay Kimber, Director, Centre for Continuing Professional Learning
Faculty of Education, Mt Gravatt Campus, Griffith University
176 Messines Ridge Road, Mt Gravatt, Brisbane Qld 4122

Professor Claire Wyatt-Smith, Dean
Faculty of Education, Mt Gravatt Campus, Griffith University
176 Messines Ridge Road, Mt Gravatt, Brisbane Qld 4122

Please cite as: Kimber, K. & Wyatt-Smith, C. (2010). Secondary students' online use and creation of knowledge: Refocusing priorities for quality assessment and learning. Australasian Journal of Educational Technology, 26(5), 607-625.

[ PDF version of this article ] [ AJET 26 ] [ AJET home ]
HTML Editor: Roger Atkinson []
This URL:
Created 16 Jul 2010. Last revised 16 Jul 2010.

Page access count