New Jersey Institute of Technology

Online Portfolio Assessment in the Humanities

At NJIT, we use portfolio assessment in the Senior Capstones, Cultural Studies courses, Freshman Composition and undergraduate Technical Communication for program assessment and curriculum improvement. However, in the Spring of 2004, we decided to move the portfolios in two programs (undergraduate Technical Communication and the Master's of Professional and Technical Communication, MSPTC) online. This process is ongoing and the results of the readings have not been fully analyzed. We present some of the initial steps below.

Undergraduate Technical Communication

In the undergraduate course, we have long been reading and scoring best papers for an ongoing program assessment. The history of this assessment effort is published in two papers, "The Assessment of Technical Writing:" by Norbert Elliot and Robert Lynch in and "Setting the Discourse Community: Assessment for the New Technical Communication Service Course" by Nancy Coppola. As we began modifying the curriculum of the undergraduate technical communications course, we continued the holistic scoring of best papers. We read and scored best papers, holistically, every semester. Once a year we read and scored the entire paper portfolios analytically. This last semester (Fall 2004) we required that each student have an online portfolio with all of his or her work in on the web.

New Criteria

In order to give validity to the variables that we were assessing, we needed new criteria. Thus together, through an online modified Delphi (a Delphi is a method for structuring a group communication process so that it is effective in allowing a group of individuals, as a whole, to deal with a complex problem) we came up with a new set of criteria for the undergraduate technical communication curriculum.

We needed to do this to include attention to visual clarity, presentational logic, audience-awareness, appropriate use of voice, and editing competency. We then made these new criteria into an analytic scoring method for the online portfolios.

This December, we read online portfolios, scoring them for thirteen separate variables, with these new criteria. The results and not yet been analyzed and we will share them with you when they are.

Graduate Technical Communication

In the MSPTC program, we began two forms of assessment: formative assessment, based on the students ongoing work, as discussed and recorded by professors, and a summative assessment based on criteria and portfolio readings. For this, we also created a new set of core competencies that are spread across the program.

Formative Assessment

The list of core competencies was distributed to each professor. The professor then made comments and collected examples of student work to provide evidence about the competencies of each student. Then we all got together and discussed each student individually, with these notes in front of us. The session was tape-recorded and the recording provided the basis for the final “report card” comments that are given back to the students (slide 9).

Summative Assessment

The summative portfolio assessment was scored analytically for traits. Further information will be published soon.

In sum, our assessment program in the Department of Humanities provides a basic structure for sustainable curriculum improvement. By assessing programs (not the students—that is the job of the instructors) our community can initiate changes that will be used to continually improve the program. We will continue to refine our assessment metrics: our goal is to discuss, debate, and present assumption of core competencies that drive assessment, and (in turn) to revisit, refine, and clarify the construct validity of the existing predictor variables. Our strategies will include engaging in face-to-face and electronic discussions among faculty about the design of programmatic standards and the methods used to ensure that they are being met. And, as always, we will communicate our curricular goals to those most impacted by our decisions: the students. When will the process be complete? Never.