Towards an Open Rubric – Part One

Though it seems like just a short time, almost three years ago my old workgroup at Penn State set out to do something crazy: help our faculty deal with an overnight tripling of class sizes in our college.

The College of Information Sciences and Technology had been created by University President Graham Spanier in 1999 under a protectionist model: class sizes were capped and admission to the major was restricted in order to try to create something different: a program built from the ground up around Problem Based Learning. At the same time, the administration recognized that the college couldn’t be self-sufficient with these restrictions, and provided the startup funding necessary to allow it to prosper.

When this additional funding came to an end, the college administration discovered a sobering fact: class sizes would have to nearly triple for the college to become self-sufficient. The artificial environment under which the college had prospered was coming to an end.

At the time of this inflection point, I was the Senior Technologist of the now-defunct IST Solutions Institute. SI was the education R&D arm of the college: an eLearning and technology “Skunk Works” comprised of Instructional and Multimedia Designers and Software Developers. A few months earlier, Stevie Rocco, one of our Instructional Designers and my partner in crime at SI, had come across an interesting project: a JavaScript-based rubric tool for evaluating student course work [I'm trying to find a reference to this project - BP]. There were a number of technical limitations of this prototype, but the idea was sound: have the rubric be the UI metaphor with which faculty could interact with a system that facilitated higher-quality, higher-speed grading by the faculty member by simultaneously:

  1. handling the accounting operations behind grading and giving feedback
  2. fostering the sharing of grading standards across a diverse faculty

We set about to design develop a rubric-centric application, one that would complement Penn State’s ANGEL LMS and SI’s existing Edison Services suite of eLearning tools.

In my mind, an absolute imperative behind developing such an application would be the separation of the definition of rubric documents (or data objects) from the application code of such a system. Many of the existing rubric tools (including that first JavaScript implementation) had no clear separation of data from behavior; at best, this makes them inseparable from their single, embedded rubric. In any case, the result is effectively a closed systems with little hope of sharing data with open systems in the education enterprise.

Still other rubric-based systems decomposed a rubric into multiple Relational Database tables, shattering the coherence of the rubric as a first-class part of the system. One can hardly fault such projects: this was the prime application design pattern of Web 1.0 and even Web 2.0 applications then coming into common use.

As we developed our prototype rubric tool (which we jokingly called “The Rubricator”), I made sure the design was built around a rubric as a document, at the time marked up in XML, that could be separated from the application, shared, remixed, etc. The UI was built in Adobe Flex with a server layer in ColdFusion, two technologies the SI gang was already very familiar with from previous projects. “The Rubricator” would load the rubric document payloads at runtime, ensuring a strong separation from logic and data representation.

The whole process of design at SI was one we took very seriously. To date, this project was the best example of team collaboration and iterative design and development I have experienced in my professional career. After two iterations of prototyping and design meetings, we now had a clear design and application flow:

Rubricator Application States

After the ensuing six months of back-burner and after-hours hacking, we approached the end of our third iteration and a magical “1.0″ release. Then the unthinkable happens: SI was dissolved and the team was scattered across other units in the college. While that was disappointing to all of us personally and professionally, we were leaving a big stakeholder in a really awkward position.

Next: Part Two – Finishing what we started

2 thoughts on “Towards an Open Rubric – Part One

  1. Never would have guessed PSU would dissolve you right before something was finished…I went to PSU, grew up in State College and my father works there…so I have seen a bit of how things work there…crazy.
    This is an interesting topic for sure.

  2. @Dan, in PSU’s defense, it was my college’s administration (College of IST) that dissolved the Institute. Penn State is a very big organization, and decisions are made at all levels by people with often only in the interests of their immediate unit. By and large, the folks that were part of the Institute are still serving the university, albeit in different roles and manners. Only our cadre of student workers and few of the full-time folks like myself have completely moved on from PSU.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>