In April I ran some tests-of-concept for my Multimedia Competition Site during three different graphics-oriented contests: Catalog Production using big PDF files from two dozen countries; Web E-Commerce Production from a similar group of schools; Video Production and Digital Photography from 18 New York City high schools. I’ve also brainstormed the idea with teachers, administrators and student competition organziers in various fields, som beyond graphics.
I kludged together the elements of my Multimedia Competition Site from a combination of HTML pages, Excel files and email. Judges got a password-protected site with PDFs or links, Judges’ Instructions and a spreadsheet formatted for the contest’s’ rubrics. They clicked and judged and entered their raw data (for as many as nine rubrics per item) and then emailed their results. Each Judge was isolated during judging. In some cases, I shared the previous judges’ results with new judges.
OLD SCHOOL
Previously we schlepped CDs, memory sticks and FTP batches around town, often with a satchels full of the hard copies of the work. We’d have to coordinate face-to-face meetings to handle the material. International submitters were at a disadvantage since they had to ship the physical work weeks ahead of time.
FINDINGS
Even with my kludge, this year’s judging was remarkably faster and easier. A tougher-than-usual schedule was easily accommodated. The pool of judges has deepened: regular judges are now suggesting colleagues in and out of town who would now be happy to help — now that we’re more digital. The results seem remarkable consisten judge-to-judge, which was true of previous years’ paper-based competitions. Even when judges had no other results, they seemed to reach very similar ratings.
SOFTWARE
I’m evaluating two different PHP “YouTube clones” to see which is more extensible for my needs. Both have the security I need, and both allow sharing and rating videos, JPGs and GIFs. I need to add PDFs, PPTs, MP3s and DOCs. I also need to auto-create thumbnails of the new formats.
Most canned apps use a simple five-point rating routine. I need to configure n ratings per item so I’ll have to tweak the code. I’m also concerned about the complexity of uploading. It seems none of the teachers I work with can follow instructions.
SIZZLE
I’ll be happy if I can get a system that manages gigabytes of diverse entries and streamlines the judging process., a simple administrative win. But if I can make the contest live so that every judge sees all the other results as they’re posted, if judges can comment on work while it’s still coming in, if teachers and students can be integrated into the process, I’ll really have something transformative. Just in New York City, we have over a dozen competitions that could use this system, in whole or in part: http://council.icte.us/competitions.
–Jack
A quick note on something that may make your current system easier and then another idea.
Have you thought about using Google Forms (part of Google Docs) instead of emailing spreadsheets. you can get somethign like this:
https://spreadsheets.google.com/viewform?hl=en&formkey=dEpHa3FpX2tQQWk5dkxMMFl5eVd5b3c6MQ#gid=0
and then it will automatically populate that date into a Google Spreadsheet (forms are really just an extension of spreadsheets). By emailing your judges the link to the form they don’t have to have a Google account. You would have to have a separate way of getting them the media and all that.
I agree with Joe that there are broader use case potentials, but also that for now it is good to focus on the specific case you have which is very well defined. Joe’s comment got me thinking about another tool that is specialized right now but could have broader applicability, VOCAT (https://www.baruch.cuny.edu/vocat/). Right now it’s designed to assess video taped speeches but it could assess any kind of video and with some extending, other media. It seems that with some tweaking it could handle the kinds of things you’re looking for. It’s also nice that it is another education and CUNY related project. I believe they are planning on open sourcing it.
I can help introduce if you like. In general I think your project has a lot of potential both as you’ve currently defined it (and I think it’s good to stick with the specific use case you have now) and in the future should you decide to broaden it.
I wonder if there’s a way to include some other tools (since you’re developing anyway!) that could make this more useful for teaching, beyond the judging and competition. Annotation and shared annotation of multimedia files is a largely unexplored area right now, with few exceptions. If these files could be annotated (with specific time- and place-markers on the screen, the ability for annotations to be shared and annotators’ contributions to be tracked–and even better for the annotations to be responded to and annotated themselves, I could see how judging would be the only beginning–the competition could be a much larger teaching and learning moment.
Jack, what are the youtube like systems you’re evaluating? Off the top of my head I thought of Kaltura, http://www.kaltura.org/, and Media Core, http://getmediacore.com