Educational software I wish existed: StackExchange for teaching in the Humanities and Social Sciences

Here’s an idea I’ve been mulling over for the past week. (Mulling over enough to have spent many hours in an abortive attempt to create a prototype. But even simple web application programming seems to be a nontrivial problem that requires more time investment than I can muster right now). I have a large bank of questions for some of the courses I teach. These are mostly essay questions (short and long) of varying degrees of complexity, that ask students to make arguments, provide evidence, point to real-world examples, etc. I usually select the questions for final exams or other forms of assessment from this pool; a student who could answer all of them well would have essentially mastered the content of the course.

I would like to have a web application that draws on this pool of questions to do this:

1. In “quiz mode” a student would either select a question or be served a random question from the pool, and then he/she would answer it. They could then move on to a different question, for as long as they wished. (Perhaps the questions could be served in such a way that students can vote for the questions they most want answered, as in Google Moderator). The software would also allow the students to rate both the quality of their own answers (how good they think their answer is) and/or their level of confidence in their answer (how certain they are that they have a good answer), as well as the level of difficulty of the question. Their answers and ratings would go into a database; as they accumulate, the instructor could see which questions are rated as “hard” by students, or display characteristic problems, and focus teaching efforts there.
2. In “rating mode” students would either select a question or be served a random question from the pool, which they would view along with any (anonymized) answers from themselves, other students in the course, or even the instructor. They could then vote on which answer is best (if there is more than one answer) or rate the quality of existing answers. Perhaps they could also comment or edit existing answers if they want to add something to them, or feel a correction is in order. As these ratings accumulate, students would get a better sense of what counts as a good or a bad answer (assuming the “wisdom of crowds” works its magic; the courses I have in mind for this sort of application have around 100 students, which seems like it would be enough).
3. In “asking mode,” the application would allow students to submit questions, which then would go into the pool and could be answered by other students in the class. (Administrators could edit the questions for clarity or reject questions that are not sufficiently related to the topic of the course). 
4. The application could even have something like the reputation management features of the StackExchange family of sites. Students who answer questions would gain reputation “points,” so long as their answers are rated relatively highly (fewer points for unrated or lower-rated answers); asking questions or rating answers would also get them some reputation points, though fewer. (For purely illustrative purposes, imagine that asking a question nets you 2 reputation points, so long as it is not rejected by the instructor, rating an answer nets you 1 point, and answering a question nets you between 5 and 10 points, depending on how highly rated your answers are). Perhaps these reputation points could be translated into actual grade points at the end of the term in accordance with some appropriate formula, though that would depend on the design of the course.

As I imagine it, an application like that would offer students extensive practice in writing, especially if combined with say, a requirement that they answer at least one question every week (in fact, this system could displace one of the traditional two essays we ask students to complete in many courses). It would also help them practice the entire material covered in the course: since the questions for the final exam would be drawn from the pool (or be very similar to some of the questions there) students who use the tool would be essentially studying for the final every time they use it. (“Quizzing” yourself is one of the most useful study techniques available, and the system would be designed so that you would get relatively quick, and eventually accurate, feedback on your answers, without the instructor having to grade hundreds of essays). And I (the instructor) would in turn get feedback on how well they understand the material, as well as on what aspects of the course they are having difficulty with. 

What do people think? What problems would you foresee emerging with a system like this?

As far as I can tell, nothing quite like that exists, though in some ways this would be like a private version of StackExchange, seeded with a pool of questions on some specific course topic and open only to people taking the course. Google Moderator has some useful features, and I’ve been thinking about using it as a study tool for students in my course this term, but it would not be fully integrated with the rest of the assessment in the way I would want. Or is there something out there that I’m missing? How difficult would it be to develop the system I've described above?

Using CiteULike for Teaching

I recently rediscovered CiteULike, a free web service for bibliography management. I know I had used it in the past (I still had an account, though I had not logged in for four years) but stopped using it in favor of Zotero and Endnote (because of the latter's integration with Word). But then I stopped using Zotero, partly because it tied me to Firefox (I now use Chrome almost exclusively, and love it), and partly because it was often slow and occasionally crashed (I think it's better now - it even has word processor integration! - but I had one too many bad experiences).

Anyway, there's no software to install with CiteULike and no browser dependencies - it's just a bookmarklet - and you can easily transfer your data to and from Endnote and BibTex. (Incidentally, LaTex is great for Greek-heavy writing on Ancient philosophy with one of the Betacode packages. It's too bad most of the places I submit my work to tend to demand Word files, and that converting a file from LaTex to Word can be such a pain). But it also occurs to me that something like CiteULike would be great for teaching as well.

I typically maintain a large bibliography of recommended works and suggested readings for the essays in my courses, and it's a bit of a pain to add new entries to it and make sure the new entries appear in all the appropriate places and are appropriately organized and formatted (partly this is just the fault of the Blackboard course management system we use at Victoria - which, despite some good points, is still a pretty cumbersome piece of software). So, for example, for my POLS 209 course on Dictatorships and Revolutions I might have some recommended readings on Venezuelan politics (which I suggest the students consult before writing an essay on the relevant topic in the course), or a set of readings on the comparative economic performance of dictatorships vs democracies (for students who are writing on the topic, or might want to learn more). But every time I need to add a new reading, I have to change things in four or five different places, and manually add links and other things. Moreover, though students can add items to the general bibliography (it's maintained in an internal wiki - though the Blackboard wiki tool is terrible) they may add things formatted in odd ways, or in the wrong categories, and they cannot easily download the sources to a bibliography manager.

But with CiteULike I can maintain the entire course bibliography in one place, and then provide links to specific topics using tags (you can even subscribe to the whole thing or to specific tags using RSS, if you want to keep up with the latest additions). Readings on Venezuelan politics? Here! Or I might tag some readings as recommended for the first week of the course, or some particular lecture, and post the link in the course outline and in the appropriate place in Blackboard. And students can of course download the citations to the citation manager of their choice, click on the handy links to read the actual articles (the point of the exercise, after all), or simply copy and paste the plain text of the citations they used to their bibliographies when they finish writing their essays (too much student time is, I think, spent worrying on minor details of bibliography formatting rather than actually reading the sources).

Moreover, students who create an account with CiteULike can easily contribute readings to the common course pool, perhaps with their own review notes (I can always tag them appropriately); they just have to join the class group. (I already incentivize this sort of thing in the course outline - adding to the course bibliography in appropriate ways, or contributing to the course blog/discussion board/wiki, are all things that help students get points in my courses). In fact, that's the part I'm most interested in: I keep trying to foster collaborative research in the classroom, and if students are easily able to contribute sources to the common pool of readings, they can basically help each other in constructive ways. And they can help me as well - I have found some very interesting sources for the course (even whole new literatures) in essays that students have written in the past - sources which sometimes end up in the course bibliography (you really do learn lots from your students).

I am actually considering requiring my honours students (in POLS 401, a course on contemporary political thought) to actually create a CiteULike account and contribute readings to it periodically, perhaps with some review notes. (I tried something like this once with a wiki, but the wiki is too cumbersome for this). To be sure, there's some upfront investment on their part (I probably will have to demonstrate how to use the software in the first few classes, or write a detailed handout with an explanation and pictures) but once they learn, I imagine it could be helpful to them (both in this and other courses).

Has anybody else done similar things? What potential problems am I missing?