The Rubric.

[We Despise The Wretched]

Visla already explained how I feel about this, for the most part, but I thought I'd say my two cents. I simply believe that none of us are perfect writers, therefore it's near-impossible for any judge - of any skill - to correctly pass judgment on another's writing. That isn't to say that a Judge can't touch on the gist of a player's pros and cons. So honestly, I'd say it's as good as it's going to get.

Though, one possible way that judgments can improve would be to have more than one judge convene on a single thread. That way, scoring won't be based on the eclectic views of one reader, but rather, a general consensus.

Addendum: But then again, that would just make things much more time-consuming, and we're all just a bunch of lazy-asses anyway. So I repeat, I think it's as good as it's going to get. :P

[Cydnar]

Workshop judging tends to work on a more collective basis, it was implemented to address similar concerns following previous endeavours to address the 'issues' with the rubric.

[Silence Sei]

Okay, well I'm not going to say whether or not I think the rubric is flawed, its just the system we currently have. You point me in the direction I'm supposed to judge, I'll use whatever tools I'm given.

As far as the guilty until proven innocent theory, I have hardwired it into all the new judges that we start all categories, with the exception of mechanics (Start at 10 and work down) and Wildcard (as that's opinionated) with a score of 5, and go either way. I assume the writers are average and not poor or excellent at the start of the story, and woprk from there, and I believe Rev and Zerith do the same (Least I've told them to a couple of times)

[Saxon]

I really have like this urge to take the hook and run with some kind of argument against the methods used here, but in reality, this is a fight that has been hashed and rehashed more times then I can remember. And I don't really have time right now to stand around and beat a dead horse. Especially when nothing is going to come of it.

Visla made a lot of good points and I agree with most of them. Writing is not an exact science, and the methods we use to treat it that way are not only wrong, but the reasons why we do it are wrong to. We use judging as a way to score and apply experience to a person's account so they can run through the gambit and keep up the illusion that Althanas is a game and to treat it as such. Don't get me wrong, while the idea of peer review/editing of other people's work is a brilliant idea, I've never gotten behind the whole campaign of treating Althanas as a roleplaying game and claiming it to be a writing workshop as a front.

But, for the sake of argument, lets forget the glaring philosophical and ethical argument of using literature to lure people into a bad game. Lets focus on instead the flaws of Althanas' judging system;

1) There is little standardization in how judges apply the rubric in judgments. There is no form of training or level of expertise required to be considered for becoming a judge. And while that's a little ambitious for a site like this, I think its a fair argument to make since judges are protected inherently by site rules from public scrutiny or reprimand for the methods they choose to use to judge your threads and mine. I think if you'd like a better rubric and better commentary, I'd suggest trying to build some sort of program for judge trainees to become accustomed to judging and to learn how to do it right as well as ethically.

As far as I know when I was a judge, there was a thread you could use for formulas when calculating experience and gold, a little pamphlet from an old member over the philosophies he applied to judging, and your hand could be held for awhile by other judges during your first couple of judgments until they decided you were fit to go off and peruse the backlog on your own. That's all. Other then that, your requirements for being a judge were that you needed to be able to form an opinion of your own and have a pulse. You'd be amazed on how hard it is for some people to have BOTH of those things at once.

I'm just saying. Throw in a bit of an orientation for judges, construct a program to teach new judges how to judge and provide a little more oversight, and things might run a bit smoother. The sink or swim method for handpicking judges is and has always been a terrible idea.

2) The weights of the points assigned in particular areas of the rubric needs fixed. For no reason should something as fundamental as setting weigh the same as something as subjective as continuity. As it stands, a lot of the rubric follows the method of 10 points across the board from what I can remember. And it implies that all things in the rubric are equally important. They're not. While I'd argue that Setting, Mechanics, and Action should all have more weight of some where around 12-15 points a piece, things like Clarity and Continuity should be worth around 5 a piece.

The reasons for this is while having a good setting, knowing proper grammar and being able to control the tension in your story are all very important to a good story, a person's ability to tether there thread to other adventures is less so. And being able to write clearly is entirely dependent on the judge that you have. They just do not seem like the cornerstones to writing here when evaluating someone.

3) The Wild Card. I didn't lump this in with # 2 because this area of the rubric is so absurd it deserves its own time in the spotlight. It has a special place here because from the time I joined, the time I judged, to the time I left the staff I have never figured out why anyone on this site would need 10 bonus points in order to get a perfect 100 score. It really takes the bonus out of it, now doesn't it? While arguments have been made for this area to be used at the judge's discretion as a way of letting the player recoup points he or she lost in other areas, I think this area of the score just sucks up other points that could be weighted in other areas.

Lets also not forget that you as a member are also entirely dependent on the person judging your thread to find something he or she enjoyed in your writing to assign nonbonus bonus points to you in order to balance out the fact you sucked in either grammar or continuity. Because God help you if they don't and you're stuck with 0 points in Wild Card because at your absolute best all you will ever get is a 90. I think that alone breeds subjection and gives Judges a little too much power in determining your score. These points would be better served if they were redistributed else where.

If we'd like to have a judge have the authority to award bonus points, instead of devoting an area of the rubric to it, make a clause. Allow X number of free points to be awarded to the score for a legitimate reason. At most 5. That way a member can still achieve a 100, they have the option for bonus points for exceptional work, and the judge's judgment is kept in check. Everybody wins.

[Visla Eraclaire]

Just as a point to how different people view things, I think Saxon picked the wrong items to weight. That's because we clearly have different writing styles and emphasize different things. A computer can be programmed to write in mechanically proficient English. Mechanics is just an area for judges to knock points for typos and bad editing. Guess what? Editors do editing. Writers do writing. Action is also one of the more poorly judged areas. Not every story has action in the literal sense and it's rare for someone to recognize that a lack of action can be deserving of a good score.

This isn't me attacking Saxon, this is just me pointing out that even between people who agree a lot there's a lot of disconnect on weighting. I don't think there's a one size fits all weighting for threads. That's why I'd emphasize the objectives of the thread. Honestly, I'd scale back the rubric to fewer areas that are more general and applicable to all threads, put them at 20 a piece, rather than trying to get into too much granularity.

We can throw ten areas out but that's really just obfuscating a failed system.

Also, I'm reading the descriptions that follow the numbers in the poll and don't feel they represent the numbers. That's what you get with a ten point scale trying to encompass subjective impressions. 6 and 7 seem backward as do 3 and 4.

[Lord Anglekos]

Also, I'm reading the descriptions that follow the numbers in the poll and don't feel they represent the numbers. That's what you get with a ten point scale trying to encompass subjective impressions. 6 and 7 seem backward as do 3 and 4.

Actually, I have to agree with Visla on this. I re-read my poll options, but only after I'd posted them, and some things just didn't sound the way they did in my head. So yeah, this further proves his point on the ten-point system. So would it be better or worse if there were, say, 20 points for each selection? Setting, continuity, etc...

[Saxon]

Actually, I have to agree with Visla on this. I re-read my poll options, but only after I'd posted them, and some things just didn't sound the way they did in my head. So yeah, this further proves his point on the ten-point system. So would it be better or worse if there were, say, 20 points for each selection? Setting, continuity, etc...

I think trying to tailor your error to his argument of adjusting the rubric is a dangerous undertaking. Rather then weighing down the bandwagon, I'd say take your lumps and move on.

[Lord Anglekos]

I think trying to tailor your error to his argument of adjusting the rubric is a dangerous undertaking. Rather then weighing down the bandwagon, I'd say take your lumps and move on.

I have. I simply agree with Visla's point, and seeing as he used my own poll to prove it, I don't see the "dangerous undertaking" of voicing my agreement. If you believe it is such, then congratulations. We are now all forearmed to the dangers of agreeing. And we thank you profusely for that.
In any case, I ask the same question from two posts ago. If the numbers were raised from 10 to 20 points, and the number of "categories" were reduced, would it be better, worse, or the same?

[Hysteria]

Scales should opperate on an 'average' point in the middle. So on a 1-10 scale the 'average' make should be 5, with anything 0-4 being below average and 6-10 being above.

The poll has 3 and 4 as 'alright' so you only get two options if you don't like it. The standard scale of 1 to 5 that is always used for survays has 1 (Strongly dislike) to 3 (neutral) to 5 (strongly like) with 2 and 4 being dislike and like without the 'strong' aspect.

Whatever the number that is picked as long as the spaces between each number are equal and the 'sides' of the rating are equal (you can be just as 'good' as you can be as 'bad') then it should be alright. Your poll doesn't offer opposing points for all the options and the space between each one is not the same (it really doesn't matter in this sort of poll, but for the rubric it would).

Someone before (can't remember and can't be bothered looking who it was) said that they started everyone from a 10 and if they made mistakes it moved down. Thats alright, i'd start at a 5 and go up and down, but meh. In the end the rubric is just there so I can lvl up.

[Visla Eraclaire]

In the end the rubric is just there so I can lvl up.

This is another major problem. Althanas still serves two masters: gamers and writers. Judges have to make sure gamers get enough exp to keep their grind moving along while writers get enough feedback and legitimate criticism to keep them satisfied.

I think 5 twenty point categories would be better. I brainstormed what those five might be with Saxon last night. The problem is, this won't really solve anything. There are a lot of problems and solving any one of them isn't going to make a big impact. That's why much of Althanas gets shuffled around and painted over every now and then without significant improvement. The foundation and core construction are flawed and without a unified fix to that, all the small fixes are pointless.

Althanas lacks a singular goal and direction, it lacks dedicated staff, and it lacks a sufficient member base. You won't get dedicated members without decent judging. You can get good judges without a large pool to pull from. You lose significant numbers of both if you choose a single goal for Althanas instead of having it be mediocre at two things.

It's intractable. You cannot tract it.