I'd like to say something about the state of the Level Judges.

For topics related to SuperMarioBrosX.org. Suggest anything that you would like to see on the site here.

Moderator: Userbase Moderators

HeroLinik
Larry Koopa
Larry Koopa
Posts: 3464
Joined: Fri Dec 20, 2013 1:28 pm
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby HeroLinik » Sat May 28, 2016 1:42 pm

AeroMatter wrote:Look, nobody on this forum is important enough for "people have lives" to be an excuse for anything otherwise you wouldn't be here. It's an excuse for laziness, because there's plenty of time for LJs to do everything else on here minus judging levels. You guys are just not needed, and stand in the way of people who want reviews so they can improve their levels or just simply want feedback at all. I said in the other thread that level topics only get a few replies and then die because of the system we have now, and it's just a stupid way of running things.
I'm going to throw in the fact that they actually got rid of the Level Judges on Knux's site back in 2013, and I was around to witness it. I can't remember what was said exactly, but it was probably the fact that there's no need for judges if anyone could review them (and probably something to do with inactivity, but I was reviewing quite a lot), but according to the system we have here, it may seem like Level Judges are almost necessary to move levels to their respective sections. However, considering a regular user can do that, and judging by the discussion taking place, the mods won't be tasked more, can't they just do the same here?

zlaker
Reznor
Reznor
Posts: 2844
Joined: Fri Dec 20, 2013 1:46 pm

Re: I'd like to say something about the state of the Level Judges.

Postby zlaker » Sat May 28, 2016 1:47 pm

Where are we suppose to get any new "active" judges when the only thing people do on a level thread is "Nice gfx. 7/10"

Emral
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 9890
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Phoenix

Re: I'd like to say something about the state of the Level Judges.

Postby Emral » Sat May 28, 2016 1:51 pm

zlakergirl357 wrote:Where are we suppose to get any new "active" judges when the only thing people do on a level thread is "Nice gfx. 7/10"
To add to that, WHEN someone decides to give an elaborate review in an attempt to get the position, the reason for turning that person down seems to be incredibly vague and unhelpful.
I'm not gonna call anyone out, but if a review is flawed, there's more feedback someone can give than "the review doesn't hold up to our standards". That's basically the equivalent to "This level needs work 3/10".

Mable
Luigi
Luigi
Posts: 5806
Joined: Sat Dec 21, 2013 4:23 am
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby Mable » Sat May 28, 2016 1:54 pm

zlakergirl357 wrote:Where are we suppose to get any new "active" judges when the only thing people do on a level thread is "Nice gfx. 7/10"
Remember when you would get level judge but they said "Not perfect english no judge position"

HeroLinik
Larry Koopa
Larry Koopa
Posts: 3464
Joined: Fri Dec 20, 2013 1:28 pm
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby HeroLinik » Sat May 28, 2016 2:07 pm

Enjl wrote:To add to that, WHEN someone decides to give an elaborate review in an attempt to get the position, the reason for turning that person down seems to be incredibly vague and unhelpful.
I'm not gonna call anyone out, but if a review is flawed, there's more feedback someone can give than "the review doesn't hold up to our standards". That's basically the equivalent to "This level needs work 3/10".
Hang on. I applied for the position about a few months ago, when I was in the reviewing craze, doing levels left and right. Just about all of my reviews had been approved and the levels moved, so I pitched mine into Valtteri to get my position back, only to be turned down because there are too many judges right now, rather than because of a flawed review? Come on, what's the harm in adding another person to the roster? You can never have too many Level Judges!
Last edited by HeroLinik on Sat May 28, 2016 2:08 pm, edited 1 time in total.

Imaynotbehere4long
Boomerang Bro
Boomerang Bro
Posts: 1389
Joined: Thu Jan 23, 2014 3:00 pm

Re: I'd like to say something about the state of the Level Judges.

Postby Imaynotbehere4long » Sat May 28, 2016 2:08 pm

Re-posting what I said in the other thread:

Because being a Level Judge is something you have to apply for, it should be something you not only want to do, but also something you know that you'll be able to do on a consistent basis in the future. Requests shouldn't be necessary to get levels reviewed in a timely manner. If you don't want to review levels anymore, you should resign. If you find that you don't have enough time to review levels anymore (or at least be able to prioritize an average of one review per day), you should also resign. We're not expecting level judges to watch the levels forum 24/7, but at least watch it 0.015/7.

Yes, real life can be time consuming, but you know what isn't time consuming? Reviewing levels. With few exceptions, it only takes ~5-10 minutes to play a level and ~30 minutes to type a decent review if you're slow like me (maybe 60 if the level is relatively complex). I highly doubt anyone's life is so busy that they can't take out an hour per day to review a level.

Plus, it should go without saying that the priority should go toward reviewing older levels.
Valtteri wrote:Why does a level judge's review weigh more than that of a regular user's?
Because they're the ones who have the ability to move the level to another sub-forum. If five regular users like a level, post a well-made review about it, and give it a 6-8/10, but the level judge who reviews it hates the level and gives it a 2/10, the level is going into The Trash Can. Also, what Kuribo said.

Mable
Luigi
Luigi
Posts: 5806
Joined: Sat Dec 21, 2013 4:23 am
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby Mable » Sat May 28, 2016 2:38 pm

Imaynotbehere4long wrote:but the level judge who reviews it hates the level and gives it a 2/10, the level is going into The Trash Can.
That would be dumb and unfair as shit to the creator imo. If five regular like and give it a score like this and the judge gives it a 2 it doesn't mean he has the immediatly right to do this. The Creator will most likely complain about this and the other judges should decide if what he did was ok or not.

Don't tell me you wouldn't get mad if people like this and give a good score like a 8 and the judge gives it a 0 and moves it to trash even tho it's a good level.

zlaker
Reznor
Reznor
Posts: 2844
Joined: Fri Dec 20, 2013 1:46 pm

Re: I'd like to say something about the state of the Level Judges.

Postby zlaker » Sat May 28, 2016 2:48 pm

Except that the situation where a judge gives a level a 2 and other people gives it 6-8 is would never happen.

aero
Palom
Palom
Posts: 4787
Joined: Fri Mar 28, 2014 2:51 pm

Re: I'd like to say something about the state of the Level Judges.

Postby aero » Sat May 28, 2016 5:38 pm

zlakergirl357 wrote:Where are we suppose to get any new "active" judges when the only thing people do on a level thread is "Nice gfx. 7/10"
Open up the level forum to more than just Level Judges, that's where. No need to find anyone, no need to have judges, and reviews get done.
Enjl wrote:To add to that, WHEN someone decides to give an elaborate review in an attempt to get the position, the reason for turning that person down seems to be incredibly vague and unhelpful.
I'm not gonna call anyone out, but if a review is flawed, there's more feedback someone can give than "the review doesn't hold up to our standards". That's basically the equivalent to "This level needs work 3/10".
It's like there is no "standard" and it's completely arbitrary through and through while pretending to be objective. You give essays on levels pretty much, while others have color coded and categorized 10-scores for levels. This whole idea of leaving "constructive" reviews and utilizing professionalism jams the gears of the system when all you really need to do is be moving levels around in the first place. If the LJ group were just gotten rid of or re-purposed as "Level Moderators" for everyone to discuss, then all of the problems with the current system (inactivity, poor reviews, vagueness, long waits, inefficiencies, """"""""""real life"""""""""") would be dealt with in one go.

PixelPest
Link
Link
Posts: 7111
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby PixelPest » Sat May 28, 2016 6:59 pm

AeroMatter wrote:
zlakergirl357 wrote:Where are we suppose to get any new "active" judges when the only thing people do on a level thread is "Nice gfx. 7/10"
Open up the level forum to more than just Level Judges, that's where. No need to find anyone, no need to have judges, and reviews get done.
Enjl wrote:To add to that, WHEN someone decides to give an elaborate review in an attempt to get the position, the reason for turning that person down seems to be incredibly vague and unhelpful.
I'm not gonna call anyone out, but if a review is flawed, there's more feedback someone can give than "the review doesn't hold up to our standards". That's basically the equivalent to "This level needs work 3/10".
It's like there is no "standard" and it's completely arbitrary through and through while pretending to be objective. You give essays on levels pretty much, while others have color coded and categorized 10-scores for levels. This whole idea of leaving "constructive" reviews and utilizing professionalism jams the gears of the system when all you really need to do is be moving levels around in the first place. If the LJ group were just gotten rid of or re-purposed as "Level Moderators" for everyone to discuss, then all of the problems with the current system (inactivity, poor reviews, vagueness, long waits, inefficiencies, """"""""""real life"""""""""") would be dealt with in one go.
Just to put it straight up: I think that's an awful idea. Who do you expect to discuss things in detail enough to come to a good conclusion? Who cares about a perfect standard as long as each of the judges' different styles addresses both the positive and negative aspects of a level and gives constructive criticism as well as a fair score. What do you really think will come out of this?

aero
Palom
Palom
Posts: 4787
Joined: Fri Mar 28, 2014 2:51 pm

Re: I'd like to say something about the state of the Level Judges.

Postby aero » Sat May 28, 2016 7:13 pm

It's not a problem on any of the other forums here. What will be different about the levels forum?

PixelPest
Link
Link
Posts: 7111
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby PixelPest » Sat May 28, 2016 7:31 pm

AeroMatter wrote:It's not a problem on any of the other forums here. What will be different about the levels forum?
Are you referring to having levels still be moved or no?

Artemis008
Boomerang Bro
Boomerang Bro
Posts: 1346
Joined: Mon Jul 06, 2015 4:34 pm

Re: I'd like to say something about the state of the Level Judges.

Postby Artemis008 » Sat May 28, 2016 7:33 pm

What if we just have a poll in every level that decides its category based on the community's vote. This would not only mean faster reviewing but would also mean less biased reviews. Yes, I understand biased LJ's aren't too much of an issue, but it's better safe then sorry IMO.

PixelPest
Link
Link
Posts: 7111
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby PixelPest » Sat May 28, 2016 7:39 pm

Artemis008 wrote:What if we just have a poll in every level that decides its category based on the community's vote. This would not only mean faster reviewing but would also mean less biased reviews. Yes, I understand biased LJ's aren't too much of an issue, but it's better safe then sorry IMO.
Wouldn't that bring about more bias? Also, if we poll levels, people can vote on their own polls, people may vote poorly because they don't like the person for some reason, or people will vote highly for the wrong reasons. All in all, I don't think polls are the way to go

aero
Palom
Palom
Posts: 4787
Joined: Fri Mar 28, 2014 2:51 pm

Re: I'd like to say something about the state of the Level Judges.

Postby aero » Sat May 28, 2016 7:47 pm

PixelPest wrote:
AeroMatter wrote:It's not a problem on any of the other forums here. What will be different about the levels forum?
Are you referring to having levels still be moved or no?
I'm referring to reviews. Read what I said in the other thread, reviews would emerge from the attitude of the thread and not have the same nonsense behind them that they do now. Level Judges would only be necessary for moving the threads and nothing more. The subforums themselves are labeled with attitudes so it carries over well from the attitude of a thread. For example if there's a lot of criticism, it would be up to you to move it to the Trash Can or Average and if it gets a lot of good feed back then you would move it to The Best or Pretty Good. In between this is where the pros and cons of a level would be discussed solving Linik's issue with the total focus on the cons and it gives better insight to the level maker, and would bring activity to the forum itself so it doesn't look like the Introductions forum used to. Everybody wins.

HeroOfRhyme
Boomerang Bro
Boomerang Bro
Posts: 1382
Joined: Sat May 17, 2014 12:47 pm

Re: I'd like to say something about the state of the Level Judges.

Postby HeroOfRhyme » Sat May 28, 2016 8:38 pm

Just for my two cents, when I was a level judge I did more moderating than reviewing. Best LJ 2014.
Spoiler: show
‎________$$$$
_______$$__$
_______$___$$
_______$___$$
_______$$___$$
________$____$$
________$$____$$$
_________$$_____$$
_________$$______$$
__________$_______$$
____$$$$$$$________$$
__$$$_______________$$$$$$
_$$____$$$$____________$$$
_$___$$$__$$$____________$$
_$$________$$$____________$
__$$____$$$$$$____________$
__$$$$$$$____$$___________$
__$$_______$$$$___________$
___$$$$$$$$$__$$_________$$
____$________$$$$_____$$$$
____$$____$$$$$$____$$$$$$
_____$$$$$$____$$__$$
_______$_____$$$_$$$
________$$$$$$$$$$

PixelPest
Link
Link
Posts: 7111
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Contact:

Re: I'd like to say something about the state of the Level Judges.

Postby PixelPest » Sat May 28, 2016 8:49 pm

AeroMatter wrote:Reviews would emerge from the attitude of the thread and not have the same nonsense behind them that they do now.
How is there any nonsense in the reviews there are now?
AeroMatter wrote:Level Judges would only be necessary for moving the threads and nothing more. The subforums themselves are labeled with attitudes so it carries over well from the attitude of a thread. For example if there's a lot of criticism, it would be up to you to move it to the Trash Can or Average and if it gets a lot of good feed back then you would move it to The Best or Pretty Good.
This seems even more innefficent than the current method. What you seem to be wanting is discussion over every single level in which multiple people come to the same conclusion, the proper conclusion too, and then a Level Judge will move it. Who's going to be having these discussions? How are they in themselves going to be accurate or unbiased?
AeroMatter wrote:Everybody wins.
Except for the people whose levels are unpopular, the people whose levels are unfairly (and unrightfully in this manner) judged by the community, the people whose levels are judged by the screenshots, and the people who approve of the current system now. Some people will win too much who just throw a crap-ton of interesting graphics in their level or who are well-known level designers whose work people will automatically say, "Sure. It's probably good. Let's put it in The Best."

I have no idea how you conceive this to be a good idea

Artemis008
Boomerang Bro
Boomerang Bro
Posts: 1346
Joined: Mon Jul 06, 2015 4:34 pm

Re: I'd like to say something about the state of the Level Judges.

Postby Artemis008 » Sat May 28, 2016 9:08 pm

PixelPest wrote:
Artemis008 wrote:What if we just have a poll in every level that decides its category based on the community's vote. This would not only mean faster reviewing but would also mean less biased reviews. Yes, I understand biased LJ's aren't too much of an issue, but it's better safe then sorry IMO.
Wouldn't that bring about more bias? Also, if we poll levels, people can vote on their own polls, people may vote poorly because they don't like the person for some reason, or people will vote highly for the wrong reasons. All in all, I don't think polls are the way to go
It would be like voting for president, a few people might be bias but the majority wouldn't. Basically, one person wouldn't effect the poll enough to change results significantly. It would take a large group of people to change the score.
So, it would work like this.
A poll would be held for your level, it would consist of three options.
-Trash
-Average
-Best
The poll would end after 14 people have voted, so it cant be split 5/5/5

Let's say Steve wants the creator of this level to have a bad time. He votes Trash.
However, what Steve didn't realize was that he is just 1 out of 14 people. So well Steve might have voted Trash, 13 other people could vote the other options. Steve is not part of the majority vote so the creator of the level didn't have a bad time.

PersonNamedUser
Reznor
Reznor
Posts: 2882
Joined: Fri Feb 27, 2015 8:07 pm

Re: I'd like to say something about the state of the Level Judges.

Postby PersonNamedUser » Sat May 28, 2016 9:11 pm

I'll just join into this huge debate about the level judges:

I honestly think the judging system is ok as it is but I'm all for making a poll about how it should be set up, I also think we honestly should fire some of the judges, because some barley ever review levels or are too inactive to have that kind of position though I won't name any people. I once was going to try to apply for level judge but considering how important of a job it is, I problely would be inactive like most of the level judges are. And Also consider I eventually plan to host TMC2 I just wouldn't have time.

aero
Palom
Palom
Posts: 4787
Joined: Fri Mar 28, 2014 2:51 pm

Re: I'd like to say something about the state of the Level Judges.

Postby aero » Sat May 28, 2016 9:12 pm

If you spent as much time judging levels as you do asking rhetorical questions there wouldn't be that much of a problem as there is now. Level Judges are the arbiters of objective, fair, and accurate reviews, and this pathological disdain for improvement in this community shows this.

As I told Valtteri: run a poll.


Return to “SuperMarioBrosX.org”

Who is online

Users browsing this forum: No registered users and 4 guests

SMWCentralTalkhausMario Fan Games GalaxyKafukaMarioWikiSMBXEquipoEstelari