System of the next contest

[Raster]

It's probably a bit too early to talk about how the next contest will be run, but anyway, here's what I think: I think you should get rid of the blind judging system and remove the 'don't post any detail about your level' rule.

Ignoritus said that the judges might give certain users (who are famous for their good levels) a higher score although their levels' quality is questionable. I think it's time we mature as a community and get rid of these misconceptions. It's the host's duty to hire judges who are impartial and judge levels objectively. If those judges prove to be inefficient, count them out.

If we keep going on with this system the contests will never really evolve.

Also, another reason why the 'blind judging' system was put in place is because judges might rig the score to aid their friends. That point is moot because if a judge wants to rig the score they can just ask that user what their level is called.

<Emral> EVEN IF JUDGES CAN TELL WHICH LEVEL IS YOURS IT DOESN'T MEAN THEY'RE BIASED AND WILL CHANGE THEIR SCORE DEPENDING ON THAT

So, what do you think of my idea? Should we implement this new system or not?

[zlaker]

I can't see the reason for not posting any screenshots or anything like that of the level. And we should remove the blind judging system. So I'm agree with ya Raster, this system should be implemented.

[glitch4]

because SMWCentral does 80-point system on the contest (each 8 worts 1 score), I want 100 score like this because it's better then some "." on it since they're annoying.

I agree with zlakerboy357 as well.

[Raster]

because SMWCentral does 80-point system on the contest (each 8 worts 1 score), I want 100 score like this because it's better then some "." on it since they're annoying.

I agree with zlakerboy357 as well.

Ah, you mean scores from 1 to 100? I can agree with that. It gives the judge more freedom when scoring a level.

[glitch4]

Yeah sure. And it would be awesome if designer would put judge's scores like SAJewers did in MAGLx.

[Ignoritus]

Ignoritus said that the judges might give certain users (who are famous for their good levels) a higher score although their levels' quality is questionable. I think it's time we mature as a community and get rid of these misconceptions. It's the host's duty to hire judges who are impartial and judge levels objectively. If those judges prove to be inefficient, count them out.

I have told you repeatedly it has nothing to do with impartiality. It has to do with natural inclinations people have and can not control. If you go into a level expecting it to be good, you're in a good mood, and view the level in a more positive light. If you go into the level expecting it to be bad, you're going to take greater notice of nitpicky flaws in order to verify your prejudiced belief. Some people may be affected by this less than others, but it still a factor no matter who is judging.

Furthermore, the host didn't hire judges based on any factors other than "they're one of our level judges". Which I might add is a really questionable system to begin with.

If we keep going on with this system the contests will never really evolve.

Back up this claim with evidence.

Also, another reason why the 'blind judging' system was put in place is because judges might rig the score to aid their friends. That point is moot because if a judge wants to rig the score they can just ask that user what their level is called.

This is a meaningless point, as I never said this was a reason why it was put in place to begin with.

So, what do you think of my idea? Should we implement this new system or not?

No. I feel it removes the participants sense of security and makes them feel like they don't legitimately have a chance against the "big guys" in the community who are practically assured good placements before their level is played.

I don't want to start drama with anyone so I won't name names, but I can say there was at least one instance I know of where a designer was known in a contest and a judge gave a very questionable score on very questionable logic compared to their other reviews.

[FanofSMBX]

Evidence? I'll give you evidence. Note how the Tower of Biased series is pretty much always "(notable user) makes a hub and all the levels are randomly put in tada I made an episode". That's proof that these contests are all alike, because Tower of Biased can be made alike.

[Ignoritus]

Evidence? I'll give you evidence. Note how the Tower of Biased series is pretty much always "(notable user) makes a hub and all the levels are randomly put in tada I made an episode". That's proof that these contests are all alike, because Tower of Biased can be made alike.

Uh.

Your evidence is that our level design contests are composed of people submitting levels they designed?

[MistakesWereMade]

Evidence? I'll give you evidence. Note how the Tower of Biased series is pretty much always "(notable user) makes a hub and all the levels are randomly put in tada I made an episode". That's proof that these contests are all alike, because Tower of Biased can be made alike.

That has literally nothing to do with what ignoritus said, we're talking about the /contest system/ not the fucking hub

[Raster]

I have told you repeatedly it has nothing to do with impartiality. It has to do with natural inclinations people have and can not control. If you go into a level expecting it to be good, you're in a good mood, and view the level in a more positive light. If you go into the level expecting it to be bad, you're going to take greater notice of nitpicky flaws in order to verify your prejudiced belief. Some people may be affected by this less than others, but it still a factor no matter who is judging.

You're looking too much into this and you also ignored the points I made in the OP. I brought up impartiality because a lot of members think the contest is 'biased' and I wanted to dismiss that.

If someone is incapable of judging a level objectively then that person shouldn't be a judge.

Furthermore, the host didn't hire judges based on any factors other than "they're one of our level judges". Which I might add is a really questionable system to begin with.

And I can agree with that, though not all of the judges are actual Level Judges as far as I'm aware.

This is a meaningless point, as I never said this was a reason why it was put in place to begin with. You said that was why, I corrected you, and apparently you ignored my correction and are standing by your belief you conceived before you had knowledge of the truth. Hmmmmm, kind of exactly what I'm talking about isn't it?

That's kind of meaningless considering I have direct proof of a judge saying he'd be inclined to rig the score of a friend's level if they knew what the level is called.

That's also one of the main reasons why most of the users think the system was implemented.

No. I feel it removes the participants sense of security and makes them feel like they don't legitimately have a chance against the "big guys" in the community who are practically assured good placements before their level is played.

That's ridiculous. You don't need to remove the blind judging system to make certain users feel that way. Not to mention all contests are based on competition and you can't change that.

Evidence? I'll give you evidence. Note how the Tower of Biased series is pretty much always "(notable user) makes a hub and all the levels are randomly put in tada I made an episode". That's proof that these contests are all alike, because Tower of Biased can be made alike.

Tower of Biased has nothing to do with this.

[Ignoritus]

You're looking too much into this and you also ignored the points I made in the OP. I brought up impartiality because a lot of members think the contest is 'biased' and I wanted to dismiss that.

Please show me exactly which points I've ignored?

If someone is incapable of judging a level objectively then that person shouldn't be a judge.

Ok, now you're just ignoring me. My point is that no one, no matter how "objective" can overcome these sorts of notions.

That's kind of meaningless considering I have direct proof of a judge saying he'd be inclined to rig the score of a friend's level if they knew what the level is called.
That's also one of the main reasons why most of the users think the system was implemented.

Neither of those change the fact that that isn't one of the reasons blind judging was instated. Though I certainly agree it is concerning that a judge would say that. Has said judge been dealt with?

That's ridiculous. You don't need to remove the blind judging system to make certain users feel that way. Not to mention all contests are based on competition and you can't change that.

This doesn't really make sense at all as a response to what I said. You might want to reread what I said, and if that's still your response perhaps reword it?

[Raster]

Please show me exactly which points I've ignored?

I think it's time we mature as a community and get rid of these misconceptions. It's the host's duty to hire judges who are impartial and judge levels objectively. If those judges prove to be inefficient, count them out.

Ok, now you're just ignoring me. My point is that no one, no matter how "objective" can overcome these sorts of notions.

What evidence do you have though? Even if that really is true, the score wouldn't be affected much, if at all (considering the judge in question is fit for the job).

Neither of those change the fact that that isn't one of the reasons blind judging was instated. Though I certainly agree it is concerning that a judge would say that. Has said judge been dealt with?

Well, most people thought that the blind judging system was implemented to avoid 'bias'. That's a commonly accepted idea in the SMBX community. Also, I don't think that judge was penalized. It is likely that Valtteri didn't even read that post.

This doesn't really make sense at all as a response to what I said. You might want to reread what I said, and if that's still your response perhaps reword it?

Well, you said:

No. I feel it removes the participants sense of security and makes them feel like they don't legitimately have a chance against the "big guys" in the community who are practically assured good placements before their level is played.

That was your answer to my question (do you think blind judging should be removed?). I don't think blind judging removed that issue. Certain participants are still going to be frightened by the idea of competing with the 'big guys' anyway. That's a good thing in my opinion because it builds up confidence over time and encourages participants to design better levels.

[Ignoritus]

I think it's time we mature as a community and get rid of these misconceptions. It's the host's duty to hire judges who are impartial and judge levels objectively. If those judges prove to be inefficient, count them out.

I did respond to this. It doesn't matter who you hire, there's an extent of prejudice that they can't control.

What evidence do you have though? Even if that really is true, the score wouldn't be affected much, if at all (considering the judge in question is fit for the job).

A combination of basic psychology and experience. Bear in mind I've judged a fair deal of SMBX contests at this point.
And while I could argue the extent to which it affects scores with you all I want, I'll jump straight to the point: Blind judging is virtually harmless, so if you're admitting it can have an effect on the scores, why would we remove it?

Also, I don't think that judge was penalized. It is likely that Valtteri didn't even read that post.

I think it should be brought to his attention then. To be honest, I'm beginning to see several questionable practices being employed in this contest.

I don't think blind judging removed that issue. Certain participants are still going to be frightened by the idea of competing with the 'big guys' anyway. That's a good thing in my opinion because it builds up confidence over time and encourages participants to design better levels.

Yes, they will still be intimidated under blind judging, but only because they're legitimately outskilled by the more experienced designers. Not because the more experienced designers get to ride their reputation.

[Raster]

A combination of basic psychology and experience. Bear in mind I've judged a fair deal of SMBX contests at this point.
And while I could argue the extent to which it affects scores with you all I want, I'll jump straight to the point: Blind judging is virtually harmless, so if you're admitting it can have an effect on the scores, why would we remove it?

I also speak from experience as I judged and hosted a contest with over 30 participants. Frankly I think blind judging is unnecessary and just additional work for the host, and the results of such contests are likely the same as the results from contests that do not utilize the blind judging system. It also means we cannot trust our own judges, which is unsettling to me (and others, probably). Contests from other Mario level-making communities don't have this system and things seem OK there.

Yes, they will still be intimidated under blind judging, but only because they're legitimately outskilled by the more experienced designers. Not because the more experienced designers get to ride their reputation.

Not entirely true; it's possible that the judges recognize the levels' authors anyway. For example, a judge recognized one of the SMBX:R contest level creators because the level title used alliteration. Designers likely use gimmicks and mechanics that they previously used so it's not really impossible for judges to recognize the creator.

[Ignoritus]

I also speak from experience as I judged and hosted a contest with over 30 participants. Frankly I think blind judging is unnecessary and just additional work for the host, and the results of such contests are likely the same as the results from contests that do not utilize the blind judging system.

To establish my credentials, I've judged 3 of these community contests with around 30+ contestants each, as well as MaglX with a whooping 134 contestants. And I'd say that the results are not totally negligible and, having hosted 4 of these contests myself, that the amount of extra work from blind judging is not significant enough to justify not employing it.

It also means we cannot trust our own judges, which is unsettling to me (and others, probably). Contests from other Mario level-making communities don't have this system and things seem OK there.

I reiterate once more that it has nothing to do with trusting our judges. This is a function that they cannot control. Furthermore, this isn't the type of thing that's easy to see at a base level. You'd need to know all of the contestants, all of the judges, read the reviews carefully, play the level, and even then there would be outliers that just happened to amuse the judge for reasons you can't identify. It's hard to make a solid case that someone's opinion was formed illegitimately because it's an opinion.

Not entirely true; it's possible that the judges recognize the levels' authors anyway. For example, a judge recognized one of the SMBX:R contest level creators because the level title used alliteration. Designers likely use gimmicks and mechanics that they previously used so it's not really impossible for judges to recognize the creator.

Oh yes, it's totally possible and usually happens in a few cases in each contest. But there's nothing we can do about that. We do what we can though. We don't just say "well we can't do it perfectly so why bother try at all".