How to host a contest

Share guides you have written for SMBX, and related tools here.
Emral
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 9722
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Phoenix

How to host a contest

Postby Emral » Tue Nov 16, 2021 6:21 am

Hello! Welcome to this guide of assembled thoughts on hosting contests. I've observed, helped with and hosted quite a number of those over the years, and before the common pitfalls and considerations are forgotten I would like to archive them here.

Each spoiler contains one action that must, should or could be performed when it comes to contest hosting, and some guidelines for how to efficiently work with them. This list is assembled from the things I usually think about in the process of hosting a contest, and every step should be taken with a "if your idea is new and different, go for it!" disclaimer.
Most importantly: Remember to have fun and make it fun. We're all gamers, not bureaucrats.

Pre-Production: Settle on the format (theme)
Spoiler: show
So you're looking to host a contest, are you? That's awesome! Contests are competitions, usually in level design, where community members face off to present their prowess to the rest of the community.

When starting out with your contest, you probably already have an idea for what it should look like; a "theme", in other words. Some popular themes include the vanilla contest, where only pre-installed assets are allowed to be used, or the world map contest, where you face off in a competition on world map design. But the contest format isn't a stranger to weird ideas, such as contests where levels need to be built based off reference images, or where levels need to be built around certain word prompts, or combinations of predetermined items.

Regardless of what idea you go with, do keep the following in mind: Your idea for the contest's theme will drive every other decision in the process. Who to hire for judges? - A contest all about kaizo levels needs more skilled players to do the judging! What should the rubric look like? - If you include a special gimmick, it should be accounted for in the voting.

A good theme sparks creative ideas in the participant's mind while being open enough to be interpreted in many ways. With a theme such as "night", you know immediately what kind of levels to expect - standard levels set at night. However, a theme like "three" will lead to a much more diverse pool of levels. It is, in my opinion, a better theme because of that.

So choose your theme wisely! Theorycraft about how it may potentially turn out. Account for what might happen in the worst case: some themes (collab contest needs friends, lua contest needs coding skills) may receive a lower turnout due to favoring a specific subset of community members. Keep in mind that not all such downsides are inherently bad - just make sure you know what you're getting yourself into.

I also recommend always asking some friends about their thoughts on your ideas. Other people can offer up interesting new perspectives that can eliminate critical issues at an early stage, or make your initial ideas even better.
Pre-Production: Settle on the format (rules)
Spoiler: show
The rules of the competition come next, and are often quite simple means of allowing you to restrict and safeguard the content people are going to submit. Many contests restrict the SMBX versions allowed to level the technical playing field, or exclude certain gameplay elements like characters to avoid buggy submissions. For deciding on the rules, think of what you would NOT want to see under any circumstances.

Common rules include the ban of things that go against the forum rules, a restriction on the submission count, collaboration guidelines, permitted exit types, and a notice on joke levels.
Pre-Production: Settle on the format (judging method)
Spoiler: show
You also need to decide on what happens after submission. Most commonly, a roster of judges is gathered via direct invite or auditions, who will play through the levels, give their thoughts and assign a score to them with regards to a judging rubric (see below). Other forms of voting may be more suited for your competition, however. Contests may also only be judged by the host, or by the public. There are also further creative formats that recontextualize the idea of a contest as a whole. Medals (see below) can be a good incentive for turning a contest into a collaborative medal hunt.

Whichever method you choose, however, be prepared for its implications on the event. Judges may have to drop out if life throws them a curveball. You may have to intervene if one turns out to be incredibly rude in their reviews. The public vote needs moderation to prevent rigging, and easy methods of participation.
Pre-Production: Settle on the format (time frame)
Spoiler: show
The timeframe of the contest is commonly split into the participation-time and the judging-time. The participation time could be between a day and several weeks, and for highest participation is recommended to start on a friday and end on a sunday or monday, to give participants an extra weekend. The judging time is difficult to estimate ahead of time, but should be at a time when the judges are available (or in turn: hire judges that are available at the end of the scheduled participation period).
To ensure higher interest by community, the contest's participation period should also not overlap with common exam periods or submission periods of other contests.
Pre-Production: Settle on the format (medals)
Spoiler: show
Medals are fun userbars distributed for semi-arbitrary achievements throughout contests. They've been very popular since their introduction, and are frequently done in official contests since their introduction. However, they are neither mandatory to contests, nor are they restricted to official contests. If you want to go through the extra effort of distributing medals, you can decide to do so. However, before making that decision you should probably read the segment on creating these medals further below.

On top of deciding on whether you want to make medals in the first place, you should decide on how they will be distributed. Official contests usually go with a format where each judge can give up to one medal of their choice to each level (including a "favourite" medal). Depending on the format of your contest, a different approach might make more sense, though, so make sure to decide on what would suit your idea best.
Pre-Production: Create the rubric
Spoiler: show
The rubric is the metric by which levels are judged. While not every contest needs one, the vast majority do, and the rubric is often tightly interlinked with the theme, and the kinds of levels you wish to see in the contest. A story contest might put a large focus on a "narrative" scoring category, while a level with a gimmick may award points to use of that gimmick.

Rubrics should be easy to interpret by both the participants and judges. For example, historically the rubric "aesthetics" has been misunderstood as purely relating to visual fidelity, when by definition aesthetics also encompasses visual style, audio, and the interplay between audio and visuals. Similarly, "design" may put a disproportionately large focus on moment-to-moment gameplay, whereas "structure" may do the opposite.

Usually a rubric ends up with up to 5 or 6 parts, but don't be dissuaded from stepping outside that rough guideline. If you want to explicitly subdivide an "aesthetics" category of 20 points into 10 points for "fidelity" and 10 for "little touches", go nuts! Do make sure however that the totals are easy to quantify. Your judges may have a difficult time scoring categories with totals that aren't multiples of 5 or 10.
Pre-Production: Deciding the submission guidelines
Spoiler: show
In what format do you expect to be receiving levels? Should users include their usernames? Do you want readmes with additional information? Are there restrictions on the total size of the folder (in megabytes)? If you don't give clear instructions for submissions, you may end up with a very messy batch of submissions. Personally I like to include an image of the "ideal folder", for easy reference. Example:

Image
Pre-Production: Anonymous participation
Spoiler: show
When deciding on the rules, you may choose whether you would like participants to be anonymous or not to the judges. There has been much discussion around this rule, and ultimately it is a decision up to each host. However, when making your decision, make sure to keep the following in mind:
- Anonymity used to be argued for by the promise of guaranteeing an increased measure of fairness due to erasing a user's name impacting the judge's opinion on a level. However, from recent non-anonymous and anonymous contests, there is no evidence to support this
- Participants often have very distinct design styles and a judge who has been in the community for a while will be able to determine an author from the patterns in their submission, so even if there was a measurable bias, anonymous judging would not be able to erase it
Pre-Production: Anonymous judging
Spoiler: show
Since anonymity among participants is something contests occasionally do, you might have the idea to anonymize your judges (as well/instead), to make it harder for participants to cater to their tastes. Much like anonymous participation, this is something you as a host have to decide for yourself. However, when making your decision, make sure to keep the following in mind:
- Friends of chosen judges may send them updates to their work privately or in shared communities, and it will be hard for the judge to avoid it. Even in contests without anonymous participation it is wise to keep judge exposure to levels before their playthroughs to a minimum, to avoid preconceived notions. You will have to make sure that your judges do not have friends, which may be a difficult ask.
- It is still possible to appeal to the judge roster as a whole, even if judges are anonymized. The type of level that historically has been doing well in contests doesn't change much with a changing judge roster. If you intend to have different kinds of levels rise to the top of the rankings, you may be better off designing your rubric in such a way to favour those levels. See: Create the rubric (above).
Pre-Production: Deciding on penalties
Spoiler: show
Sometimes, participants don't play by the rules in some way or another, and it might feel right to penalize them for it. The early contests on the SMBX forums all had late-submission penalties, and point deductions for breaking various rules. Personally, I think all this accomplishes is that the results become inauthentic.
Penalties are often used as a means to push people to follow the rules, but what works just as well is to announce a penalty of disqualification for breaking a contest-specific rule (something like "level must be submitted before the deadline"; forum-rules-rules such as "no plagiarism" and "no mature content" should always result in disqualification), but then make an exception for any participant who breaks it, resulting in the push to follow the rules working, yet no penalties being handed out.
Pre-Production: Prepare additional materials (devkit)
Spoiler: show
Depending on the theme it might make sense to include a devkit with which levels are to be designed. Beware, however, that devkits always introduce new elements of human error, where people may not know how to apply it, may forget to not inlclude it in their submissions, or may - due to inexperience - copypaste devkit code into their own lua files, creating more work for quality control pre-judging to ensure every level works.
Pre-Production: Gather a roster of judges
Spoiler: show
If you are using judges, you must gather them sooner or later. This can be achieved by auditions or direct invite.
Directly inviting judges involves asking people in DM if they would like to judge. You should make sure to ask people who you think would do a good job, and who have time when the judging period starts.
You can also hold judge auditions. This process requires judges to volunteer and provide an example of their work, before being voted on by either you or the community. Commonly, judge auditions ask judges to use your rubric to judge a set of levels selected to demonstrate different kinds of levels that the judges may encounter in a competition. At the end of such an audition process, each judge had a chance to express their judging style and handling of different kinds of common levels. Common kinds included are "low-effort", "hard", "long", "janky" and "low gameplay".

When posting judge auditions, always make sure to communicate the judging timeframe and expected workload.
Furthermore it is important to note that when using judges, the judges should be banned from participating in the contest, to ensure an even playing field for all participants.
Pre-Production: If not using judges: Prepare your alternative voting method
Spoiler: show
Decide on what to do when levels are submitted early on so that you aren't scrambling for a solution later. For example: When using public voting, make sure to sketch out the format of a google forms form that will be used, and think about what fields you need for confirming a vote's integrity.
Pre-Production: Judging preparations
Spoiler: show
To shorten the time between the submission period and judging period, it is ideal to take a moment in this step to prepare the judging format. If you are using judges, I like to prepare a spreadsheet in this step through which reviews should be sent.
Here's an example in the form of the judging template for the chocolate contest 2021: LINK
This template contains statistic fields for every level for easy lookup, as well as columns for judging content in the middle, and some meta stuff on the right. Furthermore, a second spreadsheet for a rubric explanation (since the contest's rubric was pretty unique) and a third spreadsheet for judging guidelines.

If you are using public voting, I recommend setting up the format of the google forms form in this step.

In either case, the level names can obviously not be added yet. They will come in the step "Pre-voting: Judging preparations".
Pre-Production: Creating graphics for presentation
Spoiler: show
If you want to be fancy, you can create graphics for a logo, for headers in the forum posts, for the trophies that would be won by first/second/third place, and whatever else you can come up with. It adds flavour to the posts.
Pre-Production: Contest episode
Spoiler: show
If you want to make a contest episode, here is a good time to decide the form it should take. Since you are the host, you will not be participating in the contest. If you are also not a judge, you have the submission period and the judging period available to work on the contest episode, assuming no external obligations. Factor your availability in with your decision to make one, since if you aren't available during this period there is a high chance the episode may end up delayed indefinitely. And at that point, it's not worth announcing.
Announcement: Announcing the event ahead of time
Spoiler: show
If you want to, you can announce the event ahead of time, providing information necessary for people to gauge whether they'd be interested in it. I like to use this time to already collect signs of interest, and then I like to ping everyone who showed interest when the contest begins. Kind of like a manually set reminder.
Announcement: Introducing the judges
Spoiler: show
Either in the ahead-of-time announcement or in the eventual submission phase you can also optionally ask your judges for brief introductions to their style, preferences and history. A post including these introductions can be made so that people know what the judges are like and to make them more human (which might not be what you want in every situation).
Announcement: Creating the rules thread
Spoiler: show
For creation of contest threads I like to use the "drafts" feature on the forums.

The rules thread is the main place of discussion regarding the contest's guidelines, as well as the place for hosting explicit signups, should your contest theme require you to know when a user signs up (for instance when having to send them a PM with an individualized theme). As such, this thread should serve as a hub for all information participants might need to know. These commonly include (unless your contest has a theme that breaks the convention):
- The submission period
- The rules
- An explanation of the contest's theme, if any
- The judging method (& rubric)
- If you have a devkit, the devkit download and installation guide
- If you don't have a submissions thread, this thread also needs to house the submission guidelines
- If you made graphics for the contest's presentation, you can use them to adorn the thread

If you prepared your contest appropriately, creating this thread should be just a matter of putting everything you prepared into words easily understandable by the uninitated.
Announcement: Creating the submissions thread
Spoiler: show
For creation of contest threads I like to use the "drafts" feature on the forums.

The submissions thread is an optional thread for people to submit their levels in. The submission guidelines you prepared earlier should now be arranged here in an easy to parse format. Expect people to read these 0-1 times at the end of their submission, and never before then. As such they should be intuitive and easy to follow. The more complex they are, the more likely you will have some outliers that didn't follow the format and which you need to fix the format for yourself. That's generally fine, though, since the overall reduction in post-submission workload is still worth it.
Deadline: Reminders
Spoiler: show
It's good practice to post regular updates on how much time is left for submissions. Reminders can serve as reminders of the contest's existence for new participants as well, so it's worth posting them before the immediately encroaching deadline as well.
There is no set-in-stone formula for when posting a reminder is good, but one that I try to aim for is:
- one reminder at the halfway point
- one reminder one week before submissions end
- one reminder two or three days before submissions end (at the start of the final weekend)
Deadline: Considerations for a deadline extension
Spoiler: show
If multiple people show interest in a deadline extension, it may be worth considering. Shifts in the timetable have a ripple effect throughout the rest of the event, so when considering a deadline extension, ask yourself the following:
- Will the delayed judging period affect my judges' ability to judge all the levels? If so, is that effect negative or positive?
- Which delayed submission deadline will interfere with my ability to gather the submissions for judging the least?
- How long should the deadline extension be? (Commonly: 1-7 days)

If the deadline extension has a positive impact on the event, go for it.
Deadline: Halting submissions & late submissions
Spoiler: show
Once the submission deadline arrives, an announcement is to be posted informing everyone that submissions are now over.
Since participants are often allowed to edit their submission(s) before the deadline arrives, this is also the point where it's best to gather all the submitted files to prepare a package for judging.
The act of gathering all submissions will usually take a while, so when I announce the end of the submission period, I tend to do so with a leniency period during which late submissions are accepted. When I host, I tend to allow late submissions between 1 hour after submissions close, and the point where the package is sent to the judges. You don't have to accept late submissions at all, but if you do, I recommend not penalizing them, even if stated otherwise in the rules. In my opinion, the scoring of a contest should be entirely dictated by its scoring system.
Pre-voting: Quality control
Spoiler: show
The host should give the submissions one playthrough before relaying them to the judges and ensure the following:
- does all custom music play?
- is the start point in the correct position?
- are there easy-to-encounter lua errors?
- are all levels beatable? (theoretically - even if there is lag or unfairness. the lag/unfairness is for the judging to decide)
- if you have a readme system: does every level come with a readme? or: is the player character/exit count made clear to the judges for every level?

After fixing any issues you come across, the level pack is ready to be uploaded and soon distributed for judging.
Pre-voting: Judging preparations
Spoiler: show
Before uploading the levels, you can now look back on your documents from the Pre-Production step on judging preparations and add all the level names, exit counts, etc... to the document you prepared, finalizing it for judging use.
Pre-voting: Considerations for public vote
Spoiler: show
If you're using public voting, there are a few more considerations to make in this step. For a start: Public votes are a lot more chaotic and and prone to abuse. Voters may not have played the levels themselves, or try to sway the vote by voting with several accounts.
Ensure that your form has precautions in place to prevent duplicate voting, and, if possible, try to assemble a playlist of youtube videos showcasing playthroughs of all the levels, which people can watch as they cast their votes.
It should be as easy as possible for someone to vote, but impossible for someone to rig it. A good step to prevent rigging the vote is to ask people to include their discord tag with the vote, so that you can ask them if anything seems fishy.
Voting: Starting the judging period
Spoiler: show
Upon completion of the pre-voting steps, the judging period may begin. Post all relevant info either to the judges or to the public, and be ready to answer any questions that come up. Generally, though, if you're not judging yourself, this is an ideal time to finish a contest episode if you're doing one.
Voting: Gathering the results
Spoiler: show
If you are using judges, the results will trickle in one set at a time, allowing you to do some work on the results while other judges are still working. Consider using a master sheet to assemble the results and have spreadsheet calculations do the math for you:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0 (Chocolate Contest 2021 example)
In the case of using judging, you will also need to ensure that the results are formatted in a readable, searchable and complete manner. For Chocolate Contest 2021, I trialled Google Slides for this, which worked out wonderfully, so I'm putting it up as a recommendation:
https://docs.google.com/presentation/d/ ... XInUM/edit
Other forms of presentation include word documents, txt files, pastebin links and custom-coded html webpages (scope creep!).

Once the judging period is over, either via a set date in the public voting section or via all judges having submitted their voting documents, you can fill in such a master document with the final results.
Pre-Results: Creating userbars from medals
Spoiler: show
To create userbars, I use rixithechao's userbar editor tool, and paint.net/any pixel art tool for creating the medal graphics. By no means are you restricted to the format I use here, but the userbar editor is great for getting results quickly.
Once you have exported your userbars, you need to upload them all (I use imgur) and link them in the results thread using bbcodes. For easy navigation, I like to sort them alphabetically.
Pre-Results: Creating a video for the results
Spoiler: show
If you want to, you can record and post a video announcing the results, like I have done for contests this year. This is a couple of hours workload, potentially more if you have little experience with video production. However, if you have fun making videos, I would go for it! Just make sure it doesn't significantly delay the results. On the flipside, footage can be obtained at any point after the start of the judging period, and if you did public voting you likely already have enough.
Results: Posting the results thread
Spoiler: show
Once you have everything you want to post, it's time to format the results thread. Contents should include:
- The top placements
- Either embedded or linked-to detailed results breakdowns (master sheet)
- Judge comments if you have judges (uniform format is ideal)
- Medals, if you have them
- The results video, if you have it
- A thank you notice
- If no contest episode is released at the same time, a pack containing all submissions should be linked here as well

If you also made a contest episode, you can cross-reference it between here and the episode thread. If you aren't done with the contest episode yet, post it later.
Post-contest: Creating a contest episode
Spoiler: show
If you decided to make a contest episode early on, you should have most, if not all of it finished at this point. The last steps are the laborous tasks of adding all the results you just copypasted to a million spreadsheets into SMBX. I heavily recommend using lua for this, as the SMBX message system was not designed for large loads of text.
If you are only now deciding to have a contest episode, I recommend using something like Contestmapsystem to input all the data quickly.
Post-everything: Wrapping up
Spoiler: show
Congratulations! You have hosted a contest. How was it? Did everyone have fun? Would you like to do it again? Give everyone a pat on the back, and yourself two. Then, take a break.

Return to “Guides”

Who is online

Users browsing this forum: No registered users and 2 guests

SMWCentralTalkhausMario Fan Games GalaxyKafukaMarioWikiSMBXEquipoEstelari