BN Test Review (osu!taiko, catch and mania)

Hello! As you're probably all aware, we have recently welcomed a new batch of promising modders from osu!taiko, osu!catch and osu!mania to the Beatmap Nomination Group!

Information on what goes into the Beatmap Nominator test and selection process can be found in this thread, but this time we thought it would be cool to share with you some of the results of the recent round of tests, such as the total number of applications, average scores, and common mistakes.

Additionally, we'll share some thoughts on how the testing process went from our perspective, what we think people did well at, and what needs to be improved for the next round. Hopefully this should be of interest to those of you thinking about applying for the BNG in the near future!

So without further ado, let's get right into it!

osu!taiko (by DakeDekaane)
Total # of candidates: 15
Passing candidates: Chromoxx, snowball112

I was kind of excited after having 15 people solving the first test created by me in it's totality. It's worth saying that I liked this new format, as it simulates the real deal, instead of open questions which made applicants guess what we wanted them to write. Well, they still have to write what we want, but I consider it isn't as restrictive as before, yadda yadda.

Most of applicants did pretty good on Part A, as the average score was 39.8 points, demonstrating that they had a very good knowledge of Ranking Criteria, or know where to consult it.

Sadly, the average score for Part B was 22.2 points. Most of the test score was distributed on kind of obvious issues for people who know how osu!taiko works: wrong note snapping, inconsistent uninherited timing points (red lines), ridiculous difficulty spikes, notes being at the same place, mapping ignoring song pace, difficulty settings, kiai placement were the more notables. There were a considerable amount of candidates who missed one or two of said issues, the most common being the timing points.

The rest of the test was focused on minor issues or specific things that could be improved, like spinner placement: apparently it's okay to end spinners on a particular beat that could fit better as the beginning of a pattern/section (hint: it's not, check that spinner near the end in [Difficulty 2]), 1/6 pattern usage, monotonous mapping, awkward/wrong note placement or patterns, overusage of big notes and the capability of map being richer in notes (only one person got this). Unfortunately, these issues were hardly mentioned.

There were some some things based on general Ranking criteria, like unused files, metadata, difficulty naming, etc., which gave the incredible score of zero - you were supposed to get those points in Part A, right? Despite that, a small bonus was awarded for that, along with a bonus for mod quality.

Oh boy, the mod quality. Some mods were really detailed in every point improving the beatmap and fixing issues, some were brief but directly mentioned the issues and a detailed way to fix it (thanks for making the checking easier!), some more were balanced, and some more improved the beatmap but didn't mention the issues (some were really tiring!). Very few mods lacked explanations or were confusing. This overall perception about applicants' mod quality makes me happy, their ability to identify issues not as much, sadly.

I take the opportunity to congratulate the passing candidates, you did a good job! Those who did not, don't be afraid to try again - the more we practice and the more we ask, the better we become. To finalize, the results of this test helped to identify one of the many issues that hinder the osu!taiko community and there will be a surprise mechanic aiming to combat this, but I'm not going to give you any spoiler, so stay tuned!

osu!catch (by JBHyperion)
Total # of candidates: 5
Passing candidates: -Sh1n1-, Absolute Zero, Electoz

I had mixed-to-positive feelings about this round, which was my first in charge of test creation along with Deif. Considering 3 of our existing BNs have moved on to focus on other things in recent weeks, it was great to minimise the impact by bringing in some new guys to replace them. However, it was a bit of a shame to see that the number of candidates was quite low this round. A number of people who I expected to take the test failed to meet the required activity score, so I hope they get back to actively modding over the next few months in preparation for a test in the new year.

Most candidates demonstrated a reasonably good grasp of the Ranking Criteria, with an average score of 36.6 on Part A. Questions regarding backgrounds, hitsounds, metadata and BNG rules were generally answered particularly well, whereas candidates tended to struggle with questions related to skins, storyboards and how to handle mapsets with multiple contributors. Some common problems included confusion as to how to handle storyboard load > 5.0x, acceptable creator-to-guest mapper contributions, and also the use of widescreen support.

Candidates didn't perform quite as well in the modding section of the test, with the average score for Part B being 28.1. Additionally, no candidate scored more on Part B than their respective score for Part A. Whilst Cup and Salad difficulties were generally handled well, candidates seemed to struggle with the higher diffs, with a number of major errors such as overmapping, broken flow and improper edge/hyperdash usage being missed or ignored. Perhaps worryingly, only one of the five candidates pointed out a 1/4 hyperdash in the Platter difficulty, which at the song's BPM of 175 is not only an established unrankable issue, but an easily detectable one. Make sure to keep the difficulty-specific Ranking Criteria on hand in future guys! Saying that, it wasn't all doom and gloom however, as the overwhelming majority of suggestions provided were clearly explained and well-reasoned, and [General] problems such as incorrect metadata, inconsistent timing and poor difficulty setting usage were almost all pointed out, with the only important miss being a delayed hitsound file.

All candidates who expressed interest in receiving feedback on their tests have received it, and we'll be working closely with the new BNs over the coming weeks and months to help them settle into their new role. To those who were unsuccessful, either at the application or test stage, don't fret! You'll get another chance to apply in a few months, provided your modding activity and behaviour stays above board. Ask questions, strive to improve your skill, and burn the Ranking Criteria into the retina of your eye! (:

Congratulations to our passing candidates, and good luck to everyone applying next round - I look forward to working with each and every one of you, both now and in the future, to make osu!catch mapping and modding a diverse, exciting and friendly place for all!

**osu!mania (by Blocko)
**Total # of candidates: 25
Passing candidates: Critical_Star, Dellvangel, DixonBlackwing, juankristal, Protastic101, Rivals_7, SanadaYukimura

I was really surprised when I heard that there will be 25 candidates applying for BN in osu!mania. That's the highest number we've gotten so far over the last 2 tests, and I was very excited because so many people applied! With that said, for the second part of the whole test, instead of ten 10-second segments, the candidates had to mod a full mapset to fully test their modding capabilities. Part B contains issues that should be prevented when it comes to ranking a mania mapset, which can include wrongly snapped notes, absurd difficulty spikes, unpredictable slider velocities, inconsistent timing points and hitsound errors, to name a few. We'll get to how people did on the test right about... now!

The average score is 33.6 for Part A, which is not too shabby. Based on the results, people in this round have a fairly good amount of knowledge with the Ranking Criteria. Storyboard, metadata, drain time and hitsound related questions were a bit of a bump on the road, but others, including backgrounds, skinning and handling beatmaps with multiple creators, were generally done fairly well. Not bad!

Part B's average is 25.6 though, which isn't up to par with Part A. A few number of applicants submitted their test but left it unfinished, so that hindered the score average by a margin. This also reduced the average for Part A to a small extent. If you're planning to take the BN Test in the next round, be sure to take the test with enough time on your hands and with the best of your ability next time!

The majority of the candidates did pretty well on the harder difficulties, covering errors such as huge difficulty spikes and extreme slider velocities, which can be easy to detect judging by the results. The struggle can be seen in the easier difficulties, as some of the most important things missed by a number of candidates were hitsound errors, relatively high snap usage and unconventional pattern usage. Furthermore, the score average is higher for EZ than NM, which is quite worrisome considering not a lot of people did so well on that difficulty. Even with that in mind, snapping errors in the whole mapset were detected by a good number of candidates, and those can give you quite a number of points considering how the ranking system of osu!mania works.

Mod quality really depends on how the candidates explain their suggestions, and there are a few that I found outstanding in quality because those candidates provided explanations to their suggestions that were clear and concise. Some kept their suggestions short and to the point, and some only suggested to move, delete or add a note. Remember to flesh out your mod with some clarification to your suggestions so it's clear what idea you're trying to convey to the mapper!

A small bonus is awarded for checking metadata, timing points, giving suggestions outside of the mapset (which include HP, OD, countdown and storyboard, etc.) and generally giving good suggestions. Even reminding the mapper to check Aimod in case there are errors around the mapset would give a bonus as well. Some of these concepts may have already been graded in Part A, but it helps to show that you have the Ranking Criteria in mind for Part B and applying that knowledge to your mod.

I've received some responses from candidates that said it took 3~5 hours for them to finish Part B, and possibly even more than that, but fret not! From what I can tell you, the next test will not be as long as this one (I hope), so stay tuned. Also, Feerum gave me a hand with test preparations and checking, so I'd like to give him a huge thanks for helping me out!

I'm proud that there are people I know that have passed the test and are currently bubbling and qualifying maps as BNs, but for those who failed the test, it doesn't hurt to try again to become a better modder, right? As a word of advice, keep modding. Doing that in turn will improve your modding skills and perhaps will let you see another perspective toward a mapset. Remember that there's always a next time for these kinds of things! Once again, congratulations to the passing candidates, good luck, and remember to have fun as full-fledged Beatmap Nominators!

So, this brings our recap of the latest BN applications to a close. Hopefully you found this interesting and informative, and of course feel free to ask if you have any questions! For those of you who are planning to try again next round, or perhaps attempt the BN test for the first time, be sure to keep the test creators' comments in mind and give it your best shot! All of us look forward to seeing even more success from taiko, catch and mania modes on the next round in a few months time, but for any of you who mod standard, your chance will be coming up very soon, so stay tuned!

—JBHyperion