BN Test Review #2

Hello! Our modding community has been hard at work for the past four months, pushing the quality of mapping forward every day in order to give the playerbase new and exciting content to enjoy.

For those that want to go the extra mile and have the final say on what maps go on to become official osu!content, the Beatmap Nominator test is at the top of their to-do list. Through tribulation to eventual triumph, we are proud to again welcome a new batch of promising modders from osu!taiko, osu!catch and osu!mania to the Beatmap Nomination Group! As always, information on what makes up the Beatmap Nominator test and selection process can be found in this thread, but if you want to see the nitty gritty details on all the applicants, responses and results, then look no further!

Additionally, the test creators will again share their insight on the creation and testing process, noting what people excelled at, and what people should brush up on for future tests. If you're considering an application for the BNG in the near future, be sure to give this a read and prepare yourself for the trials ahead!

So without further ado, let's get right into it!

osu!taiko (by OnosakiHito)

Total # of candidates: 19

Passing candidates: Arrival, Doyak, Jonawaga, Kin, Skylish, Stefan, Surono, thzz

While the outcome of this test could have been different at any time, I would say that everything was according to the Keikaku. In the last months and years, since the new evaluation system kicked in, we had the chance to observe and gain data from the past Taiko BN Tests which showed a very high fail rate. I was always wondering why that is, until I actually took one of these tests by myself and noticed how hard they really were. I can't say if it was intentional, but there were many points I had to think about twice or thrice for myself whether they were an issue or not, and that made me actually realise how much of a problem for a fresh modder it could be to take these tests in Part B and why the fail rate might have been that high.

According to this knowledge, we changed the difficulty to make it closer to what we expect for entry into the BNG. That's why Part B contained mostly basic issues, like for example excessive overmapping, two notes at one spot, broken Slider Velocities or just very poor flow. The thing is, being a Beatmap Nominator is also a learning process that doesn't end with passing the Test. It's a process that goes far beyond that and which actually forms a Nominator over time. This is what made us reconsider the current structure of the Part B test - modders should prove that they know at least the very basics in modding and have the possibility to prove themselves as BN later on through their additional suggestions.

Now I can say that this change has obviously given us some fresh air in the Taiko ranking process, as we got seven new BNs which I want to congratulate at this point. Overall, most of applicants did pretty well on Part A, as the average score was 32.3 points, demonstrating that they had a good knowledge of the Ranking Criteria, or know where to consult it.

As for Part B, the average score was 25.6 points. While we have seven new BNs, most people still missed a few crucial points which must have not been ignored. For example, only 4 out of 19 people managed to find the broken barlines / Slider Velocities in the test, despite these being very obvious errors noticeable by simply test playing the difficulties. Others ignored the metadata checks, or simply the fact that we were using too quiet volume samples (only 6 out of 13 noticed this!). Another problem area we observed was with the modding itself: Some people lost impression points by simply not explaining or giving reasons why they suggested a certain change. Sometimes the mods contradicted themselves too, by criticizing a single pattern as being overmapped, while suggesting a whole new part on another place which is much harder. What worried us the most though was the suggestion to disable widescreen support. I will disable everyone who writes that again! This has no effect when unticking, aaaaaah!

On a more positive resonance, most people actually know a good portion of the basics in Taiko, such as having lower SVs rather for lower difficulties on high BPM songs, or not using ¼ beat snap in these. Also when thinking about certain pattern constellations like oddly placed finishers or the very obvious overmapping In the Inner Oni difficulty, nearly everyone found these out.

Overall, I'm fine with the outcome of this month's test and will wait and see what is going to happen in the next weeks and months. The BNs have overcome their first hurdle. Now the real work begins!

osu!catch (by JBHyperion)

Total # of candidates: 8

Passing candidates: Benny-, F D Flourite

For my second round in charge alongside Deif It was great to see a few more applicants, but I still hope to see further improvement in this area ready for the next cycle, since in my opinion our gamemode is lagging behind osu!taiko and osu!mania quite considerably in this aspect. Overall I have to draw on the positives though: we might have lost one BN in Electoz, but we have gained two more this cycle, so we're at least moving in the right direction.

Overall, all candidates demonstrated a reasonable to good understanding of the Ranking Criteria, with an average score of 37.4 on Part A, which was a slight improvement over the previous round of tests. Candidates were on average, more knowledgeable regarding spread management and metadata. Additionally, custom skin elements were handled rather well in contrast to the previous test cycle. On the other hand, questions related to the nomination process and consistency of settings (e.g. widescreen support, combo colours, etc.) were handled poorly overall, with correct use of timing points appearing to be a noticeable deficit in knowledge.

Despite a lower score on Part B of 29.4, I found that the modding section was generally handled well, especially when compared to results of the previous round where many marks were lost on fairly obvious unrankable issues. In contrast to the previous test cycle, the spread used in this round was Salad-Overdose, with modders performing best on Overdose and especially Platter difficulties. This was nice to see, considering the Platter difficulty caused modders a number of problems in the previous test. In general there were fewer obvious mistakes or unrankables missed this time around, though some that persisted included failure to identify insufficient note-to-spinner spacings, poor flow between regular dashes and hypers, and strangely, unfitting overmapping / undermapping. Additionally, many instances of only "partial" marks were awarded where candidates failed to fully explain the problem in question, such as not actually highlighting the location of unsnap errors reported by AIMod, or clarifying the BPM condition for hyperdash rankability. Remember to explain each issue in as much detail as possible, and then provide an explanation as to why exactly the presented solution resolves the issue. Lastly, it should come as no surprise, but the position of Beatmap Nominator is one of great responsibility, and as such the test should be taken seriously. Memeing on the test will ensure your application is taken about as seriously as you expect - that is, not at all.

Both our passing candidates have already received their feedback review and are preparing to dive into the world of nomination at their leisure. As always, we'll be working closely with them over the coming weeks and months to help them settle into their new role, answer any questions they may have and hone their skills to keep striving for ever-higher standards of excellence. To those who were unsuccessful in the test, we encourage you to review your performance also so that you can overcome the mistakes and limitations of this round to succeed next time around. Even if you didn't meet the required activity threshold but feel like you'd be a good addition to the team, you'll get another chance to apply in a few months, so be sure to keep modding regularly, helping others in the modding and mapping community, and most importantly, gaining experience to polish your own skills. Maybe we'll be celebrating your victory sooner than you know it!?

To conclude, many congratulations once again to our passing candidates, and good luck to everyone applying next round - I hope we get can along well both now and in the future, to keep developing our beloved game mode for even more people to enjoy!

osu!mania (by Feerum)

Total # of candidates: 11

Passing candidates: Julie

As you're probably already aware, we only got one new BN this round, so please give a warm welcome back to Julie! This was a bit of a surprise following the last test where seven people passed! We didn't expect people to find the test so hard this time, maybe it's because 7K+1 is a mode which rarely gets mapped, but we wanted to try something different this time around. The song we used for this round was "Saiyajin", from our Featured Artist, Thaehan.

The average score is 29.9 for Part A, which was quite a reduction compared to last round. This indicates that many people's knowledge in the Ranking Criteria is lacking, with questions about spread, skinning and metadata getting a lot of missed or wrong answers. It wasn't all bad though, as most people managed to get a good score on the timing, guest mapping and nomination rules questions.

Part B's average was 23.5 though, which was really disappointing. We had 3 people which got over 60 Points in total and, sadly, only one over 70. The EZ and NM difficulties featured some bad patterns and were overall not very consistent between same sections. We are still kinda surprised that a lot of people missed hitsound issues over these difficulties. This was very noticeable in the MX difficutly, but thankfully most of the other problems in this difficulty, like broken SV changes and difficulty spikes were detected by most people. The difficulty that affected the results the most was the HD however. This difficulty had a huge number of errors, such as unbalanced patterns, hand bias, up to wrongly snapped notes and placed notes which didn't follow any kind of specific sound at all.

Whilst I am sad that we only got one new BN this round, there's no need to be disappointed - we are sure the next round will end better, so good luck to everyone next time too!

~

With this, I would like to conclude our recap of the latest BN applications for osu!taiko, osu!catch and osu!mania. Hopefully you were able to take some information away from this, whether in preparation for your own Beatmap Nominator test in the future, or just as a casual observer interested in the development of the BNG and ranking process. Of course we couldn't cover every tiny detail in this review, so if you have any questions, please feel free to ask! For those of you who are planning to tackle the next round of BN tests, either as a new or returning candidate, think hard about the test creators' comments and try to apply them to your own modding. We all wish you the best of luck in your endeavours and will be eagerly awaiting even more success from osu!taiko, osu!catch and osu!mania modes in the next round of tests.

P.S. To anyone who is awaiting responses to your questions in the "Ask Me Anything" feature, I'll be answering them in the next issue of The QAT Gazette (general update article) to avoid cluttering these focused articles too much, so please wait patiently!

—JBHyperion