Patch sub-categories for leaderboards
6 years ago
South Australia, Australia

So, a patch finally hit for console. We all knew this was coming, now it's time to start the discussion about what to do with the leaderboards. So far it hasn't hit Switch, but it's on PS4 and XBone, so we can assume it'll hit Switch soon. The build is the same as the current PC build, 1.03.0919 (where the default build would be 1.0). Note that PC recently got patched from 1.03.08 to 1.03.0919, which changed a few minor things (including allowing Studio 2 skip). So here I'm going to be outlaying some options about what to do.

Variable vs Sub-Category:

In the long term, sub-categories for separation on builds will be the go to. But in the short term, that might not be the best option, as we don't exactly know if there will be further patches, in between patches, small patches, patches that impact mechanics, etc. I'll talk more about that in the future, but for now I'll just show what the difference is.

A variable is like what we have for the consoles. You can see the info on the leaderboard, and choose to filter it as you view, but the default listing is just 'whatever is fastest'. The issue with this long term is that someone playing on a patched version, even if they were to filter the leaderboards, would still have their submission competing with the unpatched version. For example in this picture, the submission would still say "6th" on their personal profile, which isn't particularly accurate for competition on that version.

A Sub-Category is what we already have for "Beat the game" and "All Emeralds". This will be the long term solution. But how to actually handle the sub-categories is something that needs to be discussed. For example, how to distinguish the sub-categories. This for both ILs and full game runs, and choosing an option that works for both simultaneously may prove tricky

One option is to simply use the build number, e.g. the sub-categories would be V1.0 and V1.03. The two obvious issues with this option, would be then someone on 1.0 would be excluded from competing without using the glitch. Take for example Alluusio's run for Studiopolis 2, which was done on PS4 on Sept 1st, using build 1.0. However, he did not use, or attempt to use, the glitch in that level, which is available in 1.0, and 1.03.0919 but not 1.03.08. For all intents and purposes, it should be competing with the runs on build 1.03.08. You could still name them as such and just submit to the one that applies to you, but that would seem rather confusing to a submitter, and incorrect labelling. The other obvious issue would be if there where multiple builds, it would create a potentially huge amount of sub-categories, separating competition unnecessarily and making the middle builds barren and uncompetitive (and potentially unaccessable in future)

Another option would be to label them as "Patched" and "Unpatched". This would allow distinguishing between using what is available on unpatched versions, vs what is not. However there's a few issues with this still, what if there's another patch, and a 1.06 comes out? If that were the case, and there's a level with 2 bugs, both work on 1.0, 1 works on 1.03, and none work on 1.06, then where do I submit a run that was done on 1.03? This is more likely to happen in full game runs, where there could be 10 bugs usable on 1.0, then 5 on 1.03, then 2 on 1.06, etc.

My personal preference is to have only 2 sub-categories if possible. However, if they make a build past 1.03, then there's going to be some mess one way or another. My personal preference is to have to two sub-categories, and have them be "Newest Version" and "Any Version". This also means if you do a run on the newest version, you could technically submit to both. This is actually a good thing though, imo, as it means if the situation arose where even though the newest version is at a disadvantage, if it gets optimised more and ends up with a lower time, the other category still accurately shows the best run. Instead of for levels like Studiopolis 1, where the over the roof has been patched out, but the times are worse, so the "Unpatched" times would actually be far slower.

The issues with this are this would mean for full game runs, if a build was made beyond 1.03 then all 1.03 runs would be grouped together with 1.0 runs, at a disadvantage. It also means runs which were allowed in the "Newest Build" may then have to be moved later on to the other sub-category. However I find this preferable to them having their own sub-categories, because a) I don't think anyone will optimise a good run before the last build is released anyway (they won't be patching this in 2020), and b) the competitiveness for an in between build will almost certainly be 0, so it'd just be sitting there as dead weight. The same would apply to ILs under the same unlikely criteria as described above, but I think anyone who is using a bug that ends up getting patched just has to be aware to know that could be happening in the future, as has to suck it up if it does.

Last point on the IL charts, but the "overall board" (This one: https://www.speedrun.com/sonicmania/individual_levels ) will only display the default sub-category. I'm amusing the community response based on actions from what I've seen so far is that the majority would prefer the default in that case to be the "Newest Version" runs. So that for example, that board would display Level_Select's 2'51"77 on Titanic Monarch 2 Sonic, instead of his 48 second time on console, even though both could be submitted under different sub-categories. Either way not everyone will agree no matter what we choose, but I'd like to hear everyone's thoughts so we can make the most people happy.

Tl;dr: Will be using sub-categories in future. May use Variable for now if an overwhelming consensus on best option isn't reached. Personal preference is for "Newest version" and "Any Version" to be the sub-categories. Would like to know which you would all prefer to be the default option, and any other opinions.

Edited by the author 6 years ago
Lemin and Jaypin88 like this
New York City, NY, USA

I like "Newest" and "Any". Gives me more excuses to keep running the game when it gets patched.

jocloud31 likes this
Michigan, USA

I won't make a stink about whatever we end up doing, but I also like just having newest and any for versions. It adheres to the doctrine of "keep it simple, stupid" and is fair enough overall. As for whether newest or any ends up the default, I have no strong opinion so long as it's easily visible which any given run is.

jocloud31 likes this
Germany

To me Newest and Any sounds reasonable with the addition that runners on Newest also can competed against the older patches. For those who dislike the patches to pick their most favourite one to compete with everyone else, no matter how many patches we get and which one will be picked as the most favourable to speedrun sounds great. Also i have to quote the words of AlecK "keep it simple", which make the suggestion of 2 subcategories favorable for me.

jocloud31 likes this