Leaderboards Update/Change (December 2018 Edition)
5 years ago

There's been discussion in the smash 64 speedrunning community about the potential of merging "versions" together. I'll separate them in order of priority:

------------------------------ 1. MERGING VC INTO ITS REGIONS ------------------------------

The current layout of VC is flawed in so many ways. VC actually has multiple different versions mixed into one, as each region (NA, J, PAL) has its own VC version, and are all put in the same leaderboards. For what it's worth, the VC version and the N64 console version are essentially identical games, save for an announcer screen difference. The proposed solution, which seems to be generally agreed upon in the Smash 64 speedrunning community, is to merge VC into its own individual regions, while "subtracting" away the time difference lost in the announcer screen (currently estimated to 25 seconds). This way, VC runs will be displayed on the same leaderboards as the N64 console runs of the same region, and on an equal playing field to N64 console runs.

The one thing that cannot be compared exactly between versions is the lag during the game that the N64 experiences, while VC does not. I compared various instances of potential lag advantages between NA and VC under normal circumstances (e.g. Master Hand moves), and most seemed to be mostly identical (if a difference exists, it's extremely minor). The only instance that would truly cause a difference is if, say, a Charizard spawns on Polygon Team, or Fire Flowers are used on that stage. That said, both those instances are not common occurrences, and both are 100% avoidable by the runner if played properly.

For bonus mode runs, all runs are ran with In-Game time, so both the N64 and VC versions would end up with the exact same time, meaning bonus mode runs are essentially unaffected by this decision.

So the current plan is to merge VC into its individual regions (NA, J, PAL), while "subtracting" the time difference of the announcer screens, and having 3 "versions" on the same leaderboard.

------------------------------ 2. MERGING PAL AND NA REGIONS ------------------------------

This next step is assuming step 1 does pass, but is split on whether or not it should be done.

After further comparison, PAL and NA run at the exact same "speed". Having looked at several PAL runs, the game runs at the exact same speed as the NA version, meaning that the 50 FPS vs 60 FPS speed difference doesn't really affect this game. So in terms of that, there does seem to be a valid argument to merge the two regions together.

That said, there are version specific differences, notably character-specific differences, that do give one version an advantage over another depending on the character ran. Some of these differences don't affect a character's time at all, while others create a large difference (notably Link). These version differences are a good argument to keep the two versions separated.

Furthermore, bonus mode runs will be affected by this change, something that should absolutely be taken into account (and is a large reason why I'm currently leaning towards not merging the two). The character-specific differences will have a major impact on several character's bonus mode runs, again, notably Link being potentially the biggest difference. If this change is made on the Classic mode side, the Bonus mode side will be impacted as well.

If this step passes, we'd have all versions (excluding J) in one leaderboard, with J in a separate leaderboard (we coud then discuss merging J as well, but the community seems heavily against merging J). If this doesn't pass, then we will have NA, J, and PAL as the three versions all separated into their own leaderboard.

This is the current state of the conversations of the leaderboard changes.

If anyone has any opinion on this, please post your opinions here. If there is nothing that is said against the merging of VC and NA, then that will happen in the next couple of days. On the other hand, merging NA and PAL is still an ongoing discussion (I'm undecided, and have gone back and forth several times myself), so if you have any input on that, absolutely say what you think.

Edited by the author 5 years ago
GoodleShoes likes this
Texas, USA

(Be warned, I only occasionally run this game)

I think there should be separate leaderboards for VC and N64, with the necessary divisions within those categories. If it were up to me I'd say have all the N64 together on one leaderboard and all of VC on one leaderboard, but if that's unpopular then that shouldn't be the way forward.

At this current moment, combining all the different regions together (notably J with anything else) is the unpopular decision, and very likely won't happen. J is just too different to be on the same leaderboard as any other region, and the majority of runners think this way as well. The only region combination that "could" happen is NA and PAL, and even that's looking questionable.

If VC is to remain separated from N64 console, then the leaderboards would essentially remain how it currently is. VC and N64 console (for each individual region) is essentially the exact same game, so if that doesn't get merged, then I don't see how combining different regions with differences in the gameplay itself would ever have a chance of happening.

No matter how much or how little someone runs a game, and no matter whether they agree or disagree with what I'm thinking, I want to hear any opinion. Maybe someone has an extremely valid opinion I never even thought of. So if you think they should remain separated, I'd absolutely like to hear why.

So... I guess that said, why is it that you think VC and N64 should not be merged?

Texas, USA

I think the leaderboards should be separated along VC and N64 lines because VC is a faster piece of hardware that has lower lag overall. I don't think it's a huge deal considering the length of the IL's, but for consistency's sake I think they should be ranked on separate boards.

The different versions, though, I think should be ranked all on one. Much like the NTSC, NTSC-J, and PAL versions of SM64 and Goldeneye are all on the same leaderboard while players are allowed to use any version they want.

Overall, though, I don't think this is that big of an issue. The practical effect this will have on the leaderboards is not that big considering that NA N64 is the most popular.

GhillieGuide
He/Him, They/Them
5 years ago

A difference I haven't seen mentioned is the gamecube controller, VC has a modified control mapping to accommodate the GC controller and it affects the game even if you use an adapter to use an n64 controller. I find the sensitivity makes it more difficult to complete runs on and the layout is different on the default controller for buttons. I prefer n64 vastly more as someone who speedruns on both consoles sometimes and wouldn't compare my VC runs in earnest to my N64 runs.

also the bonus stages are darker, most important difference ever!

I don't really care about this too much, you all do what you want. -Ghillie

Switzerland

I'm happy with this solution. It separates the versions that have actual gameplay differences (JP, NA, PAL) and merges the versions that only have different screen transitions and lag (VC).

Edited by the author 5 years ago

While VC is normally faster than N64 in many cases for many games, having done tests of individual incidents between the two, the difference was pretty much non-existent. The only case where lag would truly be a disadvantage in the N64 version are situations that runners are completely in control over. Usually, these involve something like a Polygon picking up a fire flower and using it, or a Charizard appearing (which both at higher level play never happens except MAYBE very hard 1 stock). So while they are different hardwares, they actually do play out the game in a very similar manner.

The current difference calculation is 25 seconds of difference. If it is determined that this difference is inaccurate or wrong, it doesn't take much to just add or remove an extra second to re-balance all the runs properly.

As far as combining the three regions (NA, J, PAL), J is completely different from the rest, and often considered to be a completely different speed game, so for the forseeable future, J will not be combined with any other region. An argument could be made to combine PAL and NA, which is currently split on whether it should take place, but it doesn't look likely that combination will pass in the immediate future.

The issue of different controllers between N64 and VC could be seen as both a valid reason, and as an invalid reason, depending on the point of view and circumstances. For someone who is natural to the 64 controller, the VC version is a major disadvantage. However, for someone who is natural to a gamecube controller smash game, it may be the preferred method of running for them. If anything, this lets people who run on both N64 controllers and gamecube controllers to have equally viable runs to each other. There's a difference between running at a clear disadvantage (NA vs J), and running a game a slightly different way while still on equal footing (N64 vs VC).

As far as the bonus stages being darker, I think it is even more of a reason to combine the versions. Runners on VC are running on a darker bonus stage, meaning they're already on the path to the dark side. If we combine the versions together, then we may show them the light side of the force, and may still have a chance of saving them from running down that path. However, if we do separate the versions, then there's nothing stopping their run to the dark side, causing imbalance in the force. Then again, would combining the versions allow the dark side an easier path to eliminate the light side? We should probably think more about this...

Being more serious though, the two versions are basically the same game (for each region), and aside from what seems to be runner-controllable differences, are able to be played on an equal playing field. The majority of the community seems to be in favour of combining VC and N64 (I think the Discord server was unanimous in combining the versions, but there might have been one person I'm forgetting about that wasn't in agreement). So unless there's a large amount of support to keeping N64 and VC separated, or a very significant reason is brought up to keep them separated, then the two will end up being merged.

If anyone does have any thoughts on this, please say your reasons. It's better to say your thoughts before we make a change than after we make the change, when we realize too late that what you were going to say actually is the problem we are stuck with.

GhillieGuide
He/Him, They/Them
5 years ago

Does this mean IQue can be legal now?? If we are gonna ignore the lag reduction on VC then would the same logic apply to IQue as well considering it's most likely comparable to VC with the announcer stage difference removed?

Is the lag difference that major between VC and N64? I timed a few different things, and it seemed like VC did not save any time over N64. From what I remember on iQue, it's faster than both VC and N64 by a wide margin (more specifically the Master Hand move from the background being significantly noticably faster on iQue, while it timed the same on VC and N64). If any of this is wrong, let us know.

It's very well known that iQue is MUCH faster than N64 to the point of not even being comparable. If VC is faster by the same amount, then VC should not be combined with N64.

GhillieGuide
He/Him, They/Them
5 years ago

The IQue version in my comparison was extremely comparable to the N64 given comparison of footage in just the announcer cutscenes, this would suggest that without the introduction of lag inducing variables N64 and IQue run pretty much the same speed. The IQue version,NTSC-U and J, and VC version are capped to run about 59.97fps or approx 60hz.

The game runs the same on all 3 versions unless lag inducing variables like bombs, samus' upb etc. are present, if anything on paper the Wii should run a little faster/with more stability than IQue(assuming you ignore the announcer cutscenes). I haven't seen real world comparisons where that lag difference has been accurately tested and the amount of lag would depend on the actions in each individual speedrun.

In my comparison the differences in IQue, J, and U are all down to human error margins for mashing:

Hardware Stats:

Wii Processors: CPU: PowerPC-based Broadway processor, made with a 90 nm SOI CMOS process, reportedly† clocked at 729 MHz GPU: ATI Hollywood GPU made with a 90 nm CMOS process,[131] reportedly† clocked at 243 MHz Memory: 88 MB main memory (24 MB internal 1T-SRAM integrated into graphics package, 64 MB external GDDR3 SDRAM) 3 MB embedded GPU texture memory and framebuffer Graphics: 410 million polygons/second, 16.7 million colors

IQue Processor: R-4300 64-bit CPU, 140.625 MHz Memory: 16 MB DDR (8 MB available) Graphics: 100,000 polygons/second, 2.09 million colors

Both consoles have no problem running the game without little interruptions from lag present on N64 in my anecdotal experience. The VC emulation is known for being a bit off like any N64 emu however and I may have confirmation bias.

Edited by the author 5 years ago

I guess this means more tests and timings will be needed to be performed before we make any major decisions then. I don't have the time to make any serious tests for a while, so unless someone else will, I guess we're leaving the leaderboards as is for the forseeable future.

VC is still in a weird situation though, as it has been brought up that every region on VC is being put in the same leaderboard, so PAL and NA are running against J. So VC already isn't in a great spot to begin with.

GhillieGuide
He/Him, They/Them
5 years ago

Here is a rough comparison of 3 of my PBs using the exact example you listed pikashy, I couldn't match it to the frame doing it quickly like this but the VC and IQue runs in particular look to be only a couple frames off perfectly matched. https://viewsync.net/watch?v=InaRVFV7V44&t=359&v=rBiLe0yR3wc&t=375&v=kT1JxOUkk04&t=397&mode=solo

Going frame by frame, all three had the exact(ish) same time for the move to happen (from take off to "hitting DK"), so my tests on Master Hand have clearly not been a good thing to test at all.

GhillieGuide
He/Him, They/Them
5 years ago

Ideally a comparison would compare footage from loading zones and countdown, the countdown is the only place I notice consistent lag showing up on n64 depending on stage. I was ironically appreciating the lowered lag on VC compared to n64 enough to notice it when I recently did some VC runs before this was brought up. It could have just been confirmation bias but I feel pretty comfortable spotting lag frames especially in this game. Maybe I will do some testing but not tonight/tomm.

I do know VC ports on the wii almost infamously and unanimously save time over their N64 counterparts in loading times across all of the communities that I am aware have tested it. The texture buffer in specific IIRC is a big difference in addition to the GPU which were both not present on the N64 console. I haven't seen any other testing videos other than the couple I created for announcer cutscenes and timer on/off, if there is comparisons anyone can link that I missed maybe I am ignorant but it seems like VC would have a similar advantage as IQue given their superior hardware from 2003/6 vs 1996 if not for the announcer cutscenes. In my announcer cutscene comparison video I remarked 3 years ago in a comment that I believed it would be faster due to load times if not for the announcer difference.

Edited by the author 5 years ago
GhillieGuide
He/Him, They/Them
5 years ago

I have no internet on my computer right now so I can’t upload videos of testing until mon-Tuesday.

I did testing that compared load times and the countdowns on NTSC-U,VC-U, and IQue.

Here are the results over the course of a 1p mode speedrun @ 30fps US N64: 1;18;14 IQue: 1;18;12 VC U: 1;18;03 Edit: PAL EU: 1;18;17 This suggests that the Wii has far better load times than the N64 and gives a slight advantage(1/3 a sec) just on load times.

I haven’t tested the ingame lag differences because I am not aware of a way to induce lag the same way consistently with just one player(the IQue only allows 1p unless I get a rare 500+ dollar multiplayer setup). If anyone has some ideas of how to properly test this let me know.

Back when I was somewhat relevant one disagreement Pikashy and I had was the amount of time saved by playing on IQue over the course of a 1p run. It was asserted by Pikashy that the difference was 10-12 sec iirc based on what now looks to be undocumented and flawed testing. My theory was that it was about 5-8 seconds but I didn’t put in words why my estimate makes sense to me based on my experience.

The way this game lags on 1p mode it usually only drops about 1 frame @30fps any time it lags up to about 3 frames when it gets really bad. A time difference of 10 seconds for example would require 100 instances where 3 frames are dropped in the course of about 6 mins. More realistically it would require 300 instances of a single frame drop or 150 instances of 2 frame drops. My 5-8 number is rough but it’s much more realistic if you actually look at how many times a run has dropped frames on an N64. An estimate is gonna be inaccurate no matter what, it’s just an estimate. No matter what specific testing is done to measure lag frames on moves we will struggle with two variables when making an accurate comparison.

  1. the N64 does not appear to lag at a consistent rate
  2. the amount of lag produced is variable based on the moves used on CPUs and how many hits of multi hits they receive, if a Pokémon comes out, someone gets a star/bomb etc.

I have been playing all 3 of these versions for years and my inclination is that the lag differences of IQue vs N64 are comparable to the lag differences of VC vs N64. In addition I have now proven a slight advantage of loading times for the wii. I think my original estimate will hold up somewhat if the difference isn’t even less(3-5 might actuallly be more accurate).

I am willing to do more testing to prove my hypothesis but ultimately there will never be an accurate comparison of how much faster an IQue run will be vs an N64 run that has not occured yet. Tackling a single run’s differences might allow for a accurate comparison just for that run given enough work but the difficulty of getting very similar runs on all 3 versions + extensive editing to match all the moves in each specific area with everything matched seems like estimate 30 hours of unpaid and meaningless, repetitive work that would introduce hundreds of opportunities for human error.

Edited by the author 5 years ago
GhillieGuide
He/Him, They/Them
5 years ago

TL;DR version VC saves 1/3 sec ish on loading time in a 1p mode speedrun

No conclusive testing on IQue vs VC vs N64 has been done to quantify the lag produced on each console and/or competitive advantage gained.

My estimate on competitive advantage for VC/IQue vs N64 outside of the testing already done ranges from about 3-8 seconds depending on the character selected, strategies used, random elements, and natural variation on console slowdown.

Edit: I forgot to mention that my loading times are specific to my Wii and on one of the fast black ones VC would theoretically save a bit more on loading

Edited by the author 5 years ago

I think even just showing that VC has an advantage lag-wise, even if the exact amount isn't calculated, is enough to show that it should not be combined. While it would be nice to have exact values for the sake of "rough" comparisons between versions, I think an estimated time difference will be plus or minus 5 seconds away from the actual time difference.

VC is still in a weird situation by itself though. It's comparing 3 different "regions" all in the same "VC" category. Unless someone starts actively running on "non-NA" VC regions, I guess it's ok for now, but once someone starts doing actual runs on Japanese, we may run into an issue with VC itself. Due to the essential lack of any PAL and J VC runners at the moment, splitting VC into three separate categories (making a total of 6 altogether) is not only version overkill, but pretty much unnecessary as both regions would pretty much be a leaderboard for 0 or 1 person.

Of all the games I've played or followed speedrunning-wise, why is 64 one of the worst when it comes to version differences? Unless everyone runs on a single version and doesn't run on any other version (not going to happen, nor should it be enforced), someone's going to be running on a faster version. Oh well, at least the majority all run on the same version instead of an even spread. That would be a bigger nightmare.

GhillieGuide
He/Him, They/Them
5 years ago

I did more testing and comparison yesterday and have some more results to share.

The next set of testing I did measured the difference between announcer cutscenes with no inputs. This was a good demonstration of specifically the difference between N64 and IQue. Below are the results, again the videos for this testing will be uploaded tomm when I get internet on computer.

NTSC U = 1;32;28 VC-U = 1;34;20 IQue = 1;31;20 PAL = 1;32;26

The VC results just show how messed up the announcer cutscenes are on that version. The PAL timing was a bit of a surprise, not really sure what to make of it.

The IQue difference from N64 when extrapolated to a run of 5:40 length ends up being about 3.9 secs on the base framerate for the full run. If we were able to accurately determine the amount of times the N64 lags in addition to that during a run would give a better estimate. That means my estimate of 5-8 is again right around where I see the IQue advantage over N64. I am gonna do some rough ingame comparisons between versions but having problems finding something that lags consistently w 1p still, I was hoping green shell glitch would suffice but right now I just have a couple moves in 1p mode so any advice would be appreciated.

GhillieGuide
He/Him, They/Them
5 years ago

Here's a post with video links for proof of the above testing, haven't got around to any ingame lag testing, I have some stuff recorded but probably need more for any kind of conclusive analysis.

Here is a majora's mask video showing VC lagging less than N64 I found, should be same demands on the console and hardware used for this game.

Edited by the author 5 years ago
GhillieGuide
He/Him, They/Them
5 years ago

"I think even just showing that VC has an advantage lag-wise, even if the exact amount isn't calculated, is enough to show that it should not be combined."

I think this video should suffice to show the difference in times of heavy lag at least until I can get something more solid together.

I also compared VC to IQue directly and it's a bit less useful of a comparison.

Edited by the author 5 years ago