Castlevania: Harmony of Dissonance - Forums - Proposed new Rules and timing adjustment

https://docs.google.com/spreadsheets/d/1il4q4AjNEPI0WpcEABWLf7hkTE2sshJrmlEwoNnQj5Q/edit#gid=0

Hello!

Maxim Any% has grown to be really competitive, with a multitude of runners from many countries. The runs that are submitted uses Both Hardware (DSLite), as well as Emulators (VBA RR-V-24, Bizhawk, WiiUVC, others). Concidering that the top times are now hunting for mere frames, I felt the need to do some extensive investigation into the accuracy of emulators, since a frame could mean the difference between #1 and #2 at this point.

The main focus of my investigations has been the amount of frames it takes to transition from one screen to the other, showing without a doubt that there are many differences between emulators. My full research is pretty selfexplanatory and can be found here:

The new proposed rules for run submissions of ANY Maxim Cathegory are:

IF Emulator on computer is used, Bizhawk 2.2.1 OR later, along with the GBA BIOS and mGBA core MUST be used. VBA-NEXT has been proven to be inaccurate.
WiiUVC is an exception to above rule, it is perfectly fine to use it
The runs must (at least at the end), show the Bizhawk window with the mGBA core in the bottom left corner, to make it easier for the mods to verify the runs.
No Turbo or Autofire-option may be used. This rule has been in effect before but never explicitly stated.

These rules conform well with the Metroid: Zero Mission scene and has worked well for them. The investigation into the preferred emulation has also been the same. Reference: https://www.speedrun.com/mzm/thread/syvkl

In Addition to the investigation done, the need to adjust some times on the leaderboards are well motivated. Runs made prior to this post has been made will have time-adjustments (Explanation can be found in the above linked document):

SORP Yu #1: 30,67 -> 31,07

NOVel #2: 31.93. No change, performed on DsLite (Original Hardware)

Charleon #3: 32.11 -> 31.93 (Tied for #2)

Gossyo and T2-CVHD: Unchanged, played on WiiUVC

Any feedback and/or objections to this proposal will be welcome, but if no solid argument can be made, these new rules and timeconversions will be made in a couple of days. The rule for Emulator is in effect now when this message is posted

Edited by the author 6 years ago

So basically the standard you are converting to is WiiU-VC and DSlite(whose load times appear to be the same)? And even though VBA-next is more accurate than mGBA, you want mGBA to be used since its accuracy is also comparable to those two platforms?

I think this is fine, but I don't think VBA-next should be banned since it is more accurate than mGBA, or that people should have to show which core they are using for verification since neither of them are less accurate than the platforms with w/ the proposed load time standard that other runs are being converted to. We can just say mGBA is preferred, and if someone happens to do a run on VBA-next it can just be converted down to mGBA/VC/DSlite; same thing if someone does a run on Wii-VC or the GBA console. It might seem more intuitive to convert everything to console's load times, but since hardly anyone runs on console I think it's fine to just convert console to mGBA/VC/DSlite to save time for whoever's doing the conversions.

"So basically the standard you are converting to is WiiU-VC and DSlite(whose load times appear to be the same)? And even though VBA-next is more accurate than mGBA, you want mGBA to be used since its accuracy is also comparable to those two platforms?"

Correct. I'm interested to know your sources of how VBA-Next is more accurate though, since my spreadsheet basically states otherwise, at least for this very specific application.

As for VBA-Next being pardoned for runs, I can agree to that, but in that case I'd like to change

• The runs must (at least at the end), show the Bizhawk window with the mGBA core in the bottom left corner, to make it easier for the mods to verify the runs.

to:

If the run is made on Bizhawk, you must specify what core (VBA-next, mGBA) you are using to help the verifiers do the time-conversions, alternatively show the bizhawk window before/after the run so the core can be recognised. Otherwise it will be rejected.

One of the things I really want to achieve with this rule is to minimize work for the verifiers. You should not have to download the run and do frame-by-frame analysis to figure out which of the two the player has used.

Also I would actually say DSLite = WiiUVC = Bizhawk (mGBA), since I expect an errormargin of 1 frame all the time since 60FPS game footage is truncated to 30FPS Youtube/Whatever video.

By more accurate, I mean more accurate to original hardware, as I am assuming all of these platforms have faster load times than original console since I know for a fact mGBA isn't console-accurate, and less accurate than VBA-next from other games I have TASed. Your tests are consistent with this since they show your VBA-next run to have 112 total load frames and the mGBA run to have 107.

I just don't think runs should be rejected under the premise that they might be using a more accurate emulator core if they forget to show it. I agree with the sentiment of simplifying the verification process, but there's no harm done if a run ends up on the leaderboards where somebody loses time because they neglected to read rules that say something like "mGBA is preferred since its accuracy is consistent with Wii-U VC and DSLite".

Good idea to get frame data and convert times to some standard. I'm fine with WiiUVC being the standard.

But I think something is wrong with your frame data. I don't believe that your time is as fast as NOVel's based on the different Dracula kills. You walk back to get into position for the damage glitch and NOVel doesn't so his should be faster, but on your chart you list both frame counts as 88. If I remember correctly VB mentioned in the discord that VBA-Next and original hardware have slowdown for the damage glitch that most emulators don't, so maybe that's it.

Based on the frame research I've done, VBA-Next and mGBA are BOTH inaccurate compared to the original GBA in terms of load times. But I haven't done much testing yet so that could be wrong. Also comparing different emulators, the biggest difference in load times in BizHawk seems to depend on whether or not you have a proper gba bios. It doesn't let you run the VBA-Next core without it, but it does let you run mGBA either way, and the timing for mGBA is very different depending on it.

So I'd say have runners tell what emulators they used, including BizHawk core and if they had a proper gba bios, but for now I'd like to get more extensive frame data to be able to do proper comparisons, so don't start converting or rejecting things yet. Also please err on the side of accepting runs rather than rejecting them. There's communities in China/Taiwan and in Japan for this game and there's language problems in communicating between them to let all the rules be easily known and agreed on. With a good conversion method it shouldn't be difficult to convert most of the different emulators anyway.

Brooke

She/Her

6 years ago

I think the lag thing Sopha touched on is very important as well. Lag for Maxim Any% should be fairly consistent and I feel like it should be tested for all the different versions. Other than that I have no problems with the changes to the rules suggested.

Glad that I was able to spark such enthusiasm from you guys!

@sopha: correct, it seems like a copy paste error regarding mine and NOVel's dracula fight. I just added it since I was sort of curious of how much the different top runs differed on the dracula fight. I'll correct it when I'm at home on my work PC. That being said, the dracula fight time played no part in any calculations done in the document. And of course I welcome all effort put in to proofreading the document as well, that is why I have it publicly shared.

@VB: I completely agree that this lag should be examined as well; How it behaves on WiiuVC, VBA RR, DSLite, Bizhawk(mGBA) and Bizhawk(VBA-Next) respectively. Additional adjustments may be needed with these findings.

Additionally I would like to personally put in some time to really dissect SORPS run on VBA RR to see what else might differ, and how he somehow saves 0.9 seconds over NOVel with a worse dracula fight (Again, the dracula fight need to be re-evaluated in case I made an error there).

For the record, I use Quicktime player for all the research. It has the option to display #Frames instead of time and can frameadvance using the left/right arrowkeys while paused.

I look forward to a clean board with clear adjustments and clear rules :)

Edited by the author 6 years ago

sorp 6 years ago

I almost agree with Sohpa. Frame data making with our 30FPS video may be inaccurate. I think if you want to show differences between VBA-RR V24 and Bizhawk with VBA-next, you can run the game in 60FPS on the two emulators,record a video from the same pixel and the same frame, then move in the same frame and count loading time of transition, put them together and play at the same time. You can upload the original video or slowdown(for example play as *0.25 speed)to Youtube. That will make me convince.Because both you and I can't say we did perfectly in the run, so we had different stauts during transition. In addition,I don't the way you count loading time. When transition happens, camera start moving from the specific position.Then when the camera catch position of Maxim in the screen, the screen turns bright. (I don't know whether i explain it exactly in English. )So if you just see the screen, the loading time is perhaps not exact enough.That's the another reason why I suggest test in same stauts(pixel, frame, movement).

About who kills the Dracula faster, I think Novel and Charleon are almost same. I know 4 ways to get damage glitch on Dracula and all ways I practise a lot. In Novel's run,Dracula moved FORWARD, STOP, so he choosed vertical jump immediately to get second damage glitch. But use this way,he didn't throw the subweapon at the first time, throw it in midair that will get more hits on Dracula (because of the flying path). In Charleon's and my run, Dracula moved FORWARD, FORWARD. In this RNG, it's better to move back or you will be hit by the boss. In this way, we jumped forward and threw the subweapon at the first time, then get the damage glitch. So it doesn't mean moving back will be slower. My standard of killing Dracula's speed is watching the position of Dracula's Claw when Dracula died, (exclude damage glitch's lag). I think Dracula's claw moves in a fixed path, so it's easy to refer to. In this standard, I think I got several frames faster in my Dracula' s fight (my divekick hit on a lower position, so I could start the second damage glitch faster). The intutive feeling may be inaccurate. By the way, I watched the clip of new Dracula’s fight Gossyo posted on twiiter, I think this way is infirmtive and can't be much faster in my opinion, I think the slower part of Charleon is in the SKY WALKWAY room, he moved to right a little, then rolled to left. That would cost more frames.

WiiUVC being the standard is the best. If you want to get a 60FPS WiiUVC video, you can download T-2's run on Niconico. About differences between emulators, I think you can hear some TASer's advice. They may know more about it.To be honest,I plan I would get a 30.9x run finally. Based on 31.93 run,and count all improvements save the frames i made (besides some obvious improvements, there are some small technic). So when I saw I got 30.71, it was faster than I thought in fact. Adjustment is necessary and reasonable, but should be convincing.

I gave up this run 2 years ago, because I couldn't do the small jump from CHAPEL to SKY WALKWAY and Dracula's bad RNG. During these 2 years, I played Juste and learn more knowledge about this game. When I watched T-2's run a month ago, I learned how to do the small jump immediately. So I picked up this run and wanted to do better. One day,a TASer discussed with me about diffrent part between TAS and RTA. That made me think the whole short run again. What should/shouldn't and what I can do in this run? So I made these improvements. OWATA's run was posted yesterday, I think that whiplaunch is a really cool improvement (i think it will be 0.2s faster). I tried the whiplaunch several days ago,but success rate of zipping in the wall is too low.That means it would cost me a lot of time in the beginning part. So I don’t use it now. And I won't run Maxim any% again until that whiplaunch is used widely or something new comes out. Because there are nothing new in this short run now. I don't feel fun in this repeated job. I'm waitting for your rules of emulator, I would like to practise Maxim all bosses for fun, and Juste any% with a new route(developed by another TASer, but there are something hard, I'm not sure whether i can complete it...) in the furture. (I'm sorry my poor English can't express my opinion exactly)

Edited by the author 6 years ago

I finished the second page of https://docs.google.com/spreadsheets/d/1il4q4AjNEPI0WpcEABWLf7hkTE2sshJrmlEwoNnQj5Q/edit#gid=1585218115 where I list everyone's Gameplay frames in all the screens. From my listings I can conclude that SORP saves 14F (0,46s) in the Dracula Elevator room over the second fastest (gossyo). This is purely because of roll horisontal momentum and the divekick putting him on the elevator.

Sorp 828F -> 918F = 90F Reaching the bottom of dracula's chamber 918F -> 947F = 29F Bottom -> Elevator

gossyo 468F -> 560F = 92F Reaching the bottom of dracula's chamber 560F -> 601F = 41F Bottom -> Elevator

For the boss fight itself, both WiiUVC and VBA RR applied 1 new shuriken damageglitchtext per frame, showing no real behaviorchange or slowdown. I can find no evidence at all that suggests that Any emulator (or hardware) differ in pure gameplay portions of the game (This, of course, might need a second eye). All frame differences I've seen can be logically explained by the execution of the rooms per player.

As a bonus: Owata's newly found dracula kill: https://twitter.com/th1018495/status/969558254593298432/video/1 saves 13 frames (75F total) vs NOVel's 88F, which is 0,44 seconds!

@sorp While I do agree that it's not optimal to analyze 30 FPS footage of 60 FPS capture, all my findings and statements has been taken that into consideration. I always expect +/- 1 frame difference because of 60 FPS -> 30FPS Truncation. Looking at the numbers of the document, this method seems to be working, since both of the WiiUVC runs AND mGBA core run had the exact total of transition frames. DSLite hardware can be included based on this +/- 1 frame rule.

Edited by the author 6 years ago

The current rules I see are "If running on Emulator, Bizhawk 2.2.1 or later with the mGBA core is a must, otherwise your run will be rejected. Current runs using other emulators will have their time converted"

For now, I'm going to change them to "If running on Emulator, please specify what emulator you used along with its version, whether or not you're using the GBA bios, and what core (VBA-next or mGBA) you used if running on Bizhawk. If this information can't be easily found out, your submission might be rejected. Bizhawk 2.2.1 or later with the GBA bios and mGBA core is preferred since its accuracy is consistent with Wii-U VC and is closer to original hardware in terms of loading times. Current runs using other emulators will have their time converted." ...which is a long mishmash of everything suggested so far. Hopefully it can be shorter in the future.

I don't know if these rules were supposed to be for Maxim any% only or for all categories. Currently it's for all categories but those are not so much of a frame war that they matter nearly as much. It's also much easier to convert times for Maxim any% than for other categories, which I wouldn't personally want to do. I'll remove the "your submission might be rejected" part from other categories since it's not a frame war and the "current runs will have their time converted" part since I think it was only meant for Maxim any%.

It also holds off on banning certain emulators which it's still too early to do if at all with the limited data we have. First, there's a setting in VBA-RR that lets you use the GBA bios, which based on a little early testing I did makes the loading times closer to the mGBA core with GBA bios. Potentially, VBA-RR with GBA bios could be closer to WiiU VC than Bizhawk with mGBA core without GBA bios. I'd still like to do more testing though.

Second, Charleon: "DSLite hardware can be included based on this +/- 1 frame rule." - It shouldn't with just one trial. The transition frames often only differ by 1-2 frames, so just one trial of original hardware doesn't account enough for the random variation of your method. It might be better with more trials, but it still wouldn't be as good as doing it in 60 fps. Since most of what you're comparing are emulators anyway, it's not that hard to record TASes frame by frame for them.

The method I used to compare the GBA with emulators is to repeat easy screen transitions and see if the GBA desyncs from the emulator. First, I moved Juste through the door between Entrance and Marble Gallery, which gave me enough time to move back toward the door on the first possible frame. Since I did it on the double pack version and I know the door glitch works differently on that, I tried two more setups. In the second, I activated the two warps in the Entrance with Maxim and held down to warp between them. In the third, I candle-zipped into a wall in the Entrance with Maxim which is from the start two rooms right and one room up, which warps you around to the same room continuously until you slide out. The results are: 1: VBA-Next with bios > mGBA with bios > original hardware > VBA-RR v24 without bios 2: VBA-Next with bios = mGBA with bios > original hardware > VBA-RR v24 without bios 3: VBA-Next with bios = mGBA with bios = original hardware > VBA-RR v24 without bios (longer transition > shorter transition)

This was just a quick and inconclusive test to make sure that the original hardware isn't the same as any of the emulator setups. I'd like to pin down exactly where the original hardware is compared to the others more, but I've been busy with other things recently and I'll continue to busy for a few weeks, so I won't be able to do that very soon.

(Also, my runs have been mostly made on VBA-RR v24 without bios, and Maxim All Bosses has been made on Bizhawk v2.2 mGBA without bios. Depending on things, maybe that run is actually slower than PK's on the leaderboards.)

charleon likes this

Fully agree on everything you say Sopha! welcome to the investigation squad!

Oh yeah, sorry, I forgot to clear the rules after the discussion started, looks much better portraying where we are in the investigation right now

I'm still confused as to what data is suggesting mGBA is closer to original hardware than VBA-next when Charleon's spreadsheet shows mGBA to have faster load times, which is consistent with VBA-next being more accurate to console than mGBA in literally every other game I've compared them in.

By the way, BizHawk 2.2.2 is supposed to be coming out soon which contains an update to the mGBA core(v0.6.1) which according to the changelog contains many accuracy fixes. I compared this core in the recent mGBA emulator version itself to VBA-next, and it's still nowhere near as accurate as VBA-next.

"I'm still confused as to what data is suggesting mGBA is closer to original hardware than VBA-next"

I based this deduction on the data available in the table to the right on the first page;

The 2 runs made with WiiUVC consistently has a total of 107 transition frames. DSLite has 106 total transitionframes. Bizhawk with mGBA core has 107 total transitionframes (same as WiiuVC and only 1 more than DSLite), Bizhawk with VBA-Next has a total of 112 frames, which is 6 more than DSLite and 5 more than mGBA and WiiUVC.

So if I consider "DSLite" as "Original hardware", then mGBA is more accurate than VBA-Next, since the total transitionframes differ by a single frame, as opposed to VBA-Next's 6.

Edited by the author 6 years ago

DSLite isn’t original hardware; the Gameboy Advance is. mGBA and VBA-next aren’t emulating the DSLite; they’re emulating the Gameboy Advance, and VBA-next is doing it more accurately.

I agree with standardizing mGBA since its inaccuracies are effectively equal with VC/DSLite, but the “mGBA is closer in accuracy to original hardware than VBA-next” statement should be removed from the rules since it’s not true.

Also, I think using the proper bios should just be required for any emulator submission. If people don’t know where to get it, we can just include it in resources.

charleon likes this

Here's a plan for doing the conversion. Tell me if you think there are any problems with it.

1- Loading times for TASable emulators: To get an exact comparison of loading time between different emulators, I plan to record a TAS starting from when timing normally starts up to the earliest frame you can attack in the Dracula fight. There's no need to kill Dracula to do the comparison. For different emulators, I'll do the same TAS but add and remove idle frames to make it work with the different loading times, and the difference in the number of those frames will be how many frames are added or subtracted in the conversion. I think this will account for sorp's worries that the screen black-out time might not reflect the actual loading time, and we'll be able to tell if there is any difference between them. Since to play Maxim you have to clear a Juste file I'll do this part. I don't really want to do the other parts though.

2- Loading times for Wii-U VC: 30 fps isn't the best, but I think if there's a large enough sample size of videos being measured it shouldn't be a problem. Charleon's method of doing it should be fine, it should just have more samples, then take the average of the total loading time. I feel like about 6 would be enough but more would be better, so I'd say 6-12 or something. (I just made these numbers up.) There's enough of those those on the leaderboards for Wii-U VC. Also, every frame in a 30fps video should count for 2 frames of game time to make comparison easier. If I don't encounter the problem sorp mentioned in doing part 1, then we probably don't have to worry about it here and everything should be fine. If the black-out time really isn't the same as the loading-time, then it might still be always off by the same amount when doing the same transition from one room to the other, so it would be fine then too. But if it's off due to subpixels or something then I think we're screwed. My plan is to just hope it's not a problem.

3- Loading times for original hardware: I guess the best way is to record more samples and take the average like with Wii-U VC, but that will be annoying. Can anyone play on original hardware and record some Maxim runs on it? I only have the double pack version and a cell phone camera, and I don't want to do this at all. Also the original GBA doesn't have a backlight, so that makes things harder. Maybe as long as there's just one original hardware run it's not that big of a problem? DS Lite is backward compatible with the GBA, so I think it can count as original hardware rather than an emulator.

4- Damage-glitch lag: First figure out which things have the damage lag and which don't. I guess VBA-Next has it and mGBA doesn't, but it would be nice to make sure with a TAS. I don't know how to make sure for original hardware or to check if the lag is exactly the same, but I think it should be fine if we confirm it by feel and assume that it's the same amount of lag as VBA-Next if we decide there is any. Then since Dracula always has the same amount of health and the number of critical hits tends not to vary too much, for simplicity we can assume that it's the same number of lag-frames each time for the whole battle, and this number of frames will be the amount added or removed for the conversion. To find out the number of frames we can make a TAS without the lag, then change it to run with the lag by adding idle frames until Maxim is at the same pixels before doing the next input.

That's my plan. There might be some imprecision in the conversion, especially in the damage lag, but I think this should be good enough.