The longstanding issue of emulator submissions
7 years ago
Germany

Hello! I think I can speak on behalf of a large part of the GB speedrun community and here is what's been bothering us for a long time now: We're in the great situation that with BGB and Gambatte there are 2 very accurate emulators when it comes to GB speedrunning. Therefore, emulator submissions are treated no different than console runs on the leaderboards for a vast majority of games (there are, however, emulation differences in some games/categories: memory-related glitches, e.g., where emulator submissions are excluded). Anyways, there would be no issue if runners would stick to accurate emulators, but sadly VBA and its derivatives are still a widespread annoyance for leaderboard moderators and, eventually, runners of GB games alike. Having comprehensive joint leaderboards with emulator and console runs side by side is a great thing. It encourages new competitive runners to give speedrunning and sharing their results a try without having to lock in and spend a triple digit amount of money on a hobby they're not really sure about yet. However, with VBA still being so widespread and not everyone aware of its inaccuracy, many leaderboards are currently filled with invalid runs, including lower ranks and top positions alike. This issue has been known for many years already and there are still VBA submissions coming in left and right. My stance is that I don't want to exclude emulator runs in general, at all, but rather spread the word about accurate emulation and help to fix the current leaderboards accordingly. Above all, leaderboards should be fair and achievements compareable. Past submissions obviously would have to be filtered "manually". That's pretty much every single game community's individual choice and it can be tedious depending on the number of submissions. However, for the present and future I'd like to request a feature that tackles the issue before such runs land on the leaderboards. There IS already the checkbox to mark a run as played on emulator. It would help a lot if the submitter had to specify the emulator they used during the submission process (not a dropdown, but a text field, I suppose, since a fixed list would be rather long and likely still not comprehensive), coupled with a note that VBA is NOT a valid option - maybe appended with a recommendation of BGB and Gambatte. If there was a way to make that mandatory for all GB games that would be ideal. This could, of course, be applied to any platform for which accurate emulators exist. I'd like to hear a few opinions on this - from speedrunners and site admins alike, preferrably.

Thanks & see you around!

AndreaRovenski, Aureus_Lunae and 4 others like this
Pennsylvania, USA

If both BGB and Gambatte could both run under Wine in Linux (I already run VBA under it), then I'd give them consideration. That being said, I don't really see what the problem is with VBA.

Germany

The problem is inaccurate emulation, both in clock speed and in lag emulation (and I think load times are affected as well in some games). With this combination of inaccuracies it is impossible to "adjust" such a time to be in accordance with console runs. They're just not comparable.

(An example from just today: VBA emulates the GB game F-1 Race ~0.92 % too fast over the segments I timed.)

Edited by the author 7 years ago
pld and kobepilgrim like this
New Jersey, USA

@Cool2man

"BGB is a GameBoy emulator/debugger which runs on Windows and Wine"

http://bgb.bircd.org/

Edited by the author 7 years ago
Theguesst, Oh_DeeR, and pld like this
Belgium

Using VBA is literally like doing sports under the influence of performance enhancing drugs. It didn't fly for Lance Armstrong, so it shouldn't fly for anyone.

Check out http://www.speedrun.com/Speedrunning/thread/4sdl9 if you want more information about differences between emulators.

Oh_DeeR and pld like this

I really like Oh_Deer's idea and I would like to see the feature being implemented as it would make the work as gb moderator much easier.

@Cool2man

I would recommend to check this thread :) http://www.speedrun.com/Speedrunning/thread/4sdl9

Edited by the author 7 years ago
Oh_DeeR likes this
New York, USA
Theguesst
He/Him, They/Them
7 years ago

With due respect moelle your intentions are good but your analogy is simply inaccurate. The fact of the matter is that VBA has multiple revisions and versions with different purposes such as recording, multiplayer, and redos to the coding. This complicates things of course, but no version of VBA is accurate in emulating gameboy games; there are frame inaccuracies compared to consoles and programming errors aren't correctly emulated in some cases. Dedicated gameboy emulators are superior in accuracy the main two being BGB and Gambatte. Making it easy for runners unaware to get the emulator and submit with is is very important.

Oh_DeeR likes this
United States

I think for games where it matters, it makes sense to have a custom variable with a list of emulators. Make it mandatory, and if people out in a bad one, then you know to reject it.

TheGreatToddman, pld, and Oh_DeeR like this
Germany

Interesting suggestion, PresJPolk. After testing that, I have to say that it doesn't seem like a viable solution, at least right now. First and foremost it requires moderators who know and care about it. As pointed out earlier, many simply don't know what they are doing. Secondly, unless I'm doing something wrong, the feature of custom variables doesn't seem to be in a fully functional state. I can't seem to change the column order, i.e. where the new variable shows up. I can also not set the default view to show runs of any option of the new variableĀ¹. Am I missing something?

Ā¹Edit: I found a workaround for the default view: set a default option, apply it to all existing submissions, then remove the checkmark on the default option. Makes this more viable.

I stand by my point that the issue is most effectively tackled globally instead of on a per game basis.

Edited by the author 7 years ago
New Jersey, USA

Is VBA even worth using at all? When I play GBA games, using save states causes it to crash. Does anyone else experience these issues? Maybe it's different for game boy games.

This is perhaps slightly off-topic, but I no longer use VBA for GBA games. mGBA is better, for me.

Pennsylvania, USA

I haven't had save states crash VBA on me.

New Jersey, USA

@Cool2man

That could be because of the version that you use. I should have mentioned that I'm using the Windows version on Windows 10.

Edited by the author 7 years ago
United States

If the moderators of a game you care about don't know what they're doing, contact them? I don't see the problem here.

kobepilgrim likes this
New Jersey, USA

@PresJPolk

I agree, though it is still helpful to also create a more generalized awareness.

Edited by the author 7 years ago
Belgium

Although contacting runners or moderators (which is also still a hassle as it can't be done on the site itself yet) usually ends up in an agreement, there are still some rotten apples in the bunch.

I still remember the person who didnt want to leave VBA behind "because now the game feels slower and I have to relearn it completely".

We have been trying to spread the word about the emulator problem for months, heck even years, but it's impossible to reach everyone so we still have to check every video of new runners or games thoroughly to see what is being used. Having an option to let us know in advance would solve a lot of unnecessary work in the future.

If it was up to me, the moment you select VBA as an emulator option, the run should be denied with a message showing up why.

pld, xDrHellx, and kobepilgrim like this

"If it was up to me, the moment you select VBA as an emulator option, the run should be denied with a message showing up why"

^This!

Of course people who want to cheat will simply lie and pick a different emulator so these runs will be sorted out manually but such an option would at least automatically filter out the "fair" submissions, i.e. people who really simply didn't know about VBA.

xDrHellx likes this
United States

Mostly agree with first post. Tracking emulators used modularly would certainly be useful and desirable, but it's some amount of work to implement. Does the site track an emulator list and the game chooses from it, or does the game add arbitrary emulator names?

I think a page of recommended and/or banned emulators should be created. It's pretty much consensus that VBA for GB/GBC games is bad, and PJ64 2.0 is bad. I can't personally make a complete list without some user contribution, since for example I have no idea what consensus is for Playstation emulators these days.

SpeedRunsLive page says the following:

You may not use the following emulators for racing: SNES - ZSNES, snes9x 1.4x N64 - Project64 2.x GB/GBC - VBA, VBA-M GBA - VBA-M (use VBA instead. If VBA does not support your GBA game, please contact an op.)

It's not particularly thorough for the amount of consoles out there, but it addresses the most egregious and common cases.

Edited by the author 7 years ago
France
xDrHellx
He/Him, It/Its
7 years ago

[quote] I think a page of recommended and/or banned emulators should be created. [/quote]

http://www.speedrun.com/Speedrunning/thread/19h43

While there's still no "list" so far, most posts in this thread explains wich emulators to use for most stuff, and why.

New Jersey, USA

I was wondering why so many people are drawn to VBA for GB games, in the first place. I used it initially because I was playing GBA games and noticed the extra feature, but I don't suspect that GBA games are so popular that this is a commonality. But even for GBA games, I ended up moving to mGBA, so I couldn't figure out how people choose VBA as a starting point for GB games. Then I realized that it's toted on ROM sites as one of, if not the best emulator for GB games.

Could there be a way to propose removing VBA from those types of lists, at least on the major ROM sites? Does anyone have connections with anyone affiliated with one or more of them? I would argue that VBA isn't even the best for GBA games, let alone GB.

Edited by the author 7 years ago

Well if you search for gb emulators on google VBA is the first emulator that is shown and I guess a lot of people don't bother to check if they are actually using an accurate emulator.