Ninja Gaiden (NES) Forum  /  Emulator ban: Mesen (Locked)


The mod team has decided to ban the emulator Mesen because of a potential run ahead feature, listed here: "Run ahead allows the reduction of input lag by the number of frames specified. CPU requirements increase proportionally with the number of run ahead frames specified."

We've noticed other boards for speedgames ban it too and we'd like to implement a ban across all NES ninja gaidens as well.

All currently submitted runs done with Mesen will be approved, but future submissions will be rejected

MarthSRMarthSR, twin0megatwin0mega and 2 others like this. 

Hello all! I just finished making a video in reaction to the growing trend to ban Mesen as a legal emulator for NES speedruns. This trend is motivated by the FALSE belief that the "run ahead" feature is undetectable. In the video I show how to detect it and make the case for banning the feature, not the emulator. Mesen is a high accuracy emulator, and I believe it's worth fighting for against this misinformation. Please watch the video and consider sharing it with your moderators. #SaveMesen

twin0megatwin0mega, SjohnSjohn and 2 others like this. 

Was a real shame when it got banned. The other emulators that work with the auto splitter just don’t cut it (if you want to stream)

twin0megatwin0mega likes this. 

Informative video! My only issue with MESEN from a fairness standpoint, is it would seem an unscrupulous actor could use the history viewer to fake a run with the rewind feature. Makes it very convenient and easy. Unless I am missing something and there is some way to detect such a thing.

twin0megatwin0mega and Thee_DeadguyThee_Deadguy like this. 

don't know much about history viewer and the rewind feature. just played around with it for a bit and couldn't get it to work in a way that would be useful for a cheater, but again i don't know much about it. looks like rewind has been available since version 0.9.0 and this is the first i've ever heard anyone pointing it out as a concern. the Mesen bans have been motivated by run ahead.

in terms of how to detect history viewer, i could look more in to it, but i found one way. Whenever you open Mesen, there is a game selection screen letting you choose from the last four ROMs you ran. that screen isn't captured by the history viewer. if moderators were to become seriously concerned about it, they could require every submission to start from the game selection screen (added bonus, the game selection screen confirms which version of Mesen you're using). this would require a runner to close and reopen the emulator for every attempt. alternatively, the submission need not include the screen, so long as there's a VOD showing that your series of attempts started from that screen.

And as i pointed out in the video, there are many undetectable things that a player could do (ram/memory watch, available on many EMUs) if they wanted to cheat. We ban the use of the features, not the entire emulator. ultimately the only real defense against such things is a history of attempts proving you have the appropriate skill level, and good will with the community. And even those don't guarantee anything.

there's no limits to what people "could" do, if they wanted to cheat. That would be true even if we banned all the emulators. even an "unscrupulous actor" playing on console could splice captured footage, or use a TAS chip.

twin0megatwin0mega and blacksquallblacksquall like this. 

I've been following several NES leaderboard forum discussions on this topic, and have noticed a few main points that deserve consideration.

First, the smb1 leaderboard is clearly setting the standards that many of the other NES leaderboard mods are following. Every Mesen ban I've seen seems to have been directly prompted by the smb1 ban. This is generally fine, but I think it also creates a situation where anyone who disagrees with the reasoning behind the ban has to defend Mesen against a variety of other issues that, while maybe important for other discussions, aren't really relevant to the issues that the mods actually presented.

All of the discussions, including the original one on the smb1 forum, seem to have some trouble with sticking to the main arguments, which are: (1) using runahead presents an unfair advantage, and (2) banning runahead in Mesen isn't feasible because there's no indicator that it's active; therefore, it's necessary to ban Mesen.

NES leaderboard rules generally exist to strike a balance between these concerns:
1. Provide a level playing field for runners
2. Require enough recorded evidence for mods to be able to verify runs
3. Be accommodating to accessibility issues, and make a reasonable effort to lower barriers to entry

I actually want to know: does using runahead present an unfair advantage? Has there been any compelling evidence for this argument? There is definitely some uneasiness and uncertainty that it _could_ give a significant advantage, even if it's unknowingly being used. Howeve,r, the only evidence presented in the original smb1 post (and certainly the entire reason Mesen is now under scrutiny) is that Sour, Mesen's developer, helpfully pointed out in the documentation that speedrunners should disable runahead. Here's the quote again for reference:

> Run Ahead: Run ahead allows the reduction of input lag by the number of frames specified. CPU requirements increase proportionally with the number of run ahead frames specified.
> Note for speedrunners: Using features such as run ahead to reduce lag typically counts as cheating for the purposes of speed running.

Of course, nobody would be surprised if the same warning was given for this option:

> Allow invalid input: On a NES controller, it is impossible to press both left and right or up and down at the same time on the controller’s D-pad. Some games rely on this and pressing both buttons at once can cause glitches. When enabled, this option makes it possible to press opposite directional buttons at the same time.

But I think the note about runahead strikes a nerve particularly because that feature isn't very well understood by this community. Most of us would be confused by a sudden ban on FCEUX just because it has the option for "Allow Left+Right / Up+Down", even though there is _much_ stronger evidence that it gives runners an unfair advantage as compared to runahead.

It might be helpful to give a little more insight into why Sour decided to include the note about runahead in the documentation in the first place. Here's a conversation between Sour and another well-known NES Development community member from late 2019:

> Sour: if I had known runahead would only take me a couple of hours to implement, I would have done this a long time ago. >.>
> Fiskbit: Keep in mind that this feature is a big problem for speedrunners. Some communities have outright banned RetroArch because of it. It's a super cool feature, but definitely has some competitive implications.
> Sour: it's technically pretty trivial to verify that a user isn't using it if the speedrun shows the inputs on the screen
> Fiskbit: Ah, that's an interesting point.

SpaceColonizer's video does a great job illustrating what Sour says here, which hits the second part of the original argument for the Mesen ban. It _is_ possible, and according to the Sour "pretty trivial" to verify if submissions are using runahead. They go on to discuss Sour's idea to include an indicator in the Mesen UI that shows if runahead is enabled, but he concludes that it's not really necessary since it could just be hidden or removed in the source code, and runahead can be detected anyway. Fiskbit goes on to say:

> Fiskbit: That's true, but the majority of speedrunners are not very technical, and this helps ensure that people don't accidentally have their emulators misconfigured.
> Fiskbit: One of the hurdles is that the official emulator lists used by many communities are very outdated, suggesting requiring specific old versions and ignoring newer, better emulators.

(For anyone curious, runahead is disabled by default in Mesen.)

Finally, I'd like to make one last point, echoing what Fiskbit says about outdated emulators: I think the significantly higher accuracy of Mesen should carry _some_ kind of weight in this discussion. FCEUX only passes around 50% of the standard test suite for NES emulation, but as SpaceColonizer showed in the video, Mesen passes 100%. Nestopia passes around 80%. It seems somewhat contradictory for a community that places so much value on using original hardware to throw out the most accurate NES emulator for having one feature which may or may not give a slight advantage. What kind of advantage might there be due to bugs and glitches in low-accuracy emulators, which would certainly be much harder to detect?

I think I've hit the main points here, but just to reiterate: I would genuinely like to understand what the real competitive advantage of runahead is, if anyone can provide an example.

OopsKapootzOopsKapootz, MagicKMagicK and 6 others like this. 

Agree completely with Gordea. Unless someone has hard evidence that Mesen's runahead feature gives a meaningful advantage to running NG (I highly doubt it), this seems like a pointless ban. Mesen is a great emulator and it's sad that we can't use it.

twin0megatwin0mega, SpaceColonizerSpaceColonizer and Thee_DeadguyThee_Deadguy like this.