It has come to my attention that there is a significant discrepancy in some emulators (at least one emulator). The discrepancy is a timing difference that gives a significant time advantage.

I noticed it on a run that was recently submitted and so looked into it a little deeper.

This difference is measurable on the screen below the castle screen. In each run every runner moves non-stop, right to left, without any delays or variation. This means that I could use this to time the screen and work out how many frames to get from one end to the other.

Most emulators + Original Versions take 79 or 80 frames (@30fps) to get across the screen.
Some emulators take 72 (@30fps) frames to get across the screen. I compared the clock on the screen to a stopwatch just to confirm the video wasn't sped up or anything like that.

What does this mean: A 35 second run (20th) would become a 31.89 second run (=1st).

Upon inspection I have noticed this has affected at least 1 leaderboard time. I will not provide any details of which run(s) at this time as I do not want it to influence anyone's opinion.

To be clear, I do not believe anything intentional has occurred. This seems to be a previously unknown error in emulation.

I ask this community to please weigh in on this topic. Please provide feedback on each of these items:

1) For timing to be fair, any run that is using the wrong emulation should be removed. Please agree/disagree with this.
The other alternative is re-timing them based on a time-slow calculation (but would contain the quicker video).
2) I can do a review of a number of runs and re-time and reject those that are not accurate. Would I need to provide proof of this? What would that proof look like?
3) I would prefer not to have to re-time all 59 runs. Are we happy with top 10? Top 20? etc.
4) We will need to do a review of which emulator(s) cause this and include it in the rules as a banned emulator. Please agree/disagree with this.
5) For future runs, a quick check would need to be made. I again would prefer to keep this top 10/20/etc. Do you agree? Which runs should be timed frame by frame?

Hopefully a large number of people provide feedback to this. I would prefer to make any changes with group-consent. (If no feedback is forthcoming, I will talk to PresJPolk and decide upon the best course of action.)

Thanks.

TLDR: Some emulators time too quickly. Please respond to these questions:

1) For timing to be fair, any run that is using the wrong emulation should be removed. Please agree/disagree with this.
The other alternative is re-timing them based on a time-slow calculation (but would contain the quicker video).
2) I can do a review of a number of runs and re-time and reject those that are not accurate. Would I need to provide proof of this? What would that proof look like?
3) I would prefer not to have to re-time all 59 runs. Are we happy with top 10? Top 20? etc.
4) We will need to do a review of which emulator(s) cause this and include it in the rules as a banned emulator. Please agree/disagree with this.
5) For future runs, a quick check would need to be made. I again would prefer to keep this top 10/20/etc. Do you agree? Which runs should be timed frame by frame?

1) I concur with removing runs that have an emulation advantage.
2) Whatever is giving the advantage should be enough proof, I'm not a very techy guy so that's a little out of my wheelhouse.
3) For the runs that aren't obvious, you could probably redo just the Top Ten.
4) Agree with listing either banned or approved emulators.
5) Probably reserve frame counting to the Top Ten for your sanity.

My $0.02

1) I do think any runs with the emulation advantage should be removed
2) If I read it right illegitimate runs would run faster than legitimate runs. The proof could simply be comparing an illegitimate run to a legitimate run and showing the speed difference.
3) I'm willing to go through and do all 59 runs if you guys would like but that's up to you. My main concern just lies within the top 10 or 20 or so however
4) I agree with listing banned and approved emulators
5) I'd be willing to become a verifier and check each run since I'm pretty active on this site but frame by frame timing should be reserved for top level times (top 10 probably)

I agree 100%. Full disclosure: I always use original hardware. Apples to apples, I would say.

I will wait until next Monday for any more replies before pushing forward with any the next step.

My current plan will be a top-10 check and removal of runs with non-accurate emulation.

More feedback before then is very welcome.

Rom in resources should be categorized for me