Emulator comparison results
3 years ago
Italy

Hi, everyone. So, here are five emulators being compared. Why are we doing this?

  • To see if IGT is consistent across emulators and, possibly, comparable to what we get on PSN platforms. The dream would be being able to reunite PSN and emulator in the same category, ruling them with IGT and keeping the competition nice and fair.
  • To see which emulator is faster (if IGT stays consistent).

The consists in playing the very first segment of the game (which is very linear, no surprises, not too much RNG) and saving the game, so that we get to see the IGT displayed on screen. Note: there was some lag in a couple of tests.

And yes, ePSXe, bad, oh no. I don't care, it's for science.

The results are interesting:

  • ePSXe and PSXJin are the fastest of the bunch and also have faster IGTs, which look consistent between them.
  • Mednafen, BizHawk and DuckStation are quite slower but still close one to each other. Their IGTs, again looking consistent between them, are slightly slower than what I got on ePSXe/PSXJin.
  • The RTA-IGT gap on ePSXe/PSXJin is around 22 seconds.
  • The RTA-IGT gap on Mednafen/BizHawk/DuckStation is around 26 seconds.

This test unfortunately proves that there are inconsistencies across emulators. I would suggest, and I think this is what @If_And_While wants to tend to, to find a single emulator solution. Next steps that come to mind are:

  • Gathering similar recordings of the game on PSP, PS3 and PSTV and compare the times between those and with the emulator times.
  • If IGTs are different even across PSN platforms it's going to be fun :D
  • If IGTs on PSN are consistent (= we can consider them all on the same level, comparable, competitive, fair) verifying if one of the emulators has IGT comparable to PSN.
  • If yes, considering the possibility of having that emulator as the only one allowed and reunite the categories.
  • If not, keeping the categories split (but again, in this case I would suggest having only one emulator).

Italy

A full run test sounds quite difficult to perform, but the test is still mandatory, in my opinion, for the PSN platforms. Eventually we could do the same on PS1 and PS2, since there was an argument on having those console together as well.

In my opinion, there should be at least 3 emulators which are allowed to use. Well, I think it is necessary to allow all players to select the most comfortable emulator for themselves. But there is appearing one question: what should be done with runs made on PSXjin?

Italy

I think they have to get deleted. The times are just not comparable to the other emulators. One minor thing that I hate about the others, though, is that they all emulate the game very slowly. The run already takes forever because of all the garbage between camera angles.

Ontario, Canada

You should not be deleting runs, they should be grandfathered in or moved to archived category(s) and labeled miscellaneous. I am very sure no one would want there time and effort later told to be worthless because at the time they were submitting the rules were different even if the runs no longer fight at all in the current rulesets.

If there is a way to preserve them that would be the best way to do so, deleting/rejected otherwise already verified runs should only be don't if absolutely necessary and this generally is only when the run is either missing video proof or is found to be a cheated submission, which none of those are what this issue currently is.

Valhalla

@Nazzareno Yeah rather than be deleted, old Emu runs should go under a separated category (Legacy) or Misc. like they did in the Parasite Eve 2 board.

One additional point is that the tests made so far cover only one aspect of the emulators problems. The test is limited to one version of different emulators in the same machine. The reason why in PE2 community they were forced to select DuckStation is because it was the only and the most consistent emulator across different machines and graphic settings for the Parasite Eve 2 game at the end. All the other emulators were inconsistent even in the same machine with different versions of the same emulator, same version but different machines and even same version same emulators but different graphic settings. I'm not suggesting DuckStation for DC2. @Dimension_Bomber My point is you not only need to select one emulator and one only, but also decide which version of that emulator and possibly the graphic settings, so to have a fair competition.

@MODS please don't even consider merging together PSN and Emu ever again. It is a disrespect for console runners if not for timing at the least for philosophy.

Edited by the author 3 years ago
Italy

Yeah, I agree about not deleting runs, my bad.

@RapixOnGaming, about different emulator versions – it goes without saying. In that case, I understand why ePSXe is unwanted. It is due to an easy change of the video plugin which could slightly accelerate the run... Or turning on the frameskip... Or setting up, for example, 61 fps instead of 60 fps. I know it is also possible in Mednafen, but it has to be done much harder. Don't remember about BizHawk. I think it is the same with Mednafen.

England

I know we are currently working out which emulator would be best to use for the leaderboard but could I have some idea on what Emu I should be using if I haven't got it already? Are we using PSXJin? And if so, how do you set up your controller to work with it because I am currently having no joy. And if it is Duckstation, would it be usable through RetroArch?

Edited by the author 3 years ago
Italy

Duckstation should be the one, but whait for the Mods to make it official. Same for using it through RetroArch: it sounds fine for me, but I don't know.

England

No problem. Just I want to run for WR next week and want to make sure I’m using the correct emulation and programs. Thank you for the advice!

Game stats
Followers
241
Runs
643
Players
89
Latest threads
Moderators