How to handle the issues with FPSFix
3 years ago
Netherlands

It has recently been discovered that the FPSFix mod which we decided to allow for PC runs a little over a year ago has effects beyond what we initially realized, particularly on machines with AMD processors. Below are the results of the investigation into this done by MrTyton:

"Ok, so good news and bad news.

There is a difference when utilizing FPS Fix. However, this seems to be localized only to AMD processors. As best I could tell, there is no difference between Intel, Intel w/ FPS, and AMD - the difference only arises when utilizing AMD w/ FPS, resulting in about a 4-5% speedrup in constant movement when compared to AMD, and transitively to any utilization of Intel.

Doom timer does seem to be affected by this as well - while I lacked any direct comparisons between AMD and AMD w/ FPS, by the transitive property described above the results should be the same. I am unsure as to whether or not it affects other battle issues (including, but not limited to, result screen, stagger duration, etc).

These conclusions are brought from 7 computers, 3 Intel 4 AMD. While I would have liked more data, I made did with what I did - I was unable to perform any statistical tests. The thing that I was looking for were increases in the differences between sequential events occurring, in order to try and account for slight differences in movement/inputs. If an increase was consistent over a long period of time, then this helps to eliminate random noise.

A small amount of the differences between AMD and AMD w/ FPS were due to base AMD having a large increase in sub-60 FPS. However, I do not think that this would have accounted for all of the difference seen.

Why is this happening?

Well, the likely cause is due to the removal of the framepacer, and AMD instructions operating differently than Intel instructions, such as a floating point difference. What these differences are... I'm not sure, sadly."

We now have to grapple with the question of what to do with a mod that has allowed quite a few people to have a better experience running the game, but is also causing some to gain an advantage beyond what is possible in the unmodded game, and which is hardware dependent and thus not attainable for everyone.

Potential ideas include straight up banning the mod, having runs with it hidden by default (similarly to emulator runs on Pokemon boards) or having some kind of leaderboard split. Furthermore, depending on what solution we go with, we will have to think about what to do with existing runs on the board. I will post my own opinions on these issues later.

Obviously this situation sucks for everyone, but please be civil and respectful to others' opinions on this matter. Thank you.

Portugal

The fairest solution in my opinion is splitting the leaderboard into fix/no fix, but the simplest is just banning the fix. I am one of those that stands most to lose without the fix, so this is a biased opinion. I still don't know what my PC's specs are because my brother hasn't been able to tell me, but in Nautilus my PC stays at 40-50 FPS while Soma stays in constant 60, both with the fix, as a simple term of comparison. The game will probably be unplayable for me without it, especially after I got used to using the fix, and if it is still playable, it will not be able to compete against other setups because it becomes too bad without this massive crutch. However, I've never personally cared about being in the leaderboard, so if the decision is to ban the FPSFix, I will just not submit my times and constantly run against myself only, which was always the goal to begin with. If the leaderboard is split between fix/no fix, runs that are submitted there should include the type of processor so it's clear to both runners and lurkers why some runs might/might not have an advantage.

Edited by the author 3 years ago

It's my opinion that a speedrun community should first strive to be inclusive and then make it as fair as possible for everyone involved. So from my standpoint the most elegant solution I can think of, until there is some way to either convert the times or configure the fix to not speedup AMD processors, would be the following:

  • Split the PC leaderboard into AMD and Intel boards, as this is the criteria that blocks direct comparison due to the speed difference with fix used
  • Add a variable for Fix Used Y/N so that those who really want to can filter out runs that have or haven't been done on FPS Fix

That way the two boards are within themselves comparable and everyone gets a shake at it.

Lorraine, France

Here is my though (sorry for grammars/synthaxes) I'm intel user

I ran the game before the fix came up. I always complainned during my stream like " wtf this lag " " man, i'm sure i lose time here because of lag " I was getting used to it in the end. But dodges were hard when random drop fps. Bridge 2, bart's fight are massive fps drop.

Running with fix make the game perform better, smoother. But not faster. I still got drops randomly everytime on same place and in every chapter. Since i know where i'll get it it doesn't affect me that much nor the gameplay.

It's not because i was getting used to run with massive drop, i want it back. Definitvely not.

The problem came up with AMD which made the game running faster. Not only smoother I wouldn't mind much if it was small difference like less than 1min. But i understand for fair competition and wr it's difficult to accept it.

Like Daspharao i think if the final decision is to ban it ( which is understable ). I'll keep running with it and no submit my run.

AMD speedup might need more data, research to have really precise how much it saves.

Here is an option i propose. ( probably not the best but maybe simpliest):

1_ Do not change time nor adjust/modify

Add Filter: amd/intel ( incluing that everyone is using fix ) Add in rule : For reference ad like 1% time to amd user. (which would be around 2.30min in 4h50 time.)

This allow to have everyone running, getting into ldb.

The down side is for wr time i've no clue how to handle since we don't know how much time save amd over intel.

I've not better idea to handle this problems. Unless beeing strict like pokemon ldb and force no fix.

2_ Separate AMD/INTEL like we would separate PAL/NTSC-U/JP

Best regards Soma

Tokyo, Japan

My opinion is very biased because I have never used FPS fix and 15 FPS doesn't bother me if that's how the game works. FPS fix should be banned or hidden by default. I don't think anything can justify FPS fix if it makes significant time difference.

If splitting PC category is preferred, it should be "AMD+FPS fix" and others.

(Also IGT timer when)

(edit) There are more investigation ongoing. Things written above is based on the initial information.

Edited by the author 3 years ago
United Kingdom

My (admittedly somewhat distant nowadays) opinion around this is that it mirrors the discussions which were had initially around introducing use of the fix.

Original thread here for reference: https://www.speedrun.com/ff13/thread/6tped

My perspective is if we're now in the position where there is proof that the fix can have a positive impact on times outside of just framerate stability then it should be banned. I appreciate people don't enjoy running the game on lower framerates or unstable setups but that to me is just part of PC speedrunning, as unfortunate as it is - and with the port being what it we have, there's not a whole lot that can be done in terms of other mitigations (i.e. accurate in game timer or similar).

We should have a way to indicate every run done with the fix, as that was part of the initial inclusion to the boards, so for the sake of not destroying history I would at least ask for a variable to be added to indicate if a run was done with the mod or not, and that would allow for a later stance around separation to be made, while indicating that it is not an ideal situation to continue on with.

Poland

If we were to split the leaderboards, I like the idea of splitting it AMD+FPSFix / other. It gives AMD users who want to compete and compare with Intel users an option to do it. At the same time, AMD users who need to use the Fix in order to be able to play the game smoothly or want to use it for other reasons would have a leaderboard to submit their runs to.

Another issue stemming from it is if we'd allow AMD users on the "other" leaderboard to use OneTweakNG. FPSFix is a fork of OneTweakNG that additionally removes the frame pacer and offers some graphical fixes. The removal of frame pacer seems to be the main reason why FPSFix improves the framerate. OneTweakNG alone also makes the game run a bit better (at least for me), while not offering the "speedup" that FPSFix does due to frame pacer removal. Thus, I don't see any drawbacks of allowing AMD users to submit runs with OneTweakNG to the "other" leaderboard if they wanted to compete with Intel users while still having some sort of fps improvement. If anything, this would put them at a disadvantage compared to Intel+FPSFix users.

It should be also pointed out that whatever we decide would probably have to be propagated to the longer categories' leaderboards. And then there's the very niche issue of Trilogy% leaderboard, if more AMD+FPSFix users would want to run that (and I know Pharaoh showed interest).

Overall, I prefer splitting the leaderboard in some way over banning the Fix completely, as it does allow a lot of people to fairly comfortably run the game, which would otherwise be basically impossible. Even if that's how it often is with PC speedrunning.

Edited by the author 3 years ago
North Rhine-Westphalia, Germany

I was suspicious towards the fix at the start and not sure if it really doesn't change anything important aside from FPS. In the end it was analyzed and shouldn't have made any significant difference. Now we know that was wrong and some runners save time over other runners by doing nothing better, just having a slight speedup.

Even if the speedup would "only" be 20s in the whole run, it makes the competition less fair (my console run rn is ~18s behind WR currently so 20s matter even in a run this long). That's why i think the fairest solution would be to run the vanilla game again as it used to be done before the fix was found and also done in the majority of other PC speedrunning communities who don't even have such fixes.

If the bad port results in fps drops and timelosses on lower-end hardware that's something that is an issue that is out of our control in the community, because that's the way square enix made the port. If an external mod gives a significant advantage to some runners over others though that is something we can control by either disallowing it or restricting it. Possible solutions would be:

  • ban the fix -> everyone runs vanilla, some better, some worse, but it's the same game
  • split the leaderboards to runs done with intel/intel+fix/amd and another one with only amd+fix
  • hide runs done with amd + fix by default

Overall, no matter what the decision will be, it won't satisfy everyone and that's something we have to deal with.

Rio Grande do Sul, Brazil

My opinion here probably doesn't matter much, cause i'm not even a runner, just a learner on console category and follower of this amazing community, but here it is. I see a reason to ban FPS fix, even tho we don't know for sure how much the difference is, the fact of it improving performance for certain hardwares is already a good way to justify it, but this community never cared so much about something like SSD/HDD, something that clearly makes a lot of difference in load times, so why care for this now? If we think about the possibility of the AMD+FIX ever making a difference is minimum, Final Fantasy XIII is a run about so many things, precision, fight execution, the focking dodges, even improvisation sometimes, there is so many variables in a 4+hours run that even a ~4% speed up on walking sections will probably never make a significant difference (and probably have never made until this day).

Yes, the Fix is a mod, and mods are something tricky to decide on a speedrun community, and it is different from installing a memory drive on your PC, but we all need to be honest, this port is shit... and even tho having a decent pc to speedrun is something obvious and basic, we need to see that this port is very unstable and poorly made, and the fix is something that have always helped all users (Intel and AMD) dealing with it (and we all know that the port is bad even with really good setups), and probably brought some people to the community, like, i am the only brazilian here, probably one of the few members of the community from the third world, where hardware is super expensive (eat or play games), i have a simple AMD setup, the game runs ok alone but with obs it starts to fuck up, if i buy a simple video card and use the fix i can have a really decent performance to start running pc. So for me, if we ban the FIX, we will be looking just at time, speed, competition, instead of looking at a community that reunites around the love for game that people normally misjudge.

My solution would be: One PC leaderboard but now detailing "Intel/AMD+Fix/no Fix" and "SSD/HDD".

PS: without mentioning the thousands of weird interactions that the fix and the game itself can have with different processor+video card combination, we maybe dealing with something so variable that we can't even count on.

Ontario, Canada

Though I'm not a speedrunner of this game, I'll include my opinion on the matter.

Keep the FPSfix.

Perhaps I'm thinking too much in a floaty manner, but when I ask why we speedrun, it translates "to go fast". Why are we restricting ourselves to moments of lag, poor performance, and slow gameplay? Surely everyone wants the game to run completely stutter-free. Are we speedrunning more authentically when we stare at a load-screen for an extra 5 seconds, or when the animation lags to 45 fps for a brief 1.2s? If there is way to increase the funamental "runability" of the game, whether that involves improvements in rendering, animation, processing, loading, booting, or even button prompt comfort, I say go for it.

For the sake of grand equalizing, wouldn't the answer be to force everyone to revert to a Fixed 30 FPS, minimizing performance enhancements on everyone. I feel nobody would be satisfied with this conclusion, even if on paper it looks "fair".

Having said that, I am not privvy to the inner-workings of the speedroute in this game, and how dependant they are on the FPS at critical points in the game, so this opinion holds low value. To what extent can we call a game broken/unfair? I would hope the line there is black and white (as an arbitray example, using Cheat Engine to float through the air to get to the end of the path is blatant cheating), but I do not know if the disadvantage appears when some have their games at 55fps mid-battle while others drop down to 40fps. Claiming that the "person with the best/priciest PC gets to be the fastest" is not a community-friendly notion, and I will not advocate for that. Past instances of speedcommunity tension in my history have simply been resolved through the decision to "add more categories to the speedrun leaderboard". There seems to be this thought floating around here already. An addition may involve submitted runs to post their PC specs alongside their submission, since it seems AMD vs. Intel is a crucial factor.

Stay fast speedmen.

Spain

A lot of valid opinions have been given already and I would like to give mine too.

I think that the fix should be banned, and if not that the fix runs should be hidden by default.

I don't wanna repeat what others have already said like Kaya or MLSTRM, since I completely agree with their points.

I just wanna point out the last discoveries in the discord. Through the same fight (Orphan 2 Doom timer), six people have posted their times of the doom timer with and without the fix. For everyone without the fix they have the same time, but without the fix for some it's several seconds faster and for some several seconds slower, independently of CPU brand.

There were always inconsistencies on how the game runs because of the "PC lottery", but if six runners with six different PCs get the same time without fix and different times with the fix I'd say that the problem here lies within the mod, thus should be banned.

Florida, USA

This game punishes runners enough as it is. We should not punish them ourselves - especially as existing runs and runners submitted their times in good faith.

I see two lines of thought in today’s discussion. Either your primary concern is having a more inclusive environment, where more people run the game - or your primary concern is sanctity of existing top leaderboard times.

So who matters more? The people at the top, or the number of total runners? The better answer is to solve for an answer that solves both.

Whether this is a loadless style script, a timer that counts total frames, something that fixes in game timer? I don’t know the solution.

But if the goal is to enforce arbitrary standards of what a good runner is, by gatekeeping against certain hardware, then the real monsters were the friends we’ve made along the way.

California, USA

My opinion is to allow the FPSFix in some form on the leaderboards. I'm not particular as to whether that's through splitting the boards or a hidden filter or anything else. Some distinction should definitely be made if allowed.

However, I'm still learning the run and have no interesting in grinding for a top time. When I did practice without this fix this week, while everything was doable, I definitely noticed the difference. The FPSFix provides enough of a quality of life improvement for playing the game, that I'm in the camp of continuing to use it even if it gets banned from the leaderboards. So my main bias is simply, if/when I get a run I'm happy with, it would be neat to see my name on the boards.

England

As a fan of this run, its runners and wanting to learn it myself some point down the line I am a little conflicted. I see the need for it to be banned as I share the opinion of Hoishin, anything that effects the game should not be allowed, I know the port has bad FPS issues but thats part and parcel of the game surely? Again though my opinion is biased as I have an intel build and when running a game I like to run categories that have a lot of runners, I don't feel as satisfied with a good time if there isn't much comparison/competition and I fear splitting the leaderboard would lead to making each category for PC looking barren.

EDIT: just seen a decision has been reached already so ignore my slow ass Kappa

Edited by the author 3 years ago