Forums  /  Sonic 3 & Knuckles  /  Rule changes for 2019 - Discussion
  TimpZTimpZ

There has been talk recently of the rules being outdated. This thread will be used to discuss potential changes that have been suggested.

When replying, try to separate your arguments and preface it with the category so it’s easier to follow.

-- Emulators --
The current list of allowed emulators is outdated. Many runners nowadays use varying front-ends with differing cores that blurs the definition of what is actually allowed, originally the list was made for standalone emulators.

General requirements:
- Accurate enough emulation to not gain a real-time advantage over console.
- No input-replay functionality (TAS)
- RAM watch and edit not available
- Hard coded settings that make it harder to do video splices (e.g. no setting to turn off notifications on creating savestates)
- Not open source (?)
- Complete list should cover Windows, Mac and Linux OS.

-- Input viewers --
Runs on emulators are usually easier to use for cheating, so it stands to reason they should have more requirements when submitting. At the same time the rules must require reasonable low effort and be unintrusive on the runner. Requiring the use of external input viewers for emulator-runs seems to fit this definition well. I would like to hear any concerns with requiring https://gamepadviewer.com/ for emulator runs.

-- Livesplit --
Livesplit has autosplit functionality for a number of emulators nowadays and in the future it might be possible to run it on any console using image recognition. Moving forward it might make sense to require the .lss file generated. This could be implemented in a number of ways such as always being required for emulator runs, or it could be just required on your first few submissions until you reach a trusted status in the community. Please let me know if you think this is an awesome or terrible rule and how you would like to see it implemented if so.

-- Framecounter --
Depending on the emulators allowed, it could make sense to require a framecounter visible in the game window. However it might be controversial because while it would add some extra tools for moderators to verify by it looks bad and could be distracting. Again give me feedback on this suggestion.

Note: this is not the FPS value, it is the number of frames since boot.

-- Hard reset --
While it does have use as a tool for verifying runs, many new runners are using retroarch which currently doesn’t support this meaning you need to exit and reenter the program altogether. There are also a couple emulator runs that are currently verified but doesn’t follow this rule which would mean they would have to be discarded. Finally this rule doesn’t apply to console runners and to require it has been said to increase the risk of hardware damage for serious runners.

With other suggested changes, this might be an unnecessary requirement moving forward.

-- Save files --
Starting a run from a created file that has not completed Angel Island should have little impact on the run, however it has been recently shown that softresetting a save file does have some effects in a few cases. If it can be shown that there’s no speedrun benefit to starting from a created file, this rule can be relaxed a little bit.

-- Grandfathering --
All verified runs currently submitted would be grandfathered into the leaderboards and will be judged on those terms in case suspicion is raised about them in the future.

-----

I will allow a couple weeks for responses so everyone gets to give their input. Please consider that discussion on e.g. discord might be missed so if you do please reply here afterwards. If there is a topic that seems controversial I might create a vote for that item.

Thank you for helping out the community!

BenInSwedenBenInSweden likes this. 
  kickasspancakeskickasspancakes

In reply to:

Emulators: I'm personally okay with Regen and Kega as stand alone emulators. I'm not familiar with Mac or Linux, but adding "retroarch with genesis gx plus core" as an option. Retroarch is by far the best emulator out there for Sonic speedruns, and if its not your cup of tea we have the other 2 allowed options as well. I just think the list needs a re-wording is all. More specific.

Input Viewers: I think having an input viewer on emulator could be a nice compromise for the "soft reset" change (below) and the framecounter discussion (below). thing is, how resource heavy are they? If somebody doesn't have a powerful PC i'd hate to not allow them from being verified because their PC cant handle an emulator, a capture program, AND an input viewer. just something to think about. I need more info on this, however i'm slightly leaning towards required for emulator.
edit: i have been told its not very resource heavy. again, this may be a good compromise for the other rule changes then. worth discussing.

Livesplit: I'm against the required .lss file. If someone submits something suspect, we can always ask for the file without it being a requirement. maybe making a note that a mod may request such a thing and to have one handy.

framecounter: it would be a good tool to have for verification purposes, i do agree there. however i'm firmly against it.

hard reset: even if having an input viewer is decided to be not required, i believe a soft reset is just as valid as a hard reset in this specific context. if we as mods need to look at a soft reseted run more closely, so be it. its a pain in the ass for not only retroarch users, as well as people who use the Sonic Classics cart (sonic 2) rom on any other emulator. I'm obviously 100% for showing the reset in its entirety.

Save file: more info on this is needed, but as it stands could be relaxed. so far it has shown to be a negative to start from an already started file (an outdated new CN2 strat). im not sure this rule needs changing.

grandfathering: If any rule changes should take place, we need to grandfather all verified runs prior to the change in as valid. Obviously post-rejecting a run strictly based on these rule changes alone would not be fair.

 
  BenInSwedenBenInSweden

Some comments from me:

-- Emulators --
Accurate enough emulation to not gain a real-time advantage over console. ?

No input-replay functionality (TAS) ? - I feel like other rules make this moot (other games allow these emulators as well - such as the DKC series). But I don't really feel strongly enough about it to argue against it either. what about if say retroarch gets Input Playback though? I would prefer it if we were able to cater for emulator agnosticy with basic requirements.

RAM watch and edit not available ? - Whilst the intention is good here, from working on the autosplitter, it is a moot point as there are tools out there to view and edit the ram in game etc (possibly with a lot more documentation and tutorials than the ones built in to emulators). Whilst there are cases where RAM editing would potentially be useful to cheat, I think for the most part they would be obvious. And the non-obvious ones would be hard to detect, but again, excluding emus for it is rather moot.

Hard coded settings that make it harder to do video splices (e.g. no setting to turn off notifications on creating savestates) ? - This can be turned off in retroarch. However, I think frame counter display solves this for the most part.

Not open source (?) ? - RIP Retroarch then. This also narrows the emulator list significantly to a select few - certainly none I would consider using for runs currently. Not sure if any of the non-open source ones support frame counter display.

Complete list should cover Windows, Mac and Linux OS. ?

-- Input viewers --
Fine by me. Doesn't support keyboards though, which may/may not be a bad thing.

-- Livesplit --
This is fine by me, however, I think it's mostly a Windows only program, and the autosplitter may not work on Mac or Linux. So maybe some flexibility required here.

-- Framecounter --
I think the way me and zaxon have it set up is fine and unobtrusive in retroarch... sits in the bottom left corner with some transparancy, with a small but readable font, and sits nicely under the lives counter. https://i.imgur.com/4BEOMbH.png is the settings I use (and zaxon has very similar ones).

-- Hard reset --
I think ditch it, it could potentially open the door for 3rd party input replay as the hard reset is generally a static point. If it's to be used for the "console on" timer objects, then it needs to apply to console too. Technically I would say a soft reset shouldn't be necessary for a requirement either, but if it is, should also apply to console too.

-- Save files --
I think this is fine, its only really AI->CN that are potentially affected as we re-use the save file after that. My autosplitter rewrite also caters for Angel Island resets to handle this case. I need to built some trust in starting from an AI save though, as things like the Ceiling in CN2 were certainly an issue. 😃 Don't think it really makes a difference to a run being valid or not though.

 
  BenInSwedenBenInSweden

Cakes - the input viewer is just a browser window that uses the HTML5 "Gamepad" specification, so just changes the style of the buttons when pressed etc.
It would mean needing quite a toaster of a PC for it to degrade the performance enough to have an effect on someone already using the emu and OBS. 😃

 
  LakeLake

Just my 20 cents about one weird idea that I have.

Given the fact that different people like different things (some people don't want a frame counter, some other want to play on retro arch, etc.), and given that the purpose of each rule is to provide more evidence to the verifier (a run is not automatically legit just because it follows the rules, no matter what ruleset), I thought it might be an option to have a "select the rules you like" ruleset.

It might be interesting to set a specific "reliability score" to each rule (ex. Camera Controller 100 points, Input Viewer 25 points, non-open-source-emulator 10 points, frame counter 15 points, playing on real console 85 points, etc), and then require every runner to meet a specific number of "credibility points" in whatever way they like.

It may sound complicated but if engineered and tested correctly, this might have the following benefits:
1) It allows runners to not stick to rules they don't like and/or agree with
2) It "should" give the verifier enough material to be able to assess the legitimacy of the submission

I say this because I think that, ideally, rules should be guidelines, and not biblical commandments that automatically result in rejection if you don't follow them in their absolute entirety.
The reviewing process should still be, at least partially, "the job of the reviewer".

Zaxon96Zaxon96 likes this. 
  leandrom90leandrom90

I'm away from my personal computer now, but I'm against banning open-source emulators (the only way for PC players to use GenPlus GX) or ones with TAS functions, provided you can prove you're doing a legit run while using them.

Picking an example from another game, you can record TAS using Snes9x, but the frame counter appears differently when you're playing a movie, which makes easy for mods to verify a run. Mega Man X series board states that using a frame counter is enough for Snes9x. Dunno how RetroArch frame counter works, but if it's similar to Snes9x, I think that enabling it is enough to show a run isn't a prerecorded video.

Linking their rules about emulators to show an example: https://megamanleaderboards.net/index.php?page=emus

BenInSwedenBenInSweden likes this. 
  TimpZTimpZ

-- Emulators --
Ben and leandrom,

Just to be clear the reason I'm worried about open source emulators is that it could be easy to make small edits, such as prompting text or removing text that we make rules around.

Changing code might sound like too much effort but searching for the text "Savestate Successful" and replacing it with a space is simple enough that any aspiring cheater could do it easily in an open source project.

I cannot see why there's a good reason to open the pandoras box of allowing emulators with native TAS capabilities regardless of other safeguards that are in place. It's just a plain bad idea if we're making a list of allowed emulators.

Perhaps there could be a compromise in that if you use an open source emulator or front-end you need to enable frame counter?

Also when it comes to RAM watch and edit I mostly want to limit options where someone could for example set the score counter so splicing is easy, but as you say there are external tools for that already. Good point.

-- Lakes idea --
I don't like that. Maybe we could allow some flexibility if a trusted runner did something with no impact like forgetting to enable the frame counter in a run if that becomes a rule. But a points system seems, no pun intended, pointless.

 
  leandrom90leandrom90

TimpZ, I feel your worry is fair, and is a possibility I haven't thought. Hope there's an alternative where people can use GenPlus GX without those downsides.

My biggest worry when it comes to rule changes is putting an overwhelming amount of rules that will put new runners away. When I've started running Sonic 2, having to capture the border of the emulator made me use OBS Classic (which didn't allow some things OBS Studio does), among some other things that bothered me enough to make me put the game away and pursue other games to run. This is an example of rule who did more harm than good, in my opinion (as it was easy to fake an emulator border, since this feature in OBS Classic was so buggy that didn't capture title changes in the emulator window).

Also, I don't like the idea of "trusted runner", as someone may abuse this status in the future. Usually, boards hold top runners to a higher standard, not the opposite.

TimpZTimpZ likes this. 
  LakeLake

Well, no problem for the rejection of my idea.
Maybe it could still give some insights for allowing certain rules to be "flexible", i.e. required only when some other rules are not in place, etc.

 
  BenInSwedenBenInSweden

@TimpZ,

Compiling an emulator can be quite a hefty task, so sometimes just removing some text isn't always that simple - of course doable though.

The frame counter is fine with me, it seems like it's pretty common in other communities and it's set-and-forget in most emulators. Seems some (like the one leandrom90 linked to) allow for showing it before and after the run as well. It also seems like my previous ideas about TAS emulators also showing that it's recording is also used elsewhere too. (The frame counter in TAS emulators show a different frame display when playing back as it's normally currentframe/totalframes rather than just the count, and also if the TAS emulator was shown to be recording, it's harder to fake then.

I also agree with leandrom90, no-one should be implicitly "trusted" for runs. This has been shown multiple times in other communities where even mods have posted WR cheated runs, that have been caught out at a later date.
Forgetting to turn on the frame counter (as an example) from a long time runner would be a pretty weak excuse, as they would be well aware of the rule in place anyway.

Also agree that some rules can do more harm than good, with the issues I had with Retroarch not always working, and reluctance to use Fusion or Regen, it would have ruled me out from running the game.

 
  Zaxon96Zaxon96

I honestly think we just need a visible reset with text. This hard reset soft reset notion seems very pointless. IF the run is legit and the runner didn’t cheat. It seems pointless just to reject it because they didn’t follow a rule then didn’t change their gameplay. As long as they didn’t have level select on either. These rules do nothing but make it inconvenient for the player. A cheater will ALWAYS find a way to make it around the rules no matter the rules. The only players getting harassed and harmed are the legit runners of the game. Let me tell you, Runners just wanna play the game and have fun. Runners should get rules that are beneficial, not some rules just to turn them away. I do agree with lake though that some of these current rules shouldn’t really have to be followed because it’s only inconvenient and a chore... it just stops legit players from playing. If a cheater wanted to cheat. They will do it regardless. Hard reset, soft reset. They will cheat. They don’t care what the rules are. I feel like since retroarch is the best emulator for runners that’s non tas. It really should be up to the runner if they want to add on for more proof. The more proof you put the faster it gets verified and more trusted a runner would become. Less proof. Slower verification and more scrutiny. I feel like regardless it’s a verifiers job to figure out if it’s been cheated or not in a way. So to sum up. Retroarch=good. If you want to show frame count and or input display and or live stream etc... this should be your choice if you want to add a little bit more proof and evidence. Not a requirement. Obv the less proof you show and the less known you are in the community. You’re setting yourself up to be asked questions etc.

 
  TimpZTimpZ

@Zaxon96 we used to have issues on TSC where people would submit runs that were impossible to ascertain if they were legit or not because they were done on TAS emulators with nothing else to back them up. The reasons for not banning them were something similar to yours, they were the easiest way to record and cheaters gonna cheat anyway.

I agree with your opinion that following the rules should be simple, unintrusive and hard to misunderstand. But there is a balance where too few rules (or bad ones) makes the leaderboards lose legitimacy. If people are going to cheat I want to make it as hard as possible to increase the chances they make a detectable mistake, otherwise there's no way for verifiers to live up to their name.

I've spent all day today of my free time to try to verify a run which would have been much simpler if only for a few changes, and I still can't tell if it's good or not. So yes I agree the rules need a change and some things can probably be relaxed, while other things should be added.

Now my question for you is what would be acceptable and what would not?

 
  Zaxon96Zaxon96

I think input display is a fine add on to the rules. it takes 2 minutes to get it.
I think frame counter is fine even though it is a bit annoying.
I'd really rather not use frame counter but if there's other choice then We'd all have to.

Streaming+using livesplit should just be recommended but not enforced as a rule. (this is the more proof you provide the easier it is for the Verifying Mods)

Are there any other suggestions of adding on any other rules that i missed?

 
  TimpZTimpZ

@Zaxon96 I would like your opinion on what emulators should be allowed/not allowed. I believe there should be a definitive list of allowed verified emulators but unfortunately I'm not up to speed on what would constitute a good one.

 
  Zaxon96Zaxon96

Any emulators that have a tas function/replay movie built in should not allowed. (IMO)

So the emulators i do know about, Fusion and Retroarch, Seem like a good fit. I'm not knowledgeable on any other emulators unfortunately. But currently these are the 2 most popular emulators I'd say.

 
  BenInSwedenBenInSweden

Hi @TimpZ,

Using http://emulation.gametechwiki.com/index.php/Sega_Genesis_emulators as a reference.

The TAS capable ones in that list are BizHawk, Gens, and MAME.
Then reducing the Active List you have:
Megado (no builds of this as far as I'm aware, so not worth worrying about currently).
Genesis Plus GX, which has Retroarch, BizHawk and OpenEmu as front ends. OpenEmu is mac only so not sure if it has TASing abilities.
PicoDrive - prefers performance over accuracy, again is a core for Retroarch, should be discouraged, don't think it provides any RTA benefit, so possibly doesn't need to be outright banned.
BlastEm, Exodus and higan - all 3 strive for accuracy, nothing wrong with using them, no TAS abillities (Exodus has a RAM viewer/editor). None have a frame counter either.

Essentially, for no-TAS, show frame counter, we are left with Retroarch only, but you can decide what to do with framecounter-less ones.

 
  chronoonchronoon

Just want to weigh in on the New Save/No Save rule. It should at least include Angel Island saves as valid.

For one, with the current route, nobody is going to do No Save anymore. Even beginners can easily soft reset for cutscene skips.

Second, I've done hundreds of runs off of Angel Island files and noticed absolutely no difference whatsoever from hundreds of runs of New Save files. There is no easily discernible difference between Angel Island and New Save. The Angel Island cutscene plays the same on either type, the game doesn't save acquiring new lives until you beat AIZ2, and everything else is the same too. The CN2 mystery block also works the same on either file type. I'm aware it's different for starting from any other stage, but I'm focusing on the merits of Angel Island resets.

Third, if the original intent was so no parts of the game are skipped by means of file select, by which I mean that players start on Angel Island and play through each stage to the fadeout after Death Egg 2 (obviously with the exception of skipping zones by glitches in-game), then that should be made clear. If the intent was to prevent runners starting with Chaos Emeralds already collected for an All Emeralds run, then that too should be made clear.
New Save covers all this, sure, but the rule doesn't say you can't switch to a different save file after starting, does it?

Fourth, why is the burden of proof not on the rule itself? I'm sorry that I missed it, Timpz, but if you could please put in this forum any evidence of influence from an Angel Island reset, it would at least be some justification for holding onto it. Otherwise, disproving influence seems like a gargantuan and unpleasant task for the handful of S3K ASM experts we have.
As is, despite having no visible nor clear justification and such a high standard to remove it, this rule remains as an inconvenience for every runner. Just to reset the first stage, you waste at least 5 seconds each reset all to avoid a 'what-if' scenario.

It's silly 😛

BenInSwedenBenInSweden likes this. 
  TimpZTimpZ

@chronoon I hear everything you say, but I just want to avoid a potential future situation where we do find a difference that does make an impact and need to remove it again. Perhaps the gain is worth the hassle though, so this is why I want community input on the topic.

At this time I'm in your camp that there is no discernable difference except for the fringe cases like Tails in CN which isn't exactly helpful to a run but I do wonder if someone for example double checked if there is a load time difference between New save and existing save.

 
  BenInSwedenBenInSweden

How about wording it.
"Run must start from Angel Island, but not from a slot that has cleared the game". You can't go back to AI from an uncleared file anyway, so an incomplete file from HC onwards is invalid anyway.

I would imagine the only difference we find in future, is the same as the CN2 ceiling, where a potential trick is discovered that only works from an empty file, but given that CN2 seems to be fine from an AI incomplete file, I think it's unlikely we will find other differences that are major to the run.

 
  BenInSwedenBenInSweden

So I was poking around at Fusion and Regen, and found out that Regen has Input Replay functionality, along with slowdown and frame advance etc, so it can be used for TASing runs.
e.g. I did a recording of a (poor) Angel Island, and then played it back from the recording, with the input display (just for "look no hands!" dramatic effect).

So seems like it shouldn't be allowed? and possibly retroactively void runs using it as they probably can't be trusted?