I would avoid accepting runs that finish off-track missing a lapline.
Sure, this might get fixed for 1.3, and this might make some tracks faster (like XR591 became faster when the checkline was extended), but validating runs where the game itself doesn't say the race has been completed is weird and a bad precedent, beyond requiring significant work.
About this :
" Normally, I should have rejected your ghost as well, but chose to accept it anyway as there is no way to know if Debug was enabled or not in the actual run (which is also yet another reason to ban Ghosts as it is really prone to loopholes like that)."
One thing ghosts have over the pure video is that they are auditable in-game. The downside of ghosts is that one could try and edit the file to produce a sort of TAS, but on video alone it can be hard to differentiate stuff like "trajectory optimization" and cheated speed, if the cheater is not too greedy. I remember when some WR from a new runner (Andet ?) came out, I asked for replay files because I had real doubts. But testing them in game, I got convinced that it was indeed perfectly feasible to achieve what he did. More advanced tools that detect impossible trajectory/speed changes within the game's physics could be built (the main issue would be collisions, but if the tool flagged instances as "impossible unless collisions", manual review would make it work).
Basically, an available ghost replay file makes it easier to detect attempts at altering the game rules, while a live replay is better at preventing "TAS-ing" attempts.
In the end, in normal operations, runs won't be subject to such scrutiny, as it requires time and energy. But if doubt arises, having the tools available would be helpful. And if rules required runners to submit a replay file for ILs upon demand by the game's moderators, it would cover all bases.
Hello, As I managed to record ghost replays online, I guess those ghost replays are not accepted by default (please correct me if it isn't true). My question is, how to distinguish now whether a ghost replay is a valid offline ghost replay/ghost replay recorded by a server/server with changed code/a ghost produced by some broken offline code? I've read that ghost replays will be probably rejected since 2.0, but I assume that's not soon.
(Not that the problem of distinguishing valid and invalid ghosts never existed before, but now (almost) the same code can produce online ghosts which are probably invalid)
PS. Yeah I didn't intend to upload them anyway because I prefer "broken" and "boring" addons more :D But someone could try it anyway...
Ghosts will indeed be rejected starting from 2.0 (the Alayan remarks in the previous post make sense, but in all cases a ghost without the corresponding live recording will be rejected). That would be a mod not listed in the allowed mods (and the "doing actions impossible to do normally" example also apply here), so it would be rejected by the rules. However, unless there is lag, I don't see how we could distinguish online ghosts from offline ones, though maybe Alayan has an idea. If there is no way to distinguish a submission from something that does not break the rules, then it will be accepted, even if there rules were actually not followed, unless there are good reasons to suspect cheating. From the moderator side, there is no way to know whether the rules were broken in these cases anyway. Unless I see something off, I will always assume good faith when verifying a run, that the rules were followed. You could in theory submit these runs and I will probably not notice anything, though it would also be a bit sad.
Live Runs from Frankfurt Speedrun are now explicitly accepted in reaction to the latest submitted run. I don't really see anything against that, one should not be able to get any advantage running there and at worst getting great times there may be a bit more difficult, which is acceptable Imo. I added the Frankfurt Speedrun Subversion, be sure to select it if you run on this server.
"IL Runs done on the following SuperTuxKart servers are currently admissible, but may still be rejected if there were noticeable lags or if we detect that the server made unacceptable modifications (different physics, etc.):
- Frankfurt Speedrun Only Live recordings are accepted in these cases."
Also added a paragraph about cheating.
In reaction to the latest submission that was run online but outside of Frankfurt Speedrun, it is now allowed to submit runs done on servers other than Frankfurt Speedrun. Simply make sure to pick the "Online" option for the "Subversion" (this is not needed for Stk Addons) and mention in the description which server was used, or we may reject the run. Naturally, it remains true that the run "may still be rejected if there were noticeable lags or if we notice that the server made unacceptable modifications (different physics, etc.)", and "Only Live recordings are accepted in these cases.".
Should the 10% rule be changed ?
I don't think we've had problems with it yet, but getting within 10% of some ILs, such as theodore's Maths Class time, is already very demanding (my improved PB is still not within 10%). While on some tracks it's relatively straightforward to be within 10%, it's not so in general.
Also because of the consistency required, while reaching within 10% of the WR in most of the tracks of a full-game run is not too difficult, doing so across a full-run is. With the current rule, an Any% beating Andet's time could be submitted without video.
Of course if someone came out of nowhere with such a time, we'd ask for verification, but it's probably better to make the rule consistent with how we'd handle individual cases.
I would keep the principle of not requiring a video for weak times, but I would change the threshold.
I think that the rule is alright, it can come in handy in case a recording went wrong, if someone makes some casual runs without recording but did not expect to get a decent time, or for new players that have trouble setting up Obs, etc. And of course, we agree that it should definitively not be required for weak times.
The vast majority of runners are going to provide Videos in any case anyway, making the rule rarely used in practice, but we still have a nice fallback for the cases like the ones above. The threshold was even too high for your own case, which was still accepted as a reasonable exception. And the rules still allow us to reject dubious submissions.
In the Oliver's Math Class or Full Run cases, how I see things is that we already have several people that made it within the 10%, so honestly if someone not too suspicious claims to have made some time that is more than 10% worse, then I would believe it without proof, after all we already know that it is perfectly achievable.
If we were to tweak the rules, then I would not change the 10%, but maybe also require a Video if the player is Top 5 or so instead.
The question is not whether the time is achievable (obviously it is since it's worse than WR) or if it is achievable by that person (it may or may not be, likely it is but it might require a long grind), but whether it has been achieved. I firmly believe that 10% is too low to be comfortable with this.
If there are multiple instances of very good players with published times that are worse than the threshold, the threshold is not good.
And yes, my own time was accepted with only a short video of the final screen result, but besides online play and other TT performances supporting the notion that I could do it, it's under the provision of replacing it with a new video-recorded performance later. I have been more focused on training individual tracks than doing the all-tracks GP so I have a couple of recordings but not something I want to share yet, but I think this provision is important.
I might think like you if the threshold were 5% instead, which is what I first thought to use, but 10% is already lenient and gives a large room for unoptimizations and mistakes or bad luck. 5.7 s margin for Omc or several minutes for Story Mode are pretty massive and forgiving. We don't need to be comfortably sure that submitted times with even more mistakes were indeed achieved.
If there are multiple instances of very good players with published times that are worse than the threshold, the threshold is not good.
I see this the other way around, if times are worse than the threshold, then they are not that great and don't warrant a proof. Well, even in your own Point Ranking, you would give a terrible score of 3/40 for a time 10% worse... Runs that used to be great might have been "powercrept" by some new strategy or simply more persistence. Andet's or even my own Any% might have been great 4 years ago, but it is no longer the case nowadays.
The only case the current threshold could be an issue would be when you have some insane shortcut that just 1 or 2 insane speedrunners actually used, so you have 1 or 2 times that might be much better than all the others so that anyone within the 10% would be implied to have used that shortcut (obviously, I thought about Volcano Island here). That can be fixed with a complementary "Top 5" Rule, or an explicit Rule enforcing a Video if a complicated glitch was exploited, but I am against raising an already lenient threshold. Otherwise we might just as well drop the Rule...
As concrete comparisons with other places that have a similar rule, the Mario Kart Double Dash Leaderboard does not have a % threshold but rather a fixed time. For Full Runs, they require Video for times under 30 min Igt while the Glitchless Wr is currently 27:58.038, or only 6.8% under the threshold. The gaps vary from 6.8% to 20%, but barring a couple of Cups with some broken exploits, they just go to 12.2%. Similar Rule in Mario Kart 7, with gaps of 8.1-10.3% for the main Categories, and even only 4.1% for the Mushroom Cup. For Mario Kart 8, there is not even any requirement unless you are Top 5.
Videos are no longer required if a Ghost is provided, since a Ghost Recording does not add any proof value...
However, a reminder that Ghosts or Ghost Recordings will no longer be accepted starting from 2.0 (including Runs from previous versions that might be performed after 2.0 release).
I agree that a video of a replay doesn’t add much, but I think having a ghost replay file along with a live recording adds a lot. I’d suggest making live recording + ghost file a requirement starting with 2.0.
I’d also suggest asking for ghost files from now on. Almost all speedrunners provide them anyway. I’ve set up a Git repo with all current WRs (see resources). I’ll try to keep it up-to-date.
Sounds good, if we require a Live Video then also providing the Ghost is not much to ask...
Nice work on the Ghost Repository. If you are regularly updating it then we could use it as reference and people could submit theirs by making Pull Requests so we don't have many Discord Links scattered everywhere and that might expire at any time.
I also considered using Archive.org to save current Record Live Recordings as well... After all, a platform or account could disappear at any time in the long term...
I’m committed to keeping the repository up to date. It’s not much work. A pull request would be most convenient but shouldn't be a requirement imo. Some people don’t use git. As long as they provide a link to their ghost somewhere (google drive, discord, matrix ...) where I can get my hands on it, I’ll add it to the repository. The idea is to have them all in one place, not scattered and lost over time like Wax’s Hacienda replay ;(
I'll do Hacienda as my next WR, or maybe do first Gran Paradiso and then go direct to Hacienda! Im excited to do this!!!
I’d like to suggest requiring a live recording after the 1.5 release (which is expected soon) instead of waiting until 2.0. I also propose making a ghost replay file mandatory along with the live recording.Replay files for WR runs have already become the standard, and all recent WR runs have been submitted with one anyway. As for live recordings, any reasonably modern hardware should be able to record STK without issues.
The reasons for this change are:
- To ensure we maintain a complete collection of WR ghost files.
- To have high-quality live recordings that help promote STK speedrunning.
- To deter potential future cheaters.
Just to be clear, I don’t think we’ve had any cheating issues so far. point 3 is purely preventive.
Runs under the 10% rule wouldn’t be affected, ofc. Also, if anyone has unsubmitted ghost files lying around, they can still submit them until the 1.5 release. What do you think?
At the time I thought of the Required Live Recording Rule, 2.0 was supposed to come after 1.4 so in some way we would be doing what I initially intended... Though with the delay, I also started to think that it might after all be nice to give some more time as high requirements may also deter people from running Stk, and we are not a lot of people right now so...
But well, since we are bringing up the subject already, and I also said myself that we can expect serious Runners to make some minimal investments, we could already start enforcing this from now on without delay. I believe Graphics 2 1280x720 Live Recording is feasible with some 10+ years old Haswell with Hd 4600 Integrated Graphics or so, it is not much to ask and waiting some more for 2.0 or even 1.5 will not change the situation...
I already agreed regarding requiring Ghosts. And we can indeed have these Thresholds under which we will strictly enforce acceptable Live Recording + Ghost, over which either one is enough, and nothing required over 10%.
Rule Changes are generally not retroactive so already Verified Runs will stay.
I will do the formal Lawmaking a bit later...
Rules Updated:
Runs within 10% of the current record require a Live Recording or Ghost File/Recording for submission. Top 3 Runs, or Runs within 2% of the current record, require both a Live Recording and, when they can be produced by the Game, Ghost File (not just Recording) to be verified. Runs more than 10% worse than the current record don't need any proof, unless they are Top 3 in which case a Live Recording or Ghost File/Recording is enough. We however reserve the right to reject runs without proof if there are suspicion of abuse of this rule.
I did a tweak regarding things over 10% yet Top 3... We don't need high requirements in these situations, but it is still nice to have more than nothing in Categories that might be less popular or that were just established...
This affects every Version, except 0.8.1 I guess since it does not have Ghosts... I clarified the Rule, of course if the Ghost cannot be produced by the Game in a given Mode or Version, it will not be required.
