New profile page has incorrect place ranking
1 year ago
New York City, NY, USA

Logged in today and noticed the new look of the profile page with the placement and time layout alignment changes. Looks really sharp imho, very nice. However, it looks like the rankings for some of my records changed out of nowhere. I clicked on a few of them and noticed that on the game's ranking pages and the submission detail page, they're fine. Wasn't too big of a deal, but then I noticed that I got a new WR on a run out of nowhere so knew something was up.

Affected categories (for reference): https://www.speedrun.com/jumpking/run/zx7ww85y (555th not 564th) https://www.speedrun.com/hollowknight/run/y69582qy (9th not 1st) https://www.speedrun.com/the_legend_of_zelda/run/yo7l72dm (268th not 269th)

Edited by the author 1 year ago
SkittlesCat likes this
United States

I came here to report the same issue - it seems tied to games that use the in-game time field, as I recently had a game added that only uses in-game time and there are 2 problems. The first is that obsolete runs always show on my profile, and the second is that all runs of the game are shown as "first" on a profile. Everything works as you'd expect on the leaderboard, just not profiles.

thekrynn likes this
Somerset, England

There are also some more bugs for me, https://media.discordapp.net/attachments/847776479030935567/989572521013694584/unknown.png Pou Category Extensions' and Granny 2D's covers now don't show correctly This is because sr.c is now using their experimental API on the user's run widget, the game ordering page and featured run page which apparently has bugs that they somehow missed lol

Edited by the author 1 year ago
Gaming_64, VyPr and 2 others like this
United States

Hey all, thanks for the feedback and bug reports.

The new user runs list is the first visible feature that uses our new leaderboard backend, a complete rewrite to fix long-standing bugs, remove limitations, improve ranking accuracy, performance, etc. We've been testing it in private for a while, but deploying widely has revealed a few edge cases that we didn't manage to catch - sorry about that!

We're actively deploying fixes for these issues today and tomorrow. Additionally, we're working on tools to help runners/mods identify runs with "bad data" - invalid timer values, mismatched game/run values, orphaned categories/variables/values/etc. Please bear with us as we clean up this space - I promise we're working on it and it will be in much better shape soon!

Jubilee, KaweedFul and 7 others like this
Somerset, England

Why use the new backend when you've only had site staff to test it? It seems like you're wanting people to complain In the future, maybe make it public for Supporters so more people can test it

Edited by the author 1 year ago
Gaming_64 and Camcorder like this
United States

I hear you. For what it's worth, it's a bit more complicated than you might see from the outside. Let me elaborate...

The new backend was indeed being tested by supporters (at times), but that's a MUCH smaller audience (many many orders of magnitude), and so the level of testing/feedback we could see was limited. That being said, it did help us find and fix a lot of issues before launch! This is also a major ongoing project - we'll continue to fix and improve things over time.

Ultimately a launch like this is a bit of a judgement call: we felt that we had gotten what we could have from the beta group, had no known defects (only differences: cases where the old code is buggy and the new code fixes it), but did still expect some bugs to be revealed by deploying widely. We launched it, y'all found some bugs, and we're actively fixing them.

I don't enjoy deploying imperfect software, but sometimes its unavoidable (or at least impractical to avoid): SRC is a very old site, it contains a lot of bugs (and erroneous data as a result of very old bugs). We're in the process of updating/fixing all of it. The leaderboard system is a particularly perilous area to make changes. Sometimes it's going to be messy, and sometimes we won't catch all of the issues before launch. I promise you that we're doing our best to make improvements while making the proper tradeoff between "time we spend hunting for edge cases" and "edge cases that manifest to the public and can easily be fixed when revealed".

Ivory, diggity and 5 others like this
Somerset, England

Also what I mean is giving Supporters access to the Experimental button when the new backend could be used [quote=Camcorder]but that's a MUCH smaller audience[/quote] It's better than site staff and the devs witch are probably even smaller [quote=Camcorder]new backend was indeed being tested by supporters (at times)[/quote] When were we using it?

Edited by the author 1 year ago
Gaming_64 likes this
United States

Again it's a bit more complicated than that - the "Experimental" button drives the choice of VISIBLE backend, but there's more instrumentation/testing happening behind the scenes that we use to measure outcomes and reveal issues. That said, we certainly could have (and probably should have!) exposed that button to Supporters (or even made it the default) before the wide launch. We may do that for game leaderboards if we think it'd be helpful and appropriate (without being disruptive).

Ivory, diggity and 3 others like this
Utah, USA

Also, I don't like that multiplayer speedruns don't directly show who you did them with anymore I loved that on my profile page. And now it's gone.

CyanWes, grnts and 5 others like this
United States

We can fix the multiplayer issue, sorry about that!

Pear, Ivory and 3 others like this
Somerset, England

@SkittlesCat83 I agree @Camcorder I mean on areas that support it, obviously the whole site doesn't have the new backend coded yet

Edited by the author 1 year ago
Gaming_64 likes this
Arizona, USA

happening to me too, not only are the last 3 obsolete (without the "show obsolete runs" toggle on), none of them are first place. However, the first run, the PC category, is correct. Idk if it's like a variable problem or smth https://media.discordapp.net/attachments/884639740949856309/989608025377611876/unknown.jpeg

SkittlesCat likes this
Utah, USA

Animal Jam Play Wild is glitched and I checked. Whoever runs it has it on their profile the wolf cover. I change the Animal Jam cover every time the app gets a new cover and Animal Jam is stuck on old one from ages ago on people's profiles.

CyanWes, grnts and 3 others like this
New York City, NY, USA

Appreciate the clarification, makes a lot of sense. Out of curiosity (just being inquisitive and not stepping on toes), was curious as to why there's been a lot of issues with being able to have redundancy when it comes to development for public facing, beta testing and experimental builds. From experience, I've seen it usually comes down to architecture issues and/or available resources, which makes sense considering the comment about legacy code. Like I said, was more so just curious, since I haven't seen many areas aside from the supporter page in case there were members who wanted to provide anything in the vein of resources or expertise.

Australia

Excited to hear about a new backend... does that mean we'll be getting an API v2 after all?

Lazio, Italy

Sorry to bring the topic back up, but for me the incorrect placing is still there, as seen here: https://i.imgur.com/0udBPQR.png https://i.imgur.com/XY9G6yg.png

I think the Profile Page calculates the total position also considering the runs that shouldn't count for the actual ranking, when it wasn't like this before. Is a fix in the works?

Edited by the author 1 year ago
United States

@Brionac thanks for bringing this up with a specific example, the screenshots really help to point out the behavior here. I'll explain where we are and what we're doing to resolve this.

Backstory: we're rewriting the algorithm/backend that determines the rank and obsolescence of runs.

TL;DR: the new algorithm fixes bugs in the old algorithm. The user page uses the new algorithm, but the game page leaderboards still use the old algorithm. This will be resolved soon.

More context: for the specific case you raise, what's happening is that your user page shows a run w/ rank 15, but the game leaderboard shows it w/ rank 11. This is caused by a difference in the algorithms in how multiple runs from the same player are ranked based on game settings. You'll notice that the game leaderboard has 4 runs above yours that have no rank: they show a "-". That's because those runs are by players who have other runs with higher placements. As additional context, the game has the "Platforms obsolete each other" setting OFF, which allows for a single user to have multiple runs ranked on different platforms. The old algorithm has separate algorithms for determining "rank" and "obsolescence", which leads to the inconsistency you see on the game leaderboard: a user has multiple runs showing (because they do not obsolete each other - which is correct), but only one gets a rank/placement (which we see as a bug). The new backend algorithm unifies the rank/obsolescence logic and removes this bug: a run can either be ranked or obsolete, but cannot be in this middle state where it's shown but not ranked. Due to the "platforms obsolete each other" setting being off for the game, those runs with a "-" rank now get actual ranks, which pushes down your ranking.

The game moderators can easily change this behavior by enabling the "platforms obsolete each other" setting, which will cause the duplicate runs by those players to be obsoleted/hidden. I know it's never fun to see the rank of one of your runs go up, but we see this as an important step in improving the quality/reliability of the rankings - making sure that every situation has identified, consistent, and properly tested behavior.

This is just one of many inconsistencies we've been working to resolve as we deploy the new algorithm. We're currently hard at work on rewriting the game leaderboards to use the new backend. This will fix a lot of bugs, improve performance, and allow us to make long overdue improvements to the leaderboard UI/UX as well.

Hopefully this helps clarify the situation. Please let us know here if you have any feedback!

Brionac, KaweedFul and 5 others like this
Aquitaine, France

Hi, just reporting another tiny issue with the new profile overlay, I can't click on the person I did my coop run with anymore. Clicking on the username just brings to the run instead of their profile!

Lazio, Italy

@Camcorder Thanks for the explanation and for the update as well. About the matter (from what I understood) what you described kinda forces leaderboard mods of games with differences between platforms but that still have all the runs on a single leaderboard with a platform filter (Sega Rally Championship has different physics engine between Arcade and Home Console ports for example) to either take one of these paths in order to solve the issue:

  1. Divide platforms into subcategories (the most correct thing in this case, though it would also fragment the competition on the leaderboard)
  2. Enable the "platforms obsolete each other" toggle in the leaderboards settings (fixing the profile issue, but at the same time hiding slower runs done on different hardware, which would still be important to be shown due to possible platform differences in gameplay/physics/loading times/etc...)
  3. Leave the leaderboards like that and keep incorrect placing on the user profile page

In my opinion having runs done on different platforms on the same leaderboard shown as unranked is still fine for situations in which the leaderboard is not big enough to justify a platform category subdivision, where normally you would just use the filter in case you wanna look at a specific platform, and the profile page should follow this logic. It's of course a secondary matter because it only affects the profile page currently, but I wanted to provide my feedback regarding the issue anyway to provide a different point of view.

Either way thanks for your time explaining everything :)

Australia

I also think that there are cases in which marking them as unranked is okay. I think the option should still be given, especially as its been a feature for a while