Comments
United Stateschar_bunny1 year ago

There's definitely pros and cons :) Getting close to the next second barrier can be really rewarding, but yep we do have a lot of ties. The tricks do matter for a full game run so we do still have a place for them in our community! And it's style points :)

United Stateschar_bunny1 year ago

Pengo's answer is spot on :) For 1-1 specifically the difference is due to the pipe transitions pausing the in game timer (but obviously not pausing real-time), so the livesplit running in real time gets desynced from the in game timer in the top right corner.

United Stateschar_bunny2 years ago

Happy (belated) Mar10 Day! šŸ˜„ The Community Best Any% run for Super Mario Maker for Nintendo 3DS is now live šŸ™‚ You can choose to watch with commentary or without:

With commentary: Video only:

Enjoy! šŸ˜„ šŸŽ‰

Final time: 1:43:42

United Stateschar_bunny2 years ago

Happy Mar10 day everyone :) I was able to premiere the video tonight and the on-stream time was a 1:43:42. I'll post soon with an official time and the link to the video :)

United Stateschar_bunny2 years ago

The list has been compiled! I've reached out to everyone on it to get permission to use videos. Here it is: https://docs.google.com/spreadsheets/d/1gROleFgqnC5h3auMUw2REGzrJ7Ctse348491ej0l0WA/edit?usp=sharing

For ties, I went to the earliest submitted. Keep in mind the list looks at RTA time which is why it differs from the IL leaderboards which are in IGT.

United Stateschar_bunny2 years ago

Hello!

I've already been talking about this in the discord but I wanted to post here as well :) For Mario Day this year (March 10th = MAR10) I'm putting together a Super Mario Challenge Any% run using the community's best IL times to see what the theoretical fastest human time is for that category. I'm currently still going through and checking ties between levels, but I'm getting closer to finding the fastest video for each level.

That being said, I will only be using videos if I get permission from the owner. If you have one of the fastest times, I'll be reaching out to you hopefully tomorrow (late on 3/8) to see if you're ok with being included in the video. The plan for the video is:

  • Upload it to YouTube on Mar10 day
  • Have a watch party on Twitch to watch the video and compare it to the current WR Both of these would be on my own YouTube/Twitch channels. If for any reason you are not ok with that, please don't feel any pressure to allow me to use your video.

Things I'll be asking you if you have one of the fastest times (I'll link you to all of the levels you have fastest times in):

  • Are you ok with me using your IL video for this purpose?
  • Are you ok with me leaving your audio in or would you prefer I swap it for just level music?

I'm calculating times using RTA instead of IGT for this since Any% is run in RTA. I'll put up the full list of times I've calculated so you can see where each of your videos falls. If there's a tie, I'm going with the earliest submitted video. If I don't get permission from someone to use their video, I'll keep going down the list until I find someone whose video I can use.

Thanks everyone! :D

United Stateschar_bunny2 years ago

Sorry I missed this! Wrap is correct - as long as we can still see what's happening (and specifically the in-game timer for ILs) you're good to go :)

Gaming_64 and 3000ink like this
United Stateschar_bunny2 years ago

Good news - WE DID IT!!! Weā€™ve managed to find a way to get rid of the load times and gone through and retimed every single 100-man easy run. Weā€™re going to update the leaderboard and rules soon, but just to be clear on whatā€™s changing Iā€™ve included both versions of the rules below (itā€™s just a new section titled ā€œFor Easy onlyā€). Depending on how this goes we can look into other 100-man categories next, but I think weā€™ll take a break before doing more investigation here - as you can imagine this was a huge effort. Some fun stats:

  • Time taken for retimes and new rules: Over 2 months
  • Total runs retimed: 30
  • Total load time removed: 41 minutes
  • Total load zones timed: 1,193
  • Total level endings timed to calculate level freeze load zones: 325+
  • Total level ending combinations that affect level freeze: 64
  • Total spinners timed to calculate spinner load zones: 269+

One line in the rule changes that I want to clarify here is this one: ā€œModerators can at their discretion choose not to retime a non-competitive run if it has excessive load zones, in which case the RTA time will be used.ā€ Thatā€™s really only there so that if someone submits a troll run with 100 skips in it we donā€™t have to sit here and actually retime it - if youā€™re going for a good time in good faith we have no problem retiming a run even if itā€™s not at the top of the leaderboard :)

Finally, congrats to N00bie371 who is the new WR holder in this category! Thanks again everyone for the patience, and best of luck with the new timings :) Iā€™m hoping to see some new WRs here soon :D

Current rules

Easy, Normal, Expert or Super Expert.

  • Time starts on activating the "New Game" or "Yes" button and stops when you hit the Axe in the final level.
  • Skipping is allowed.
  • Video is required.
  • Playing off of a cia file is permitted.

As of June 12th 2021 runs that are done using an emulator are permitted for the 100 Mario Challenge category.

  • If possible, please show the Emulator border when capturing your gameplay.

New rules

Easy, Normal, Expert or Super Expert.

  • Time starts on activating the "New Game" or "Yes" button and stops when you hit the Axe in the final level.
  • Skipping is allowed.
  • Video is required.
  • Playing off of a cia file is permitted.

For Normal, Expert or Super Expert.

  • RTA timing is used for this category and loads are NOT removed

For Easy only.

  • The leaderboard is ordered by a loadless time instead of RTA. Moderators will retime each run after submission. For accurate retiming, try to keep at least the top screen of the 3DS on camera at all times and ensure that the in-game timer is legible after finishing a level.
  • Moderators can at their discretion choose not to retime a non-competitive run if it has excessive load zones, in which case the RTA time will be used.
  • You can view the full loadless retime rules and details on retimed runs here: https://docs.google.com/spreadsheets/d/10-yHDtCA4Jih2Z0Q4RuID3E2A4NjUe3lvclrPXOZzAk/edit?usp=sharing

As of June 12th 2021 runs that are done using an emulator are permitted for the 100 Mario Challenge category.

  • If possible, please show the Emulator border when capturing your gameplay.
Pengo5, grnts and 2 others like this
United Stateschar_bunny2 years ago

I'm sorry for the lack of updates here, but a lot has been going on behind the scenes. We've got a ruleset now, and we've started retiming runs, but as that's happening we're finding new things to investigate. Plus it's going to take quite a while to get through them all :( Thanks again for the patience - I promise we're still working on this!

Polonxy likes this
United Stateschar_bunny3 years ago

UPDATE: We have a draft doc explaining the rule change. It goes into a lot more detail than I think most people want, so just stop reading once it makes sense to you or once you get bored :P As long as you feel like by the time you're done reading you understand the change we're good to go. I've made this link available for commenting so if you have questions or feedback feel free to comment in the doc directly or here in this thread. Please comment by SUNDAY so we can hopefully make the rules updates official :) Thanks! Doc: https://docs.google.com/document/d/1iJ4F4uNq6MEDQ-O_1cfsahyGQBSepVdMIjFC3RABHpI/edit?usp=sharing

Tmob likes this
United Stateschar_bunny3 years ago

I know I've been quiet here for a bit, but I promise it's because we've been hard at work making this happen :) We're getting close, and now that we are I'm hoping for some help :)

*** COMMUNITY REQUEST *** If you're playing 100-man and encounter a weird level ending (anything out of the ordinary happens after you lose control of Mario) - please save the footage and send it to me if you can :) We're in the process of testing out the way-more-complicated-than-I-wish-it-was flow chart to account for end of level load times, and could use everyone's help finding the edge cases :)

United Stateschar_bunny3 years ago

Update: Poll results were an even split :P Thanks to everyone who voted and gave feedback - the next step from the mod side is to put together a draft of what the new timing rules will look like taking all of this into account and then share it back once it's ready :) Thanks for your patience! :D

United Stateschar_bunny3 years ago

Update: The first poll results are in, and the decision is unanimous! :D We'll be working on a rules update for timings that will remove the load time discrepancies caused by system/version/SD card differences :)

That brings us to our next poll! We've been working on drafting the new set of rules and we're running into a lot of situations where there's a pretty clear tradeoff between precision and simplicity. Some examples: https://i.imgur.com/BlHB45A.png

Please let us know what you think at this poll here by Friday and leave your comments below! POLL: https://strawpoll.com/pf32o4ka4

United Stateschar_bunny3 years ago

[Original thread title: "Discussion: Thoughts on changing timing for 100-man to eliminate load time discrepencies?". I changed it to reflect where the discussion landed.]

Hello!

Load times have been a headache for this game for a while. The system, game cart type (physical/digital), and even SD card all have an effect on load times. This is a problem for every category in the game, but it's especially noticeable in the shorter 100-man categories where even a few seconds can separate leaderboard spots. As a result, we'd like to consider changing the timing for 100-man categories such that load time differences are no longer a factor.

There are a number of ways we can tackle this, but the general proposal is to change the official timing to be the total time spent in levels + some set time value per level that represents a standardized load time. We would keep the RTA time on the leaderboards but add a column for this "standardized time" and sort the leaderboard on that value. This would even out the game and allow everyone to be competitive, regardless of their set-up.

Basic poll on this idea (please participate - poll closes Wednesday): https://strawpoll.com/hy6wyucqr

Some more detailed points that are up for discussion:

  1. What should the per-level standardized load time be? The reason I think we should add a per-level load time rather than removing them entirely is so that there's still a penalty for skipping levels similar to the penalty that would exist in RTA timing. My thought here would be to take an average of load times on the fastest known system/cart/sd combination so everyone gets that advantage, but would love to hear other thoughts on this.

  2. How should individual levels be timed? We can standardize on RTA timing for the level (for example, start time on the last full-black frame before the level starts to appear, and end time on the first frame where a score shows up on the flagpole/axe/goal tape), which would be the closest to today's timing but which would be the most technical to retime and could be intimidating for newer runners. Alternatively, we could use in-game time (final timer - starting timer) which is much easier to use but reduces some of the sub-second skill expression. For the in game time we could also consider adding standard loads for screen transitions (3 seconds for a pipe, 3 seconds for a keydoor, and 1.5 seconds for a door, for example) so that starts that are faster RTA are still faster IGT. Thoughts?

Please let us know by responding to the poll (https://strawpoll.com/hy6wyucqr) and leaving your comments either here in this thread or in our discord channel - this affects our most popular category so we'd love to hear from everyone who's willing to share before making any changes :) Thanks!

United Stateschar_bunny3 years ago

This is super useful information - thanks for sharing :) Appreciate your help here!

United Stateschar_bunny3 years ago

Bonus - I've been looking in forums for other active 3DS games for anything they might have found. Some threads that were relevant:

Luigi's mansion - they also see a difference between new and old 3DS: https://www.speedrun.com/lm3ds/thread/9itur Luigi's mansion - they add a 20s penalty for NTR runs due to load times (based on measuring load times between runs on different versions): https://www.speedrun.com/lm3ds/thread/8fmky Mario Kart - removing the SD card reduces load times but adds RNG to load times: https://www.speedrun.com/mk7/thread/s0kxd New SMB - they add a set time per system in a separate column on the leaderboard: https://www.speedrun.com/nsmbds/thread/5v09e New SMB - a more detailed thread on the separated load times: https://www.speedrun.com/nsmbds/thread/avj68 New SMB - interesting thread on possible software for load time removal (would only work for capture card, thread seems to have died, but may lead to something): https://www.speedrun.com/nsmbds/thread/soxwb

I've also asked the New SMB community if they ran into the same issues we did with SD cards and physical/digital differences - I'm hoping they may have some ideas they'll be willing to share: https://www.speedrun.com/nsmbds/thread/02vu1/1#y3trq

Pengo5, grnts and 2 others like this
United Stateschar_bunny3 years ago

Hi!

Hope it's ok to ask a question from a different community here, but I'm a mod for another 3DS game (super mario maker) and load time differences are killing us. I'm looking for a better way to account for them and found that this community is using a set value added based on system, which is something we've been talking about doing as well. Problem is, from what we've found it's not just old/new 3DS that matters, it's also whether the game was physical/digital, and what SD card is used in the 3DS system. If you don't mind me asking:

  1. Do these same differences show up for this game?
  2. If so, any ideas on how to account for them?
  3. How did you get your set values to add? (averaging a set number of runs? something else?)

I'm hoping we can do the same thing you're doing here with real time / load time leaderboard categories, but I'm worried with SMM that we have a matrix of times to add based on combinations of the above. Any ideas you have and would be willing to share would be much appreciated!

(for reference, here's everything I've found about load times in SMMf3DS: https://www.speedrun.com/smm3ds/thread/pon5h)

United Stateschar_bunny3 years ago

SD card update

Welp turns out they do make a difference...yay...

TL;DR: SD cards make a difference in load times. Evidence seems to recommend cards with a speed class of 10/U1 or higher, but we don't have complete data. Alternatively, Toshiba cards may just be bad.

Backstory: Once again, someone from the community (this time JuiceACD - thanks!) noticed that they were getting faster load times in challenge mode than either Polonxy or myself (char_bunny). Comparing Juice's run to my run showed that the time loss was enough to show up in world 1 alone. Polonxy then went and upgraded his SD card and verified with video comparisons that his load times were now comparable to Juice's. I then went and got 2 theoretically faster SD cards to do timing tests (and hopefully catch back up).

=== About SD cards and load times ===

Credit to Polonxy for finding this video that explains SD card properties/specs:

The key property I was guessing affected load times was something called the "speed class" of the SD card - basically, how fast it can send/receive data. While that's hard to prove with just the few cards we've tested, that's probably the thing to look for (instead of things like capacity or bus interface). We have some informal data for a few cards (faster/slower based on video comparison) as well as more formal data for a few others (frame count differences for levels in W1).

IMPORTANT: If you are looking to upgrade your SD card, check that it's compatible with your system. For example, a new 3DS only works with a MicroSD card that supports the HC bus interface. Compatibility information: https://en-americas-support.nintendo.com/app/answers/detail/a_id/274/~/what-sd-cards-and-microsd-cards-are-compatible%3F

Informal comparisons

These cards were compared by checking synched videos of runs on each card.

  • Format: Brand + Capacity + Bus interface + Speed class
  • Fast: Lexor + 4GB + HC + C4
  • Fast: SanDisk + 16GB + HC + C10
  • Slow: Toshiba + 4GB + HC + C4 Worth noting that both the Lexor and Toshiba cards were default cards that came with the owners' 3DS system. This data seems to indicate that brand makes a difference, with a theoretically slower Lexor card (speed class 4) tying a theoretically faster SanDisk card (speed class 10) for performance.

Formal comparisons

I timed out load times across 3 SD cards that I owned. Specs:

  • Format: Brand + Capacity + Bus interface + Speed class
  • Toshiba + 4GB + HC + C4
  • [SanDisk U1] SanDisk + 32GB + HC/I + C10/U1/V10/A1
  • [SanDisk U3] SanDisk + 32GB + HC/I + U3/V30/A1

The first thing I tried was timing them from Coursebot. Interestingly, there were NO significant differences between the SD card load times in Coursebot. Thankfully we'd already shown SD cards make a difference, so instead I tried timing loads during a Super Mario Challenge mode run. I originally timed in 6 segments between levels (trying to wrap around the human inputs of "Next" and "A" between levels) but the data was difficult to use and based on my timings I was consistently hitting those buttons within frames of when they were available, which was a negligible amount compared to the overall load times. So instead, I measured from the end of one level to the start of the next. Here's the data:

https://i.imgur.com/Ag2WNA3.png

A few interesting points:

  • The SanDisk cards were consistently faster than the Toshiba card
  • The faster cards were between 0.5 and 1 second faster on average
  • There was significant variance in load times between runs for the Toshiba card but not for the SanDisk cards
  • Even at its fastest, the Toshiba card was always slower
  • Assuming the minimum 0.5 second loss per level, that works out to a time save of about 44 seconds over the course of a challenge mode run
  • Surprisingly, between the two SanDisk cards, the one with the lower speed tier performed slightly better (but not by much)

There was also an additional run that I timed after updating my version of the game just to see if that made a difference (since that was an earlier theory on the load time difference between runners this time around), but I did not find any difference when timing it, so it seems unlikely that the game version makes a difference in load times (although admittedly one data point isn't really enough to say for sure).

=== Summary ===

SD cards make a difference. Try to get a faster one if possible, but don't go crazy or spend a ton for a really fancy one. Seems like after C10/A1 there may not be a benefit. Game version likely doesn't make a difference, but there's not much research either. Feel free to message me or reply here if you have questions, answers, or more data :)

Useful resources

SD cards explained (thanks again to Polonxy for the link): SD card system compatibility: https://en-americas-support.nintendo.com/app/answers/detail/a_id/274/~/what-sd-cards-and-microsd-cards-are-compatible%3F How to remove an SD card: https://en-americas-support.nintendo.com/app/answers/detail/a_id/89#DT:t1-q1a1EP:t1-q1a1-c How to transfer data between SD cards: https://en-americas-support.nintendo.com/app/answers/detail/a_id/309/~/how-to-transfer-data-between-sd-cards-or-microsd-cards

grnts, Goosper and 2 others like this
United Stateschar_bunny3 years ago

I think many of you have already seen this, but for those who haven't a number of community members sent in clips that were compiled into a montage highlight video for Mario day :) You can find the video here:

We coordinated this on Discord - if you haven't already joined our server you can find the link in the menu :) Happy Mario day everyone!

Gaming_64, Polonxy and 3 others like this
United Stateschar_bunny3 years ago

[UPDATE: You can ignore this thread if you're running 100-man easy as load times are removed for the leaderboard]

TL;DR: If you're planning on running challenge mode and haven't bought the game, buy the digital copy - it's significantly faster. If you can, you also want a New 3DS instead of an original one.

Backstory: After Polonxy first pointed out that the load times in my challenge mode runs were significantly slower than his, we've been talking about ideas as to what could be causing the difference. After a bit of research (and $25 for the digital version of the game) the answer came down to playing on a digital vs. physical copy of the game. Below is everything I've learned about load times along the way.

=== What DOES affect load times ===

Digital vs. Physical

The digital version of the game is significantly faster than running on a physical copy. I compared load times for videos I recorded on the same system with the same settings using the same capture card (so the only difference was the game) and used certain key frames to precisely measure the difference in load times between the two copies of the game. I timed the first four levels in world 1 and the transition to world 2 in challenge mode with game files that had all medals for those levels completed. For each of levels, the load times at the start of the level were about 1.5 to 2 seconds faster on digital, and the load times at the end of the level were about 0.5 to 1 seconds faster on digital. In total that worked out to about a 10 second difference over the course of world 1 alone. Assuming the rest of the game is similar with a 10 second difference for every 4 levels, that works out to a 3-4 minute time difference over the course of the any% challenge mode run. Here are my specific timings if anyone would like to check them out: https://i.imgur.com/qmN3tnu.png Polonxy also found a separate thread that touches on this issue: https://www.speedrun.com/speedrunning/thread/dv0eo/1#0pk4p

New 3DS vs. Original 3DS

This one was harder to compare since I don't have an old 3DS with a capture card, but for this one I went and did my best to hand-time every video submitted for the 1-1 IL any% category. My numbers here are less accurate since I couldn't go frame by frame and the videos had varying quality, but I still found a difference of a bit under a second between the two 3DS versions. I slowed each video down to 0.25 speed to help improve my accuracy, and I timed from when Mario lands on the loading screen to as soon as the level started to appear. For New 3DS submissions, that was generally around 9.5 seconds. For Old 3DS submissions, that was generally around 12.5 seconds. The delta at 0.25 speed was 3 seconds, or about 0.75 seconds in real time. If anyone would like my numbers on this one let me know, but I'll skip posting the full chart here since it was more of an informal timing.

Level select vs. Challenge mode

This one was unexpected, but levels load slower when playing challenge mode than they do when picking them out of Coursebot. I tested this out on a variety of levels, both early and late in the run, and they all show around a 2 second difference when loaded from Coursebot vs. any%, with Coursebot being faster. Here were my numbers for the delta: 1-1 :: 1.8 seconds 13-6 :: 1.8 seconds 15-2 :: 1.8 seconds 18-5 :: 2.1 seconds

=== What does NOT affect load times ===

Thankfully most other things. In general, load times are very consistent. When I was testing whether my capture card settings were causing the lag (they didn't) I timed a single level's load time over and over. Out of the 7 load attempts, the fastest and slowest level loads were only 2 frames (or 0.033 seconds) apart. I did the same for a second level and got loads all within 1 frame (or 0.017 seconds) apart. This is a good sign that there's no significant randomness affecting load times.

=== Where load times don't matter ===

  • IL times - since those are timed using in-game time, load times don't matter at all.
  • Pipe transitions - I timed those out between physical and digital as well but didn't find any meaningful difference.
  • Between worlds - I timed out the transition from world 1 to world 2 between the physical and digital copies and they were identical down to the frame.

=== Where there are still open questions ===

  • 100 man - I don't know if the load times vary for 100 man levels the same way they do for challenge mode levels. If anyone has ideas on timing that let me know. It's a tricky challenge since load times, while consistent per level, do vary between levels.
  • 2DS vs. 3DS - There weren't enough 2DS times for me to get good data based on the IL submissions for 1-1, but it seems that there likely isn't a difference in loads between 2D and 3D.
  • Capture card - Same as 2DS, there weren't enough capture card times to get data on the differences. Based on loose timing the loads seem to be equivalent, so it's likely it doesn't make a difference.
  • SSD - If digital vs. physical turned out to be a wash, then the SSD was the likely next source of load time variance. With just the informal times I did for the 1-1 ILs I didn't have enough data to make a call on whether there was evidence for this. If anyone has data here it would be awesome.
grnts, Goosper and 7 others like this
About char_bunny
Joined
7 years ago
Online
today
Runs
152
Games run
Super Mario Maker for Nintendo 3DS
140
Runs
LOVE 3
LOVE 3
Last run 1 year ago
11
Runs
Waluigi's Taco Stand
Waluigi's Taco Stand
Last run 2 years ago
1
Run
Games followed
Super Mario Maker for Nintendo 3DS
4,965
visits
LOVE 3
LOVE 3
Last visit 11 months ago
84
visits
Games moderated
Super Mario Maker for Nintendo 3DS
133
actions