[UPDATE: You can ignore this thread if you're running 100-man easy as load times are removed for the leaderboard]
TL;DR: If you're planning on running challenge mode and haven't bought the game, buy the digital copy - it's significantly faster. If you can, you also want a New 3DS instead of an original one.
Backstory: After Polonxy first pointed out that the load times in my challenge mode runs were significantly slower than his, we've been talking about ideas as to what could be causing the difference. After a bit of research (and $25 for the digital version of the game) the answer came down to playing on a digital vs. physical copy of the game. Below is everything I've learned about load times along the way.
=== What DOES affect load times ===
Digital vs. Physical
The digital version of the game is significantly faster than running on a physical copy. I compared load times for videos I recorded on the same system with the same settings using the same capture card (so the only difference was the game) and used certain key frames to precisely measure the difference in load times between the two copies of the game. I timed the first four levels in world 1 and the transition to world 2 in challenge mode with game files that had all medals for those levels completed. For each of levels, the load times at the start of the level were about 1.5 to 2 seconds faster on digital, and the load times at the end of the level were about 0.5 to 1 seconds faster on digital. In total that worked out to about a 10 second difference over the course of world 1 alone. Assuming the rest of the game is similar with a 10 second difference for every 4 levels, that works out to a 3-4 minute time difference over the course of the any% challenge mode run. Here are my specific timings if anyone would like to check them out:
New 3DS vs. Original 3DS
This one was harder to compare since I don't have an old 3DS with a capture card, but for this one I went and did my best to hand-time every video submitted for the 1-1 IL any% category. My numbers here are less accurate since I couldn't go frame by frame and the videos had varying quality, but I still found a difference of a bit under a second between the two 3DS versions. I slowed each video down to 0.25 speed to help improve my accuracy, and I timed from when Mario lands on the loading screen to as soon as the level started to appear. For New 3DS submissions, that was generally around 9.5 seconds. For Old 3DS submissions, that was generally around 12.5 seconds. The delta at 0.25 speed was 3 seconds, or about 0.75 seconds in real time. If anyone would like my numbers on this one let me know, but I'll skip posting the full chart here since it was more of an informal timing.
Level select vs. Challenge mode
This one was unexpected, but levels load slower when playing challenge mode than they do when picking them out of Coursebot. I tested this out on a variety of levels, both early and late in the run, and they all show around a 2 second difference when loaded from Coursebot vs. any%, with Coursebot being faster. Here were my numbers for the delta:
=== What does NOT affect load times ===
Thankfully most other things. In general, load times are very consistent. When I was testing whether my capture card settings were causing the lag (they didn't) I timed a single level's load time over and over. Out of the 7 load attempts, the fastest and slowest level loads were only 2 frames (or 0.033 seconds) apart. I did the same for a second level and got loads all within 1 frame (or 0.017 seconds) apart. This is a good sign that there's no significant randomness affecting load times.
=== Where load times don't matter ===
• IL times - since those are timed using in-game time, load times don't matter at all.
=== Where there are still open questions ===
• 100 man - I don't know if the load times vary for 100 man levels the same way they do for challenge mode levels. If anyone has ideas on timing that let me know. It's a tricky challenge since load times, while consistent per level, do vary between levels.
GoosperGoosper, DarkSRCDarkSRC and 6 others
You can go through YouTube videos frame by frame by pausing and using the <> keys. Or you can use a non-secure website
|DarkSRCDarkSRC, char_bunnychar_bunny and MilozakiMilozaki like this.|
SD card update
Welp turns out they do make a difference...yay...
TL;DR: SD cards make a difference in load times. Evidence seems to recommend cards with a speed class of 10/U1 or higher, but we don't have complete data. Alternatively, Toshiba cards may just be bad.
Backstory: Once again, someone from the community (this time JuiceACD - thanks!) noticed that they were getting faster load times in challenge mode than either Polonxy or myself (char_bunny). Comparing Juice's run to my run showed that the time loss was enough to show up in world 1 alone. Polonxy then went and upgraded his SD card and verified with video comparisons that his load times were now comparable to Juice's. I then went and got 2 theoretically faster SD cards to do timing tests (and hopefully catch back up).
=== About SD cards and load times ===
Credit to Polonxy for finding this video that explains SD card properties/specs:
The key property I was guessing affected load times was something called the "speed class" of the SD card - basically, how fast it can send/receive data. While that's hard to prove with just the few cards we've tested, that's probably the thing to look for (instead of things like capacity or bus interface). We have some informal data for a few cards (faster/slower based on video comparison) as well as more formal data for a few others (frame count differences for levels in W1).
IMPORTANT: If you are looking to upgrade your SD card, check that it's compatible with your system. For example, a new 3DS only works with a MicroSD card that supports the HC bus interface. Compatibility information: https://en-americas-support.nintendo.com/app/answers/detail/a_id/274/~/what-sd-cards-and-microsd-cards-are-compatible%3F
These cards were compared by checking synched videos of runs on each card.
I timed out load times across 3 SD cards that I owned. Specs:
The first thing I tried was timing them from Coursebot. Interestingly, there were NO significant differences between the SD card load times in Coursebot. Thankfully we'd already shown SD cards make a difference, so instead I tried timing loads during a Super Mario Challenge mode run. I originally timed in 6 segments between levels (trying to wrap around the human inputs of "Next" and "A" between levels) but the data was difficult to use and based on my timings I was consistently hitting those buttons within frames of when they were available, which was a negligible amount compared to the overall load times. So instead, I measured from the end of one level to the start of the next. Here's the data:
A few interesting points:
There was also an additional run that I timed after updating my version of the game just to see if that made a difference (since that was an earlier theory on the load time difference between runners this time around), but I did not find any difference when timing it, so it seems unlikely that the game version makes a difference in load times (although admittedly one data point isn't really enough to say for sure).
=== Summary ===
SD cards make a difference. Try to get a faster one if possible, but don't go crazy or spend a ton for a really fancy one. Seems like after C10/A1 there may not be a benefit. Game version likely doesn't make a difference, but there's not much research either. Feel free to message me or reply here if you have questions, answers, or more data
SD cards explained (thanks again to Polonxy for the link):
|GoosperGoosper, PolonxyPolonxy and ShivenShiven like this.|
Bonus - I've been looking in forums for other active 3DS games for anything they might have found. Some threads that were relevant:
Luigi's mansion - they also see a difference between new and old 3DS: https://www.speedrun.com/lm3ds/thread/9itur
I've also asked the New SMB community if they ran into the same issues we did with SD cards and physical/digital differences - I'm hoping they may have some ideas they'll be willing to share: https://www.speedrun.com/nsmbds/thread/02vu1/1#y3trq
|GoosperGoosper and PolonxyPolonxy like this.|