Hi all,

Are there any tools for analyzing game footage? Like extracting the score/lives/health from the video stream or registering events by analyzing sound and certain regions on the screen?

I'm currently looking for an interesting side project and I was wondering if 1) there is a need for such a tool and 2) does it exist already?

Any input on this topic is appreciated.

• nom

I'm not aware of anything like that, and it sounds like it would be absurdly difficult to make. Perhaps you could make one for a specific game, but making any sort of generalized one would basically be impossible. It would be easier to just grab that information from the game's memory than trying to guess based on the footage itself (assuming the game can be run on PC or an emulator, at least, no clue how you'd go about reading a console's memory).

"if 1) there is a need for such a tool" Don't really see any. It would only be able to get info that someone can readily see by just watching the video, so I'm not really sure what you could/would want to do with that data.

Actually I think you could apply this idea to automatically remove load times from just video (where games have easily defined/static loading screens). You could give it an image (of a loading screen) and have it analyse all frames that contain that image (or parts of that image?). Not sure how skilled you are to have it cater for variable resolutions/animated loading screens etc.. but that would be neat and useful I think, assuming that it's even possible ... I'll let you be the judge.

You could use such a tool to aggregate data from various streams at once for games that have an easily identifiable HUD or environments. Like for Overwatch you could list all the streams that are currently playing a specific character/map/mode. For some games such a tool could give you an unfair advantage though if opponents are streaming as it could quickly gather information you shouldn't have and display it as a HUD over your game.

@Tenka my buddy @GrimMaple over at NFSruns was working on something similar to remove load times from Carbon, it was getting pretty decent results.

@Lonne cool! .. has he made it publicly available? and could it be adaptable to other games easily? i.e. allow custom image(s) to compare against?

At the very least that proves that the idea is plausible and has application.

to my knowledge hes still working on it. and in theory it could be adaptable to other games.

Nice, I hope he decides to come forward with the finished product if/when its ready. I think a lot of boards would find it useful, depending on the game.

Thank you all for your input :D. @ShikenNuggets Reading from memory is of course the most accurate solution but I want to create something that works with any screen capture e.g. from YouTube. Everyone should be able to use it and extract meaningful data.

@Lonne @Tenka removing loadscreens should be very easy, especially if it's static. It's probably the first feature I'll implement.

I've decided to make this my new pet project :D

The hardest part right now is reducing the scope of my ideas to something simple that many can benefit from.

My current idea is to create a program that lets you load a video and:

• grab values from static huds
• timestamp events like deaths, levelups, loadscreens, mapchanges,...
• cut source material based on these events (e.g. to remove loadscreens)

In essence it'll be a tool than can be used to analyse captured video, extract meaningful data from it and export it. I'll start with old games first as they have easy recognizable graphics, sounds and static huds.

With that in mind do you guys have more ideas? What would you wish from a program that could analyze a speedrun video retroactively?

In a more advanced form it could assist with a statistical analysis for detecting cheating in submitted runs; noting anomalies/outliers, etc.