I test manually. I obviously can’t test every possible resolution, so I focus my testing on the extremes: really small window, really wide window, really tall and skinny window, and 2160p full-screen.

Automated testing for UX issues is problematic. I could write automated tests for things like “make sure the enemies spawn off-screen”, “make sure the event my character is reacting to is on-screen”, “make sure all text fits the window without going smaller than a specified font size”, but doing so would be difficult. I can’t write automated tests for more subjective elements like “make sure the player can see far enough that she can reasonably respond to threats in time” and “make sure the text is readable at all resolutions”. So I generally don’t bother with automated testing for UX issues at all.

I usually, load the level or section I want to test and change the resolution with a single key press while testing the level / section.

This is important for web games. There are a lot of problems. System zoom and browser zoom interfere with how this is handled.

Problems I usually do not see for downloadable games. A downloadable might run internally on resolution A and just scales up or down to resolution B for the screen.

Actually using the native resolution, like a widescreen display, is harder. Is there no css like function in those game engines? Where you say, this ui element is supposed to be rendered at 10% screen height? Do you really spell out the pixel height and need to test this out for different resolutions? I would assume that game engines over the years would solve this and make this more plug and play for developers.

Worth flagging the web export case, because it quietly breaks the assumption underneath the tool and nobody has said it outright yet. I ship a browser game rather than Unity, so take this as the web angle rather than a Unity answer.

In a browser there is no single thing called resolution. There are three numbers, and the bugs live in the gaps between them: the CSS size of the canvas element, the size of its drawing buffer, and devicePixelRatio, which multiplies one into the other. A downloadable game has one resolution. A web build has a CSS size times a DPR that the user can change underneath you. Browser zoom changes it. OS display scaling changes it. So you never get 1280x800, you get 1280x800 at 1, at 1.25, at 1.5, at 2, and at whatever fraction a Windows laptop happens to be set to.

That matters for your matrix specifically. Replaying a flow across a list of resolutions varies the axis that produces layout bugs, and those are the cheap ones, the ones a screenshot diff was always going to catch. The axis that produces the expensive ones is DPR at a fixed resolution. 1280x800 at DPR 1 and 1280x800 at DPR 2 are the same layout and four times the pixels, so what breaks is not the layout at all, it is memory and frame time. In a tab that is not a slowdown either, it is the tab getting killed, because there is no swap to page out to the way a native client has.

Which is why we clamp devicePixelRatio rather than honouring whatever the display reports. Uncapped on a high-DPI screen it is the easiest way there is to quadruple your buffer for no visible gain.

The other failure a same-DPR screenshot diff cannot see is the CSS size and the buffer size disagreeing. Nothing errors, the canvas is just soft, and it reads as an art problem rather than a code one 🔎

So the question I would actually put to you: does the recorder capture devicePixelRatio as part of the recorded environment, or only the pixel dimensions? If DPR is a first class axis sitting alongside resolution, this is useful to anyone shipping a web build. If it is not, it will find the cheap bugs and miss the ones that cost a weekend.