Faster Melee input lag testing samples collection (v6, 2017-04-13)
Comments
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

 
Comment only
 
 
Still loading...
ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAFAGAHAIAJAKALAMANAOAPAQARASATAUAV
1
↓Sample number→Test series, vsync
Console VGA CRT monitor 60hz
FM4.3 NTSC i7-2600K GTX960 CRT120hz
R1 FM4.3 NTSC i5-750 HD5770 Dualmonitor CRT120hz
FM4.4 NTSC Laptop Haswell-i5
R2 FM4.3 NTSC i5-750 HD5770 1monitor CRT120hz
R3 FM4.4 NTSC i5-750 HD5770 1monitor CRT 120hz vrh x1, vrh gecko code off
R4 FM4.4 PAL i5-750 HD5770 1monitor CRT 120hz vrh x4, vrh gecko code on
R5 FM4.4 PAL i5-750 HD5770 1monitor CRT 120hz vrh x1, vrh gecko code off
R6 FM4.0 PAL i5-750 HD5770 1monitor CRT 120hz vrh x2, vrh gecko code on
R7 FM4.4 NTSC i5-750 HD5770 1monitor CRT120hz 4xVRH+code, half audio rate
R8 FM4.4 NTSC i5-750 HD5770 1monitor CRT120hz 4xVRH+code, half audio rate
R9 FM4.4 NTSC i5-750 HD5770 1monitor CRT120hz 4xVRH+code, half audio rate
R10 FM4.4 NTSC i5-750 HD5770 1monitor CRT120hz 4xVRH+code, half audio rate
U8 FM4.4,vs=fast
U9 FM4.4,vs=off
U10 FM4.4,vs=off
H1 FM4.4
B150M-Pro4S FM4.4
Z77-Pro4B150M-Pro4SB150M-Pro4SB150M-Pro4SB150M-Pro4SB150M-Pro4S
B150M-Pro4S FM4.4
B150M-Pro4S FM4.4
B150M-Pro4S FM4.4
B150M-Pro4S FM4.4
Wii HDMI 5€ AliExpress
B150M-Pro4SB150M-Pro4S
B150M Dolphin-vsync on
B150M GTX-vsync fast
FM4.9 NTSC i7-2600K GTX960 CRT120hz
FM4.9 NTSC i7-2600K GTX960 CRT120hz
FM4.9 NTSC i7-2600K GTX960 CRT120hz
FM4.9 NTSC i7-2600K GTX960 CRT120hz
FM4.9 NTSC i7-2600K GTX960 CRT120hz
FM4.4 NTSC i7-2600K GTX960 CRT120hz
FM4.4 NTSC i7-2600K GTX960 CRT120hz
FM4.4 NTSC i7-2600K GTX960 CRT120hz
FM4.4 NTSC i7-2600K GTX960 CRT120hz
2
Monitor →CRT, 60HzCRT, 120HzCRT, 120Hz
Laptop-LCD, 60Hz
CRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzU2515H, 60HzU2515H, 60HzU2515H, 60Hz
HP LP2065, 60Hz
VG248QE, 60Hz
VG248QE, 60Hz
VG248QE, 120Hz strobing10%
VG248QE, 120Hz Game Mode
VG248QE, 120Hz SRGB
VG248QE, 120Hz SRGB
VG248QE, 120Hz Game Mode
VG248QE, 60Hz
VG248QE, 60Hz
VG248QE, 60Hz
VG248QE, 120Hz
VG248QE, 60Hz
CRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120HzCRT, 120Hz
3
PC →Gamecubei7-2600K, GTX 960
i5-750, HD5770
i5 (Haswell)
i5-750, HD5770
i5-750, HD5770
i5-750, HD5770
i5-750, HD5770
i5-750, HD5770
i5-750, HD5770i5-750, HD5770i5-750, HD5770i5-750, HD5770
i5-2500K, GTX 950
i5-2500K, GTX 950
i5-2500K, GTX 950
G4560 IGPG4560 IGP 2xIR
i5-2500K, GTX 950 2xIR
G4560, GTX 970
G4560, GTX 970
G4560, GTX 970
G4560, GTX 970
G4560, GTX 970
G4560 IGP 1xIRG4560 IGP 2xIRHD 5770 2xIRHD 5770 2xIR
Wii→HDMI 5€ AliExpress
G4560, GTX 970G4560, GTX 970G4560, GTX 970G4560, GTX 970
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
2600K, GTX 960
4
FM, game version, buffer →N/A, PAL?4.3, NTSC4.3, NTSC4.4, NTSC4.3, NTSC
Disabled, NTSC
4.4, PALDisabled, PAL4.0, PAL4.4, NTSC4.4, NTSC4.4, NTSC4.4, NTSCv4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 3v4.4, NTSC, 0v4.4, NTSC, 0v4.4,NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0N/Av4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.4, NTSC, 0v4.9, NTSC, 0v4.9, NTSC, 8v4.9, NTSC, 8v4.9, NTSC, 8v4.9, NTSC, 8v4.4, NTSC, 8v4.4, NTSC, 8v4.4, NTSC, 8v4.4, NTSC, 8
5
17625.641.45638.647.442.648.829.833.638.432.243.45855536845403538383140555260557633192630226760516471604566
6
27224526651.264.8536025.640.850.247.647.85034507056373342313742294044517623272129206470546051555957
7
36815.237.26246.258.449.26037.641.86836.641.84340677943322841284041473750486625252836246873516460607360
8
456.426.835.45646.464.251.276.831.240.252.434.446.44236677048412628343447324963477837203631346465546057555859
9
56823.243523653.657.860.83540.641.44551.24542626151394334343032454456526529182937385764596257596752
10
66024.856.65831.85654.459.234.641.438.447.830.45531656452302644283435354264448224272532247371486958675160
11
767.229.647.76036.252.858.466.2234745.250.442.45643447251313734373735384868486423283324296965626252666155
12
867.631.2486238.642.654.656.234.241.64136.2443433456551402435293830524460447127243542216259686453685664
13
973.621.650.45050534960.82934.24647.645.25848615940383532453043344850418041213128236448505666576558
14
1064.424.8474243.668.662.456.834.836.249.442.642.65035547440322833413041495256407326184631245559625762636068
15
115820374835.454.459.856.633.437.856.854.4335748646252423030323833284458467133363832375976626251605858
16
1260.419.256.447.253.242.650.227.24148.240.436.44456505555364735323635504263426831223439285950616562596154
17
1373.617.253.847.459.648.862.631.248.656.84141.43645517153433437393645424448516233233631286762615769526449
18
14663231544641.865.622.638.236.63847.45036567863523131393126474964528428263140224853596464675661
19
1561.228.457.84343.841.457.625.443.642.851.839.83949626447602931264437575652367624213437305259696664676450
20
1675.220.838.4525845.463.439.644.851.243.634.84956566856454037403725484465487526214142276564625959666266
21
177432.8524056.846.251.624.24039.442.449.44742586441502834373734334047567732243941326360676255586957
22
186227.23244.651.257.249.431.638393749.65845586852433130332935374461507831202236206254686957586957
23
1962.822.83838.44144.449.626.84050.441.838.25540527845433134403625584255498227264430285365585552626265
24
2061.227.232.653.250.648.670.632.643.441.850.451.64444536947453223383037305546447024123122316651616051646065
25
Maximum lag in ms7632.857.8665468.662.476.839.648.66854.451.65856677963604744454447585668568441364642387376696971687368
26
Minimum lag in ms56.415.2314231.84141.448.822.633.636.632.230.43431445540302423262925283744366223122122204848485551524549
27
Average lag in ms66.3824.7244.38555.6363636443.6953.850.4459.1430.4740.6446.6743.0642.8448.542.956.467.9549.440.9532.434.1535.0534.7535.942.345.856.547.273.728.8522.93333.527.161.8561.459.3561.8558.5561.156159.05
28
Average lag on tearing frame
Tearing not accounted for
Tearing not accounted for
44.52857143Tearing notTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag not accounted for¹No tearingNo tearingTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurementsTearing lag integrated into measurements
29
Average lag on no-tearing frame
39.88571429accounted for
30
31
Date2016-092016-092017-02-242017-02-042017-02-242017-02-242017-02-242017-02-242017-02-252017-02-252017-02-252017-02-252017-02-252017-03-072017-03-072017-03-072017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-03-182017-04-132017-04-132017-04-132017-04-132017-04-132017-04-132017-04-132017-04-132017-04-13
32
NotesTournament reference, good CRT console setup should be just like this. A Gamecube with an original DAC cable rewired for VGA 480p output and a Sony GDM-F520 monitor set to 120 Hz was used for this test. I think I used a PAL iso, but I'm not 100% sure. Will re-test.Optimal netplay reference, using a CRT monitor at 120 hz and a modern graphics card (GTX 900 series is the best, most efficient and latest that still supports analog output at high resolutions and frame rates). Lackluster speed compared to C series, possibly due to the older graphics card. Consequent test series (R2-R10) didn't show noticeable improvements, so it seems quite likely to me that the AMD 5770 card from 2009 is causing much higher latency than the GTX 960 from 2015. This was very surprising to me, I had expected that modern graphics cards should be very similar in processing time. The only big improvement was with using FM4.0, but according to Truck that build is unstable, so it's not a viable option that makes much sense to prioritize testing on.Done on a laptop a friend brought. It was a basic office model, not a gaming laptop. Slow LCD panel, probably.Control series for R1, done out of bewildering that R1 was so much higher than column C despite the same software configuration. The average of these 20 values was so close to R1 though that it seemed that the additional latency really was from hardware. Afterwards, I ran R3-R5 with PAL iso / 4xVRH codes (the essential part of FM that reduces lag) off to check whether there was an error on my part that made the 4xVRH code not take effect. GPU 0%I decided to also test the PAL Faster Melee build (pre-configured) I had lying around to see whether it was any different. Surprisingly, the average of 50 ms is noticeably higher than that of R2, which has essentially the same configuration apart from iso. Iso used was a near vanilla PAL iso (only change is access to debug menu). I wanted to use the PAL community build, but access to develop mode is removed there, so I cannot use it to set up my testing environment (pitch black to perfect white scene change on X+↓ button input).From having the R4 PAL iso results closer to NTSC with the 4xVRH disabled, I thought it was likely that the 4xVRH code was not working properly with PAL. So I did this series, which is just like R4 except for having 4xVRH disabled. Average latency was yet stronger increased, so it seems like PAL isos run with more latency than NTSC isos, at least in FM4.3 / FM4.4. Whether this is also true on consoles will be an interesting question to look into.Because TruckJitsu told me that FM4.0 had way less input lag than all the builds afterwards (at the cost of stability), I wanted to see just how much faster it would run on this machine that seemed otherwise slow. The average of 30 ms is a huge improvements over this PC's other results, but it still doesn't reach the speed of the GTX 960 PC completely. Also, we can expect that with FM4.0, the GTX 960 PC would get averages of below 20ms (will test for this next time), so these results do not contradict the impression that the HD 5770 has innate lag.The consistent difference between NTSC and PAL was puzzling to me, so I decided to do four more test series with the exact same settings, which are very close to those of R2 (except for FM4.4 instead of FM4.3 and half audio rate being enabled, which seems to not have made any difference). These were all very close to each other, with only 6 ms between the smallest and highest average, so it seems that the PAL series is too much of an outlier to be randomness variation. I'll still do two more PAL test series just to make sure, before I go into PAL vs NTSC testing on console. At this point, what do these results mean for you as a FM user? – The average of 24 ms input lag that I posted in 2016-09 is not the amount of lag you can assume to have on your system. Depending on your graphics card and monitor, and possibly other things not yet known, your setup might be much closer to 40-55 ms input lag than to 24 ms. This makes quite a bit difference in the amount of buffer you can use. With a 24 ms setup, you can have pings of up to 42 ms and still play like on console. That's enough for about 1400 km air distance in Europe. A setup with 55 ms, on the other hand, can only tolerate pings of up to 11 ms before it feels slower than console. 11 ms is only enough for about 300 km. Because of this, for now, if in doubt or you feel like your FM setup feels slower than it should be, I recommend buying a GTX 900 series card if you use a CRT monitor, or a GTX 1000 series card if you use an LCD monitor. (GTX 1000 series doesn't have VGA, so you can't possibly use it with a 120 Hz CRT monitor.) The weakest cards of their series, GTX 950 (~90$/€) and GTX 1050 (~120$/€) are easily capable enough for Faster Melee."vs=fast" means vertical synchronisation is set to "fast" in the NVidia control panel. Triple buffering is disabled. The average of 48 ms leaves 18 ms for ping, so with pings of 18 or less, you can afford to set vsync to fast. Vsync set to "on" gave ridiculous lag, both with the Dolphin setting and with the NVidia control panel setting (roughly 120 ms, unplayable). I also tried setting triple buffering to on while vsync was on in the NV panel, the lag was still roughly 120 ms. So keep vsync either at off or at fast.Vsync off gives the average of 43 ms latency again and allows for up to 23 ms of ping.3 Buffer is still a bit faster than console+CRT. Theoretically, one buffer in 4.4 should be 4.2 ms, so 3 buffers should be 12.6 ms slower than 0 buffer. The average here is 13.5 ms slower than with 0 buffer, so that's sufficiently close. With 10 more ms left until console+CRT average, we can expect that two more buffers will be fine, for a maximum of 5 buffers with this setup and vsync off (3-4 with vsync=fast).Antialiasing might have been enabled, the values in column AA should be more reliably (it was certainly disabled there)Averages forFrom this test series, it seems that even with anti-aliasing disabled, 2xIR is slightly too demanding for the weak HD 610 integrated graphics of the G4560 CPU to run with as low latency as with 1xIR. So if you want more than 1xIR, you should either opt for the G4600 instead of the G4560, or buy a graphics card to use with the G4560.This test series confirmed the assumption that the lag in the i5-750 + AMD HD 5770 setup was due to the GPU. Whether other AMD cards also have considerable lag is not yet known, I don't have any others to test with. Here, the AMD HD 5770 was roughly 11 ms slower than the integrated graphics of the G4560 CPU at 2xIR (column AA), and 14 ms slower than the G4560 iGPU at 1xIR (column Z). Compared to column T, which used the VG248QE at 60Hz with an Nvidia GTX 950, this test series was 16 ms slower even.Increasing refresh rate from 60Hz to 120Hz gave the expected 8ms improvement. Compared to NVidia GTX 900 cards, there still was noticeable lag – roughly 13 ms more. So in total, the HD 5770 is clearly a laggy GPU and not recommended. The very cheap Wii→HDMI adapters from AliExpress seem to be nearly lagless, with roughly 4ms of lag (and another ~3-4ms of lag from the Asus VG248QE over CRT). Playtesting on this setup was okay, however judging from the measurements, the original "Sewell Wii2HDMI" Fizzi tested should be 2-6ms faster than this cheap adapter. ¹ To check for similarity to column C (before I started using vsync compensation). Here, we are about 2 ms faster on average, which is within the error range for my method with 20 samples per series (roughly ±3ms). Dolphin vsync seems to work well. Use it if your combined lag (GPU + monitor + ping) is about 8 ms faster than console and you prefer a tearing-free picture.Again, setting vsync to fast in Nvidia control panel worked well and seems to be just as reliable as turning vsync on in Dolphin.Slightly better average here on 4.9 than the last similar one on 4.4, with about 2 ms improvement. 2 ms is within the statistical error of this measuring method / sample size, so these 20 samples alone vs. the older 20 samples do not prove that FM4.9 is indeed 2 ms faster.Since it has been said that 8 buffer is a good default one for FM4.9, here is a test series that checks whether that's true. Indeed, here I got 5 ms less than console+CRT even, so even with a 120 hz LCD we should stay at console levels (~67 ms on average). For 60 Hz, I'd say stay with 1-2 buffers less. So: 9 buffers for 120 Hz CRT, 8 for 120 Hz LCD, 7 for 60 Hz CRT, 6 for 60 Hz LCD seems reasonable to me.Same as column AJ, another 20 samples to increase statistical significance. Current goal is comparing 4.4 and 4.9, both at 8 buffers.Same as AJ, AK and AM, with 80 samples, significance should be okSTDEV↓↓STDEVAltogether, we have averages from 58.55 to 61.15, compared to FM4.9's averages of 59.35 to 61.85. If anything, FM4.9 is here seeming to be slower, unfortunately. However, I feel pretty confident that the differences between 4.4 and 4.9 are only due to different tearing lines. These crawl up or down very slowly, so they introduce a quite considerable error range. This could be avoided by only taking one sample per minute or so, but this would make doing these tests very time-consuming. ±2ms is the most precise I can get with the current method, and I think that should be enough.
33
Testing procedure: I compared the button input that initiated an in-game full black to full white scene change (can be accomplished within develop mode) with the green line of the RGBHV video output and two photodiodes, one mounted at 25% from the top of the screen (120 Hz CRT monitor), and one mounted at 25% from the bottom. When only using one photodiode and no green video signal, tearing leads to sometimes getting measurements that are lower than justified (if the frame tears just a few lines above the lower photodiode's position, the player does not get to perceive this frame's relevant in-game cues in time). So whenever tearing happened between the two photodiodes, I kept counting until the next 120 Hz frame's lower photodiode activation. If a CRT was used, it's always a lagless computer monitor (Sony GDM-F520 [137khz] usually, sometimes Sun GDM-5510 [130khz]).CPU 20-30%all 1006.1889933955.713283988
34
Errors in debug menu until reaching VS MODE (Probably a bug in 4.4) 1xVRH and the corresponding Gecko code set to off caused a big latency increase over R2, so it seems like R2 was configured properly. Further strengthens the assumption that the big difference in B and C columns is from the different GPU (or a different hardware aspect I'm not yet aware of).samples:
35
34.45AVG↓
36
61.1125
37
maximum:
38
47
39
40
minimum:
41
23
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
 
 
 
Sheet1