1 of 16

L33t or N00b? How Player Skill Alters the Effects of Network Latency on First Person Shooter Game Players

1

Shengmei Liu

Mark Claypool

Atsuo Kuwahara

James Scovell

Jamie Sherman

Worcester Polytechnic Institute

Intel Corporation

2 of 16

Motivation

2

Mouse

Display

Local latency

50 ms

40 ms

Upgrade

20 ms

25 ms

Quality of Experience

😃

Performance

First let’s talk about the motivation. It’s a common knowledge that decrease in latency helps game players, and network latency is an important type of latency. It is the RTT between the client and server. For example, the uplink time is 40ms and the downlink time is 50ms.

The network latency would be 40+50= 90 ms.

Now if we upgrade the network to a faster network, there would be shorter network latency. The shorter latency will benefit gamers.

Talking about affecting gamers, the 1^st aspect would be performance. This would affect your ranking in the game.

We have two matrix for performance, and they are weapon accuracy which is the hit miss ratio and in-game score.

The 2^nd aspect would be players feeling, and player feeling is represented by the quality of experience.

Latency and players has been a well-studied area. However, players may have different skills in games. Many game players pay attention to network latencies, but how much network latencies impact players based on skill is not well-known. Understanding the impact of network latency by player skill may better inform gamers about the need to upgrade their network connections and motivate developers and researchers to devise tools and systems to mitigate network latency for games and game-like applications. The question we are trying to answer here is how does latency affect first-person shooter game players in different skill groups on performance and quality of experience.

3 of 16

Related work – Latency and FPS

3

“Ok…but why?”

Skill focused

Armitage et al.:

Latency tolerance for Quake 3 is about 150~180 ms.

Amin et al.:

player experience determines network latency sensitivity for Call of Duty, with competitive gamers more adept at compensating for impaired network conditions.

Quax et al.:

Unreal Tournament 2003 players suffer with network latency and jitter as low as 100 ms.

4 of 16

Related work - Skill

4

“Ok…but why?”

Low latency < 150 ms
First-person shooters
Both performance and QoE
Considerable sample size

Claypool.:

Higher skill players are resilient to performance degradations for latencies above 350 milliseconds.

Amin et al.:

Higher-skilled players notice even small amounts of latency but are able to compensate for it better than lower-skill players.

Dick et al..:

Skill impacts score but not QoE.

Here are more literature on player skills. Claypool [4] classify 51 users into three skill groups, have them play a target selection game with latency and show that higher skill players are resilient to performance degradations for latencies above 350 milliseconds. these delays are much higher than many gamers experience in actual game plays.

Amin et al. [1] query two users with different amounts of skill infer that higher-skilled players notice even small amounts of latency but are able to compensate for it better than lower-skill players. However, the sample size here is far too small to generalize

Dick et al. [6] separate 8 users into two teams, have them play four FPS games with network latency and jitter to study the factors that impact players, finding skill impacts score but not mean opinion score (MOS).

While beneficial in understanding the impact of network latency, these works focus on high latency that may not practical for gamers, or gaming actions instead of a full game, or suffer from a low sample size. While in our work, we will focus on low latency less than 150 ms and first-person shooter games. We will study both performance and experience with considerable sample size.

5 of 16

FPS Hours

5

Higher skill players

Reaction

i. Number of participants: 25

ii. >= 100 hrs total in CSGO, average ~900 hrs

iii. Gender: 25 males, 0 females

iiii. Age: 17 – 29, mean = 20.8, std dev = 3.00

Higher skill players

Lower skill players

i. Number of participants: 11

ii. <100 hrs total in CSGO, average ~20 hrs

iii. Gender: 11 males, 0 females

iiii. Age: 18 – 28, mean = 21.0, std dev = 3.4

Mean = 205

Mean = 204

Mean = 1703

Mean = 891

Lower skill players

Higher skill players

Lower skill players

Demographics

Our intent was to get 30 skilled CS:GO�users. But pandemic made this hard. So, had to recruit from�non-CS:GO players too, but skilled with FPS. Thought they'd be the same,�but were not.

Finally, 36 users were recruited in total. The users were recruited by self-reported total hours in CSGO. Players with greater than 100 hrs in csgo are higher skill players, and with less than 100 hrs in CSGO are lower skill players. We have 25 higher skill players and 11 lower skill players in total. Higher skill players played about 900 hrs of CSGO on average, while lower skill player only played about 20hrs on average

For gender, all the participants were males in both group. We recruited from university email list thus all the participants were of typical college age.

Here are more information about the users. These two graphs are the box and whisker plot of the self-reported hours in general first-person shooter games. Both boxes depicts quar-tiles and medians for the distributions, and the black pluses show the mean values. The left one is for higher skill players and the right one is for the lower skill players. The mean value for higher skill players is about 1700 hours. For lower skill players, The mean is about 900 hours. Generally speaking all the users are experienced in general FPS games, but higher skill players are even more experienced.

These graphs on the right are for their reaction time. The mean is about 200 ms for both groups, which is typical reaction time of experiencd gamers.

6 of 16

User Study

Network latency

Shuffled total network latency (25, 50, 100, 150 ms)

Questionnaires (Demographic, In-game surveys)

6

0

100

200

300

(ms)

Good

Internet Connection

Moderate

Internet connection

Other

<1

25

50

100

150

Game customization

Deathmatch FFA
Single player with 20 bots
3.5 mins per round

Client-server architecture

To study the impact of network latency on Competitive First-Person Shooter Game Players, we designed a user study.

The game for the study was Counter-strike: Global offensive, one of the most popular first-person shooter games.

We customized the game, so the game mode was deathmatch FFA, where all the 20 bots are enemies, and the goal for players is to kill as many as they can. Each round lasts 3.5 minutes. There were 2 pilot studies and many parameters like game length, latency value etc. were adjusted.

Our user study was conducted in a dedicated, on-campus computer lab using a client-server architecture. The server hosts the game and is connected via high-speed LAN to the client. The client is equipped with a gaming mouse and high-refresh rate monitor so as to minimize local system latency. The client runs the latest Linux system and the server runs Windows 10. Both server and

client run CSGO. Users were given wired Apple airpods for audio. Social distance was conducted and all the devices were sanitized before each user. The game flow was self-contained. All the users need to do was to double click on the only button on the desktop, and the flow will be loaded automatically.

Normally speaking, network latency from 0 to 100 indicates a good internet connection, moderate internet connection may have latency between 100 and 200 ms. And bad internet connections would cause even higher network latency.

The base round trip time for the LAN in the study was less than 1 ms. It’s captured by a high speed camera with 1000 f/s.

There are 4 shuffled total latency values for the rounds, and they are 25, 50, 100, 150 ms.

After each round of game, players answers 9 questions about their experience and feelings. After completing all the games, players fill out a demographic questionnaire.

7 of 16

Results

7

Network latency has a 32% higher impact on accuracy for higher-skill players.

Network latency has a 25% higher impact on the number of headshots for higher-skill players.
Higher-skill players hit 50% more headshots per game than lower-skill players.

The graph on the left depicts weapon accuracy versus latency.The x axis is the network latency. The y axis on the left is the weapon accuracy decrease from the 25 ms network latency condition of the higher-skill group. For example, an accuracy of 20 percent at 25ms of latency compared to an accuracy of 15 percent at 125 ms of latency would be a 5 percent decrease. The y-axis on the right is accuracy percent. The points are the means for all users for that latency condition, bounded by 95% confidenceintervals. The dashed lines show a linear regression for the meanvalues. The blue points and lines denote the higher-skill users, and the red denotes the lower-skill users.

The regressions fit the means well for both group of users, with high R square and low p value for both groups. Visually, the higher-skill slope is slightly steeper than the lower-skill slope indicating etwork latency has a greater effect on accuracy for higher-skillplayers than for lower-skill players. As a take-away, a decrease in network latency by 100 ms decreases accuracy for higher-skill players by an average of about 2.2 percent, and lower-skill players by1.5 percent – network latency has a 32% higher impact on accuracy for higher-skill players

While visually, the difference in accuracy slopes in the graph on left for higher-skill users and lower-skill users maynot be dramatic, what often matters is where hits land – shots to the arms and legs do far less damage than shots to the head. For reference, a single headshot with an AK-47 in a CS:GO game will kill a full health opponent.

The graph on the right depicts the total number of headshots per minute versus latency. The axes and points are as for graph on the left but here the y axes are headshots per minute instead of accuracy. The regression fits the means well for the higher-skill group. However, the regression fits the lower-skill group less well, perhaps owing to the smaller sample size. Note the average number of headshots for the lower-skill group is higher at 100 milliseconds of latency than the average at 50 milliseconds of latency. Visually, the slopes for both groups are clearly separate, with the higher-skill group getting about 50% more headshots per round than the lower-skill group. The slopes look visually similar, but are actually slightly steeper for the higher-skill group. As a take-away, an increase in network latency by 100 ms decreases the number of headshots landed by higher-skill players by an average of about 0.4 per minute, and lower-skill players by 0.3 per minute – network latency has a 25% higher impact on the number of headshots for higher-skill players. Higher-skill players hit 50% more headshots per game than lower-skill players.

8 of 16

Results

8

Network latency has about 5X impact on higher skill players than on lower skill players.

This graph depicts player score versus latency. The axes and points are as in previous graphs, but the data is the score (2×𝑘𝑖𝑙𝑙𝑠+𝑎𝑠𝑠𝑖𝑠𝑡𝑠) per minute instead. The regression fits the means well for the higher-skill group. However, the regression fits poorly for the lower-skill group, again probably due to the smaller sample size and high point at latency 100 ms for the lower-skill group. Visually, the slope for the higher-skill group is steeper than the slope for the lower-skill group. As a take-away, a decrease in network latency by 100 ms degrades player score by about 2 points per minute for higher-skill players, and 0.4 points per minute forl ower-skill players- Network latency has about 5X impact on higher skill players than on lower skill players. For reference, often less than a single point separates the scores of the top CS:GO players in a game

9 of 16

Results (Quality of Experience)

9

Network latency has twice the impact on QoE for the higher-skill players.

For an overall measure of QoE, we compute the mean combined rating, weighting all questions equally. Figure 7 depicts the results. The x axis is the network latency in milliseconds and the y axis is the rating. The points are the means for all users for that latency condition, bounded by 95% confidence intervals. The dashed lines are linear regression fits through the mean values. The blue represents the higher-skill group and the red represents the lower-skill group. The linear regressions fit the means well for both groups.

Visually, the QoE values are similar for the two groups at 25milliseconds of latency, but the impact of latency is greater for the higher-skill group than the lower skill group as evidenced bythe steeper slope. As a take-away, an increase in network latency by 100 ms degrades QoE by about 0.3 points on a 5-point scale for lowerskill players, and 0.6 points for higher-skill players – network latency has twice the impact on QoE for the higher-skill players

10 of 16

Limitations

10

Team Effort

Sample size

Gender bias

All bots

Here are some limitations. Our user study intentionally focuses on the effects of latency on individual player performance. Many FPS games are often team games, where groups of players work together to defeat the opposing team. The impact of latency on CS:GO team efforts, perhaps even team strategies, was not assessed.

Movever, Most CS:GO games use only human players and not AI-controlledbots. While it is likely that the absolute scores observed would differ for users pitted against human players, the relative effects should be similar since the latency affects the ability to aim and shoot.

Our study has only 11 players in the lower-skill group. While thisis more than some other published studies of player skill and games,the small sample size limits the statistical power of the results.

Our study is skewed towards males (nofemales participated). While this may reflect the gender breakdownof FPS games today, the results may not be indicative of female performance in competitive FPS games.

11 of 16

Conclusions

11

Network latency has high impact on higher skill players, with both player performance and quality of experience

For higher-skill players network latency has fifty-percent more impact on accuracy, five-times the impact on score, and twice the impact on QoE than for lower skill players. �

We study the effects of network latency on the first-person shooter game CS:GO comparing two groups of players: higher-skill– those with extensive FPS experience and considerable CS:GO experience, and lower-skill– those with extensive FPS game experience but little or no CS:GO experience. We setup a testbed that allowed for CS:GO play with controlled amounts of latency, gathering objective data with log files and subjective data through surveys. Thirty-six (36) users (25higher-skill and 11 lower-skill) each played rounds of CS:GO with four different latency conditions: 25, 50, 100 and 150 milliseconds. Based on our analysis, network latency has more impact – both player performance and player Quality of Experience – on higher-skill players than on lower-skill players. In general, for CS:GO with 25 milliseconds of network latency: (A) for higher-skill players, an increase of 100 ms of network latency decreases accuracy by 2.2percent, score by 2 points per minute, and QoE by 0.6 points (outof 5), while (B) for lower-skill players, an increase of 100 ms of network latency decreases accuracy by 1.5 percent, score by 0.4points per minute and QoE by 0.3 points. Put it another way, for higher-skill players network latency has fifty-percent more impact on accuracy, five-times the impact on score, and twice the impact on QoE than for lower-skill players.�

12 of 16

Future Work

12

More games

…..

More weapons

…..

More skills

…..

Higher latencies

…..

0

100

200

300

(ms)

Good

Moderate

Other

13 of 16

L33t or N00b? How Player Skill Alters the Effects of Network Latency on First Person Shooter Game Players

13

Shengmei Liu

Mark Claypool

Atsuo Kuwahara

James Scovell

Jamie Sherman

Worcester Polytechnic Institute

Intel Corporation

14 of 16

14

15 of 16

Motivation

15

Mouse

Display

Local latency

Quality of Experience

🤨

😃

Performance

50 ms

40 ms

Upgrade

20 ms

25 ms

First let’s talk about the motivation. It’s a common knowledge that decrease in latency helps game players, and network latency is an important type of latency. It is the RTT between the client and server. For example, the uplink time is 40ms and the downlink time is 50ms.

The network latency would be 40+50= 90 ms.

Now if we upgrade the network to a faster network, there would be shorter network latency. The question is how much does the upgrade help competitive fps players?

Talking about affecting gamers, the 1^st aspect would be performance. This would affect your ranking in the game.

We have two matrix for performance, and they are weapon accuracy and in-game score.

We also want to know how would lower network latency helps players feeling, and player feeling is represented by the quality of experience.

16 of 16

Related work

16

Ok…

but why?

Low latency < 100 ms
Competitive gamers
Latency compensation

Armitage et al.:

Latency tolerance for Quake 3 is about 150~180 ms.

Dick et al.:

UT2003 players suffer with latency and jitter as low as 100 ms

Fritsch:

Players of the role-playing game Everquest2 can tolerate hundreds of milliseconds of network latency

Sheldon et al.:

Some aspects of play in the real-time strategy

game Warcraft 3 are not affected by up to a second of network latency