Golf Clash Chest Study Results

or: How I Learned to Stop Worrying About the Tinfoil and Love Data

Analysis and writing by Scott Marler, Golf Clash University admin, with contributions from multiple GC communities.

Purpose

There are several aspects of Golf Clash that at least some part of the community believes are “rigged.” The most common belief is probably that the club cards received in chests are not random. While the official Golf Clash website does not appear to explicitly state either way if the draws are random, support representatives have certainly asserted that they are random through communication with individuals.

For example:

source: a post in Golf Clash University on February 25, 2018

 

Evaluating the randomness of club cards is the primary purpose of this study. As an added benefit, with enough data we can start to either understand (or make educated guesses about) the mechanics behind chests, even beyond club cards.

Scope

The data obtained in this study allow us to draw conclusions about the following chest types:

1)      Free

2)      Pin

3)      Silver

4)      Gold

5)      Platinum

I did obtain entries for golden shot chests and king chests, but there just isn’t enough data. It is safe to assume, however, that the same general logic applies.

Summary of Findings

  1. Club card draws are random.
  2. T9 and T12 chests obtained from playing in Expert and Master tournaments are not the same as T9 and T12 chests obtained from regular tour play.
  3. What you can get from a Free chest depends on the highest tour you have unlocked.
  4. What you can get from a Pin chest depends on what tours you have been playing lately.
  5. Free chests do not contain balls.  The odds of getting balls from a Silver, Gold, Pin, and Platinum chest are 17%, 50%, 100%, and 100%, respectively. If you get a ball from a Pin, Silver, or Gold chest, there’s a 50-60% chance it will be a Marlin. If you get a ball from a Platinum chest, there’s about a 60% chance it will be a Kingmaker.
  6. Platinum chests contain no gold, but Free, Pin, Silver, and Gold chests always do.
  7. Gems are only obtained through Free or Pin chests. The number of gems received from Free chests depends on the highest tour unlocked. The number of gems received from Pin chests depends on the recent activity of the player.
  8. Spending gems to open chests does not affect their contents.
  9. Having spent real money on the game does not affect any chest outcome.
  10. To wrap up, I will attempt to explain what I believe are some of the mechanics that govern how chests work.

 

Data

Data gathering started on the 2nd of January, 2019. Volunteers were recruited on the Golf Clash University, Golf Clash Learn or Burn, and Golf Clash United Facebook groups, plus a few others. Volunteers agreed to report all contents of every chest obtained for as many of their accounts as they wished to use for the study. Data was collected via Google Forms and processed with Python script.

We had 67 individuals report data for nearly 7,000 chests across 107 game accounts. The median number of chests reported by an individual is 57. There are a few folks who deserve special mention, though, because of their volume of contribution.

  • Jack Ryan contributed nearly 2,000 chests. He had been collecting data across multiple accounts for months prior to this study starting and the quality of that data was good enough that I was able to use it directly. This allowed me to shorten the duration of the study significantly.
  • Kat Ramage contributed over 700 chests during the course of the study. She was the most active contributor by far during the collection period and I want to thank her for getting behind this study and helping out with such dedication.
  • Robert Fudge, Michael Li, and Sonja Larson all added more than 200 chests and will be remembered as contribution rockstars.
  • Mike Trozzo, Michael Froman, Duncan Semple, Fletcher Wyman, David Roseman, Ray Wright, Beaner Brown, Derick Turner, Ben Aviss, and Keither Kerr all did more than 100 chests. That’s huge. Thank you.

Key Findings

Randomness – the Naïve Approach

The most straightforward and easy to understand way to test for randomness is to simulate repeating the study as though it really is random, repeat that many times, and compare the random results to the data we observe through the study.

To “repeat” the study, I go through every club obtained from all 6,792 chests and replace that club with a random club of the same rarity from the pool of clubs that chest can possibly receive. Across all chests, there were 11,640 club draws (159,535 club cards).

 

For example, if one of the clubs obtained from a T8 Gold chest was a Castaway (Tour 1 Epic card), I replace it with any random Epic card from the game, because the chest is T7 or higher. If a T3 Silver chest contained an Extra Mile (Tour 2 Rare card), I replace it with any random T3 or lower Rare card.

 

This replacement process was repeated 100,000 times, simulating 100,000 separate experiments where the club draws are truly random. This allows us to create a distribution of the number of times each club appeared across all random experiments and then see if the actual results we observe fit into those distributions. If they do, we can be more confident that the draws are indeed random.

 

The following shows the results for all clubs, compared with their randomized distributions. Each figure is labeled with the number of standard deviations our observed appearances are from the randomized mean.

The observed results for all clubs fall within the randomized distribution. The result that deviated most from the randomized mean is that it looks like Thor’s Hammer was obtained more frequently than we might expect – but even still, it falls within what we know is possible with a random draw based on the simulations.

Now that we’ve established quite convincingly that the number of card appearances looks random, it’s worth looking at the actual number of cards received for each, because conspiracy theorists may think that the manipulation is subtler.

 

To evaluate this, we can turn to one of the most basic tools in a statistician’s toolbox: Ordinary Least Squares Regression! (I know, it sounds fun – and you’re right, it really is.)

 

The manipulation that most suspect is being committed by Playdemic (suppressing the best Epic cards, such as Apocalypse) would be immediately revealed by regressing the quantity of clubs seen per appearance on the tour of the club. In other words, if more desirable cards are being suppressed, it should look like a tour rating increase leads to a quantity decrease. Tour goes up, # of cards goes down, and the nefarious Evil Corporation tricks our brain into wanting to spend money. Right?

 

Wrong.

 

I ran regression analysis on the quantity of clubs received against the following possible contributing factors:

·         Chest tour

·         Club tour

·         If the chest was opened with gems

·         If the player has ever spent money

·         If the club is already maxed out

In every case, I held as much constant as possible. For example, Epic clubs were only compared with epic clubs, chests were only compared if they drew from the same pool of clubs, etc. Also in every case, the results were statistically insignificant, practically insignificant, and explained almost none of the variance in the data.

 

In other words, just as you can describe my status with that girl I finally built up the courage to ask to the big dance in High School, there’s just no relationship.

 

Club draws. Are. Random.

Tourney vs. Tour Chests

This one probably doesn’t deserve its own section. But early on when I started looking at the data, I saw that the average club counts from T12 and T9 chests went up during a tournament week. This is because the chests you get from Expert and Master tournaments, even though they say T9 and T12, are actually better than their tour-play counterparts (more clubs, more rares, more epics). The chests themselves even show this, but I’d never noticed. And I imagine many others didn’t either. As an example, here are T12 Silver chests from T12 and from the Master tournament:

Free Chests

Turning once again to Regression analysis, it is clear that the maximum tour unlocked determines the number of gems, coins, and club cards received.

 

Conversely, holding the max tour unlocked constant and allowing the average of the tour value for the last 8 chests opened does not have an effect. So, for free chests it’s not about what you’ve done lately, it’s just what you have unlocked.

 

Pin Chests

Pin chests work the opposite way from Free Chests. The max tour unlocked is not relevant, Rather, it’s the recent activity of the player that governs the gems, coins, and club card counts from a pin chest.

 

If someone has T12 unlocked but has been playing a bunch of friendly games in T2, the pin chest will give significantly less loot than if the player were playing in T12.

 

 

Silver, Gold, and Platinum Chest Mechanics

With nearly 7,000 chests to look at (there’s gotta be a joke here about how my wife would feel if she knew how much time I’ve spent staring at chests…), I was able to draw some conclusions about what is going on in the background. What I’ve been able to gather is as follows:

 

In general, there are two things that matter when you get a chest. The tour, and the division bonus (duh). With these parameters, it looks like the following steps take place:

  1. Determine the type of chest
  1. Approximately 77% become Silver
  2. Approximately 22% become Gold
  3. Approximately 1% become Platinum
  1. Determine card types

The base contents of a chest are determined by the roll for the type of chest, including the total number of cards and the guaranteed number of Rares and Epics.

 

Initial counts for card types start like this:

Common:                0

Rare:                         Guaranteed rare * division bonus

Epic:                          Guaranteed epic * division bonus

  1. Find pool of remaining cards: initial count * div bonus – Rare – Epic
  2. For each card in the pool, determine if the card will be Common, Rare, or Epic
  1. About 95% are Common
  2. About 4% are Rare
  3. Less than 1% are Epic (my numbers show 0.84%)
  1. Break groups of card types into chunks
  1. If a card type has 19 or fewer cards, they all get assigned to a single club
  2. If a card type has between 20 and 69 cards, they are divided between two different clubs
  1. One club gets 50-61% of the total and the other gets the remainder
  1. If a card type has 70 or more cards, they are divided between three different clubs
  1. One club gets 40-60%
  2. The next gets 50-65% of what’s left
  3. The last gets the remainder
  1. If a NEW club type is drawn for a group, one card from that group splits off to get attributed to the NEW club, and the rest of that group gets re-drawn.

Bonus Observation

Regular tour-play chests are Gold a little over 20% of the time. We’ve long understood that the odds are somewhere between 1:4 and 1:5 of getting a Gold chest, and this experiment has bolstered that claim significantly.

However, there’s more to it than that. If each chest had an independent 22% chance of being a Gold chest, we should expect to see several instances of back-to-back Gold chests. But that simply does not happen. Gold-Silver-Gold can happen, but it’s extremely rare.

Therefore, the odds of an upgraded chest increase depending on how long it’s been since getting one.

It is reasonable to assume that the same thing happens for Platinum chests. You won’t be able to say for sure when a Platinum chest will drop, but you can be pretty confident that it’ll happen somewhat close to your 100th chest after your previous one. (I imagine there’s a pretty wide range here, but we don’t have enough data to venture a guess on how it works in detail).

Conclusion

The bottom line is that the game is not out to get you. Please remove your tinfoil hats and carry on smartly.

Addendum: Practical Application

The numbers proposed above for the probability that a non-guaranteed card will become Rare or Epic has some real, practical implications. We can figure out the odds that any given chest will contain an Epic card. And since we know that club draws are random, we can figure the odds of getting a specific epic card from any given chest.

 

Let’s use the example of a T12 Silver chest and a division bonus from Master III of +140% (multiplier of 2.4). This gives a chest with 74 cards and no guaranteed Rares or Epics. 74 cards in the pool.

 

The odds of getting ONE epic card can be found like this:

The probability of any individual card being turned into an epic is 0.84%, so the probability of NOT being an epic card is 99.16%. The odds of having NONE in that silver chest is .9916 to the 74th power, which gives 53.6%. That makes the chance of getting one epic about 46%.

 

So, our T12 Silver chest has a 46% chance of producing exactly one Epic card. Since we know the epic(s) can be one of 20 possibilities, the odds of our single T12 Silver chest producing one Apocalypse card is 46% * 5% = 2.3%.

 

It’s a little bit awkward to work with probabilities like this. For the numbers to be correct, we have to keep qualifying the statement to say “one” epic card. Because sometimes you’ll actually get 2, or 3 from that same chest.

 

To produce results that are easier to understand and more practically useful, we turn to Expected Values and the Binomial Distribution. I’ll spare you the details, but the short version is that we can calculate how many Epic cards we can expect from a chest (as opposed to the chance of getting at least 1). For our T12 Silver chest in M3, that expected value is 0.612. Note: this is higher than “46%” because it takes into account that if an epic is received at all, there could be more than 1. The expected value is not the same thing as the probability of getting one – it’s the combination of all possible outcomes multiplied by their individual probabilities, whether that’s 0 epics, 1 epic, 2 epics, and so on.

 

Taking that a few more steps further, we can actually start to figure out things that have practical real-world application. We now know what percentage of chests are silver, gold, and platinum. We know the odds of a club being upgraded to epic. We can figure out expected values for whatever we want. So, I can hear you asking, how many damn chests do I need to open to get an Apoc card?

 

Well, I’m not a damn fortune teller, but I can tell you a number that is reasonable. The following chart shows the number of chests you should expect it will take to find a single specific epic card based on the tour you’re playing and your division bonus.

 

 

Please note: this is not the odds of getting any old epic card. It’s the expected value for a specific card, and it’s only applicable to the clubs available based on the tour of the chest.

 

Examples:

  1. If you’re playing in Tour 1 as a Beginner, the possible Epic cards you can get are Castaway, Grim Reaper, and Horizon. If you just really want that Horizon card, you can expect that you’ll have to open 30 chests, on average, to get one card for that particular club.
  2. If, somehow, you’re playing Tour 7, still as a beginner, and you still just can’t get enough of those sweet, sweet Horizon cards, you can expect that it will take more than 90 chests just to get one of them. That’s because the pool of possible epic cards goes up as you go up in tours, until you reach 7. The effect of the pool size increasing is more impactful than the effect of getting more cards per chest.
  3. If you’re playing in T12, are in Master III division, and are looking for an Apocalypse card, you can expect that it’ll take 32 or so chests for a single card.

 

If we extend that last example out a bit, we can expect it would take 32 * 384 = 12,288 chests to take Apocalypse from level 0 to 7 if all we ever did was play T12 with a Master III division bonus. This highlights the importance of tournaments and the card shop for those interested obtaining maximum levels for the best clubs in the game.