Descriptive Statistics: Summary Statistics and Data Visualizations

Investigating the data we have

�Variable Types, Revisited

Categorical Variables

• Group observations into categories
• Nominal – named categories (colors)
• Ordinal – ordered categories (military rank)

Numerical Variables

• Variables for which taking an average is meaningful
• Interval – differences are meaningful, but ratios are not; no minimum or “starting” value (temperatures)
• Ratio – ratios are meaningful (heights)

�Describing Categorical Data

Numerical Summaries

• Frequency tables
• Frequency (counts)
• Relative frequency (proportions)

Visualizations

• Bar plots
• Frequency polygons
• Waffle charts
• Pie and Donut charts (try to avoid these)

�Try It!

Answer the following questions using our credit risk data set, and interpret the results

• Determine which variables are categorical and which ones are numeric
• Are there any that are neither? Is such a thing possible?
• Use a pivot table to calculate the frequencies for the person_home_ownership variable
• Add a column to the end of the pivot table, computing relative frequencies
• In the same tab as your frequency table, use the Insert menu on the ribbon and insert a bar graph for the person_home_ownership variable (highlight the categories and counts before inserting the chart)
• Explore additional/alternative plot types
• Investigate at least one other categorical variable by building a frequency table, relative frequency table, and appropriate plot

�Describing Numerical Data

Numerical Summaries

Visualizations

• Histograms
• Density plots
• Boxplots

Note: The minimum, 25^th percentile, median, 75^th percentile, and the maximum are sometimes referred to as the five number summary of a numerical variable.

�Try It!

Answer the following questions using our credit risk data set, and interpret the results

• In a new DescriptiveStatistics sheet, calculate each of the following for the loan_amnt variable
• Minimum, 25^th percentile, mean, median, 75^th percentile, maximum, standard deviation and interquartile range
• Back on the RawData sheet, use the Insert menu on the ribbon to insert a histogram for the loan_amnt variable
• Explore additional/alternative plot types
• Investigate at least one other numerical variable by calculating the summary statistics listed above and building at least one appropriate plot

Additional Comments About Numerical Variables: �Standard Deviations and z-scores

�An Example

Scenario: Assume that the distribution of SAT scores has a mean of 1050 with a standard deviation of 210, while the distribution of ACT scores has a mean of 19.4 with a standard deviation of 5.8. Sam scored a 1210 on the SAT and Charlie scored a 24.6 on the ACT. Who scored relatively better?

Additional Comments About Numerical Data:�Skew in Distributions

Numerical data can be symmetric or skewed

• Symmetric
• Numerically: the mean and the median agree (they are nearly equal)
• Graphically: the histogram (or density, etc.) have nearly identical “tails” to the left and right of the center
• Right-skewed
• Numerically: the mean is larger than the median
• Graphically: the distribution has a stretched out right tail (may have large outliers)
• Left-skewed
• Numerically: the mean is smaller than the median
• Graphically: the distribution has a stretched out left tail (may have small outliers)

Note. If a distribution is symmetric and has a single peak in its center, it is sometimes referred to as bell shaped.

�Working with Skewed Data

When data are skewed…

• The mean is no longer an appropriate measure of center
• We should use the median instead because the median is robust against outliers
• The standard deviation is no longer an appropriate measure of spread
• We should use the interquartile range (the distance between the 25^th percentile and 75^th percentile) instead because the IQR is robust against outliers

Multivariable Investigations: Summary Statistics �and Visualizations

We are often interested in the potential for associations between pairs of variables

• Categorical and Categorical
• Numerical summaries: frequencies and relative frequencies
• Visualizations: bar plots with fill color (stacked bars)
• Numerical and Categorical
• Numerical summaries: grouped means, grouped medians, etc.
• Visualizations: side-by-side boxplots, overlayed densities, etc.
• Numerical and Numerical
• Numerical summaries: correlation
• Visualizations: scatterplots

�Try It!

Answer the following questions using our credit risk data set, and interpret the results

• Create a pivot table to calculate frequencies for the combination of loan_intent and person_education
• Build a stacked bar graph to visualize the results
• Create a pivot table to calculate relevant numerical summary statistics for loan_int_rate by loan_intent category
• Back on the main data tab, build a set of side-by-side boxplots to visualize the results
• Calculate the correlation between loan_int_rate and loan_amnt
• Build a scatterplot to visualize the association (or lack thereof)

Exit Ticket

Navigate to our MAT240 Exit Ticket Form, answer the questions, and complete the task below.

Note. Today’s discussion is listed as 4. Descriptive Statistics

Task: Describe both measures of center for numerical data and when to use one instead of the other. Provide an example to help you clarify.

�Next Time…

• What we’ll be doing…
• Discrete Random Variables and Probability

Homework: Complete the Topic 5 – Discrete Probability and the Binomial Distribution interactive prep-work activity and submit both hash codes using the Google Form from this week’s BrightSpace announcement.