LO 4.2.3.F

Learning Objective: Contrast tree-based methods and linear models.

Review:

• Linear regression function is defined by

• Regression tree function is defined by

where

ci is the mean of the responses in the region Ri .

1Ri is the indicator that is worth ‘one’ if the instance response falls in the region Ri , otherwise zero.

R1,R2,...,Rm represent a partition of feature space.

• Suppose the relationship between the features and the response is well approximated by a linear model. In that case, an approach such as linear regression will likely work well and will outperform a method such as a regression tree that does not exploit this linear structure.

Source: Assigned reading

• If instead there is a highly non-linear and complex relationship between the features and the response, then regression trees may outperform classical approaches.

Source: Assigned reading