Quantile regression (QR) provides estimates of a range of conditional quantiles. This stands in contrast to traditional regression techniques, which focus on a single conditional mean function. Lee et al. [Regularization of case-specific parameters for robustness and efficiency. Statist Sci. 2012;27(3):350–372] proposed efficient QR by rounding the sharp corner of the loss. The main modification generally involves an asymmetric ℓ₂ adjustment of the loss function around zero. We extend the idea of ℓ₂ adjusted QR to linear heterogeneous models. The ℓ₂ adjustment is constructed to diminish as sample size grows. Conditions to retain consistency properties are also provided.