As utilities push to increase the penetration of low-emissions renewable electricity, there is growing importance over the difference between whether is used during or outside of peak demand hours when electricity is more expensive and dirtier, and there is more risk of power outages where demand exceeds the available supply. Time-of-use (TOU) pricing is a policy tool that aims to reduce peak electricity demand and encourage energy conservation by charging different prices for electricity at different times of the day. However, the impacts of TOU pricing on different population subgroups, especially those with low income or limited access to cooling options, are not well understood. This work examines how the magnitude, timing, frequency, and duration of high electricity use varies across residential customers, and what the cost of different time-of-use (TOU) pricing schemes would be for these customers. We use hourly electricity consumption data from smart meters of 200,000 unique households in Southern California from 2015 to 2020. We also explore how differences in socio-economic factors such as census-tract income and racial demographics, or individual building characteristics, influence the potential for savings under current TOU pricing schemes. The results of this study provide evidence of the potential benefits or consequences of TOU pricing for different population subgroups, especially those underrepresented in most survey or volunteer-based studies. These results can inform the design and implementation of TOU pricing policies that are fair and effective at reducing peak demand.