Home energy management systems have become more widely available due to the continued emphasis on environmental consciousness, the increased implementations of smart grids/smart meters, and the desire to have more control over one's home. A key challenge in designing effective home energy management systems is understanding the underlying causes that impact home energy consumption. To address this challenge, in this paper, we present results from an interview-based study of 22 households in Baltimore City, Maryland, across a wide range of income groups, occupant types (age, number of home occupants, and occupation), and house types (rentals and user-owned). Using a semi-structured interviewing process, we present several insights regarding home energy consumption that impact the usage and effectiveness of current and future systems. As an example, we find that non-human occupants, such as pets, significantly influence home energy consumption. Additionally, household dynamics and hierarchy, as well as routine behavior and individual habits, produce significant decision making challenges for an energy management system. Finally, problems with home insulation and appliance age are seen as fiscally insurmountable, suggesting that newer, cheaper, and readily available retrofit solutions would be beneficial.