We discuss a framework for understanding why spin glasses differ so remarkably from homogeneous systems like ferromagnets, in the context of the sharply divergent low temperature behavior of short- and infinite-range versions of the same model. Our analysis is grounded in understanding the distinction between two broad classes of thermodynamic variables-those that describe the global features of a macroscopic system, and those that describe, or are sensitive to, its local features. In homogeneous systems both variables generally behave similarly, but this is not at all so in spin glasses. In much of the literature these two different classes of variables were commingled and confused. By analyzing their quite different behaviors in finite- and infinite-range spin glass models, we see the fundamental reason why the two systems possess very different types of low-temperature phases. In so doing, we also reconcile apparent discrepancies between the infinite-volume limit and the behavior of large, finite volumes, and provide tools for understanding inhomogeneous systems in a wide array of contexts. We further propose a set of "global variables" that are definable and sensible for both short-range and infinite-range spin glasses, and allow a meaningful basis for comparison of their low-temperature properties.