Dilute magnetic materials that are both random and “frustrated” possess a remarkable “spin glass” phase. Though the materials are quite disordered, and exhibit a rather dull phase diagram, there exists an internal “ultrametric” order that has startling consequences. For example, though the spin-spin coupling is in the picosecond range, the time to arrive at equilibrium is of the order of (or larger) than the age of the universe. Dynamical magnetic measurements exhibit a strong dependence on initial preparations, as expected for a complex system. They are employed to tease out the nature of the phase transition (a continuous set of phase transitions from the “glass temperature” to T=0) and extract the correlation length (a four spin quantity). Phase space is highly degenerate, but communication between states is arbitrarily slow. Recent experiments have demonstrated how equilibrium can be attained (through finite size effects) and have detected collective behavior associated with those states in phase space that have communicated with one another on experimental time scales.