Matter seen at many scales and the geometry of averaging in relativistic cosmology