The reason is that in case of empty maps the infinite starting coords are used for the final extent, since the box is not reduced by map layers.