Can very compact and very massive neutron stars both exist?