Brill-Noether divisors on the moduli space of curves and applications