There is a clear point in a woman's life when the gender pay gap gets worse