The old adage "Your only young once" is very true, and the question is are you a risk taker or are you content with your life as it is. Only you can decide whether to take the chance while you are young.
However 54 will sneak up on you and retiring at that age is still young, most folks just dream about that and don't have the means to do it. Maybe if you postpone your gratification now, you can still have a great life later on, and will have money and a home thats probably paid for.
Sorry, I know thats no help, but its the best I could do.