You'll know when you get too old to ride, but only you will know that. I get a lot of the "mid-life crisis" and "grow up" comments too at 50 years old. But the fact is I feel more like myself when I'm flying through the air 3 feet above the ground than I do sitting in an office being a "grownup". I'm not in great shape but I can sit on a Busa for 9-10 hour days without any problem, so why stop something I love so much? The only concession I have made for age is that I don't ride too long without a 10-15 minute rest (usually for gas), I keep hydrated, and I don't let myself get too hot or tired.
I many ways the 45-65+ years are the best time to be on a bike. The kids are old, you have more money to buy good equipment, insurance is reasonable, you can take time off from work, and most of all you have control of the testosterone so you don't act like an idiot.