Billy, in my experience, San Gorgonio is always tough. I usually start to really feel the altitude around 10,000 feet. My pace slows down, and I rest more frequently. But I don’t get altitude sickness.
Part of the challenge is acclimating to the higher elevation. If you live near sea level and hike San Gorgonio as a day hike, your body isn’t fully acclimated. For Mount Whitney, you want to give yourself one night at Whitney Portal or Horeshoe Meadows to acclimate before your hike.
Also, prehydrate and drink water alternating with electrolytes on a schedule. In cold air, I don’t always feel thirsty, but I force myself to drink at least once hourly.
Do some reading on the signs of altitude sickness.
And remember 5hat even with acclimatization you will slow down as the oxygen thins out.
Good luck, and let us know how it goes!