MLB Legend Reggie Jackson Recalls Racism He Faced Early in His Career: 'I Wouldn't Wish It on Anyone'

Reggie Jackson is reflecting on the racism he endured when he was a young baseball player.

While joining MLB on Fox’s lineup during the San Francisco Giants-St. Louis Cardinals broadcast on Thursday night, the Hall of Famer shared his perspective on returning to the site of the game, Rickwood Field, where he once played in the minor leagues.

"Coming back here is not easy,” Jackson, 78, told Alex Rodriguez, after the analyst asked him how emotional it was for him to be there. “The racism when I played here, the difficulty of going through different places where we traveled. Fortunately I had a manager and I had players on the team that helped me get through it. But I wouldn't wish it on anybody.”

Jackson added, “People said to me, today I spoke and I said, 'Do you think you're a better person, do you think you won when you played here and conquered?' I said, you know, I would never want to do it again. I walked into restaurants and they would point at me and say, 'the n----- can't eat here.' I would go to a hotel and they say the n----- can't stay here. We went to Charlie Finley's country club for a welcome home dinner and they pointed me out with the N word, ‘He can’t come in here.' Finley marched the whole team out, finally they let me in there. He said, 'We're going to go to the diner and eat hamburgers, we'll go where we're wanted.' "

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories. 

The former Yankees star then offered more stories — as Rodriguez, 48, seemingly waved to a producer off-camera so that the retired athlete could continue his comments — about the time.

"Fortunately I had a manager in Johnny McNamara that if I couldn't eat in the place nobody could eat, we'd get food to travel," Jackson said. "If I couldn't stay in the hotel they'd drive to the next hotel and find a place where I could stay."

He concluded, "I wouldn't wish it on anyone. At the same time, had it not been for my white friends, had it not been for a white manager... I would have never made it. I was too physically violent, I was ready to physically fight someone. I'd have got killed here because I'd have beat someone's ass, and you'd have saw me in an oak tree somewhere."

Rodriguez and the show’s host Kevin Burkhardt and the other analysts, retired baseball greats David Ortiz and Derek Jeter, then thanked Jackson for his comments.

“We love you, Reg,” Rodriguez said and gave him a hug.