Sir Idris Elba is not just stepping away from the long-running Idris Elba James Bond conversation. He is also questioning whether audiences worldwide would accept a Black African man in the role at all.

Elba has been in the Bond rumor mill for years, especially after Daniel Craig confirmed he was leaving the franchise. On paper, the idea was easy to understand: Elba had already led Luther, carried action roles with ease, and looked entirely at home in a sharp suit. Hollywood has built entire casting rumors on less.

But in a new interview with British GQ, Elba said the speculation was never as serious as some fans hoped.