The vote from NFL owners this week on whether Tom Brady should be a part-owner of the Raiders was unanimous: 32–0 yes.

Public opinion over whether he can and should continue as a Fox broadcaster now is far from unanimous. Brady signed a 10-year, $375 million contract with Fox, and just six games in, 71% of readers in a Front Office Sports newsletter survey think he won’t still be in the job three seasons from now. 

As a formally approved part-owner, Brady faces the following restrictions to his Fox broadcasting job:

• He can’t criticize teams or referees.
• He can’t enter team facilities.
• He can’t attend practices.
• He can’t join pregame production meetings with teams or players—in person or virtually.
• He’s subject to the league’s gambling and tampering policies.

I have seen other commentators point to the “no criticizing the refs” rule as the biggest problem, but Brady—who has been rather vanilla in his commentary so far, on brand with his press conferences when he was a player—shouldn’t have a hard time curbing his criticism.

Not being able to criticize the teams is a bigger problem. When Deshaun Watson throws his third pick of the game and Brady can’t say boo about how massively the Browns screwed up by giving him, at the time, the most guaranteed money in NFL history, that is a major blocker to Brady doing his job.

And it seems to me missing out on those meetings with players and coaches is the biggest blocker. Those meetings clearly matter and yield fruit—we constantly hear Buck and Aikman, for example, referencing nuggets they got from their conversations in the days before a matchup. (The Cowboys told ESPN this week that they do want Brady in their production meetings; but league policy trumps teams, so the Cowboys cannot get an exemption from the Brady Broadcast Rules.)

Of course, defenders retort, his people (Kevin Burkhardt, colleagues, Fox producers, other insiders) will feed him the intel anyway, so what’s the difference? Oh, just extra work for all the people around him to cater to his complication, and the lack of context that comes from learning things firsthand. 

Another retort from those who think this is all fine (cue the dog in a bowler hat sitting in a burning house): Brady has been operating under these restrictions already all season (since the Raiders deal was pending), and he’s been able to do the job anyway. 

O.K. Has he done it well? Your opinion probably determines whether you think the restrictions are a problem. In my view, the the color commentator having a $200 million financial interest in one of the league’s 32 franchises is a very obvious problem, and it’s not nuanced or complicated. 

At the very least, surely he won’t call Raiders games, right? Wrong: He’ll still call Raiders games, the NFL confirmed Thursday. 

To be clear, I say all this as an unapologetic Tom Brady fan. I grew up in New England, and my high school and college years coincided with the Brady-led Pats dynasty; I’ll be grateful to the guy for the rest of my life. But he’s never been a particularly charismatic personality off the field. Being the greatest or one of the greatest to ever play your position does not itself mean you will be great on television. (See: Emmitt Smith, Drew Brees, Jason Witten.)

My take: The restrictions will be his out. Maybe not this season, maybe not next season, but at some point, long before his 10-year deal runs out, Brady is going to retire early as a broadcaster; he can say he enjoyed the experience and wants to focus on his role as an NFL owner and his other business interests.

Duke basketball legend turned 20-year ESPN analyst Jay Williams’s take: Tom Brady isn’t about to give up so soon. Speaking to me onstage at our Tuned In sports media summit in September (after Brady had called just two Fox games and was not getting great reviews), Williams said, “I love that Tom Brady got flak [for his debut]. Good for Tom Brady. The one thing Tom Brady needs more than anything is ammunition. … I want Tom Brady ready to blossom and be that guy that I know he is once we have a couple of pops.”

For the Brady we see on Fox to become the Brady that friends and teammates have long said comes out in private will require Brady getting more comfortable and relaxed. It’s hard to imagine all these restrictions, and the added scrutiny that will come with them (raising the already very high level of scrutiny) will help get him there.