France forward Payet, 29, rejoined the Ligue 1 club on Sunday for £25m, after allegedly refusing to play for the Hammers for the past month.
As much as we didn’t want him out, you can’t go against the consensus of the team,” Sullivan told BBC Radio 5 live.
Sullivan went on to reveal that Payet sacrificed his wages for January
He’s not been talking to anyone in the squad,” said Sullivan of Payet, who joined West Ham in June 2015.
“He’s been sitting in the corner of the room for his meals and he’s isolated himself. Before that he was bubbly, happy, shaking everyone’s hand before the match.
“Either it was a tactic or something in his head had changed.
“The team wanted him out, the manager – with great reluctance – wanted him out. We think it’s bonded the team better together.”