If the Crew were not blabbing about what they learned then not messing with them is a good way to keep them not blabbing, if you start sending God's Knights after them and riling them up then that increases the chance of them blabbing.
Like we know Roger's crew BTFO the God's Knights, so that means the Elder Stars need to start showing up, which means a real big disaster of a fight which you can't really do secretly, this means that if you go after Rayleigh and that means the rest are going to be incentivized to gather together which means you need multiple elder stars or Imu himself AND they are now very incentivized to just start saying everything they know as a "fuck you"