After taking out Crocodile, they became interested since he was still a fresh pirate. I don't remember if they fully knew he was a rubber man yet, but it's implied the fruit had been eaten by others in the past before, but no one was able to get stronger with it and even come close to awakening it, so they probably figured he'd burn out. But he just kept going and going, and of course declared war on the World Government rescuing Robin and even beating Cipher Pol, then broke into Impel Down and bust back out, then fought the admirals at Marineford, and they couldn't ignore him any longer. They had to keep an eye on him.
And then he went up against Big Mom and lived (and Morgans documented it). He was now a danger because he could go head-on with a Yonko, and he was heading for Kaido. They were definitely sweating by then because it was only a matter of time.
But mainly, he's a "D", they knew this once they got word of him. He was already destined for greatness and being their enemy, but they didn't know the extent of it at the time. Hindsight is 20/20: if anyone could awaken the Nika fruit, it was going to be a "D". And far as we know, no other "D" after Joy Boy (if he's a "D", too) ate the fruit previously.
Also as shown by Saturn, they're kinda dumb at times. They likely underestimated him because they can't or didn't think things through in the long-run while juggling other things on the side (such as Blackbeard and choosing to dissolve the Warlord system).