From the 80s til the late 2000s, the Mario universe kinda had to be weird and out there with original characters and settings in order to expand and build the world of Mario. But then came the 2010s where Nintendo pretty much got comfy (well "comfy" in the sense of what they want to do with Mario) and decided to play it safe and have it as sterile and serialized as possible for what you would expect in a Mario game.
I believe Mario is sort of a similar situation Sonic, rather than the series just got comfy.
Mario had a defined identity in the NES and SNES era, that had to be fully changed when making the jump to 3D. Gameplay-wise, Nintendo always intended for 3D Mario to be like 3D World, but hardware limitations made it impossible to have short form levels. You can actually see this as the games got progressively more linear going from 64 to Sunshine then to Galaxy.
Many of the oddities of the series were brought about due to refinement. F.L.U.D.D. was created solely to improve platforming as many struggled in 64, so they built a game around this mechanic. Galaxy came about to create a platformer that would warp players back to the playform if they fell + help with camera. Even the side titles like Luigi’s Mansion more came about as tech demos for hardware.
The issue with this is now the series identity has been screwed with, and you fall into the Sonic dilemma of having branching fanbases. It is not a secret that many long-time Mario fans hated Sunshine upon release for being too different. Luigi’s Mansion similarly got a mass amount of backlash. If we are speaking on Paper Mario, Super Paper Mario easily killed the side brand with the numerous complaints of how bizarre it is and how fans hated there being a story.
In terms of marketing, it was extremely hard to market the IP as there was little iconography between games. This was a huge reason as to why the New and 3D games came into being. These titles provided a very specific style and power ups that can be marketed. Everyone now knows what a Mega-Mushroom looks like.
With tons of backlash and insults hurled at Nintendo during the GC, record sales during the Wii and DS and the Gen X Mario nostalgia boom they brought, this is how we ended up getting the sterilized Mario of today.