I mean really I would think a good VPN would help you out, but not with everything. When it boils down to it, I don't blame Richard Stallman for not wanting to own even a "dumb phone." He's been trying to warn us. I'm sure people are trying to find ways to jam signals and cloak themselves, to the degree of how much it will be effective is up to one's interpretation, and the competency of the companies, the governments, and the hackers involved.
You know, the funny thing about this is that dumb phones are probably more vulnerable to the most hypothetically dangerous location tracking situations that we think about.
This dumb summary data is based off Google location tracking. You can and should turn this off for your own safety- not only is it used for advertising, but if a police department asks for information about who was in the vicinity of a crime at a certain time then you could be fingered by Google if you happen to be nearby.
Now, you do need to worry about some other more basic forms of tracking. The most obvious is the fact that if the thing's turned on, it's talking to cell towers, and that locates you at every moment you walk around like that to within a few hundred yards for forever. That happens whether it's an Nokia 3310 in the hands of Sirajuddin Haqqani or a flash iPhone. As long as it's turned on, you're also vulnerable to exploits of the baseband (the radio modem) chip and the SIM card in the phone (where applicable) *, and if local LE is targeting you, they don't even need to do that, they can get your carrier to activate GPS tracking to an even higher degree than the triangulation. There's no real way to avoid this sadly.
Where it gets more interesting is if your phone is turned off. We've all seen TV shows where cell phones that are turned off are turned on remotely, but how actually plausible is this? For this to work, the baseband chip would have to actually be running and communicating with base stations when the main phone is powered off, because if it isn't running and talking then it can't get those instructions. I wouldn't be surprised if that's happening with older phones, before whacky ideas like 'airplane mode' and where the whole phone can stay powered on for a month and the baseband chip is presumably only consuming half of that at worst. But I've read expert commentary that suggests this
probably isn't happening with recent smartphones, but noone really seems to know.
That said, if you are targeted by a nation state actor or a large spy consultancy, they may well be able to remotely install malware to the baseband chip that stops it from actually turning off and either provides location tracking by staying connected to the cellular networks, or through other more direct means. But.. generally this isn't going to happen to everyone in a population, because that would become obvious, only to targeted individuals. Hence why you should try to keep yourself off those lists.
NOTE: There is a level of tracking 'above' the Google level where many ad SDKs that pay app developers to be inserted into their apps (especially things like weather apps (as they have location permissions), children's games, etc) which can also collect various forms of identifying location and personally identifying information on you if you aren't careful, but I'm not aware of cases where these have been proven to be accessed by police. That might change if data aggregators like the one international criminal Peter Thiel owns start collecting data from these en masse. It might already be happening and we just don't know.
* Some of these attacks require that a local 'stinger' fake base station is used to trick your phone, as used by the American police against organized crime and Mossad against Americans in Washington D.C., though others may not. In some cases, a 'stinger' station may be able to remotely update baseband chip and potentially other firmware.