I don’t have children so not a personal issue but I think my answer, generally, is that it depends.
Depends on the children, depends on the parents, depends on the restaurant.
There are child brats and there are parent brats. I’d prefer them not to show up at my local bistro, let alone an upscale place. But most kids and most parents are well behaved in restaurants - even when kids are just being kids. It’s hard to expect a young child to sit happily and quietly in a restaurant for an hour or so - their instinct is to run around and be noisy. And there are loads of places, many of them"nice", where that’s fine and the sort of people who would complain are not my sort of people.
As regards high end dining, it may not be the best call to take your toddler to the restaurant. But, unless the restaurant sets an age limit (and I know a few which do), then it it is your call. By way of example, we celebrated my 65th birthday at the Michelin 2* Manoir aux Quat’ Saisons. It was lovely, by the way. But, also in the dining room, was a couple with their child, aged perhaps 7. They seemed to be also having a lovely time - you could see the three chatting together, enjoying the food and getting on with life. They had chosen to eat fairly early, possibly so the child wouldnt be kept up late (their meal was coming towards the end when we arrived at 8pm. They were there at breakfast and still seemed to be having a lovely time - as were the staff who seemed very attentive towards the child.