Here's how I see this whole "cultural appropriation" matter now, as far as I can tell:
When elements of a culture itself are copied or utilized under circumstances not deliberately meant to be harmful or offensive, it shouldn't be automatically harmful per se or to most.
Sure, there may be some side effects, as people against cultural appropriation point out - like promoting misconceptions about a culture. But there are circumstances where those side effects shouldn't be harmful. It's not inherently wrong to have misconceptions and make mistakes sometimes.
Also, like it has been brought up before - an individual is not automatically responsible for the actions of others in the same group. I believe that to say otherwise should be discriminatory and may even promote or maintain grudges, regardless of the gender, race, &/or culture involved. And I believe that most people usually don't succumb to such discriminatory thinking.
Finally, if other cultures couldn't copy elements from each other, then cultural growth would be greatly reduced or even stagnant. Cultures grow and develop by copying elements from each other - in context or not - throughout history. Not only that, and as others have said, claiming that certain people can't copy elements of a culture may very well support discrimination - and a pigeonholed view of cultures. In reality, culture itself can be fluid and changing.
We're all people - and copying, in and of itself, is not automatically theft.
My sincere apologies if I'm bringing up the matter of cultural appropriation and other stuff on the farms too much.