Seems to me that “elitism” is often abused as a term in a kind of inverse snobbery, and reflects the cultural pursuits of a different social class (I guess “upper” or “middle”).
However, the true meaning is surely where people are actively excluded from partaking in an activity - either by financial constraints or lacking contacts/backgrounds/relationships (eg. some clubs).
Ignoring the case of those insufficiently wealthy to do anything requiring money, it’s not true that pursuing an interest in fine wine is actually inaccessible to anyone who has the motivation.
There are many tasting events that are cheap or free - and those who show a keen interest as “students of wine” (even informally) don’t need to be big buyers.
I think that many people are shy at straying from self-imposed cultural class-based norms. Not all, though, and I always find it refreshing to meet people from other backgrounds at events usually regarded as the province of a small section of society.
Not sure how much I care that not everyone is interested in wine. It can be a curse as much as a blessing!