People often do believe that they are choosing good for themselves at the expense of society. What they are unaware of is that they are harming themselves. Sometimes the harm is obvious and direct, and other times the harm is subtle and insidious. But it is only because of ignorance that they do this. If they could see the truth, they would understand that they are harming themselves.
It should be noted here that people who live vice-filled lives of evil are unhappy - including those who may have riches or fame and appear happy to others. They may not know why they are unhappy. Furthermore, some people's entire lives are this way and it is all they know, so they may not even realize they could be happier. They just toss it up as "that's how life is". So, it is not necessary for them to realize it in order for them to be worse off because of their vice. In this way, the evil are naive of the good life completely. They imagine good people to be dupes who are as miserable as themselves, only more so because they lack their material wealth or perceived freedom. When good people tell them they aren't happy they laugh and wonder what foolishness the dupe is talking about.
Potentially innocent people may be fooled into following evil people because they can often be naive and ignorant enough to think that the evil person really is happy. I know of a woman who doesn't take care of her children, has no responsibility, is deep into drugs, lives with various low-lifes, in and out of prison, and is basically alienated from her family. She is quite obviously miserable because every time you see her, you see she is upset or ashamed or fighting. This cannot be a happy life.
Yet, her young teen daughter, when told that her mother is miserable said, "What do you mean? Her life seems happy?" Because her mother was always "doing whatever she wanted to do" the poor girl actually thought that her mother was having an enjoyable life. She hasn't learned yet what constitutes a good life and doesn't know the misery her mother lives with.
It is true that virtue is what's good for society. I only add that, it is therefore good for the individual as well.
It works the same in reverse. Some rulers have thought that they could do what was good for society while harming the individuals that make up that society. But this always comes back to haunt them in the end.
A society is made up of individuals, and you cannot impose something on a system that is opposed to the nature of its components. It is the nature of human beings to want protected rights and fair treatment. When a social system is imposed on individuals that is abhorrent to them, they will not function well in society and the society suffers. It is like thinking that we can have something which harms individual cells without it harming us.
This is why oppressive societies eventually crumble into violent rebellion, or simply stagnate due to the lost efficiencies inherent in an oppressed people.
A society is a system, and there is no such thing as a prosperous society made up of unprosperous individuals.
Addendum / Illustration:
As an illustration of the concept, imagine a needle gauge measuring the degree to which a society is more supportive of the individual (when the needle is far right) and the degree to which a society is more supportive of the collective (when the needle is far left).
Now imagine the needle balanced in the middle. If the needle begins to veer left, we imagine the society passing laws for the 'sake of the collective', which necessarily infringe upon individual rights to some degree. As the needle moves further and further left, the people become more and more oppressed, disgruntled, unproductive, and unsupportive of the society. Thus, the further left the needle moves, the greater the harm to the society because it's constituent components (individuals) are not optimal.
In the opposite situation, with the needle moving right, we see individuals are given greater and greater liberties such that they are allowed to pollute the environment, scam people, and eventually even assault others. Thus, the individual is harmed the further right the needle moves.
On the whole, we see that, if the needle moves left the society and the individual are harmed. If the needle moves right the individual and society are harmed. There is no way to harm one without harming the other. This, makes perfect sense when we understand that 'society' is just a term for a 'bunch of individuals', and when we understand the nature of inter connectedness.