Morality exists outside of religion and outside of government.

Religion often took on the mantle of morality in part because it was the only coherent social structure at the time. Up to a point, that was not a bad thing, rules about violence, theft, rape, etc were necessary somehow. In a time of no birth control and no medical treatment, even rules about abstinence and marriage served to reduce disease and unsupported children.*

Handing morality to the government is a case of new boss, same as the old boss. In the 'secular' west, the governments have become obsessed with the details of our lives, and whether our attitudes align with the current socially prescribed narrative. Religion enforced morality with threats of damnation, government does so with threats of violence.

Logic is not fully an answer either. Sam Harris looks to logic, and certainly it can inform our judgement but complexities of human relations simply do not fit with logic. The various trolley car scenarious show that our minds do not fully buy into the Vulcan ideal (could you intentionally cause the death of one innocent person to save a car-load of others? Is that moral?). How much should freedom be abridged for the 'overall good' (and how do you measure the 'overal good'? -- maybe loss of that much freedom is an 'overall bad'. At what point does abortion become infanticide? At what point does it acquire the legal rights guaranteed to every person (regardless of age or health)?

In a way, morality evolved like the spam filter in your email. You get a spam, and tweak your rules to block future similar messages. After a while you have lots of 'rules' but there is no logical consistency to them, indeed some will directly contradict one another. And some will delete legitimate messages. Evolution does not 'care' about those messy details. If it works on a statistically significant level, it will be selected.

*This is somewhat a problem today. Statistically, corrected for age, background, income and education, married people with children do far better financially than those who have children on their own or with casual partners. This is a major problem in some of our cities, where social workers are cautiously non-judgemental--they discourage teen pregnancy, which is not the biggest issue, and are silent on everything else. Yet the children of intact parents are far less likely to get into drugs, violence and crime.