Like most Americans, I got my taxes filed before this week’s Tax Day deadline. And like most Americans, I got money back from the IRS. You probably did too. After all, nearly two-thirds of tax returns result in a refund.

That’s not a good thing.

Many Americans have come to regard their tax refund as a plum to look forward to each spring, a windfall that compensates to some degree for the hassle of getting their tax returns completed and filed. In reality, the refunds sent to them by the Internal Revenue Service are nothing more than the belated repayment of interest-free loans made to the government in the form of excessive taxes withheld from their paychecks. They’re getting back money they never owed in the first place. That doesn’t represent a windfall, it represents a loss — the loss of everything they could have done with their money if they’d had access to it, and the loss of purchasing power eroded by inflation. It shouldn’t make anyone happy to overpay their taxes.

This wouldn’t be an issue if Americans were free to pay their taxes the way they pay for electricity, tuition, child care, groceries, or anything else — namely, by drawing money from their bank account and remitting the amount due to their creditors. But when it comes to paying the bill for government, that isn’t an option. The vast majority of American workers are never paid the full compensation they’re entitled to. Instead their employers are required by law to withhold some of their earnings and send it directly to the IRS.

Most of us are so used to this way of doing things that we hardly notice what a rip-off it is. But Chris Rock noticed. “You don’t even pay taxes — they take taxes,” the celebrated comedian fumed during his “Bigger & Blacker” standup routine. “You get your check, money gone. That ain’t a payment, that’s a jack.”

Tax withholding dates from 1943. As the economic historian Robert Higgs has written, it was an emergency wartime measure designed to collect the taxes needed to defeat the Axis.

On the eve of World War II, when only the highest-earning Americans were subject to the federal income tax, fewer than 4 million individual returns were filed. Those taxpayers would write a check to the US Treasury to cover what they owed. “Beginning in 1940, however, the tax burden increased enormously,” Higgs recounts. Federal spending soared as the government mobilized for a global war. Taxes were dramatically expanded as well, so much so that by 1945, the number of individuals who had to file tax returns had reached 50 million — a 12-fold increase in just six years.

To ensure the compliance of so many Americans who had never before had to pay federal income taxes, the government devised a technique for making those taxes almost invisible and all but painless: Employers were directed to withhold their workers’ taxes “at the source,” by skimming off the funds before workers got paid. From Washington’s perspective, the new system was ideal: Employees stopped thinking about the dollars being extracted from their pay each week because they never saw those dollars. Withheld taxes were relegated to the fine print at the bottom of their pay stubs, and Americans got used to thinking of their take-home pay as their “real” income. When the tax deadline came around each April, most people owed nothing more to the IRS. Many even received a refund of overpaid taxes, which turned Tax Day into something almost pleasant.

If, to quote an old aphorism, taxation is the art of plucking the goose without making it squeal, then the Treasury Department policymakers who implemented tax withholding were virtuosi. Their brainstorm not only anesthetized Americans to how much they were paying to support the government, but also to how expensive the government was becoming. The Treasury itself boasts about what withholding accomplished. It not only “greatly eased the collection of the tax,” declares an official history of the US tax system, it “greatly reduced the taxpayer’s awareness of the amount of tax being collected … which made it easier to raise taxes in the future.”

Ironically, one of the Treasury Department economists who crafted the withholding tax was a young Milton Friedman. Tasked with supporting the war effort, as he later wrote in his memoir Two Lucky People, he never paused to consider the long-term consequences. “It never occurred to me at the time that I was helping to develop machinery that would make possible a government that I would come to criticize severely as too large, too intrusive, too destructive of freedom.”

Had withholding been repealed after 1945 and Americans gone back to paying their taxes directly, political pressure to shrink the government and its war-swollen budget would have been irresistible. Alas, withholding remained. As a result, the federal government has grown into a massive and ruinously expensive behemoth, unloved by Americans but paid for by them anyway, with dollars they work for but never get to see.