The United States jails more prisoners than any nation on earth – about 2.3 million, or more than 1 percent of all American adults. Our gigantic penal system is regularly characterized as a national disgrace. I’ve applied the label myself.

Plainly there is something deeply disquieting about a democratic superpower locking up so many people that 25 percent of the world’s reported prisoners are housed in US cells. How can a country with an incarceration rate of 716 inmates per 100,000 residents, roughly five times the global average, think of itself as “The Land of the Free?”

Yet whether America’s vast prison population really represents such a scandalous failure depends on what prison is supposed to do. In that light, consider a trove of data released last month by the Bureau of Justice Statistics, an agency of the US Department of Justice.

In the first major federal study of recidivism since 1994, BJS statisticians tracked nearly 405,000 inmates in 30 states who were released from prison in 2005. Within six months, 28 percent of those freed prisoners had been arrested for a new crime. After three years, 68 percent had been arrested. By the end of the five years (the period covered by the study), the percentage had grown to a whopping 77 percent.

The report breaks down these new crimes by category. Five years after regaining their freedom, 29 percent of the prisoners had been arrested for a violent offense, 38 percent for a property crime, 39 percent for a drug offense, and 58 percent for public-order offenses. (Many released inmates were arrested on multiple charges.) Only 23 states could provide researchers with complete data on inmates who returned to prison; but among the released prisoners in those states, more than half – 55 percent – ended up behind bars once more.

The Justice Department’s earlier recidivism study, though organized and presented differently, came to similar findings. It found that 67 percent of former inmates released from prisons in just 15 states had been rearrested for at least one serious new crime within three years. Those included, the bureau noted, “2,900 new homicides, 2,400 new kidnappings, 2,400 rapes, 3,200 other sexual assaults, 21,200 robberies, 54,600 assaults, and nearly 13,900 other violent crimes.”

In April 2011, meanwhile, the Pew Center on the States issued its own report on recidivism. Its conclusion: “More than four out of 10 adult American offenders … return to prison within three years of their release.”

Such recidivism rates are terrible. It is heartbreaking and alarming that so many criminal offenders emerge from prison only too ready to offend again. Too many inmates come out hardened and more antisocial than they went in. For decades, scholars, policy makers, social workers, and public-safety experts have searched for the holy grail of rehabilitation and effective sentencing that would give us a more humane corrections network – one less congested, less expensive, less unfair, and less of a revolving door for the addicted and the unstable.

The problem with holy grails is that they are more easily sought than found. Bill Keller of The New York Times recently described a range of seemingly promising strategies for addressing what has become “the hottest subject in criminal justice,” the US system of mass incarceration. Among them: Easing mandatory-minimum sentences and three-strikes laws. Diverting nonviolent drug offenders to specialized courts focused on treatment. Counseling for inmates about to be paroled. Repeal of rules that bar felons from getting many kinds of occupational licences.

But would they work? Prison reform and rehabilitation programs have been earnestly advanced for decades, but that holy grail remains elusive – and recidivism remains sky-high. American sociologist Robert Martinson made waves 40 years ago with an influential essay that concluded: “With few and isolated exceptions, the rehabilitative efforts that have been reported so far have had no appreciable effect on recidivism.” Decades later, reformers are still trying to figure out what works.

Prison is awful, there’s no question about it. It doesn’t turn criminals into model citizens. It can’t be expected to cure dysfunction whose roots go back to a broken home or a lousy school.

But one thing we know prison can do: It can isolate criminals from society, and thereby make society safer. In the 1980s we began locking up more convicts for longer terms. Now we have the largest prison population on earth – and crime rates at 30-year lows. When it comes to crime and punishment, there’s always a trade-off. At least until we find that holy grail.

(Jeff Jacoby is a columnist for The Boston Globe).