In the ongoing debate over how accountable care organizations can lower costs and improve health, two new studies offer insights into the nuances of how ACOs affect spending, utilization and quality.

Authors of both studies emphasized the importance of time in assessing the effectiveness of ACOs. But this resource is in short supply as President Donald Trump and his new health secretary, Tom Price, look to dismantle the Affordable Care Act, which launched numerous initiatives to pay for healthcare on the basis of value, not volume.

Both of the ACOs studies were published Monday in JAMA Internal Medicine. One looked at post-acute care spending in the Medicare Shared Savings Program, finding that those ACOs did appear to save Medicare dollars by reducing utilization in the post-acute care space without sacrificing quality. The other study compared Medicaid ACOs in Colorado and Oregon, finding that the two states performed similarly on certain metrics, despite their divergent approaches to the ACO model.

“We still have much to learn,” wrote Carrie Colla and Dr. Elliott Fisher in an accompanying editorial in JAMA Internal Medicine. “Accountable care organizations have been established across diverse market settings, using a multitude of organizational structures and approaches to governance and operations, and this heterogeneity is reflected in the heterogeneity of their performance,” they added, and the variation in these programs’ results defy generalization.

“We know little about the effects of ACOs on patients’ health and quality of life,” Colla and Fisher continued. “We know little about the key ACO capabilities that are important to ensuring their success in different organizational or market contexts.”

Dr. J. Michael McWilliams, a professor of healthcare policy at Harvard Medical School in Boston and the lead author of the study on Medicare ACOs, examined the effects of those organizations on post-acute care, considered one of the areas with the most excessive unnecessary spending. Spending on post-acute care has more than doubled since 2001, in large part because of increased reliance on skilled-nursing facilities.

McWilliams and his co-authors found that participating in Medicare’s Shared Savings Program was associated with reduced spending in the post-acute space, without appearing to hurt the quality of care.

The 114 ACOs that joined the Medicare Shared Savings Program in 2012 saved $106 per beneficiary in spending on post-acute care facilities in 2014 compared to a control group and $77 per beneficiary on inpatient spending, the researchers found. The ACOs achieved these savings by reducing the proportion of patients discharged to acute care facilities, discharging patients home rather than to post-acute care facilities, and shortening stays in skilled nursing facilities.

For ACOs that joined the program in 2013, participation was associated with reductions in SNF spending of $27 per beneficiary in 2014. But for those that entered in 2014, participation in the program was not linked to significant changes in such spending.

McWilliams attributed the results of the 2014 entrants to the fact that it takes a while for savings to kick in. The results of the study suggested that more time and patience is needed to judge the effectiveness of ACOs, he added.

“The savings appear to be growing over time,” he said. “Many people have written off ACOs as not working, but over time, the savings have materialized.”

One reason ACOs appeared successful in lowering post-acute spending was because they had significant financial incentives to do so, McWilliams said. Most ACOs do not provide post-acute care, and so reducing utilization in that area does not affect their fee-for-service revenue.

“For a lot of services, the incentives are not that strong for ACOs to lower services,” McWilliams said. “Studying post-acute care gives us insight into how well this model might work when incentives are strong.”

Quality, as gauged by mortality, 30-day readmissions and utilization of four or five-star SNFs, did not decrease as these organizations lowered costs.

A major limitation of the study was the fact that the Shared Savings Program, along with other Medicare ACO programs, is voluntary. If the ACOs that choose to join are self-selecting, likely on the basis that they expect to succeed, then the results cannot be extrapolated to all ACOs.

“We can’t generalize what we’re seeing here to future waves of ACOs or other areas of the delivery system,” said McWilliams.

The other study examined the results of Medicaid ACOs in Oregon and Colorado, which launched in 2012 and 2011, respectively. It analyzed data from July 1, 2010 to Dec. 31, 2014, for 452,371 Oregonians and 330,511 Coloradans for changes in outcomes and spending. Researchers chose those states because of data availability and because they were two of the earliest states to launch Mediciad ACOs.

The researchers found that the models performed similarly, even though they took contrasting approaches to building ACOs.

Oregon’s ACO aggressive model covered some 90% of Oregon’s Medicaid population, and providers took on two-sided risk under a global budget. It received about $1.92 billion in federal funding from the CMS.

Colorado took an opposite approach. It had no downside or upside risk for providers, and it kept fee-for-service payments in place. Instead, it used collaborative organizations to coordinate care. The state invested about $155 million in the initiative.

Although Oregon improved in some areas of care compared to Colorado, including measures of access and avoidable visits to the emergency department, it didn’t fare any better in others. Colorado did as well in reducing inpatient care days, for instance. Both programs saw a decrease in primary care visits — a finding the researchers called a potential “cause for concern if [the reductions] reflect restricted access.”

Both states reduced spending over the study period, the researchers noted, although that also coincided with a national slowdown in health spending.

“One lesson might be: Doing something seems to be a good start,” said K. John McConnell, the lead author of the study and a health economist at Oregon Health & Science University. “It may be that just doing something for your population” — like trying to coordinate care better — “is generally beneficial,” he said. “There’s a lot of low-hanging fruit there.”

Oregon’s ambitious and expensive ACO model “did not generate savings that might be anticipated,” the researchers wrote in their study. One reason could be that its model simply needed more time before their changes would show greater reductions in spending.

At the same time, they noted, Colorado’s “focus on manageable, incremental steps has been followed by growth in enrollment, reductions in utilization, and improvement in some key performance indicators,” all without the heavy investments that Oregon received. “Colorado’s approach may represent a promising delivery system reform that may be more feasilbe for other states to adopt,” they added.

Medicaid, a joint federal-state program, covers more than 20% of the U.S. population. As a result, it’s a major budgetary challenge that is driving states, like Medicare, to find ways tocurb spending. As of 2016, nine states have launched Medicaid ACOs, and eight more are working on it.

Future studies aim to compare spending and outcomes of Medicaid beneficiaries in states with Medicaid ACOs and those without, McConnell said.