Fifteen mortgage and banking groups expressed concern that there are efforts to derail a comprehensive overhaul of mortgage giants Fannie Mae and Freddie Mac.

The groups on Thursday wrote a letter to Treasury Secretary Steven Mnuchin and Mel Watt, director of the Federal Housing Finance Agency, calling on them to help usher through bipartisan legislation in Congress to reform the housing finance system and end government control of the two entities.

“As you both have made clear on numerous occasions, housing finance reform must go through Congress to create stable, sustainable housing markets that best serve our nation’s communities,” the groups wrote in the letter also sent to members on the Senate Banking and House Financial Services committees.

“We fully endorse this point of view, and believe the debate over recapitalizing a broken system distracts from the critical structural issues that Congress must address to ensure that the federally supported secondary market serves key, bipartisan objectives,” they wrote.

The housing and mortgage groups argue that Congress must pass legislation that makes key structural reforms before a decision is made about how Fannie Mae and Freddie Mac will retain capital.

“Allowing Fannie Mae and Freddie Mac to create a capital cushion simply converts a potential future draw on federal funds into an immediate draw and such an action would effectively increase the size of taxpayer exposure to future losses,” they wrote, noting a Congressional Budget Office report from 2016.

There is concern that there’s a push aimed at allowing the government-sponsored enterprises to recapitalize, potentially without congressional approval.

Building capital buffers could further slow already glacial efforts to pass legislation in Congress with the 10-year anniversary of the financial crisis a year away.

The capital buffer is expected to fall to zero next year, meaning that taxpayers would be on the hook for any losses by Fannie Mae and Freddie Mac.

Congress had made several attempts since the 2008 financial crisis to unwind Fannie Mae and Freddie Mac but there has been no agreement on a final plan.

The groups that sent the letter were American Bankers Association; American Land Title Association; Asian American Real Estate Association; Committee on Healthcare Financing; Consumer Bankers Association; Consumer Mortgage Coalition; Habitat for Humanity; Housing Policy Council of the Financial Services Roundtable; Mortgage Bankers Association; National Affordable Housing Management Association; National Association of Hispanic Real Estate Professionals; National Association of Home Builders; National Association of Housing Cooperatives; National Housing Conference; and Real Estate Services Providers Council.