A Congressional panel heard testimony Thursday in a decades-long effort to return seized land to mainly African-American families on the Georgia coast.
The US military took the land in McIntosh County during World War Two.
The former fishing village of Harris Neck is now a wildlife refuge.
Its former residents have been arguing since the 1970's that their land was seized illegally.
Savannah minister and former Harris Neck resident Robert Thorpe was among those who spoke to the Subcommittee on Fisheries, Wildlife, Oceans and Insular Affairs.
The Rev. Thorpe said that his family was forced to leave with little notice.
"We were pushed out, not asked out," Thorpe said. "We were pushed out. It's time for justice."
A government report concluded that African-American landowners were compensated less for their lands than white landowners.
Harris Neck supporters and refuge supporters, however, disagree over whether the difference was based on race.
Dave Kelly of the Harris Neck Land Trust told the subcommittee that families now want to build low-impact homes to return to the land while keeping public access and wildlife conservation efforts.
US Fish and Wildlife Service Southeast Regional Administrator Cynthia Dohner also testified.
Her agency oversees the 2,762 acre wildlife refuge.
Dohner testified that she offered the residents an annual "Heritage Day" at the refuge but that she doesn't have the authority do more.
"The service is charged by statute under the National Wildlife Refuge Administration Act with managing these lands for the benefit and enjoyment of the public now and in the future," Dohner said. "Under our legal mandate, the service does not have the administrative authority to dispose of the refuge, given its significant ecological value."
Dohner and a representative of a non-profit group, Friends of the Savannah Coastal Refuges, stressed the land's value to wildlife and recreation.
Savannah Republican Jack Kingston told the panel that he was "disturbed" by the US Fish and Wildlife testimony.
"No one would argue the beauty of Harris Neck," Kingston said. "No one would argue the ecological importance. That's not relevant right now. What we're talking about is, was the process followed?"
Kingston said he asked for the hearing to get answers to decades-old questions.