The jail, which will operate in addition to the current county jail, had been scheduled to officially open Tuesday.
Anderson said it will be at least two weeks before the facility opens.
"We will open it soon," Anderson said. "We had some adjustments to make. We want to make it as safe and secure as possible, for the inmates and for our officers."
One redo that's causing the delay: Doors to cells didn't have any "pie holes" - narrow slots through which jail personnel can pass items without opening the door.
The facility has eight large dormitory-style cells, and each has two doors. So 16 doors are being re-outfitted with the slots, Anderson said.
Built largely to ease overcrowding at the current jail, the new 33,000-square-foot jail was patterned after a workhouse in Greene County.
Anderson said he doesn't fault anyone for the items that need changing because the facility was built exactly as its plans called for.
He said the problems that are being corrected were noticed before an incident at the current jail over the past weekend, but that incident made jail officials take an even closer look at the new facility.
On Sunday, inmates at the current jail became unruly after jail staff began a search for missing keys, took away inmates' privileges, and released pepper spray.
Anderson said the inmates have since calmed down and have been behaving well, but the delay in opening the new facility will also allow a "cool down" period before inmates are transferred from one jail to the other.
"We want to make sure this work is completed first," Anderson said. "We want to make sure everything is right. We want to get in that building, but I want to make sure it's safe."
The Greene County workhouse that served as a model for the new jail is strictly a minimum-security facility. Sullivan County plans to house both minimum- and medium-security inmates at its new jail.
It will operate in addition to the current county jail at an estimated cost of about $1.2 million per year - money that will have to be included in the county's budget year beginning July 1.
Anderson hired 25 new employees to work at the new jail, and they are all already on the job, working at the current jail until the new facility opens.