Maurice Shults was demoted by a unanimous vote on Thursday at a meeting of the Newport City Council, according to City Attorney Terry Hurst.
Mayor Connie Ball placed Shults on administrative leave last month because of allegations of work place harassment and displaying a bad temper.
Shults said on Friday he believes the remarks about his temper have been exaggerated and plans to appeal to the Civil Service Board.
A former detective and narcotics investigator, Shults was appointed in 2004 as chief after the death of longtime chief Clay Webb.
Newport is located about 50 miles east of Knoxville.
Following Shults' placement on administrative leave, The University of Tennessee's Municipal Technical Advisory Service investigated the complaints, interviewing over 30 people.
The agency's report said Shults is known for having a bad temper and that his "outbursts are legendary."
"Most employees state they avoid going to the station because 'you never know what is going to send him off.'"
The reported noted, however, that the city did not follow its own charter and personnel policy when investigating the allegations against Shults.
Ball had solicited two complaints against the chief and held one of them for more than two months before it was disclosed to anyone, according to the report.
"It is improper for a supervisor to intimidate an employee," the report said. "It is just as improper for an elected official to coerce an employee to file a negative statement on a supervisor."
Hurst said Assistant Chief Don Ball would remain acting chief while the Civil Service Board interviews candidates for the position.