Daniel Walsh, 31, faces several charges in the incident, including first-degree murder and attempted first-degree murder.
Sullivan County Sheriff Jeff Cassidy said Walsh fired a pistol through a bedroom door seven times to fatally strike Joshua Moyers, 33.
A pair of UPS workers reported the incident at 1:18 p.m. Near Vermont United Methodist Church, they were approached by a frantic woman, Michelle Lacy, who exclaimed that her boyfriend had been shot.
When officers converged on the scene of the crime, 1901 Latimer St., they found Walsh still on the porch. He was taken into custody without incident and made statements to detectives.
Citing an ongoing investigation, Cassidy declined to divulge many of Walsh's statements. He did say that Walsh admitted to shooting the pistol "until it wouldn't fire any more rounds."
Walsh was reportedly at the home to retrieve property. When he found Moyers and Lacy together, according to police, an altercation ensued and Moyers hit Walsh in the head with a hammer.
As Moyers and Lacy took refuge in a bedroom, Walsh allegedly retrieved a pistol that he knew was in a dresser drawer. He then opened fire on the bedroom door while aware that Moyers and Lacy were on the other side.
Cassidy said that Walsh had made threats prior to Wednesday's incident. A friend of Walsh transported him to the home, thinking Walsh just needed to get some of his belongings. That individual was interviewed and released.
Meanwhile, Lacy was treated by EMS for unspecified medical issues, then transported to the hospital by ambulance.
Lacy had reportedly lived at the home for at least two years. Investigators are still sorting through timelines of the romantic relationships: how long Lacy and Walsh dated, when they separated, and when Moyers came into the picture.
Along with murder and attempted murder, Walsh is charged with use of a firearm in the commission of a felony and possession of a firearm by a convicted felon. The latter is related to a previous burglary conviction.
Walsh was arraigned Thursday morning, with his bond set at $250,000. A next court appearance was not immediately known.