The team confirmed the hiring in a release Thursday night, saying Arians received a four-year contract with a club option for a fifth year.
The 60-year-old longtime assistant went 9-3 as Colts interim head coach while Chuck Pagano was undergoing treatment for leukemia last season.
Arians arrived in Arizona on Wednesday night, had dinner with top team officials, then interviewed on Thursday and met with reporters to indicate his interest in the job.
He was offered and accepted the job Thursday night. Arians also was a finalist for the Chicago job that went to Marc Trestman.
Arians was the sixth known candidate interviewed to replace Ken Whisenhunt, who was fired after six seasons and was introduced on Thursday as the new offensive coordinator for San Diego.
The Cardinals said Arians would be introduced on Friday afternoon.
The team posted a photo of Arians getting a congratulatory call from Cardinals wide receiver Larry Fitzgerald, who is vacationing in Belize and undoubtedly will welcome anyone who can do something to revive a dreadful offense.
In his meeting with reporters earlier Thursday, before he was hired, Arians said his stint as Pagano's replacement "answered all questions I ever had" about whether he could succeed as an NFL coach.
"I hope it answered all the questions everybody else has had for all these years," he said.
It apparently did for the Cardinals, who lost 10 of 11 games this season to finish 5-11 for the second time in three years.
Arians said there are young coaches for teams still in the playoffs that will be denied a chance to be a head coach now because of their teams' success.
"Maybe I was a victim of that a couple of times," he said. "Hey, I'll take the Super Bowl ring and look back later. You can't worry about why you never got one or why have you got one this time. I'm just happy it happened, and I look forward to seeing what's made of it and what turns out."
Arians has been a coach since his days as a graduate assistant at Virginia Tech, where he played quarterback, in 1975 and he has 20 years of experience as an NFL assistant. He also was head coach at Temple for five seasons.
Wearing a Super Bowl ring from his days in Pittsburgh prominently on his right hand, he said his stint as head coach in Indy taught him that being a head coach is "not as hard as it's supposed to be."
"It's really not," Arians said. "I think it's all about building relationships. Coaching is all about relationships. As long as it's built on trust, loyalty, and respect, anything is possible."
Arians was wide receivers coach in Pittsburgh and he succeeded Whisenhunt as offensive coordinator there. He was the Steelers' offensive coordinator when Pittsburgh beat Whisenhunt's Cardinals in the 2009 Super Bowl. Arians left the Steelers to become Pagano's offensive coordinator last year, taking over as interim coach when Pagano left for treatment for leukemia.
"It has been an unbelievable 12 months for me personally," Arians said.
McCoy was among those interviewed by Cardinals President Michael Bidwill. Other known Arizona candidates were Cardinals defensive coordinator Ray Horton, Seattle offensive coordinator Darrell Bevell, Steelers offensive coordinator Todd Haley and Cincinnati offensive coordinator Jay Gruden.
Horton has a year left on his contract and his future with the organization was in question.
Asked Thursday afternoon if he would keep Horton as defensive coordinator if he got the Arizona job, Arians declined to discuss the prospect.
"Right now it's way too early in the process," Arians said. "Obviously, I know Ray. I've got a history with Ray. But all those things would be way down the road. Guys are under contract. You can't really comment on staff members at this point in time."
Horton was defensive backs coach with the Steelers when Arians was offensive coordinator there.
The Cardinals had the worst offense in the NFL, and Arians downplayed any similarities to his offensive scheme and the one used by Whisenhunt. He said that after Whisenhunt left the Steelers for Arizona, he and Pittsburgh quarterback Ben Roethlisberger sat down and simplified the playbook.
Arians said his offense has some similarities, but many differences from the one implemented by Whisenhunt.
Arians emphasized the importance of building a close relationship with the quarterback.
"As a head coach or an offensive coordinator, you're tied to that guy," Arians said. "Your future is him. You better realize that, and you better have a heck of a strong relationship so that when it's really tough in the game, he's telling you what he's seeing, not what he thinks you might want to hear."
Quarterback is a major problem for Arizona. The only one of the four who got a start in the Cardinals' 5-11 season that had any success was Kevin Kolb, and he went down with a rib injury in Week 6 and never returned.
Arians said the Cardinals must either improve what they have at the position or find a new one behind what he called "Door No. 2."
He said he would call the plays as head coach "until I can find someone who is going to do it better than me. I haven't found him yet."
He said he would not leave Indianapolis for just any head coaching job.
"I said early when some teams contacted my agent about setting up interviews," he said, "I'm going to have a heck of a feeling about that organization, owner, general manager, team, to leave where I am right now."
Did he have that feeling with Arizona?
"Yes," he said. "I do."