Trump said on Twitter that Pompeo was "in the air" and was with "the 3 wonderful gentlemen that everyone is looking so forward to meeting." The president, who had been hinting about an imminent release, said he would greet them at Andrews Air Force Base at 2 a.m. Thursday.
The release of the detainees came as Pompeo visited North Korea on Wednesday to finalize plans for a historic summit between Trump and the North's leader, Kim Jong Un. Trump said on Twitter that there had been a "good meeting with Kim Jong Un," adding: "Date & Place set."
North Korea had accused Kim Dong Chul, Kim Hak Song and Tony Kim, all Korean-Americans, of anti-state activities but their arrests were widely seen as politically motivated and had compounded the dire state of relations over the isolated nation's nuclear weapons.
They had been held for periods ranging between one and two years. They were the latest in a series of Americans who have been detained by North Korea in recent years for seemingly small offenses and typically freed when senior U.S. officials or statesmen personally visited to bail them out.
Pompeo, in his visit, discussed the agenda for a potential Trump-Kim summit in a meeting with Kim Yong Chol, vice chairman of the central committee of North Korea's ruling party.
Later, Pompeo said the senior official had been a great partner in working to make the summit a success.
"For decades, we have been adversaries. Now we are hopeful that we can work together to resolve this conflict," Pompeo said, adding that "there are many challenges along the way."
Kim noted the improved relations between the Koreas, as well as the North's policy to "concentrate all efforts into economic progress" in the country.
"This is not a result of sanctions that have been imposed from outside," he added, citing the will of the Korean people. That contradicted Trump, who has said repeatedly that his pressure tactics brought North Korea to the negotiating table.
Pompeo's trip, his second to North Korea this year, had not been publicly disclosed when he flew out of Washington late Monday aboard an Air Force jetliner. Trump announced the mission Tuesday afternoon as he laid out his case for withdrawing from a landmark nuclear deal with Iran, another bitter U.S. adversary.
Minutes later, Pompeo arrived in Japan to refuel before flying on to Pyongyang. Accompanying him were a handful of senior aides, a security detail and two journalists — one from The Associated Press and one from The Washington Post, both given roughly four hours' notice of his departure.
When the flight arrived Wednesday morning in Pyongyang, North Korean officials were on hand to greet Pompeo. A motorcade took Pompeo and his delegation to the Koryo Hotel, the main hotel for foreigners in Pyongyang.
The trip came just days after North Korea expressed displeasure with Washington for comments suggesting that massive U.S. pressure had pushed Kim Jong Un to the negotiating table.
Pompeo, who first traveled to North Korea as CIA chief in early April, is only the second sitting secretary of state to visit the reclusive nation with which the U.S. is still technically at war. The first was Madeleine Albright, who went in 2000 as part of an unsuccessful bid to arrange a meeting between then-President Bill Clinton and Kim Jong Un's father, Kim Jong Il.
"Plans are being made, relationships are building, hopefully a deal will happen and with the help of China, South Korea and Japan a future of great prosperity and security can be achieved for everyone," Trump said at the White House as he announced Pompeo's visit on Wednesday.
Pompeo's first trip to Pyongyang, over Easter weekend before he was confirmed as secretary of state, was a closely held secret. News of it did not emerge until just before his Senate confirmation vote less than two weeks ago. Shortly afterward, the White House released photographs of Pompeo and Kim.
Pompeo told reporters that his first visit was to test the North's seriousness of pledges to South Korea on easing tensions. This trip was "to put in place a framework for a successful summit," he said.
The Trump-Kim meeting is expected later this month or in early June.
"I think it would be a great gesture if they would agree to do so," Pompeo said, adding that it would be difficult to hold a leaders' summit if the prisoners remained captive.
Pompeo's visit to Pyongyang aimed to lock down the date and venue for a formal announcement. It came just a day after Kim returned from China, his second trip to the neighboring country in six weeks to meet with President Xi Jinping.
A Trump-Kim meeting seemed a remote possibility just a few months ago when the two leaders were trading threats and insults over North Korea's development of nuclear weapons and ballistic missile tests. But momentum for diplomacy built this year as North and South Korea have moved to ease tensions, including with their own leaders' summit late last month.
In March, Trump unexpectedly accepted an offer of talks from Kim after the North Korean dictator agreed to suspend nuclear and missile tests and discuss "denuclearization." According to South Korea, Kim says he's willing to give up his nukes if the United States commits to a formal end to the Korean War and pledges not to attack the North.
Kim was quoted by China's official news agency Xinhua as saying on Monday, "I hope to build mutual trust with the U.S. through dialogue." He added that a political resolution of tensions on the Korean Peninsula and denuclearization should proceed in stages, with all sides moving in concert.
But his exact demands for relinquishing weapons that his nation spent decades building remain unclear. Previous U.S. efforts to negotiate an end to the North's nuclear weapons program failed under Presidents Clinton and George W. Bush.
Pompeo and officials traveling with him said the Trump administration would not repeat mistakes of the past, which they described as accepting gradual, incremental and long-term disarmament in exchange for immediate benefits.
"We will not relieve sanctions until such time as we have achieved our objectives," Pompeo said. "We're not going to do this in small increments where the world is essentially coerced into relieving economic pressure."
Trump has said that withdrawing U.S. forces from South Korea is "not on the table." Some 28,500 U.S. forces are based in the allied nation, a military presence that has been preserved to deter North Korea since the war ended in 1953 without a peace treaty.
After weeks of positive signals, though, North Korea on Sunday dismissed what it called "misleading" claims that Trump's policy of maximum political pressure and sanctions are what drove the North to talks. A North Korean foreign ministry spokesman warned the claims are a "dangerous attempt" to ruin a budding detente on the Korean Peninsula after the summit between Kim and South Korean President Moon Jae-in.
Associated Press writers Jill Colvin and Ken Thomas in Washington contributed to this report.