At one point, when asked if he would fire Price, the President said, "We'll see".
The HHS inspector general's office began a review last week to determine if Price complied with federal travel regulations, which generally require officials to minimize costs.
The letter reminded Price and other recipients that official travel for all federal employees must be "by the most expeditious means of transportation practicable", and be in line with an employee's function.
A department official said both trips were paid for by HHS and that Price took both on official government business. "I'm not happy about it, and I let him know it".
"On September 19th, Politico reported that you had chartered a private jet on several short official trips, including a $25,000 private flight from Washington, D.C.to Philadelphia and back - a trip that would at most cost $725 round-trip on a commercial airline", the letter reads.
Treasury Secretary Steven Mnuchin - a former Goldman Sachs banker - is also under investigation by his agency's inspector general for potentially improper use of a United States government jet to view the solar eclipse with his wife last month.
Days later, Mnuchin flew on a government jet from New York City to Washington following a news conference with Trump at Trump Tower.
Price's use of private flights stands in contrast to his immediate predecessors, Kathleen Sebelius and Sylvia Mathews Burwell, who would typically fly commercial.
Politico threw another grenade at Price over his jet travel last night although it was lost in the hubbub over the Alabama Senate race. "Wasting four hours in an airport and having the secretary cancel his event is not a good use of taxpayer money", Yoest said.
Republican Sen. Ron Johnson, chairman Homeland Security Committee said he wanted to "wait for the information" from the HHS inspector general before commenting. In theory private jets should be chartered by government officials only if they need to be somewhere immediately, for important reasons, and there are no commercial routes that will get them there in time.