The charity also faces a challenge to hang on to major corporate partners, and the Duke of Edinburgh's Award has confirmed it is reviewing its association with the organisation.
Cafod director Chris Bain said the charity had been unaware of the allegations until it was contacted recently by The Times newspaper.
It's a betrayal of all who put their faith in them: those who most needed their help.
They have also let down the vast majority of individuals working in humanitarian relief who do so with "great passion and integrity", OSCR said.
I still believe that such programmes are necessary.
Mordaunt has confirmed that a new unit has been set up within the department to review safeguarding across all parts of the aid sector, including United Kingdom and worldwide charities, suppliers, the UN and other multilateral organisations. Oxfam must do everything in its power to heal the damage to those who depend on both its work and the good faith and generosity of its supporters.
In a speech to an aid conference in Stockholm, Ms Mordaunt tore into Oxfam over its response to the revelations about aid workers in Haiti in 2011.
Firth added: What they were doing struck me as hard, but vitally important work.
Oxfam has neither confirmed nor denied that specific account but has said an internal investigation in 2011 had confirmed sexual misconduct had occurred and has apologized. "But I never gave into these advances", he said.
The charity claims there was no attempt at a cover up, but that it is in the midst of investigating these allegations after its deputy chief executive Penny Lawrence resigned.
She admitted she could not guarantee there were not sex offenders now working at the organisation but vowed Oxfam would "build a new culture that doesn't tolerate that behaviour".
Hollywood star Minnie Driver has become the first celebrity to quit as an Oxfam ambassador following allegations that senior staff working in crisis zones paid for sex with vulnerable locals.
"Sexual abuse and exploitation is an issue the entire development sector needs to confront", Ms. Mordaunt said, calling for a culture that "ensures victims and whistleblowers can come forward without fear".
Last week, The Times reported, citing its sources, that a number of Oxfam workers, including van Hauwermeiren, had been spending time in their official Haiti residence with prostitutes.
"The hardest thing is that my family has rejected me", said Van Hauwermeiren, who was previously head of Oxfam's mission in Chad.