The fire broke out in a disused warehouse building near the site of the main Olympic stadium Waterden Road, Hackney Wick, near Stratford yesterday. Around 75 firefighters were sent in to tackle the blaze which sent a huge plume of smoke above London.
The minister, who is in Beijing on an Olympic fact-finding mission, said it was too early to say what effect the blaze would have on work on the site.
Ms Jowell said: "At the moment there is nothing to suggest this is anything sinister such as arson rather than an accident. The Olympic Delivery Authority will conduct a full inquiry and they are working very closely with the fire brigade on necessary aspects of public safety.
"There was asbestos in the building and that fact is being managed by the fire brigade in terms of advice to the local community."
The fire broke out at a one story warehouse and no Olympic workers were there at the time.
The Olympic Delivery Authority said: "The ODA can confirm that a fire in east London is within the Olympic Park site.
"The fire is on the western edge of the Olympic Park on the site of an empty warehouse, behind the bus depot on Waterden Road. The warehouse is in the process of being demolished. The London Fire Brigade is on site and the ODA's site staff are working closely
The North London rail line was closed between Dalston and Stratford because of the smoke affecting Hackney Wick station - full service resumed this morning.