He told BBC's This Week: "She should have been there with the residents, which is what Jeremy Corbyn was".
He said there was still "simmering anger" among residents as well as "hostility and concern" over whether it Downing Street might offer no more than "warm words".
He said that feeling of being ignored is a source of much "deep frustration", and told how people have been left feeling "that decisions are taken about their lives and their homes that they are not party to".
She has faced growing criticism after failing to meet victims during a visit of the base of the tower site following the blaze. This fire could have been stopped, a long, long, long time ago.
And writing on Twitter, broadcaster Jeremy Vine agreed that the prime minister's failure to speak to was "starting to look like a defining moment for Theresa May". Exact same way. There is basic stuff, there is not even sprinklers in there.
"Why are Sadiq Khan and Corbyn coming down here to speak to people and Theresa May is coming here with police, walking around, not meeting no one [sic], not meeting families?" a young local asked the cabinet minister.
"In 2009, the last block was on fire".
"It's disgusting. She should maybe come down and help out", Dee Baxter told the Standard. What has changed since then?
Nadir said: "I'm angry at the state of the tower".More news: Senate passes new sanctions on Iran, Russia
More news: Bookies´ favourite Jo Swinson turns down chance of Lib Dem leadership
More news: Ronaldo 'wants to leave' Real Madrid after being accused of tax fraud
"I mean, it's too late to be honest, right now".
She said she has ordered daily progress reports on housing for those affected, and vowed the public inquiry into the disaster will be "open and transparent". "Because of people saving money, people are dying".
"The Prime Minister is absolutely heartbroken".
"So the people could have some realistic understanding of what was going on".
"The first person who spoke was a survivor from the 19th floor and - not quite my caricature of the Prime Minister - but to see her welling up, the response in the meeting, and to see her holding the hand of a sobbing lady next to her in the meeting was quite shocking I think to most of us there".
"What we need to do is to get a grip of this and make sure we are meeting their immediate needs as well as their ongoing needs and that is really the priority for Government".
Government minister Tobias Ellwood suggested that security concerns had prevented Ms May from speaking to local people.