Apisai Koroisau gave them a 2-0 half-time lead with his 14th minute penalty and the Kiwis already looked to be in trouble.
They levelled the scores with a Shaun Johnson penalty early in the second half but Fiji refused to let them get a grip.
It was a shock result but a deserved one in what was only the second try-less match in World Cup history.
New Zealand, who lost star forward Jason Taumalolo to Tonga and were without Jesse Bromwich and Kevin Proctor for disciplinary reasons, made a bright start to the tournament but Tonga knocked the stuffing out of the 2008 champions and Fiji finished them off.
In the 10 games under Kidwell since he took over September a year ago, the Kiwis have won three, drawn one and lost six.
Thomas Leuluai has been dropped by David Kidwell
As it was, the two points from hooker Koroisau's successful 15th minute effort were all that divided the teams.
"We slayed a giant tonight", he said.
He said: "It's indescribable".
"I am so proud of these boys".
"I didn't get any sleep last night and really thought about what I want to do, what I want to achieve". I never stopped believing.
"I don't think (there was any more we could have done)". We laid the foundations in '08 and we have seen the fruits in 2017. "It's as simple as that.".More news: Moody's ups ratings of SBI, HDFC Bank, EXIM India and IRFC
More news: Chris Coleman resigns as Wales manager and set to join Sunderland
More news: West Brom manger Tony Pulis defends his record as pressure mounts