The zoo announced the death on Wednesday night, saying the calf — whose name, Amara, had been set to be revealed shortly — died suddenly on Friday afternoon.
"The 'little watermelon on legs' as she was affectionately known, was born in late November and in a relatively short time captured the hearts of guests and keepers alike," the zoo said, on social media.
"She was found unresponsive on Christmas Eve afternoon and was sadly confirmed to have passed away.
"Preliminary findings from Taronga's veterinary team indicate a potential problem with her heart."
Veterinarians are still investigating and keeping a close eye on the calf's parents, Kambri and Fergus.
The zoo said there were "no indications of trauma or misadventure."
"She was a much-loved member of the Taronga family and her passing is understandably heartbreaking for all those who knew and cared for her," it said.
The pygmy hippo, Kambri's second, had made her public debut just 20 days ago, looking tiny splashing around in the water with her mum.
"Our mother and calf duo are doing really really well," Taronga Zoo senior ungulate keeper Renae Moss said at the time.
"Kambri is an excellent mother. She's extremely maternal and very protective and our calf is getting more confident every day, getting more energetic every day.
Baby animals celebrate their first-ever Christmas
"At the beginning she was sleeping most of the day — sleeping and feeding like any newborn baby but now she's exploring her surroundings and today she's really having a good look at her exhibit."
Ms Moss said there were fewer than 3000 pygmy hippos left in the wild, making zoo births "extremely valuable".