A co-founder of Cream in 1966, Baker also played with Blind Faith, Hawkwind and Fela Kuti, the BBC reported.
"Dad passed away peacefully," his daughter Nettie Baker said in a statement to CNN. "He was in no pain and had recently been able to see and speak to his children, close family and special friends."
"We are very sad to say that Ginger has passed away peacefully in hospital this morning. Thank you to everyone for your kind words over the past weeks," the drummer's Facebook confirmed Sunday.
In 2016, Baker had open-heart surgery and had to cancel a tour with the band Air Force, Rolling Stone reported.
Peter Edward Baker, nicknamed "Ginger" because of his flaming red hair, formed Cream with guitarist Eric Clapton and bassist Jack Bruce. The trio combined 1960s blues and psychedelia in memorable songs like "Sunshine of Your Love," "White Room," "Strange Brew," "Badge," and "I Feel Free."
Cream sold more than 35 million albums, and the group was awarded the world's first platinum album for their LP "Wheels of Fire," the BBC reported.
Baker and Bruce had played previously in the Graham Bond Organization but had a contentious breakup, the BBC reported. But Clapton insisted on having Baker and Bruce together in Cream, and the two agreed to co-exist in the supergroup.
"Ginger was pretty dismissive and anti-social, seriously anti-social but he had the gift, he had the spark, the flair, the panache ... he had it in spades," Clapton said in the 2012 documentary, "Beware of Mr. Baker."
Baker, known for his cantankerous personality, rarely lasted more than a few years with any group, Rolling Stone reported. However, Baker's ability to draw upon his jazz training and combine it with a powerful drumming style made him a rock 'n' roll icon.
Hawkwind tweeted its condolences Sunday on Twitter.
"Fly high Ginger! You were a one-off and a true legend...We were honored to work with you...RIP,"