Good thing Brickleberry doesn't pronounce the name wrong like Family Guy.
Cool Whip is a brand of imitation whipped cream hardly containing any milk product other than caseinate, so it's not real whipped cream. However, the reason why it sells a lot better in the United States than any brand of whipped cream is it has a more solid, frigid taste allowing it to taste partially like ice cream.
In "Old Wounds", Woody and Malloy were forced to spend a few nights at Firecracker Jim's place, and Jim demanded that they "get the Cool Whip" because "shit's about to get weird between him and Buella". At the end, Jim screamed at Woody "YOU LEFT THE TOP OFF THE COOL WHIP, ASSHOLE!!". Angry over this, Jim got out a grenade and pulled the pin out in an attempt to kill Woody, which failed after he used Steve as cover.