In a large bowl, cream butter and sugars together.
Add dry pudding mix and mix well.
Beat in eggs and vanilla.
Add baking soda and flour and mix until well incorporated.
Grab enough dough to make a 1" ball and flatten it out in the palm of your hand.
Place one 0.5" caramel cube and place it in the center of the flattened dough. Fold up the edges of the dough around caramel and roll into a ball, making sure the caramel is sealed in the center of the dough ball. Place on greased cookie sheet.
Repeat with remaining dough. 9-12 cookie dough balls per cookie sheet.
Bake at 350 degrees F for 8-10 minutes. Allow cookies to cool on pan for about 5 minutes, then transfer to a wired cooling rack.
Once cookies have completely cooled, frost with Cool Whip frosting.
Sprinkle the tops of the frosted cookies with Heath bits.