Beat the pudding mix and milk until combined and starting to thicken, about 2 minutes. Put in the refrigerator until ready to use.
Slice the bananas into ½-inch slices.
Add ¼ cup of the melted butter, ¼ cup of the brown sugar, and ¼ teaspoon of salt to a large skillet and cook on medium heat until melted and bubbling, about 3 minutes.
Add half of the banana slices in a single layer and cook without moving until the undersides are golden brown, about 3 minutes.
Flip and cook the banana slices for 2 more minutes.
Transfer the bananas to a large plate and repeat the cooking process with the remaining melted butter, brown sugar, salt, and banana slices.
Fold 1 container of the whipped topping into the pudding mixture.
Spread about 1 cup of the pudding mixture over the bottom of a 9x13-inch baking dish.
Arrange a single layer of graham crackers over the pudding, breaking sheets as needed to fill gaps.
Spread half of the remaining pudding mixture over the graham crackers.
Top the pudding with a single layer of cooled banana slices.
Repeat with another layer, adding graham crackers, then pudding, then more bananas.
Spread the remaining container of whipped topping over the cake.
Drizzle the caramel sauce or syrup over the whipped topping
Cover loosely, without letting the cover touch the topping, and refrigerate for at least 6 hours or overnight.
Slice and serve the cake, optionally garnishing with fresh banana slices, crushed graham cracker crumbs, and more caramel sauce.