Disclosure: I run a DevOps consultancy (MeteorOps). Nothing for sale here; my co-founder wrote the long version, link in a comment.
We set CPU limits for every container, but if there's a latency-sensitive service we avoid setting any limits. When I say "we" it's actually most of the team, but some opt for setting limits on everything.
The obvious follow-up on the no-limits school is that we give the developers dashboards to be aware of the apps' resource utilization, and over time it improves, but it's more of a "culture" fix.
For the CPU-limits-setting engineers in the crowd, what do you do to prevent over-utilization crashes?