Troubleshooting high CPU issues

  • We are looking for tips on troubleshooting high CPU usage on a pair of 5440s. We reach 95% CPU (as indicated by “show cpu overall” in the shared partition). We do run quite a few AFLEX scripts, but even after simplifying them we’re still running into 90%+ CPU regularly when doing around 95K requests/sec (HTTP ±55K, TCP±40K). Note that these are not new TCP/HTTPS connections, but requests over existing connections.
  • Total system throughput is under 10Gbps.
    • No gzip compression
    • A few HTTP templates doing header insert
    • TLS offload being done by the A10s

The question is - how do we determine what’s using all the CPU so we know what to target for optimization?

Update in case someone else comes across this thread.
Be sure to “no axdebug” after doing packet captures - even if you stop the capture with CTRL+C, it appears that whatever causes the extra CPU usage during a capture remains after it is stopped. I suspect that most hardware offload is disabled when axdebug is active.
After running “no axdebug” in all partitions, CPU usage dropped well over 50%.