When I first started working with Go, I thought my code was fast because the language itself is designed for performance. But soon, I hit a wall. My application was slowing down under load, and I had no idea why. That’s when I discovered profiling. Profiling is like having a detailed map of your code’s behavior. It shows you exactly where time and resources are being spent, so you can fix the slow parts instead of guessing.
In Go, profiling is built right into the standard library. You don’t need extra tools to get started. I’ll walk you through seven key techniques that have helped me optimize my Go applications. Each one focuses on a different aspect of performance, from CPU usage to memory management. I’ll include code examples that you can try yourself, and I’ll share some stories from my own projects to make it relatable.
Let’s begin with CPU profiling. This tells you which functions are using the most processor time. Imagine your code as a busy kitchen. CPU profiling shows you which cooks are working the hardest. In Go, you start by importing the pprof package and writing a bit of code to capture the data.
Here’s a simple way I set it up in one of my web services. I added an HTTP server to serve profiling data, so I could check it live while the app was running.
package main
import (
"net/http"
_ "net/http/pprof"
"os"
"runtime/pprof"
"time"
)
func main() {
// Start a separate goroutine for the pprof HTTP server
go func() {
http.ListenAndServe(":6060", nil)
}()
// Run the main application logic
processData()
}
func processData() {
// Create a file to save CPU profile data
f, err := os.Create("cpu_profile.prof")
if err != nil {
panic(err)
}
defer f.Close()
// Start CPU profiling
err = pprof.StartCPUProfile(f)
if err != nil {
panic(err)
}
defer pprof.StopCPUProfile()
// Simulate some heavy work
for i := 0; i < 1000000; i++ {
doHeavyCalculation()
}
}
func doHeavyCalculation() {
// Simulate CPU-intensive task
time.Sleep(time.Microsecond * 10)
// Allocate some memory to see effects
_ = make([]byte, 512)
}
After running this, I used the go tool pprof to analyze the CPU profile. It pointed me to functions that were hogging the CPU. In one case, I found a loop that was recalculating the same value repeatedly. By caching that value, I cut the CPU usage in half.
Memory profiling is next. It helps you understand how your application uses RAM. Memory issues can cause slowdowns or even crashes if your app uses too much. I remember a service that kept restarting because it ran out of memory. Using memory profiling, I spotted a function that was creating large slices and not releasing them.
Go makes it easy to capture memory profiles. You can do it manually or through the HTTP endpoint. Here’s how I often check memory stats in my code.
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
// Monitor memory usage periodically
go monitorMemoryUsage()
// Your application code here
simulateMemoryWorkload()
}
func monitorMemoryUsage() {
for {
var memStats runtime.MemStats
runtime.ReadMemStats(&memStats)
fmt.Printf("Allocated memory: %v MB\n", memStats.Alloc/1024/1024)
fmt.Printf("Total allocated: %v MB\n", memStats.TotalAlloc/1024/1024)
fmt.Printf("System memory: %v MB\n", memStats.Sys/1024/1024)
fmt.Printf("Number of garbage collections: %v\n", memStats.NumGC)
time.Sleep(5 * time.Second)
}
}
func simulateMemoryWorkload() {
// Create a slice that grows over time, simulating a potential leak
var data [][]byte
for i := 0; i < 1000; i++ {
chunk := make([]byte, 1024*1024) // 1 MB
data = append(data, chunk)
time.Sleep(time.Millisecond * 100)
}
}
By watching the allocated memory grow without dropping, I identified a memory leak. It turned out I was holding references to old data in a cache that never cleared. Fixing that made the app stable.
Block profiling looks at where goroutines get stuck waiting. In concurrent programs, goroutines might block on mutexes or channels, slowing everything down. I once had a service where responses were delayed because too many goroutines were waiting on a shared resource.
Enable block profiling with a simple setting, and Go will track these events.
package main
import (
"runtime"
"sync"
"time"
)
func main() {
// Set the block profile rate to record blocking events
runtime.SetBlockProfileRate(1)
// Example with a mutex that causes blocking
var mu sync.Mutex
var data int
// Start multiple goroutines that contend for the mutex
for i := 0; i < 10; i++ {
go func(id int) {
for j := 0; j < 100; j++ {
mu.Lock()
data++ // Critical section
time.Sleep(time.Millisecond) // Simulate work
mu.Unlock()
}
}(i)
}
// Let it run for a bit
time.Sleep(10 * time.Second)
}
After running this, I generated a block profile and saw high contention on the mutex. By reducing the lock scope or using a read-write mutex, I improved throughput.
Goroutine profiling shows you the state of all goroutines in your program. It’s great for finding leaks or too many goroutines. In a chat app I worked on, goroutines were piling up because they weren’t being properly closed.
You can capture a goroutine profile easily.
package main
import (
"fmt"
"runtime/pprof"
"time"
)
func main() {
// Start some goroutines that might leak
for i := 0; i < 100; i++ {
go leakyGoroutine(i)
}
// Wait and then profile goroutines
time.Sleep(2 * time.Second)
profile := pprof.Lookup("goroutine")
profile.WriteTo(os.Stdout, 1)
}
func leakyGoroutine(id int) {
// This goroutine runs forever, simulating a leak
for {
time.Sleep(time.Second)
fmt.Printf("Goroutine %d is still running\n", id)
}
}
The profile showed hundreds of goroutines that should have exited. I added proper context cancellation to stop them when no longer needed.
HTTP pprof endpoints let you profile running applications without stopping them. This is incredibly useful for production systems. I’ve used it to diagnose issues in live services without any downtime.
Just by importing net/http/pprof, you get endpoints like /debug/pprof/ for various profiles.
package main
import (
"net/http"
_ "net/http/pprof"
)
func main() {
// Your application routes
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, World!"))
})
// pprof endpoints are automatically available
http.ListenAndServe(":8080", nil)
}
Once deployed, I can curl /debug/pprof/ from another machine to get CPU, heap, or goroutine profiles. In one instance, I caught a memory spike during peak traffic and fixed it before users noticed.
Benchmark profiling helps you measure the impact of changes. Go’s testing framework has built-in benchmarks. I use them to ensure my optimizations actually help.
Here’s a benchmark test I wrote for a data processing function.
package main
import "testing"
// Function to benchmark
func processData(input []int) int {
sum := 0
for _, v := range input {
sum += v * v // Some computation
}
return sum
}
func BenchmarkProcessData(b *testing.B) {
// Prepare test data
data := make([]int, 1000)
for i := range data {
data[i] = i
}
// Reset timer to exclude setup time
b.ResetTimer()
// Run the function b.N times
for i := 0; i < b.N; i++ {
processData(data)
}
}
I run this with go test -bench=. -cpuprofile=cpu.prof to see if my changes improve speed. Once, I optimized a function and the benchmark showed a 30% improvement, which was confirmed in production.
Trace profiling gives you a timeline of your program’s execution. It shows how goroutines interact and where delays happen. I used it to optimize a service that had uneven workload distribution.
Generating a trace is straightforward.
package main
import (
"os"
"runtime/trace"
"sync"
"time"
)
func main() {
// Create a trace file
f, err := os.Create("trace.out")
if err != nil {
panic(err)
}
defer f.Close()
// Start tracing
err = trace.Start(f)
if err != nil {
panic(err)
}
defer trace.Stop()
// Simulate work with goroutines
var wg sync.WaitGroup
for i := 0; i < 5; i++ {
wg.Add(1)
go worker(i, &wg)
}
wg.Wait()
}
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(time.Millisecond * time.Duration(id*100))
// Some work
}
After running, I opened the trace in Go’s trace viewer. It revealed that some goroutines finished much later than others. By balancing the work, I made the whole process faster.
These seven techniques have become part of my daily routine. I start with CPU and memory profiling to find obvious issues, then move to more specific ones like block or trace profiling. The key is to profile under realistic conditions. I always test with data that mimics real usage, not just simple examples.
Profiling might seem daunting at first, but it’s a powerful way to make your Go applications efficient and reliable. I encourage you to try these methods in your projects. Start small, perhaps with CPU profiling on a test service, and gradually incorporate others. You’ll be surprised how much you can improve performance with a data-driven approach.
