请教一个 go tool ppof 的 heap inuse_space 和 Linux 系统 top 命令显示的内存占用差距过大的问题

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

已注册用户请登录

这是一个创建于 1631 天前的主题，其中的信息可能已经有所发展或是发生改变。

现在一个 go 程序存在内存泄露的问题，于是用 go tool ppof 查看了 heap 的 inuse_space 信息

Build ID: 22891c5544961cf88d62b13741ef587dc95d0dcb
Type: inuse_space
Time: Nov 4, 2019 at 8:31pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1.78GB, 96.47% of 1.85GB total
Dropped 84 nodes (cum <= 0.01GB)
Showing top 10 nodes out of 47
      flat  flat%   sum%        cum   cum%
    1.59GB 85.87% 85.87%     1.59GB 85.87%  github.com/derekparker/trie.(*Node).NewChild
    0.03GB  1.64% 87.51%     0.04GB  1.91%  gopkg.in/olivere/elastic%2ev5.(*BulkIndexRequest).Source
    0.03GB  1.43% 88.94%     0.03GB  1.43%  github.com/segmentio/kafka-go.readNewBytes
    0.03GB  1.40% 90.34%     0.03GB  1.40%  bufio.(*Scanner).Text
    0.02GB  1.35% 91.69%     0.02GB  1.35%  main.reverse
    0.02GB  1.19% 92.88%     0.02GB  1.19%  github.com/segmentio/kafka-go.newWriter
    0.02GB  1.11% 93.99%     0.02GB  1.11%  bytes.(*Buffer).String
    0.02GB  0.91% 94.90%     0.02GB  0.94%  github.com/segmentio/kafka-go/snappy.(*xerialReader).readChunk
    0.02GB  0.87% 95.78%     1.65GB 89.49%  main.(*A).Load
    0.01GB  0.69% 96.47%     0.01GB  0.69%  bytes.makeSlice
(pprof)

也看了相应的 alloc_space 信息

Build ID: 22891c5544961cf88d62b13741ef587dc95d0dcb
Type: alloc_space
Time: Nov 4, 2019 at 8:31pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 77.60GB, 73.03% of 106.27GB total
Dropped 259 nodes (cum <= 0.53GB)
Showing top 10 nodes out of 90
      flat  flat%   sum%        cum   cum%
   23.02GB 21.66% 21.66%    23.02GB 21.66%  bytes.makeSlice
    9.13GB  8.59% 30.25%     9.13GB  8.59%  github.com/segmentio/kafka-go.readNewBytes
    7.42GB  6.99% 37.23%    24.70GB 23.24%  gopkg.in/olivere/elastic%2ev5.(*BulkIndexRequest).Source
    6.15GB  5.79% 43.02%     6.15GB  5.79%  github.com/gogo/protobuf/proto.unmarshalBytesValue
    6.08GB  5.72% 48.75%     6.30GB  5.93%  github.com/mailru/easyjson/buffer.(*Buffer).BuildBytes
    6.05GB  5.69% 54.44%    17.28GB 16.26%  encoding/json.Marshal
    5.81GB  5.46% 59.90%     5.81GB  5.46%  bytes.(*Buffer).String
    5.74GB  5.40% 65.30%    11.47GB 10.80%  gopkg.in/olivere/elastic%2ev5.(*BulkService).bodyAsString
    4.54GB  4.27% 69.57%     4.54GB  4.27%  github.com/gogo/protobuf/proto.unmarshalStringPtr
    3.67GB  3.45% 73.03%     3.67GB  3.45%  reflect.New
(pprof)

可以看到 inuse_space 一共 1.85G 左右，而实际用 top 看进程使用的内存一直是在缓慢增加的(系统一共 64G 内存，最后可以吃满), 我想知道怎么去看这些增加的内存被用到哪去了，应该怎么排查这个问题。

cum

nodes

top

2 条回复 • 2019-11-06 10:35:55 +08:00

reus

2019-11-05 21:52:27 +08:00

既然 go 堆没有增长，那肯定是 C 堆了，看看 cgo 哪里忘记释放内存了。

能不用 cgo，就没必要用 cgo，编译慢，难调试难优化。

piapia123

2019-11-06 10:35:55 +08:00

@reus 多谢大佬，的确是用了 cgo