一. exemplar是什么
exemplar译为“样本”、“范例”。
exemplar最早被用在Google的StackDriver中，前面成为了 OpenMetrics 规范的一部分，即: 能够为metrics额定减少属性。
典型利用是为metrics增加trace信息，这样metrics和tracing就能够关联起来。
OpenMetrics introduces the ability for scrape targets to add exemplars to certain metrics.
Exemplars are references to data outside of the MetricSet. A common use case are IDs of program traces.
指标对象通过 /metrics 接口裸露metrics 和 exemplar信息，prometheus在 pull 时，会一起拉取并保留。
prometehus反对对exemplar的采集和存储，启动时须要额定减少参数：
--enable-feature=exemplar-storageprometheus中exemplar对象的定义：
- 跟一般的metrics相似，由t/v、labels组成；
// Exemplar is easier to use, user-facing representation of *dto.Exemplar.type Exemplar struct { Value float64 Labels Labels // Optional. // Default value (time.Time{}) indicates its empty, which should be // understood as time.Now() time at the moment of creation of metric. Timestamp time.Time}二. client侧暴漏metrics和exemplar
对于metrics，个别应用prometheus/client_go减少本人的指标；
prometheus/client_go同样反对裸露本人的exemplar，故咱们用prometheus/client_go编写client。
以application中最常见的http_request_duration_seconds指标为例：
- 定义Histogram类型变量：requestDurations；
应用requestDurations.ObserveWithExemplar()更新变量的值：
- value = time.Since(now).Seconds()
- lables = { "dummyID": rand.Int(100000) }
requestDurations := prometheus.NewHistogram(prometheus.HistogramOpts{ Name: "http_request_duration_seconds", Help: "A histogram of the HTTP request durations in seconds.", Buckets: prometheus.ExponentialBuckets(0.1, 1.5, 5),})go func() { for { // Record fictional latency. now := time.Now() requestDurations.(prometheus.ExemplarObserver).ObserveWithExemplar( time.Since(now).Seconds(), // value prometheus.Labels{"dummyID": fmt.Sprint(rand.Intn(100000))}, // labels ) time.Sleep(600 * time.Millisecond) }}()http_handler中减少EnableOpenMetrics：true参数：
http.Handle( "/metrics", promhttp.HandlerFor( registry, promhttp.HandlerOpts{ EnableOpenMetrics: true, }),)启动client，通过curl采集原始数据：
# curl -H "Accept: application/openmetrics-text" http://localhost:8080/metrics# HELP http_request_duration_seconds A histogram of the HTTP request durations in seconds.# TYPE http_request_duration_seconds histogramhttp_request_duration_seconds_bucket{le="0.1"} 5006 # {dummyID="80815"} 4.097e-06 1.6735926303881307e+09http_request_duration_seconds_bucket{le="0.15000000000000002"} 5006http_request_duration_seconds_bucket{le="0.22500000000000003"} 5006http_request_duration_seconds_bucket{le="0.3375"} 5006http_request_duration_seconds_bucket{le="0.5062500000000001"} 5006http_request_duration_seconds_bucket{le="+Inf"} 5006http_request_duration_seconds_sum 0.02248467300000002http_request_duration_seconds_count 5006能够看到，exemplar的信息与一般的metrics用#距离，对于：
# {dummyID="80815"} 4.097e-06 1.6735926303881307e+09其中：
- {dummyID="80815"} 为labels;
- 4.097e-06 为value，即code中的time.Since(now).Seconds()；
- 1.6735926303881307e+09 为timestamp；
三. prometheus拉取client的数据
应用prometheus采集client的metrics，而后就能够在prometheus UI上看到exemplar的信息
也能够将prometheus作为数据源，导入grafana查看：
参考：
1.https://vbehar.medium.com/usi...
2.client-demo: https://github.com/prometheus...
3.prometheus exemplar: https://prometheus.io/docs/pr...
4.OpenMetrics Specification: https://github.com/OpenObserv...