aggr_space_physical_used vs. aggr_total_physical_used

[rmilkowskiGSA]

I can't find a description of aggr_space_physical_used nor how it is diffrent from aggr_total_physical_used.
The former is not listed in https://github.com/NetApp/harvest/blob/main/cmd/tools/generate/counter.yaml

[cgrinds]

hi @rmilkowskiGSA the counter.yaml file is used to supplement the list of ONTAP's defaault metrics. We use counter.yaml to list metrics that Harvest generates or metrics that are missing ONTAP documentation.

The better resource to check is our public documentation:

• https://netapp.github.io/harvest/nightly/ontap-metrics/#aggr_space_physical_used
• https://netapp.github.io/harvest/nightly/ontap-metrics/#aggr_total_physical_used

Both metrics are listed there with their accompanying ONTAP endpoint, counter, and Harvest template(s).

In case you aren't aware of it, the bin/harvest grafana metrics will show you what metrics are used in each dashboard.
The following grep shows that the two metrics you're interested in are used in the following dashboards:

• cmode/aggregate.json
• cmode/cluster.json
• cmode/datacenter.json
• cmode-details/volumeDeepDive.json

bin/harvest grafana metrics | grep -E 'cmode|aggr_space_physical_used|aggr_total_physical_used'

cmode/aggregate.json
- aggr_space_physical_used
- aggr_space_physical_used_percent
- aggr_total_physical_used
cmode/cdot.json
cmode/changelogMonitor.json
cmode/cluster.json
- aggr_space_physical_used
- aggr_total_physical_used
cmode/data_protection_snapshot.json
cmode/datacenter.json
- aggr_space_physical_used
- aggr_total_physical_used
cmode/disk.json
cmode/external_service_op.json
cmode/flexcache.json
cmode/flexgroup.json
cmode/fsa.json
cmode/headroom.json
cmode/health.json
cmode/lun.json
cmode/mcc_cluster.json
cmode/metadata.json
cmode/namespace.json
cmode/network.json
cmode/nfs4storePool.json
cmode/nfsTroubleshooting.json
cmode/nfs_clients.json
cmode/node.json
cmode/power.json
cmode/qtree.json
cmode/quotaReport.json
cmode/s3ObjectStorage.json
cmode/security.json
cmode/shelf.json
cmode/smb.json
cmode/snapmirror.json
cmode/svm.json
cmode/volume.json
cmode/vscan.json
cmode/workload.json
cmode-details/volumeBySVM.json
cmode-details/volumeDeepDive.json
- aggr_space_physical_used

[rmilkowskiGSA]

The links you posted describe the metrics as: aggr_space_physical_used Total physical used size of an aggregate in bytes. aggr_total_physical_used Total Physical Used Still not clear what’s the difference between the two. However, the aggr_total_physical_used has the following metric listed: space.efficiency.logical_used, space.efficiency.savings Which suggests that despite its name it might not be reporting physical space used?

[cgrinds]

Hi @rmilkowskiGSA

Are you seeing a discrepancy that you want explained or are you trying to understand what each metric is collecting? If it's the former, can you let us know what version of ONTAP you're monitoring and whether you are using Zapi or Rest?

Here's what I gathered on these two metrics

aggr_space_physical_used

REST: api/storage/aggregates space.block_storage.physical_used
ZAPI: aggr-get-iter aggr-space-attributes physical-used
REST docs: Total physical used size of an aggregate in bytes.
ZAPI docs: This field represents physical used space in bytes. This differs from 'total-used' space by the space that is guaranteed for future writes. The value includes blocks in use by Snapshot copies.
ZAPI to REST mapping: https://docs.netapp.com/us-en/ontap-restmap/aggr.html#aggr-get-iter
physical-used → space.block_storage.physical_used

aggr_total_physical_used

REST: missing
ZAPI: aggr-efficiency-get-iter aggr-efficiency-cumulative-info total-physical-used
REST docs: missing
ZAPI docs: This contains the physical size used by the aggregate.
ZAPI to REST mapping: https://docs.netapp.com/us-en/ontap-restmap//aggr.html#aggr-efficiency-get-iter
total-physical-used → space.efficiency.logical_used - space.efficiency.savings

The aggr_total_physical_used metric is collected via ZAPI, but REST does not provide it. The ONTAP REST documentation says this metric can be calculated like so: total-physical-used → (space.efficiency.logical_used - space.efficiency.savings)

And that's how Harvest calculates it for REST aggr_total_physical_used. That value should match watch you collect from the ZAPI AggregateEfficiency template.

Back to your original question, I think the difference between aggr_space_physical_used and aggr_total_physical_used is the former does not take efficiencies into account, while the later does. aggr_space_physical_used is the total physical used size of an aggregate in bytes. aggr_total_physical_used is derived from space.efficiency or aggr-efficiency-get-iter, both of which include efficiency calculations, compression, deduplication, etc.

This KB has more information on ONTAP space usage.

[rmilkowskiGSA]

I'm just trying to understand why/how the two metrics as provided by Harvester differ. This is on ONTAP 9.14.1P4, last/current values reported by Grafana for the same array:

aggr_space_physical_used 408 TiB
aggr_total_physical_used 386 TiB

::*> aggr show-space -aggregate *_aggr*
      Aggregate :xxx_aggr1

      Feature                                          Used      Used%
      --------------------------------           ----------     ------
      Volume Footprints                             213.5TB        75%
      Aggregate Metadata                             1.51TB         1%
      Snapshot Reserve                                   0B         0%
      Total Used                                    196.4TB        69%

      Total Physical Used                           194.7TB        68%


      Total Provisioned Space                       11.33PB       4067%


      Aggregate : xxx_aggr2

      Feature                                          Used      Used%
      --------------------------------           ----------     ------
      Volume Footprints                             220.8TB        77%
      Aggregate Metadata                             2.99TB         1%
      Snapshot Reserve                                   0B         0%
      Total Used                                    214.6TB        75%

      Total Physical Used                           212.9TB        75%


      Total Provisioned Space                        6.29PB       2260%

2 entries were displayed.

::*> aggr show -fields usedsize,physical-used,availsize,size
aggregate            availsize size    usedsize physical-used
-------------------- --------- ------- -------- -------------
xxx_01_aggr0         5.73GB    159.9GB 154.2GB  81.42GB
xxx_02_aggr0         5.73GB    159.9GB 154.2GB  75.42GB
xxx_aggr1            88.82TB   285.2TB 196.4TB  194.7TB
xxx_aggr2            70.39TB   285.2TB 214.8TB  213.1TB
4 entries were displayed.

Given 196+214=410 this seems to match the aggr_space_physical_used stat (408TiB) and seems to include any disk space efficiences due to compression, etc. The system manager shows 411TiB used and 159TiB available which matches the above commands and the aggr_space_physical_used variable. According to this doc it already should include any storage fficienncy savings:
https://docs.netapp.com/us-en/ontap/concepts/capacity-measurements-in-sm-concept.html
Physical capacity: Physical space refers to the physical blocks of storage used in the volume or local tier. The value for physical used capacity is typically smaller than the value for logical used capacity due to the reduction of data from storage efficiency features (such as deduplication and compression).

I think the aggr_total_physical_used essentially is physical used minus any savigns from efficiencies, but given the physical used seems to already take into account efficieny savings due to dedup, compression, anyway, this variable doesn't seem to make sense....

[cgrinds]

The Harvest metrics differ because the ONTAP metrics are different. Here is the mapping of the Harvest metric to the matching ONTAP CLI.

Harvest metric	ONTAP CLI	ONTAP Description
aggr_space_physical_used	aggr show -fields physical-used or aggr show-space -fields physical-used	Total Physical Used Size. This differs from total-used space by the space that is guaranteed for future writes. The value includes blocks in use by Snapshot copies.
aggr_total_physical_used	set advanced; aggr show-efficiency -fields total-physical-used	Physical Space Used by the Aggregate. Displays the physical size used by the aggregate.

I verified that the values Harvest collects match the ONTAP CLI values from the table above.

aggr_space_physical_used{aggr="test1",cluster="umeng-aff300-01-02"} 2.17 TiB
aggr_space_physical_used{aggr="umeng_aff300_aggr2",cluster="umeng-aff300-01-02"} 4.95 TiB

aggr_total_physical_used{aggr="test1",cluster="umeng-aff300-01-02"} 2.17 TiB
aggr_total_physical_used{aggr="umeng_aff300_aggr2",cluster="umeng-aff300-01-02"} 4.84 TiB

aggr show -fields physical-used
aggregate physical-used
--------- ---------------
test1              2.17TB
umeng_aff300_aggr2 4.95TB
2 entries were displayed.

aggr show-efficiency -fields total-physical-used
aggregate total-physical-used
--------- ---------------
test1              2.17TB
umeng_aff300_aggr2 4.84TB
2 entries were displayed.