New TrueNAS Core widget support in 0.8.11

[ZoltrixGFC]

Thank you for bringing support for pools in the TrueNAS widget for TrueNAS core.

The only thing I've noticed is that it seems to report on raw space, and not usable space.

Eg, for TrueNAS 1 and Pool1 shown in the screenshot below, it's actually 71TBs of usable, not 94TBs.

[glime-ai[bot]]

Thanks for posting a discussion about homepage here on GitHub. Homepage now uses a fancy new AI tool thanks to our friends at Glime, so the answer below is the AI bot's attempt to answer your question. If you feel it's sufficient you can mark the answer and close this post, otherwise feel free to leave it open of course!

Thanks for using Homepage! ❤️

---

To address your concern about the TrueNAS widget reporting raw space instead of usable space, it's important to clarify how the widget is currently designed to report storage metrics. Based on the context provided, the widget's behavior is likely tied to the data it receives from the TrueNAS API.

The TrueNAS API provides various storage metrics, including raw space, which is the total physical capacity of the disks without accounting for redundancy, parity, or other factors that affect usable space. Usable space, on the other hand, is the actual storage capacity available to store data, which considers the RAID configuration, overhead, and other factors.

If the widget is indeed showing raw space instead of usable space, this could be due to the widget configuration or how the data is being interpreted from the API response. To resolve this, you may need to check if there's an option to display usable space in the widget configuration or if the widget needs to be updated to calculate usable space based on the raw space and RAID configuration.

Unfortunately, without direct access to the widget's code or the API response, I cannot provide a specific solution. However, I recommend reaching out to the developers of the widget or checking the TrueNAS API documentation to see if there's a way to obtain and display usable space instead of raw space.

If you're part of the development team or have the ability to modify the widget, you may need to adjust the widget's code to calculate the usable space based on the raw space and the specific RAID configuration of the pools.

For further assistance, you may want to check the TrueNAS documentation or contact TrueNAS support for more details on how to interpret the API data correctly.

This comment was generated by Glime.

[ZoltrixGFC]

Thanks glime bot dude. I think what you want is /pool/dataset endpoint. Every pool should contain top level dataset with the same name as the pool. There you'll find both .used and .available value in raw and parsed formats.

Reference:

https://www.truenas.com/community/threads/trying-unsuccessfully-to-get-space-usage-percentage-using-pool-api-api-v2-0-pool.97135/

[shamoon]

@rgon10 any thoughts on this?

[DangoNetwork]

Any update on this? And can we report this as a bug?

[shamoon]

Since the author of the PR is not responding, if you can post all of the relevant output from the API I can take a look. I don’t have one of these devices

[ZoltrixGFC]

So the following API call below outputs the following info on my TrueNAS Core server (ignore Pool2, my screenshots are regarding Pool1).

From what I can see the Homepage Widget must be displaying:

  "size": 94025424044032,
  "allocated": 65709031284736,

That appears to match:

But the real space is:

From the total output below, I can't spot what TrueNAS is showing?

root@truenascorevm[~]# curl -H "Authorization: Bearer " -X GET "http://192.168.0.6/api/v2.0/pool" -H "accept: application/json"

[
 {
  "id": 1,
  "name": "Pool1",
  "guid": "11336913976074413241",
  "encrypt": 0,
  "encryptkey": "",
  "path": "/mnt/Pool1",
  "status": "ONLINE",
  "scan": {
   "function": "SCRUB",
   "state": "FINISHED",
   "start_time": {
    "$date": 1729947600000
   },
   "end_time": {
    "$date": 1729998282000
   },
   "percentage": 100.00002384185791,
   "bytes_to_process": 65508438048768,
   "bytes_processed": 65508398776320,
   "bytes_issued": 65508338712576,
   "pause": null,
   "errors": 0,
   "total_secs_left": null
  },
  "topology": {
   "data": [
    {
     "name": "raidz1-0",
     "type": "RAIDZ1",
     "path": null,
     "guid": "13134207767123198237",
     "status": "ONLINE",
     "stats": {
      "timestamp": 3570769861044930,
      "read_errors": 0,
      "write_errors": 0,
      "checksum_errors": 0,
      "ops": [
       0,
       1730575625,
       241880615,
       0,
       0,
       0,
       0
      ],
      "bytes": [
       0,
       393600991760384,
       5397758087168,
       0,
       0,
       0,
       0
      ],
      "size": 94025424044032,
      "allocated": 65709031284736,
      "fragmentation": 1,
      "self_healed": 0,
      "configured_ashift": 12,
      "logical_ashift": 9,
      "physical_ashift": 12
     },
     "children": [
      {
       "name": "gptid/e631b971-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e631b971-14a6-11ed-acb9-b15618a524db",
       "guid": "1387491547155512937",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861098286,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         221513177,
         31039051,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49313466617856,
         679890915328,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da4p2",
       "disk": "da4",
       "unavail_disk": null
      },
      {
       "name": "gptid/e63ccf59-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e63ccf59-14a6-11ed-acb9-b15618a524db",
       "guid": "10788927442800907938",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861131273,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         201180353,
         29400524,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49100513693696,
         669610762240,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da5p2",
       "disk": "da5",
       "unavail_disk": null
      },
      {
       "name": "gptid/e61bed7a-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e61bed7a-14a6-11ed-acb9-b15618a524db",
       "guid": "3727487378521757695",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861164813,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         278022959,
         31073918,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49310415278080,
         679908356096,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da6p2",
       "disk": "da6",
       "unavail_disk": null
      },
      {
       "name": "gptid/e60406c8-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e60406c8-14a6-11ed-acb9-b15618a524db",
       "guid": "9331504310054041619",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861198588,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         224231270,
         29388229,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49081962881024,
         669408415744,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da7p2",
       "disk": "da7",
       "unavail_disk": null
      },
      {
       "name": "gptid/e6552d26-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e6552d26-14a6-11ed-acb9-b15618a524db",
       "guid": "6292534023382569749",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861228801,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         158812371,
         31025526,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49317534658560,
         679928537088,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da8p2",
       "disk": "da8",
       "unavail_disk": null
      },
      {
       "name": "gptid/e627f2fc-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e627f2fc-14a6-11ed-acb9-b15618a524db",
       "guid": "1574050989924520825",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861261479,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         195998632,
         29356456,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49105394069504,
         669196124160,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da9p2",
       "disk": "da9",
       "unavail_disk": null
      },
      {
       "name": "gptid/e64859c9-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e64859c9-14a6-11ed-acb9-b15618a524db",
       "guid": "4631521434768990138",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861313922,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         196964121,
         31108059,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49314688176128,
         679956934656,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da10p2",
       "disk": "da10",
       "unavail_disk": null
      },
      {
       "name": "gptid/e611b7e8-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/e611b7e8-14a6-11ed-acb9-b15618a524db",
       "guid": "12393927721474376487",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570769861347470,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         253852742,
         29488852,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         49057016385536,
         669858041856,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da11p2",
       "disk": "da11",
       "unavail_disk": null
      }
     ],
     "unavail_disk": null
    }
   ],
   "log": [],
   "cache": [],
   "spare": [],
   "special": [],
   "dedup": []
  },
  "healthy": true,
  "status_detail": null,
  "autotrim": {
   "value": "off",
   "rawvalue": "off",
   "parsed": "off",
   "source": "DEFAULT"
  },
  "encryptkey_path": null,
  "is_decrypted": true
 },
 {
  "id": 2,
  "name": "Pool2",
  "guid": "9888327122173996755",
  "encrypt": 0,
  "encryptkey": "",
  "path": "/mnt/Pool2",
  "status": "ONLINE",
  "scan": {
   "function": "SCRUB",
   "state": "FINISHED",
   "start_time": {
    "$date": 1729947600000
   },
   "end_time": {
    "$date": 1729971507000
   },
   "percentage": 96.5370237827301,
   "bytes_to_process": 15685826101248,
   "bytes_processed": 16248502026240,
   "bytes_issued": 15685790195712,
   "pause": null,
   "errors": 0,
   "total_secs_left": null
  },
  "topology": {
   "data": [
    {
     "name": "raidz1-0",
     "type": "RAIDZ1",
     "path": null,
     "guid": "1809627599875334615",
     "status": "ONLINE",
     "stats": {
      "timestamp": 3570758066853911,
      "read_errors": 0,
      "write_errors": 0,
      "checksum_errors": 0,
      "ops": [
       0,
       205609740,
       26149993,
       0,
       0,
       0,
       0
      ],
      "bytes": [
       0,
       96229571452928,
       7027445207040,
       0,
       0,
       0,
       0
      ],
      "size": 39170101739520,
      "allocated": 16243162267648,
      "fragmentation": 0,
      "self_healed": 0,
      "configured_ashift": 12,
      "logical_ashift": 9,
      "physical_ashift": 12
     },
     "children": [
      {
       "name": "gptid/fa1e8ca8-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/fa1e8ca8-14a6-11ed-acb9-b15618a524db",
       "guid": "7839527807299880964",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570758066919250,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         68379267,
         6587825,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         24101255815168,
         1756892815360,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da0p2",
       "disk": "da0",
       "unavail_disk": null
      },
      {
       "name": "gptid/fa287957-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/fa287957-14a6-11ed-acb9-b15618a524db",
       "guid": "3301109252512310648",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570758066958830,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         48975530,
         6513481,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         24013376479232,
         1756828446720,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da1p2",
       "disk": "da1",
       "unavail_disk": null
      },
      {
       "name": "gptid/fa054ffb-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/fa054ffb-14a6-11ed-acb9-b15618a524db",
       "guid": "11070109894070102769",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570758066998041,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         47298904,
         6539093,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         24100178407424,
         1756891037696,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da2p2",
       "disk": "da2",
       "unavail_disk": null
      },
      {
       "name": "gptid/fa11a78d-14a6-11ed-acb9-b15618a524db",
       "type": "DISK",
       "path": "/dev/gptid/fa11a78d-14a6-11ed-acb9-b15618a524db",
       "guid": "8842422056332793798",
       "status": "ONLINE",
       "stats": {
        "timestamp": 3570758067035845,
        "read_errors": 0,
        "write_errors": 0,
        "checksum_errors": 0,
        "ops": [
         0,
         40956039,
         6509594,
         0,
         0,
         0,
         0
        ],
        "bytes": [
         0,
         24014760751104,
         1756832907264,
         0,
         0,
         0,
         0
        ],
        "size": 0,
        "allocated": 0,
        "fragmentation": 0,
        "self_healed": 0,
        "configured_ashift": 12,
        "logical_ashift": 9,
        "physical_ashift": 12
       },
       "children": [],
       "device": "da3p2",
       "disk": "da3",
       "unavail_disk": null
      }
     ],
     "unavail_disk": null
    }
   ],
   "log": [],
   "cache": [],
   "spare": [],
   "special": [],
   "dedup": []
  },
  "healthy": true,
  "status_detail": null,
  "autotrim": {
   "value": "off",
   "rawvalue": "off",
   "parsed": "off",
   "source": "DEFAULT"
  },
  "encryptkey_path": null,
  "is_decrypted": true
 }
]# 

[johnagil]

I'm on Truenas Scale but get the same type of output as ZoltrixGFC when calling the pool endpoint. I think the problem is, as mentioned in #3493, pool gets the full capacity of the pool regardless of the raid setting. It might be getting other things as well like snapshots/backups etc to determine the total size, not really sure but it never seems to get a size that makes sense. Getting the used and available space from /pool/dataset/id/{id} like zkvvoob did in #3493 would be the easiest way to get the correct values, but it would be making extra api calls as shamoon said.

My Truenas setup has three pools and each has a different setup. One a mirror, one raidz2, and one just a single disk. Factor in the endless ways other people will have their pools setup and I'd imagine trying to figure it out by adding raid support would be a headache to maintain.

Not sure if there's a clean way this can be done without making multiple api calls. I can work on it next week if the maintainers are willing to go the multiple api call route, otherwise i'm not seeing an easy way this can be done unless someone has better ideas or knows of an api endpoint we're missing.

[shamoon]

Yea this is my worry, both the multiple api calls and the infinite setups. I actually do have a bunch of work done from a few weeks ago, but as I asked at the end of that other thread it might be helpful to ask over at truenas to get a definitive answer

[ZoltrixGFC]

Did you want me to ask over at the TrueNAS forums? It is a bit of a busy environment over there lol, but I can try.

[shamoon]

Sorry to hear that, it would be helpful if you have the stomach for it. Just want to know what the “best” (only?) options are here

[chrisjenx]

Just noticed this today, sounds like there is no easy fix, wondering if we can ask TrueNAS to report via the api a more consumer friendly number?

[chrisjenx]

I have 6TB actually usable FYI

[ZoltrixGFC]

I asked on the forum a month ago, but got no reply: https://forums.truenas.com/t/api-calls-for-pool-stats-only-shows-raw-data/22638