Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage: Add Pure Storage storage driver #14599

Merged
merged 51 commits into from
Feb 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
44db9b7
api: Add Pure Storage driver API extension
MusicDin Jan 9, 2025
7a10271
lxd/storage/drivers/pure: Initial scaffolding for storage driver pure
MusicDin Sep 27, 2024
53cac8d
lxd/storage/drivers/driver_types: Add driver option PopulateParentVol…
MusicDin Dec 4, 2024
44d05ad
lxd/storage/backend_lxd: Ensure parent volume UUID is populated when …
MusicDin Dec 4, 2024
b6b8876
lxd/storage/drivers: Configure PopulateParentVolumeUUID for all drivers
MusicDin Dec 6, 2024
668ff1c
lxd/storage/drivers/load: Add pure as storage driver
MusicDin Nov 20, 2024
2313ef3
lxd/storage/drivers/pure: Add basic pool and volume configuration val…
MusicDin Sep 30, 2024
332f955
lxd/storage/drivers/pure: Add login and request wrapper
MusicDin Sep 30, 2024
26320b3
lxd/storage/drivers/pure: Create and delete storage pool
MusicDin Nov 20, 2024
4c9c611
lxd/storage/drivers/pure: Add utils for handling Pure Storage hosts
MusicDin Nov 20, 2024
bfb237c
lxd/storage/drivers/pure: Function to resolve Pure Storage volume name
MusicDin Nov 20, 2024
bc70f8e
lxd/storage/drivers/pure: Test volume name generation
MusicDin Dec 11, 2024
0460682
lxd/storage/drivers/pure: Add util function to retrieve network inter…
MusicDin Jan 9, 2025
dc9992b
lxd/storage/connectors: Add iSCSI connector
MusicDin Dec 18, 2024
0950016
lxd/storage/drivers/pure: Add iSCSI support and volume mapping utils
MusicDin Nov 20, 2024
685bbd4
lxd/storage/drivers/pure: Ensure multipath device is properly removed
MusicDin Jan 17, 2025
60c2ab0
lxd/storage/drivers/pure: Create, mount, and unmount volume
MusicDin Oct 3, 2024
9d37bd3
lxd/storage/drivers/pure: Add NVMe/TCP support
MusicDin Dec 5, 2024
ff78ff4
lxd/storage/drivers/pure: Delete volume
MusicDin Nov 20, 2024
c55b252
lxd/storage/drivers/volume: Add util function to get parent volume
MusicDin Feb 3, 2025
9f05ea1
lxd/storage/drivers/pure: Create and delete volume snapshots
MusicDin Oct 9, 2024
b9c5eb7
lxd/storage/drivers/pure: Add utils for retrieving storage arrays
MusicDin Nov 20, 2024
2224238
lxd/storage/drivers/pure: Extract storage pool, volume, and array spa…
MusicDin Nov 20, 2024
b217d75
lxd/storage/drivers/pure: Report resource usage of storage pools and …
MusicDin Oct 10, 2024
5931f83
lxd/storage/drivers/pure: Restore volume snapshots
MusicDin Oct 10, 2024
cf938f3
lxd/storage/drivers/pure: Get volume usage from Pure Storage
MusicDin Nov 20, 2024
f9d28da
lxd/storage/drivers/pure: Set or update volume quota
MusicDin Oct 10, 2024
1984b93
lxd/storage/drivers/pure: Allow volume copy with snapshots and optimi…
MusicDin Oct 15, 2024
050390e
lxd/storage/drivers/pure: Volume refresh
MusicDin Oct 16, 2024
a4c9246
lxd/storage/drivers/pure: Mount/unmount volume snapshot
MusicDin Oct 18, 2024
bba1c1b
lxd/storage/drivers/pure: Volume migration
MusicDin Oct 22, 2024
a8ca309
lxd/storage/drivers/pure: Handle cluster member volume move
MusicDin Nov 22, 2024
d59b924
lxd/storage/drivers/pure: Allow changing storage pool quota
MusicDin Nov 20, 2024
a41a5d4
lxd/storage/drivers/pure: Delete default protection groups when stora…
MusicDin Dec 22, 2024
2a93472
lxd/storage/drivers/pure: Wait for desired disk size after resize
MusicDin Jan 30, 2025
36c7cc9
lxd/storage/drivers/pure: Allow setting custom target addresses
MusicDin Jan 31, 2025
4b43b63
lxd/storage/backend_lxd: Ensure volatile uuid is set for regenerated …
MusicDin Jan 31, 2025
2a0051a
lxd/storage/utils: Add Pure Storage to common volume rules
MusicDin Dec 6, 2024
d573f87
test/backends: Helper functions for creating Pure Storage pools
MusicDin Nov 12, 2024
f30463e
test/includes/storage: Include Pure Storage driver if gateway and api…
MusicDin Dec 6, 2024
6b1a04b
test/storage_driver_pure: Add basic Pure Storage tests
MusicDin Nov 8, 2024
f7a0a9d
test/container_move: Use helper function to create Pure Storage pool
MusicDin Nov 12, 2024
12508d4
test/storage_local_volume_handling: Test Pure Storage with other avai…
MusicDin Nov 14, 2024
e6aaa1b
test/storage_snapshots: Use helper function to create Pure Storage pool
MusicDin Nov 14, 2024
a353baa
test/backup: Skip recovery tests for Pure Storage driver
MusicDin Nov 15, 2024
729aaac
docs: Add Pure Storage driver docs
MusicDin Oct 22, 2024
9418590
docs: Add example on how to create Pure Storage storage pool
MusicDin Oct 22, 2024
b5159d3
docs: Add explanation of Pure Storage remote storage
MusicDin Oct 22, 2024
c79502c
docs: Add Pure Storage and its features to table of supported storage…
MusicDin Oct 22, 2024
d0f8f5a
docs: Update wordlist
MusicDin Dec 6, 2024
40b6b51
docs: Update metadata
MusicDin Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/.wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@ EBS
EKS
enablement
favicon
FlashArray
Furo
GDB
Git
GitHub
Grafana
IAM
installable
iSCSI
JSON
Juju
Kubeflow
Expand Down
12 changes: 12 additions & 0 deletions doc/api-extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -2587,3 +2587,15 @@ The following new pool level configuration keys have been added:
The following configuration keys have been added for volumes backed by PowerFlex:

1. {config:option}`storage-powerflex-volume-conf:block.type`

## `storage_driver_pure`

Adds a new `pure` storage driver which allows the consumption of storage volumes from a Pure Storage storage array using either iSCSI or NVMe/TCP.

The following pool level configuration keys have been added:

1. {config:option}`storage-pure-pool-conf:pure.gateway`
1. {config:option}`storage-pure-pool-conf:pure.gateway.verify`
1. {config:option}`storage-pure-pool-conf:pure.api.token`
1. {config:option}`storage-pure-pool-conf:pure.mode`
1. {config:option}`storage-pure-pool-conf:pure.target`
15 changes: 8 additions & 7 deletions doc/explanation/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The following storage drivers are supported:
- [CephFS - `cephfs`](storage-cephfs)
- [Ceph Object - `cephobject`](storage-cephobject)
- [Dell PowerFlex - `powerflex`](storage-powerflex)
- [Pure Storage - `pure`](storage-pure)

See the following how-to guides for additional information:

Expand All @@ -36,12 +37,12 @@ See the following how-to guides for additional information:
Where the LXD data is stored depends on the configuration and the selected storage driver.
Depending on the storage driver that is used, LXD can either share the file system with its host or keep its data separate.

Storage location | Directory | Btrfs | LVM | ZFS | Ceph (all) | Dell PowerFlex |
:--- | :-: | :-: | :-: | :-: | :-: | :-: |
Shared with the host | ✓ | ✓ | - | ✓ | - | - |
Dedicated disk/partition | - | ✓ | ✓ | ✓ | - | - |
Loop disk | - | ✓ | ✓ | ✓ | - | - |
Remote storage | - | - | - | - | ✓ | ✓ |
Storage location | Directory | Btrfs | LVM | ZFS | Ceph (all) | Dell PowerFlex | Pure Storage |
:--- | :-: | :-: | :-: | :-: | :-: | :-: | :-: |
Shared with the host | ✓ | ✓ | - | ✓ | - | - | - |
Dedicated disk/partition | - | ✓ | ✓ | ✓ | - | - | - |
Loop disk | - | ✓ | ✓ | ✓ | - | - | - |
Remote storage | - | - | - | - | ✓ | ✓ | ✓ |

#### Shared with the host

Expand Down Expand Up @@ -71,7 +72,7 @@ You can increase their size (quota) though; see {ref}`storage-resize-pool`.
#### Remote storage

The `ceph`, `cephfs` and `cephobject` drivers store the data in a completely independent Ceph storage cluster that must be set up separately.
The same applies to the `powerflex` driver.
The same applies to the `powerflex` and `pure` drivers.

(storage-default-pool)=
### Default storage pool
Expand Down
31 changes: 31 additions & 0 deletions doc/howto/storage_pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,24 @@ Create a storage pool named `pool5` that explicitly uses the PowerFlex SDC:

lxc storage create pool5 powerflex powerflex.mode=sdc powerflex.pool=<id of sp1> powerflex.gateway=https://powerflex powerflex.user.name=lxd powerflex.user.password=foo

#### Create a Pure Storage pool

Create a storage pool named `pool1` that uses NVMe/TCP by default:

lxc storage create pool1 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token>

Create a storage pool named `pool2` that uses a Pure Storage gateway with a certificate that is not trusted:

lxc storage create pool2 pure pure.gateway=https://<pure-storage-address> pure.gateway.verify=false pure.api.token=<pure-storage-api-token>

Create a storage pool named `pool3` that uses iSCSI to connect to Pure Storage array:

lxc storage create pool3 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token> pure.mode=iscsi

Create a storage pool named `pool4` that uses NVMe/TCP to connect to Pure Storage array via specific target addresses:

lxc storage create pool4 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token> pure.mode=iscsi pure.target=<target_address_1>,<target_address_2>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uses NVMe/TCP to connect

pure.mode=iscsi

copy pasto? Please can you follow up in separate PR these small fixes.


(storage-pools-cluster)=
## Create a storage pool in a cluster

Expand Down Expand Up @@ -240,6 +258,19 @@ Storage pool my-remote-pool2 pending on member vm03
Storage pool my-remote-pool2 created
```

Create a third storage pool named `my-remote-pool3` using the Pure Storage driver:

```{terminal}
:input: lxc storage create my-remote-pool3 pure --target=vm01
Storage pool my-remote-pool3 pending on member vm01
:input: lxc storage create my-remote-pool3 pure --target=vm02
Storage pool my-remote-pool3 pending on member vm02
:input: lxc storage create my-remote-pool3 pure --target=vm03
Storage pool my-remote-pool3 pending on member vm03
:input: lxc storage create my-remote-pool3 pure pure.gateway=https://<pure-storage-address> pure.api.token=<pure-storage-api-token>
Storage pool my-remote-pool3 created
```

## Configure storage pool settings

See the {ref}`storage-drivers` documentation for the available configuration options for each storage driver.
Expand Down
117 changes: 117 additions & 0 deletions doc/metadata.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5925,6 +5925,123 @@ Specify either a cron expression (`<minute> <hour> <dom> <month> <dow>`), a comm
```

<!-- config group storage-powerflex-volume-conf end -->
<!-- config group storage-pure-pool-conf start -->
```{config:option} pure.api.token storage-pure-pool-conf
:shortdesc: "API token for Pure Storage gateway authentication"
:type: "string"

```

```{config:option} pure.gateway storage-pure-pool-conf
:shortdesc: "Address of the Pure Storage gateway"
:type: "string"

```

```{config:option} pure.gateway.verify storage-pure-pool-conf
:defaultdesc: "`true`"
:shortdesc: "Whether to verify the Pure Storage gateway's certificate"
:type: "bool"

```

```{config:option} pure.mode storage-pure-pool-conf
:defaultdesc: "the discovered mode"
:shortdesc: "How volumes are mapped to the local server"
:type: "string"
The mode to use to map Pure Storage volumes to the local server.
Supported values are `iscsi` and `nvme`.
```

```{config:option} pure.target storage-pure-pool-conf
:defaultdesc: "the discovered mode"
:shortdesc: "List of target addresses."
:type: "string"
A comma-separated list of target addresses. If empty, LXD discovers and
connects to all available targets. Otherwise, it only connects to the
specified addresses.
```

```{config:option} volume.size storage-pure-pool-conf
:defaultdesc: "`10GiB`"
:shortdesc: "Size/quota of the storage volume"
:type: "string"
Default Pure Storage volume size rounded to 512B. The minimum size is 1MiB.
```

<!-- config group storage-pure-pool-conf end -->
<!-- config group storage-pure-volume-conf start -->
```{config:option} block.filesystem storage-pure-volume-conf
:condition: "block-based volume with content type `filesystem`"
:defaultdesc: "same as `volume.block.filesystem`"
:shortdesc: "File system of the storage volume"
:type: "string"
Valid options are: `btrfs`, `ext4`, `xfs`
If not set, `ext4` is assumed.
```

```{config:option} block.mount_options storage-pure-volume-conf
:condition: "block-based volume with content type `filesystem`"
:defaultdesc: "same as `volume.block.mount_options`"
:shortdesc: "Mount options for block-backed file system volumes"
:type: "string"

```

```{config:option} size storage-pure-volume-conf
:defaultdesc: "same as `volume.size`"
:shortdesc: "Size/quota of the storage volume"
:type: "string"
Default Pure Storage volume size rounded to 512B. The minimum size is 1MiB.
```

```{config:option} snapshots.expiry storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `volume.snapshots.expiry`"
:scope: "global"
:shortdesc: "When snapshots are to be deleted"
:type: "string"
Specify an expression like `1M 2H 3d 4w 5m 6y`.
```

```{config:option} snapshots.pattern storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`"
:scope: "global"
:shortdesc: "Template for the snapshot name"
:type: "string"
You can specify a naming template that is used for scheduled snapshots and unnamed snapshots.

The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name.

To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`.
Make sure to format the date in your template string to avoid forbidden characters in the snapshot name.
For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second.

Another way to avoid name collisions is to use the placeholder `%d` in the pattern.
For the first snapshot, the placeholder is replaced with `0`.
For subsequent snapshots, the existing snapshot names are taken into account to find the highest number at the placeholder's position.
This number is then incremented by one for the new name.
```

```{config:option} snapshots.schedule storage-pure-volume-conf
:condition: "custom volume"
:defaultdesc: "same as `snapshots.schedule`"
:scope: "global"
:shortdesc: "Schedule for automatic volume snapshots"
:type: "string"
Specify either a cron expression (`<minute> <hour> <dom> <month> <dow>`), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default).
```

```{config:option} volatile.uuid storage-pure-volume-conf
:defaultdesc: "random UUID"
:scope: "global"
:shortdesc: "The volume's UUID"
:type: "string"

```

<!-- config group storage-pure-volume-conf end -->
<!-- config group storage-zfs-bucket-conf start -->
```{config:option} size storage-zfs-bucket-conf
:condition: "appropriate driver"
Expand Down
35 changes: 18 additions & 17 deletions doc/reference/storage_drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ storage_cephfs
storage_cephobject
storage_ceph
storage_powerflex
storage_pure
storage_dir
storage_lvm
storage_zfs
Expand All @@ -27,23 +28,23 @@ See the corresponding pages for driver-specific information and configuration op

Where possible, LXD uses the advanced features of each storage system to optimize operations.

Feature | Directory | Btrfs | LVM | ZFS | Ceph RBD | CephFS | Ceph Object | Dell PowerFlex
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
{ref}`storage-optimized-image-storage` | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized instance creation | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized snapshot creation | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅
Optimized image transfer | ❌ | ✅ | ❌ | ✅ | ✅ | ➖ | ➖ | ❌
Optimized backup (import/export) | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ➖ | ❌
{ref}`storage-optimized-volume-transfer` | ❌ | ✅ | ❌ | ✅ | ✅[^1] | ➖ | ➖ | ❌
{ref}`storage-optimized-volume-refresh` | ❌ | ✅ | ✅[^2] | ✅ | ✅[^3] | ➖ | ➖ | ❌
Copy on write | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅
Block based | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ✅
Instant cloning | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ❌
Storage driver usable inside a container | ✅ | ✅ | ❌ | ✅[^4] | ❌ | ➖ | ➖ | ❌
Restore from older snapshots (not latest) | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ➖ | ✅
Storage quotas | ✅[^5] | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
Available on `lxd init` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌
Object storage | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌
Feature | Directory | Btrfs | LVM | ZFS | Ceph RBD | CephFS | Ceph Object | Dell PowerFlex | Pure Storage
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
{ref}`storage-optimized-image-storage` | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized instance creation | ❌ | ✅ | ✅ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized snapshot creation | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅ | ✅
Optimized image transfer | ❌ | ✅ | ❌ | ✅ | ✅ | ➖ | ➖ | ❌ | ✅
Optimized backup (import/export) | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ➖ | ❌ | ❌
{ref}`storage-optimized-volume-transfer` | ❌ | ✅ | ❌ | ✅ | ✅[^1] | ➖ | ➖ | ❌ | ❌
{ref}`storage-optimized-volume-refresh` | ❌ | ✅ | ✅[^2] | ✅ | ✅[^3] | ➖ | ➖ | ❌ | ❌
Copy on write | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ✅ | ✅
Block based | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ➖ | ✅ | ✅
Instant cloning | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ➖ | ❌ | ✅
Storage driver usable inside a container | ✅ | ✅ | ❌ | ✅[^4] | ❌ | ➖ | ➖ | ❌ | ❌
Restore from older snapshots (not latest) | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ➖ | ✅ | ✅
Storage quotas | ✅[^5] | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅
Available on `lxd init` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌
Object storage | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌

[^1]: Volumes of type `block` will fall back to non-optimized transfer when migrating to an older LXD server that doesn't yet support the `RBD_AND_RSYNC` migration type.
[^2]: Requires {config:option}`storage-lvm-pool-conf:lvm.use_thinpool` to be enabled. Only when refreshing local volumes.
Expand Down
Loading
Loading