Releases: openebs/openebs
v4.1.1
OpenEBS Release 4.1.1
Release Summary
OpenEBS v4.1.1 is a patch release with bug fixes for Replicated PV Mayastor, dependency updates and helm chart changes for LocalPV Hostpath, LocalPV ZFS, and LocalPV LVM.
Changes summary:
openebs-4.1.1
Containerized Attached Storage for Kubernetes
openebs-4.1.0
Containerized Attached Storage for Kubernetes
v4.1.0
OpenEBS Release 4.1
Release Summary
OpenEBS version 4.1 provides replicas to be distributed across different nodes and pools within a cluster, enhancing redundancy and performance. The ability to restore from snapshots across multiple replicas has been added in this release. In addition, this release has fixes related to snapshots, upgrades, availability, stability, and supportability along with solutioning for deploying OpenEBS in various kubernetes platforms.
Change Summary
Local Storage
- LocalPV analytics changes
Replicated Storage
- Placement of replica volumes across the different nodes/pools
- Snapshot across multiple replicas
- Restore across multiple replicas
- Resizing a volume with snapshots
Upgrade and Backward Incompatibilities
Before deciding to upgrade, review the list below:
- Kubernetes 1.23 or higher is recommended to install this release.
- Upgrade to OpenEBS 4.1.0 is only supported for the selected engines namely Local PV Hostpath, Local PV LVM, Local PV ZFS, and Mayastor from the earlier editions of OpenEBS (3.10.x or below).
Replicated PV Mayastor Release Notes
Features
Placement of Replica Volumes Across Different Nodes/Pools: Mayastor now uses topology parameters defined in the storage class to determine the placement of volume replicas. This allows replicas can be controlled via labels from the storage class.
Snapshot Across Multiple Replicas: Mayastor has enhanced its snapshot capabilities to ensure file-system consistency across multiple replicas before taking snapshots. This ensures that snapshots are consistent and reliable across multiple replicas
Restore Across Multiple Replicas: The capability to restore from snapshots across multiple replicas has been introduced in recent releases, enhancing data recovery options.
Expansion of volumes with snapshots: The latest release, Mayastor v2.7, includes support for volume expansion even when snapshots are present.
Snapshot Fixes:
openebs/mayastor-extensions#500
openebs/mayastor-control-plane#826
openebs/mayastor-control-plane@53500c0
Upgrade Fixes:
openebs/mayastor-extensions#512
Enhancements:
openebs/mayastor-control-plane@9dfb153
openebs/mayastor@c3816b9
openebs/mayastor@6d6fc65
openebs/mayastor@5575858
openebs/mayastor-control-plane@07ca344
openebs/mayastor@5575858
openebs/mayastor@349a07c
openebs/mayastor@7ad7a33
Solutions:
- Mayastor support in GKE with local SSD’s
- Mayastor support in Talos
- Provisioning RWX PVC’s
- Velero DR for LocalPV ZFS
Known behavioural limitations
- As with the previous versions, the Mayastor IO engine makes full utilisation of the allocated CPU cores regardless of I/O load. This is the poller operating at full speed, waiting for I/O.
- As with the previous versions, a Mayastor DiskPool is limited to a single block device and cannot span across more than one block device.
Known Issues
- Mayastor does not support the capacity expansion of DiskPools as of v2.7.0.
- Under heavy IO and constant scaling up-down of volume replicas, the io-engine pod has been observed to restart occasionally.
LocalPV Release Notes
What’s Changed
openebs/lvm-localpv#318
openebs/zfs-localpv#508
openebs/dynamic-localpv-provisioner#177
openebs/dynamic-localpv-provisioner#188
openebs/zfs-localpv#548
openebs/zfs-localpv#535
openebs/zfs-localpv#508
Known Issues:
ZFS- localpv/LVM-localpv on a single worker node encounters issues after upgrading to the latest versions. Specifically, the problem is related to the controller manifest being changed to a Deployment type, causing new controller pods to not enter the Running state.The issue appears to be due to the affinity rules set in the old pod, which are not present in the new pods. As a result, since both the old and new pods have relevant labels, the scheduler is unable to place the new pod on the same node, leading to scheduling failures when there's only a single node
Workaround is to delete the old pod so new pod can get scheduled:
Please check issue for more details #3751.
New Contributors
Thanks to our new contributors !!
@pando85 - openebs/zfs-localpv#508
@aep - openebs/zfs-localpv#535
@GreyXor - #3725
@mdonoughe - openebs/lvm-localpv#297
Contributors
Assets 2
v4.0.1
Release Summary
This patch fixes an issue with Replicated PV Mayastor on older Linux kernels. Click here to go to the v4.0.0 release notes.
Changes
- Read the Replicated PV Mayastor release notes.
openebs-4.0.1
Containerized Attached Storage for Kubernetes
v4.0.0
Release Summary
OpenEBS v4.0.0 is a major milestone change in entire project.
Important
This release increases and simplifies the overall user experience, adopter experience, repo/developer experience and CNCF community experience. With the guidance of the Cloud Native Computing Foundation TOC (Technical Oversight Committee), CNCF Exec Leadership, CNCF Storage TAG (TOC Technical Advisory Group).... v4.0.0 has implemented major & significant changes to the OpenEBS project structure and architecture. - These changes are critical and necessary for the long-term health, maintenance and community experience of the project.
Detailed info on this topic is tracked by OpenEBS and CNCF. Please view it here...
In a nutshell, the major project changes can be broadly classified into the following categories:
| Level | Type of restructuring effort undertaken |
|---|---|
| 1 | Deprecation of older and legacy engines |
| 2 | Restructuring of the technical documentation |
| 3 | Restructuring of the project repos, code, dependencies and assets |
| 4 | A unifying experience during installation |
These are in addition to the vast majority of super cool features 😎 and enhancements 🚀 to the Local-PV and Replicated-PV storage Engines, that OpenEBS is well-known for.
With this release, OpenEBS project Maintainers intends to re-enter the CNCF Storage landscape as a CNCF Sandbox project and re-establish itself as a contributing member of the CNCF Storage Landscape... in respect of our responsibility to the 1.2 Million active users (in our community) that have installed OpenEBS 9 million times over the past few years. They have spoken and they want OpenEBS to remain part of the CNCF project Landscape.
- OpenEBS is the leading Cloud-native storage solution for stateful persistent Kubernetes workloads.
Deprecation of Legacy Solutions
Prior to the v4.0.0 release, OpenEBS contained the following solutions or engines (note: These are the pre-v4.0.0 Legacy names):
Local Storage Engines
| Num | Storage engine name | Mode | Type of provisioner |
|---|---|---|---|
| 1. | LocalPV Hostpath | Out-of-tree | non-CSI provisioner |
| 2. | LocalPV Device | Out-of-tree | non-CSI provisioner |
| 3. | LVM LocalPV | Native plugin | CSI provisioner |
| 4. | ZFS LocalPV | Native plugin | CSI provisioner |
| 5. | Device LocalPV | Native plugin | CSI provisioner |
| 6. | Rawfile LocalPV | Native plugin | CSI provisioner |
Replicated Engines
| Num | Storage engine name | Mode | Type of provisioner |
|---|---|---|---|
| 1. | Jiva | Native plugin | CSI provisioner |
| 2. | CStor | Native plugin | CSI provisioner |
| 3. | Mayastor | Native plugin | CSI provisioner |
RWM (Read Write Many)
| Num | Storage engine name | Mode | Type of provisioner |
|---|---|---|---|
| 1. | NFS provisioner |
Auxiliary storage services
| Num | Storage engine name | Mode | Type of provisioner |
|---|---|---|---|
| 1. | Node Device Manager (NDM) | ||
| 2. | Monitoring | ||
| 3. | Velero Plugin for Data Protection | ||
| 4. | Data Populator |
Decision to Unify the project
In v4.0.0, the maintainers have decided to trim down the above list, for improved focus and sustained future maintenance.
This was a tough but strategic decision. Several factors like aged of old project code, adoption levels, relevance to K8s trends, available expertise, community involvement, compliance with CNCF project guidelines; and a bit of common sense (😉) have been taken into account in deciding which OpenEBS projects and technologies to retain, which to deprecate, which to migrate out of the parent project and which components to move forward with.
- In all decisions, the community is the primary deciding factor
Note
The NEW and re-organized structure of OpenEBS parent (Umbrella) project now comprises a set of very specific repos, projects, tech stacks, assets and Storage engines. Note: Some of the v4.0.0 Storage engines have been slightly renamed to provide a better community experience and more unified consistency across all Storage services.
| No. | Local PV Storage engine | Mode | Type of provisioner |
|---|---|---|---|
| 1. | Local PV Hostpath | Out-of-tree | non-CSI provisioner |
| 2. | Local PV LVM | Native plugin | CSI provisioner |
| 3. | Local PV ZFS | Native plugin | CSI provisioner |
| 4. | Local PV RawFile* | Native plugin | CSI provisioner |
*
Currently under review
| No. | Replicated PV Storage engine | Mode | Type of provisioner |
|---|---|---|---|
| 1. | Replicated PV Mayastor | Native plugin | CSI provisioner |
Auxiliary storage services
| Num | Storage engine name | Mode | Type of provisioner |
|---|---|---|---|
| 1. | Monitoring | ||
| 2. | Velero Plugin for Data Protection | ||
| 3. | Data Populator |
With the unification of the above Storage engines and auxiliary services, OpenEBS can now serve a large base of 900,000 users. For the users of the legacy engines (the v4.0.0 release formally deprecates the old storage engines, services, tools and tech stack). They will soon be physically migrated out of the OpenEBS CNCF GitHub org as indicated by OpenEBS Issue #3701 and OpenEBS Issue #3709.
- We would like to offer our
♥️ whole-hearted♥️ gratitude 👍 for being the early adopters and supporters of OpenEBS and contributing to the maturity that OpenEBS has reached today. Migration procedures are documented here to migrate data from legacy OpenEBS engines to OpenEBS 4.0.0 storage.
ℹ️ INFO
The code, tools, assets and dependent repositories associated with the deprecated Legacy projects and Storage engines will soon be transferred our of the CNCF GitHub openebs organization to a new CNCF owned GitHub openebs-archive Archive organization as per discussions and guidance provided by CNCF.
📗 Restructuring Tech Docs
Before the v4.0.0 release, the OpenEBS technical documentation was distributed across two main websites:
🔖 https://openebs.io/docs - Hosts tech docs for all engines except Mayastor
🔖 https://mayastor.gitbook.io/ - Hosts tech docs for Mayastor
Release 4.0.0 integrates OpenEBS Local and Replicated Storage into a new modern and updated documentation set. It is published here: https://openebs.io/docs website. The release also updates and clarifies documentation related to the legacy engines that are now deprecated and will be migrated out of the project.
Since the project documentation is versioned, users of deprecated legacy engines can still view all legacy documentation content as follows:
📒 Documentation for non-Mayastor engines (v3.10.x or earlier) will be available at https://openebs.io/docs/3.10.x/.
📒 Documentation for Mayastor (v2.5 or earlier) will be available at https://mayastor.gitbook.io/introduction/v/version-2.5.
⏩ One Installer
OpenEBS v4.0.0 Helm chart is the OFFICAL way to install both OpenEBS Local-PV and Replicated-PV Storage engines.
The OpenEBS 4.0.0 Helm chart bundles sub-charts and are now officially named as the following components:
| Num | Helm Installer - Storage engine name |
|---|---|
| 1. | Local PV Hostpath |
| 2. | Local PV LVM |
| 3. | Local PV ZFS |
| 4. | Replicated PV Mayastor |
| 5. | Kubernetes Snapshot CRDs |
Important
ALL of the above storage components are now INSTALLED as part of the default Helm installation process. The Helm chart provides an option to exclude Replicated PV Mayastor, if the user is interested in only installing Local PV storage (non-replicated storage).
Note
The method of using the operator.yaml file to control the installation of individual OpenEBS engines/components is now DEPRECATED and NOT supported.
Using operator.yaml is not recommended for independently installing Helm sub-charts. Please do NOT install the products using this method.
Component Versions
OpenEBS is now a Unified project that contains Storage engines and operators to deliver different types of K8s Storage services (Local and Replicated Persistent Volumes (PVs) for Kubernetes stateful workloads). - Kubernetes PVs can be provisioned via Container Storage Interface (CSI) Drivers or using Out-of-tree Provisioners. The status of the various components as of v4.0.0 are as follows:
| Type | CSI Driver name | Version | Status |
|---|---|---|---|
| CSI Drivers | |||
| 1. | [Replicated PV Mayastor](https:/... |
openebs-4.0.0
Containerized Attached Storage for Kubernetes
v3.10.0
Release Summary
🎉 🎉 🎉 OpenEBS 3.10 introduces new Mayastor features, and bug fixes. LocalPV-Provisioner (hostpath), NFS-Provisioner, ZFS-LocalPV and LVM-LocalPV welcome enhancements which improve QoL.
Upgrade and Backward Incompatibilities
Please review this list prior to deciding to upgrade:
- Kubernetes 1.21 or higher release is recommended as this release uses features of Kubernetes that will not be compatible with older Kubernetes releases. Kubernetes 1.19.12 or higher is recommended for using Rawfile Local PV.
Component versions
OpenEBS is a collection of data engines and operators to create different types of replicated and local persistent volumes for Kubernetes Stateful workloads. Kubernetes volumes can be provisioned via CSI Drivers or using Out-of-tree Provisioners. The status of the various components as of v3.10.0 are as follows:
- CSI Drivers
- Mayastor 2.5.0 (stable)
- cStor 3.6.0 (stable)
- Jiva 3.6.0 (beta)
- Local PV ZFS 2.4.0 (stable)
- Local PV LVM 1.4.0 (stable)
- Local PV Rawfile 0.8.0 (beta)
- Local PV Partitions 0.9.0 (alpha)
- Out-of-tree provisioners
- Local PV hostpath 3.5.0 (stable)
- Local PV device 3.5.0 (stable)
- Dynamic NFS Volume 0.11.0 (beta)
- Other components
Change Summary
A detailed changelog is available under the component repositories listed above. Here is a quick summary of what has changed since the last release.
- Mayastor changes: https://github.com/openebs/mayastor/releases/tag/v2.5.0
- LocalPV-Provisioner: fix quota calculation for XFS and EXT4 quota options
- NFS-Provisioner: add support for disabling garbage collection of the backend volume
- ZFS-LocalPV: add support for showing used-capacity info on ZFSNode resource
- CI fixes, depedency updates, and other maintenance to cStor, Jiva, and external-provisioner based projects.
Show your Support
Are you using or evaluating OpenEBS? You can help OpenEBS in its journey towards becoming a CNCF Incubation project by sharing your OpenEBS story and join the league of OpenEBS Adopters.
Shoutouts!
MANY THANKS to our existing contributors and for everyone helping OpenEBS Community going. @Abhinandan-Purkait @avishnu @blaisedias @chriswldenyer @niladrih @tiagolobocastro @w3aman @abhilashshetty04 @rohan2794 @dsavitskiy @sinhaashish @datacore-vvarakantham @dsharma-dc @hrudaya21 @pchandra19 @milanhajek @r1jt @AJDatacore @avishnu
A very special thanks to our first-time contributors to code, tests, and docs: @ukd1 @trunet @omric-runai @emiran-orange @cmontemuino @maxs-rose @MingZhang-YBPS @njuptlzf @testwill @pentago
Documentation
Install
OpenEBS can be installed via kubectl or helm. Follow the installation instructions here.
Upgrade
The upgrade instructions for various OpenEBS engines are here.
Do not upgrade if you are using legacy cstor or jiva provisioners. You have to first migrate those to the corresponding CSI Drivers. Please reach out to us for support
Known Issues
Check our open issues uncovered through e2e and community testing.
Check out the Mayastor release notes for known Mayastor issues.
Support
If you are having issues in setting up or upgrade, you can contact:
- OpenEBS Community for Support on Kubernetes Slack
- Already signed up? Head to our user discussions at #openebs
- Would you like to contribute to the OpenEBS project? Head to our contributor discussions at #openebs-dev
- Raise an issue
- Subscribe and reach out on our OpenEBS CNCF Mailing lists
- For OpenEBS project updates, subscribe to OpenEBS Announcements
- For interacting with other OpenEBS users, subscribe to OpenEBS Users
- Join Community Meetings
Contributors
Assets 2
v3.9.0
Release Summary
🎉 🎉 🎉 OpenEBS 3.9 introduces new Mayastor features, and bug fixes. LVM LocalPV sees new enhancements to monitoring, refactors and other bug fixes.
Upgrade and Backward Incompatibilities
Please review this list prior to deciding to upgrade:
- Kubernetes 1.21 or higher release is recommended as this release uses features of Kubernetes that will not be compatible with older Kubernetes releases. Kubernetes 1.19.12 or higher is recommended for using Rawfile Local PV.
Component versions
OpenEBS is a collection of data engines and operators to create different types of replicated and local persistent volumes for Kubernetes Stateful workloads. Kubernetes volumes can be provisioned via CSI Drivers or using Out-of-tree Provisioners. The status of the various components as of v3.9.0 are as follows:
- CSI Drivers
- Mayastor 2.4.0 (stable)
- cStor 3.5.0 (stable)
- Jiva 3.5.0 (beta)
- Local PV ZFS 2.3.0 (stable)
- Local PV LVM 1.3.0 (stable)
- Local PV Rawfile 0.8.0 (beta)
- Local PV Partitions 0.9.0 (alpha)
- Out-of-tree provisioners
- Local PV hostpath 3.4.0 (stable)
- Local PV device 3.4.0 (stable)
- Dynamic NFS Volume 0.10.0 (beta)
- Other components
Change Summary
A detailed changelog is available under the component repositories listed above. Here is a quick summary of what has changed since the last release.
- Mayastor changes: https://github.com/openebs/mayastor/releases/tag/v2.4.0
- LVM LocalPV: Monitoring enhancement with more visibility through the metrics exporter
- LVM LocalPV: Bug fixes and refactors
Show your Support
Are you using or evaluating OpenEBS? You can help OpenEBS in its journey towards becoming a CNCF Incubation project by sharing your OpenEBS story and join the league of OpenEBS Adopters.
Shoutouts!
MANY THANKS to our existing contributors and for everyone helping OpenEBS Community going. @Abhinandan-Purkait @avishnu @blaisedias @chriswldenyer @mtzaurus @niladrih @tiagolobocastro @w3aman @abhilashshetty04 @rohan2794 @dsavitskiy @sinhaashish @datacore-vvarakantham @dsharma-dc @hrudaya21 @pchandra19 @milanhajek @r1jt @AJDatacore @avishnu
A very special thanks to our first-time contributors to code, tests, and docs: @Rajpratik71 @dnugmanov @kro-cat @TECHNOFAB11
Documentation
Install
OpenEBS can be installed via kubectl or helm. Follow the installation instructions here.
Upgrade
The upgrade instructions for various OpenEBS engines are here.
Do not upgrade if you are using legacy cstor or jiva provisioners. You have to first migrate those to the corresponding CSI Drivers. Please reach out to us for support
Known Issues
Check our open issues uncovered through e2e and community testing.
Check out the Mayastor release notes for known Mayastor issues.
Support
If you are having issues in setting up or upgrade, you can contact:
- OpenEBS Community for Support on Kubernetes Slack
- Already signed up? Head to our user discussions at #openebs
- Would you like to contribute to the OpenEBS project? Head to our contributor discussions at #openebs-dev
- Raise an issue
- Subscribe and reach out on our OpenEBS CNCF Mailing lists
- For OpenEBS project updates, subscribe to OpenEBS Announcements
- For interacting with other OpenEBS users, subscribe to OpenEBS Users
- Join Community Meetings