Project Journey Report
etcd is a highly reliable key-value store for distributed systems that provides a way for applications of any complexity — from simple web applications to large software platforms — to store and access critical data.
Originally created at CoreOS, which Red Hat acquired in 2018, etcd offers users consistency guarantees and the ability to tolerate machine failures, even in the leader node. Applications can read data from, as well as write data into, etcd. It can monitor specific keys or directories for changes and automatically react, enabling applications to reconfigure themselves accordingly. It is the primary datastore for CNCF’s highest open source velocity project Kubernetes.
etcd joined the CNCF in December 2018 and remained in incubation for nearly two years before graduating in November 2020. CNCF published the first etcd project journey report in March 2021. Now, after more than three years of impressive project growth and community expansion, we are proud to present the second etcd project journey report, updated for 2024.
In this project journey report, we guide you through the remarkable growth journey etcd has experienced under CNCF. We can’t attribute every data point to specific inputs, but we can document and explore the correlations in search of successful decisions and actions. This report is part of a series of project journey reports published by the CNCF.
NOTE: These statistics were collected with the DevStats tool, which CNCF built in collaboration with CNCF project communities. When we refer to “Contributor,” we mean somebody who made a review, commented, committed, created a PR or issue.
Project Snapshot
Xiang Li made the first commit to the etcd GitHub repository on June 6, 2013. Since joining CNCF on December 11, 2018, etcd has added:
etcd has been adopted by organizations across the globe, including leaders in financial services, technology, telecommunications, artificial intelligence, and others, such as Fidelity Investments, Ant Financial, and Huawei. Additionally, Kubernetes clusters use etcd as their primary data store, meaning end users relying on etcd include Google, Alibaba Cloud, Amazon, Booking.com, Spotify, OpenAI, SquareSpace, Visa, Salesforce, Uber, and many others
“When etcd was donated, it had been the storage system for Kubernetes for years. Kubernetes depends on etcd to the point where problems in etcd have deep and lasting impact to Kubernetes and the ecosystem. Looking back, should [etcd’s adoption by CNCF] have happened earlier? Absolutely. But the project is in a healthy state, and its donation to CNCF was a critical step toward getting there.”
Joe Bertz
Google Technical Staff Member and Emeritus etcd Maintainer
1/7
etcd has increased velocity and adoption thanks to healthy coordination and shared vision among its major vendor and end-user contributing communities. From its roots at CoreOS, the project has grown to incorporate meaningful contributions from over 870 organizations, as well as many independent contributors, who have collectively shaped it through user-driven innovation.
The diversity of contributors is likewise expanding; in addition to its creator CoreOS, contributing organizations also include some of the world’s largest tech companies, such as Google, Red Hat, VMware (by Broadcom), Amazon, Alibaba, IBM, and Meta, as well as fast-growing mid-sized enterprises, like Indeed and Cockroach Labs. Contributions also come from dozens of smaller businesses and innovative startups, including Heptio and Weaveworks. Contributing organizations are well distributed between vendors and end users, demonstrating how users themselves can help foster and sustain fast-growing, successful projects.
CoreOS has made the most contributions to the project since its inception, with many additional companies contributing during that time. As of the August 2024 reporting date, CoreOS still led with 36% of contributions, while Google held second place with 14%. The total number of contributing companies has increased by 75% since etcd joined CNCF, rising from 499 to 873. This growth corresponds to a rise in individual contributors during the same period from 2,328 to 5,667 — a 143% increase.
All time contributions to etcd now total over 168,800, up 127% from 74,327, when it joined CNCF. CoreOS’ share of contributions decreased from 2018 until the company ceased operations in 2020, but it still leads in overall contributions. Today, Google, VMware by Broadcom, Red Hat, and Amazon are the most prolific contributing companies. These numbers indicate a healthy dynamic, with early joiners continuing to contribute while other organizations and individuals come onboard to share stewardship and grow the community.
The diversity of company contributors and the healthy balance of vendor and end-user input continue to fuel growth and give users confidence in choosing etcd as their distributed key-value store.
Cumulative growth of etcd contributions by company 2014-01-01 – 2024-08-01
Percentage breakdown of etcd contributions by company 2014-01-01 – 2024-08-01
Cumulative growth of contributing companies 2014-01-01 – 2024-08-01
Cumulative growth in contributors 2014-01-01 – 2024-08-01
of Contributors
2/7
Percentage contributions to etcd by country 2014-01-01 – 2024-08-01
Velocity
3/7
Evaluating velocity allows the CNCF, the Linux Foundation, and other stakeholders to track trends and technologies resonating with developers and end users. By observing the velocity of various projects, we can understand which ones are growing, which are maturing, and which are cultivating large communities. This information can then be used to guide decisions and strategies.
Based on velocity metrics since joining CNCF, etcd appears set to continue growing at a robust pace.
One way we track developer velocity (as defined by us) is with a simple formula: commits + PRs + issues + authors. We also look at cumulative numbers of contributors throughout etcd’s history. Both metrics illustrate that etcd’s formula of multiple vendors and end users fosters healthy grassroots support.
Monthly velocity of etcd 2014-01-01 – 2024-08-01
Growth of etcd pull requests, code commits, issues, and authors 2014-01-01 – 2024-08-01
& Sponsorship
4/7
The more a project’s community engages in education, events, and sponsorship, the healthier the project tends to be.
CNCF’s flagship KubeCon + CloudNativeCon events in North America, Asia, and Europe have spotlighted etcd since it joined CNCF in 2018. Overall, etcd has been the subject of 38 keynotes, talks, sessions, meetings, and workshops at these events, attended by tens of thousands each year.
etcd holds weekly meetings online, alternating between general community meetings and triage meetings. Community meetings give maintainers and end users the chance to discuss project updates, hear feedback from the community, and collect end-user requirements and use cases to develop new features. Triage meetings gather contributors to address the backlog of PRs and issues.
etcd also conducts biweekly robustness tests to allow members to collaboratively explore etcd’s correctness under pressure, share knowledge, foster a rigorous testing culture, and mentor new test reviewers and approvers.
etcd has recently formed a new working group to plan and develop an etcd operator for supporting applications in Kubernetes. The etcd Operator Working Group holds biweekly meetings and has its own channel in the Kubernetes Slack workspace.
etcd tutorials and courses have emerged to meet the demand for skills in using the project’s software. For example, DevOpsSchool offers an etcd training and certification program to online, classroom, and corporate learners. Additionally, training and consultancy group NobleProg offers a course titled “Distributed Systems Storage with etcd,” available virtually and onsite.
In 2018, Mechanical Industry Press published the first full-length book on etcd, titled “Cloud Native Distributed Storage Cornerstone: etcd In-depth Analysis,” for the Chinese market. Produced by the Huawei Cloud Container Service Team, with Du Jun as lead author, the book closely analyzes etcd as the top distributed key-value storage system in use.
& Programs
5/7
Since etcd joined CNCF in 2018, we’ve worked diligently to spread awareness of the project and expand its community through two core principles:
- Solving real problems and being genuine with our audiences.
- Listening to all inputs from vendors and end users to maintain balance and generate excitement.
Since accepting etcd, CNCF has covered news related to the project and featured it in case studies on its website. We have also presented 12 etcd-related video programs on our website, including live and on-demand webinars, which have garnered over 14,000 views to date.
The etcd.io website, co-managed by CNCF, has grown robustly since early 2019. Google Analytics 4 (GA4) reports 144,000 pageviews for etcd.io during July 2024.
Google Analytics 4 (GA4) 144k Monthly etcd.io Pageviews
Note: Google’s privacy settings can impact its analytics reporting. In CNCF’s own experiments using the privacy-first analytics platform Plausible, we measured 2x more traffic to our websites than Google Analytics reported. This should be kept in mind when viewing the above etcd.io pageview statistics from GA4.
When etcd joined CNCF, its Twitter account had fewer than 100 followers. Today, it boasts over 4,450 followers, having grown its following by over 45x since then.
The project also has its own active Google Group, a YouTube channel, and a dedicated channel — SIG-etcd — in the Kubernetes Slack workspace, with more than 4,700 members.
The formation of ‘SIG-etcd’ in the Kubernetes project could not have happened if etcd were not donated to CNCF. Both the donation and the formation of SIG-etcd have attracted talented contributors and helped sustain the project.”
Joe Betz
Google Technical Staff Member and Emeritus etcd Maintainer
CNCF has offered etcd merchandise on store.cncf.io.
Documentation
6/7
Cloud native projects can’t exist without robust documentation to educate new users and helping existing users solve problems. A healthy velocity of documentation changes is a strong indicator of the health of a project’s community.
To date, more than 350 contributors from diverse backgrounds and motivations have added to and polished etcd’s documentation.
NOTE: Documentation for Harbor is authored in .md files. CNCF uses the DevStats tool to automatically collect and count statistics of all relevant .md files in the Harbor repositories in GitHub.
Growth in participation in etcd project documentation 2014-01-01 – 2024-08-01
Cumulative growth of etcd project documentation commits 2014-01-01 – 2024-08-01
“The CNCF commissioned a technical documentation analysis for etcd, leading to recommendations for improving its documentation and making it more user-friendly. Also, the CNCF documentation experts have provided guidance and direct contributions to the etcd website.”
Wenjia Zhang
Staff Engineering Manager at Google
7/7
CNCF is committed to fostering and sustaining an ecosystem of open source, vendor-neutral projects by democratizing state-of-the-art software development and deployment practices to make technology accessible to everyone.
We hope this report provides a useful window into how CNCF fosters and sustains the growth of etcd, its team, and the diverse community of individuals and organizations using and shaping the project.
“Over the years, we have benefited from the reliable and consistent nature of the etcd project. The trust we’ve placed in etcd for our Kubernetes control planes has been rewarded with predictable performance. Now, we’re exploring its potential in other infrastructure applications, encouraged by its ongoing development and community support. Being part of the CNCF ensures that this open-source solution remains aligned with the rest of cloud-native technologies.”
Maxime Visonneau
Engineering Manager at DataDog and etcd end user