This is a proposal to fund a core development and testing package on the Cosmos Hub for 2024. The teams that would receive funding if this proposal passes are the Cosmos Hub Teams from Informal Systems and Hypha Worker Co-operative. The total budget is $5.7 million, or 809,659 ATOM 809,659 ATOM. This funding would replace both teams’ current funding from the Interchain Foundation and have these teams be directly accountable to the community. As ofSep 25, 2023, this represents approximately 20% ($5.7 million / $32,746,839.30) of the community pool, and will take approximately 2 months to replenish at current inflation and tax rates.
Notes:
This proposal is summarized by the following propositions:
The following items summarize the voting options and what it means for this proposal:
YES
You agree to fund the Informal Systems Hub team and Hypha Worker Co-operative with $5.7 million from the Community Pool for the Cosmos Hub’s continued development, maintenance and testing over the period of January 1, 2024 - December 31, 2024, held accountable by their specified oversight committee.
A ‘YES’ outcome will immediately release 809,659 ATOM to the specified wallet for conversion into USDC, after which any excess ATOM will be returned to the community pool and the USDC will be sent to vesting wallets controlled by representatives of Informal Systems and Hypha Worker Co-operative.
NO
You do not agree to fund the Informal Systems Hub team and Hypha Worker Co-operative based on the terms of this proposal.
A ‘NO’ or ‘NO WITH VETO’ outcome will not fund the Informal Systems Hub team and Hypha Worker Co-operative from the Cosmos Hub community pool. This may result in the teams ceasing their work on core development, maintenance, and testing of the Gaia core software and Replicated Security protocol, stewardship of the Cosmos Hub testnets program.
NO WITH VETO
A ‘NoWithVeto’ vote indicates a proposal either (1) is deemed to be spam, i.e., irrelevant to Cosmos Hub, (2) disproportionately infringes on minority interests, or (3) violates or encourages violation of the rules of engagement as currently set out by Cosmos Hub governance. If the number of ‘NoWithVeto’ votes is greater than a third of total votes, the proposal is rejected and the deposits are burned.
ABSTAIN
You wish to contribute to the quorum but you formally decline to vote either for or against the proposal.
Both Informal Systems and Hypha Worker Co-operative have played critical roles in the go-to-market strategy of Replicated Security, which is the core product offering of the Cosmos Hub.
The Cosmos Hub team at Informal Systems played a key role in the design and development of Interchain Security, and as of January 2023 also took on many duties related to maintaining the Cosmos Hub software and preparing new releases. Hypha has supplemented this work via thorough testing for all releases, operating the Replicated Security testnets program, and providing critical support to incoming consumer chains. Here are some of the teams’ past accomplishments:
See a more detailed list in Appendix A.
Together, these independently operating organizations touch every step in the software development and release process on the Cosmos Hub, which has resulted in the on-boarding of top-tier consumer chains such as Neutron and Stride, with many more waiting to join.
Software released by Informal Systems and tested by Hypha has enabled the Cosmos Hub to maintain a regular cadence of upgrades and stay up to date with the latest versions of its dependencies. Informal and Hypha have also led the emergency release process, smoothly bringing critical upgrades to the validator set when necessary.
The budget in this proposal will allow the Hub Teams at Informal Systems & Hypha to continue their work maintaining the Cosmos Hub, improving Interchain Security, on-boarding consumer chains, and spearheading the development and testing of new features.
During 2023 and before, development of the Cosmos Hub was funded primarily by the ICF. The Informal Hub Team’s budget in 2023 was around $4.5 million, while Hypha’s was approximately $1.2 million.
Funding development of the Cosmos Hub from its own community pool would mark a milestone of decentralization and sustainability, and would allow the Hub development team to be directly accountable to the Cosmos Hub, rather than to the ICF. Maintaining independence from the ICF enables the Hub to set its own standards for innovation, execution, and accountability.
With four years of governance experience, there is no entity better equipped to be directly responsible for funding the development of the Cosmos Hub than the Cosmos Hub community itself.
There are many exciting developments for the Cosmos Hub in progress, but the Informal and Hypha teams have a very clear focus. We work on the lowest level of the stack on the shared security, atomic composability, and routing technologies which underpin the Atom Economic Zone. Our roadmap for 2024 focuses on:
The Informal and Hypha Hub Teams will continue to contribute to stewardship of the Cosmos Hub’s core software, Gaia. This is obviously of core importance to everything else on this roadmap, and the team will continue to uphold the high standards set in 2023.
Many different features for the Cosmos Hub are being planned and developed by teams around the ecosystem. The Informal Hub Team has played a key role over the past year in advising, reviewing, and integrating features developed by other teams. Taking the Liquid Staking Module as an example:
We will continue providing this level of support and review for other new features.
Maintaining a regular release schedule for the Cosmos Hub has been core to our approach in the past year. Regular releases reduce the risk of any given release, build “muscle memory” for releases in the team and the validator set, and ensure that features are deployed on a timely and regular basis.
Informal and Hypha are on-call during both planned and emergency Hub upgrade deployments. We communicate with validators about the software, as well as troubleshoot any issues that come up. Due to this proactive management, Hub upgrades have been very smooth, and have resulted in very little downtime. This was even the case for the one emergency upgrade we have had this year, which resulted in only around 5 minutes of downtime. The smoothness of the actual upgrades is the result of weeks of preparation by our team and proactive communication with a large number of validator teams.
We are keeping the Cosmos Hub up to date with the latest versions of Cosmos-SDK, IBC, and other dependencies. This also includes deciding when a dependency is ready for deployment, given the Hub’s high standard of security. For example, we updated the Hub to SDK 47, as well as spearheaded the effort to get a 3rd party audit on critical parts of the update.
We will continue to perfect and maintain ICS, which is the cornerstone of the AEZ. Our improvements will focus on increasing reliability and performance of the protocol
ICS must work with several versions of IBC and Cosmos-SDK, since different consumer chains use different versions of these libraries. Maintaining releases compatible with these versions and porting features from one to another is not a small task but is critical to making sure that all consumer chains run smoothly.
The consumer chain validator set is currently updated every block. This is best for reasoning about security, but it generates a lot of traffic. We’d like to reduce this to a much lower frequency, for example once an hour. We have to prove that this is secure, but once this is done it will provide a more efficient implementation of ICS.
We would like to simplify the ICS protocol to work with IBC queries. This will allow for a much simpler protocol, and easier maintenance going forward, as well as making the Hub more robust to malfunctioning consumer chains. The read-only protocol will also make it much easier to implement features such as opt-in security, and to move faster in general.
We maintain the documentation at https://cosmos.github.io/interchain-security/, which provides a technical and theoretical overview, as well as a guide for validators and consumer chain teams.
The cultures of Informal and Hypha are rooted in software verification and QA, and we put a lot of emphasis on testing in our work on the Cosmos Hub. Beyond our extensive test suites for Interchain Security, we also write some of our own tooling.
CometMock is a stand-in for Comet that can be used during testing which greatly shortens test runtime. Many Cosmos projects have end to end tests that run for 20-30 minutes, and CometMock can reduce this to 1-2 minutes. CometMock also provides determinism (tests run the same exact way every single time), and complete control over time and block production. We built CometMock for use on the Hub, but we are working to bring it to a broader audience of Cosmos projects.
Hypha has developed a testing framework to export and modify the Cosmos Hub state. We use this framework to conduct rigorous tests on specific Gaia branches, simulating upcoming mainnet upgrades as closely as possible. This approach significantly reduces the risk associated with new releases, ensuring a high level of confidence when launching them on the Hub and contributing to the overall safety and security of launches. We will continue to enhance this framework, extending its coverage to critical scenarios and using it to test upcoming Gaia releases.
Every Cosmos Hub release undergoes a thorough testing process across three phases of testnets. In the development phase, release testnets are automatically deployed using Hypha's continuous integration scripts. Upon completion of the first release candidate, we conduct local testnets and execute a predefined set of tests. Finally, we upgrade two public testnets with Cosmos Hub validators, enabling extensive community testing. The public replicated security testnet supports multiple consumer chain testnets, facilitating integration testing in a production-like environment. Hypha will continue to oversee the Cosmos Hub and consumer chain releases through this testnet process while maintaining the necessary infrastructure and tools for public testnets.
Hypha has matured Cosmos Hub's testnet program, conducting over 50 testnet events in the current year. These events serve as valuable training grounds for validators, software testing, and the identification of operational challenges related to the Gaia codebase. In 2023, we introduced "Testnet Wednesdays," an initiative aimed at regularizing public testnet events. This initiative allows validators to allocate dedicated time for participation, thereby boosting validator engagement. In 2024, our plan is to further refine Testnet Wednesdays by diversifying event types to cover various testing aspects such as performance, security, and functionality.
As part of routine testing on both local and participatory testnets, we intend to introduce chaos testing patterns. This approach will help us explore operational limits, discover scale-related vulnerabilities, and identify system bottlenecks.
The items in this section of the roadmap are still under active research and are more likely to be substantially modified, added to, or deprioritized. Over the course of 2024, we will focus on one or two of these topics to begin building and putting into production.
Opt-in security is a family of techniques which could allow fewer validators than the whole set to validate a given consumer chain. This has potential cost and performance benefits, increasing the flexibility of the protocol, while still being a more “full-service” offering than Mesh Security. We will research techniques with which this can be accomplished, and use them to design a protocol that can benefit new and existing consumer chains. There are two basic techniques we are looking at to accomplish this:
In partial-set opt-in, only the stake of the opted-in validators is used to secure the consumer chain. This is simple, but to be secure it needs something to prevent incorrect execution. This can be done by fraud proofs, validity proofs, or fraud votes. This requirement also exists for mesh security.
In proxy opt-in, validators who are not opted-in need to delegate their stake to validators who are opted-in. This allows the full stake of the Hub to secure consumer chains, as well eliminating the need for fraud proofs, validity proofs, or fraud votes.
The Informal Hub Team has lent our expertise to the Mesh Security team, on topics such as slashing and fraud proofs. We will continue to collaborate with them to further the development and launch of Mesh Security and share concepts between the protocols to ensure that the Cosmos Hub can be well positioned for a mesh-security future.
Atomic IBC is a recently released plan that sets the Cosmos Hub up for long-term success. It does this by combining the scalability and sovereignty of the multichain ecosystem with the atomic composability of a smart contract platform like Ethereum. This combination of scalability and composability has long been a goal of many blockchain designers, and Atomic IBC solves it elegantly.
Atomic IBC will give participating consumer chains the ability to integrate seamlessly by composing multi-chain IBC interactions with many steps and delays into one transaction, called an “atomic bundle” which executes across many chains in one step. This can reduce the code required to implement complicated multi-chain workflows like those used by Timewave by around 90%.
One way to make Atomic IBC happen is by having consumer chains share a blockchain with bigger blocks but parallel execution of most transactions. We’re calling this the “Megablocks” architecture. The Megablocks architecture is relatively simple, will provide the best user experience possible, and will reduce the cost of running consumer chains using this architecture.
However, Megablocks limits the amount of customizations that consumer chain teams can make to their low-level Comet configuration and code, and necessitates a shared mempool. For this reason, we are also researching other techniques to enable Atomic IBC, from synchronizing block production across several Comet processes, to heterogeneous Paxos, a consensus technique which lets chains make shared blocks containing atomic transactions when required.
We will work towards building a proof of concept of the Megablocks architecture, while also exploring other techniques such as block synchronization and heterogenous Paxos to enable greater consensus protocol customization. We will also work to rigorously define how Atomic IBC will interact with the low level consensus customizations offered by ABCI++, and make sure that UX benefits offered by Atomic IBC are worth it.
We will explore what the developer interface to Atomic IBC could look like, and how it could make existing and future multi-chain composition use cases easier and better. We will work with existing and prospective consumer chains to see how these UX improvements can benefit their applications.
We will work on specifying and building the Atomic IBC protocol. This will entail changes to IBC to allow it to pass messages sequentially between Atomic IBC consumer chains as part of an atomic bundle, and calculate gas costs for atomic bundles. This will also entail changes to Comet to allow it to roll back transactions in an atomic bundle if any of the other transactions error.
One of the Cosmos Hub’s original missions is to route IBC packets between other Cosmos chains to increase the efficiency of the interchain by reducing relaying and light client update costs. This has been delayed by the lack of working routing in IBC. Recently, a workable IBC routing scheme has been proposed by the team at Polymer. The scheme allows IBC packets to be routed over the Hub without requiring any data to be written to the Hub, by aggregating IBC light client proofs from many connected chains.
This will augment the packet forward middleware already installed on the Hub by providing transparent routing – applications will have to make no distinction between direct and routed packets. For example, tokens transferred with this style of routing will be indistinguishable from, and fully fungible with direct routed tokens.
The Informal Hub Team will lead an effort to get this deployed to the Cosmos Hub, the Hub’s ICS consumer chains (including those using Atomic IBC), and offer it to other chains in the Cosmos ecosystem.
We will consult with the Polymer and IBC-Go teams to find the best path to deploying this feature. We will also help implement any additional code that is required.
We will support existing ICS consumer chains in deploying this feature to reduce their relaying costs. We will also work to educate the broader community about it and introduce the feature to non-consumer chains.
This committee exists to oversee, advise, and guide the core teams and the community in regards to the Informal Systems and Hypha Co-op Hub teams’ work throughout their fulfillment of their Cosmos Hub development, maintenance, and operational work related to this proposal.
This committee is ratified by the passage of this funding proposal. The committee represents various stakeholder groups in the community that are equipped to perform oversight to these teams, are impacted by their work, or able to provide an outside perspective. It includes representatives from consumer chain teams (representing the Hub’s “customer” voice), Hub validators (representing the Hub’s “operator” voice), and prominent projects in the industry who can provide an external point of view.
The committee’s power ultimately comes from the confidence and support of ATOM token holders. If, at any time, the token holders believe that the committee is not providing sufficient oversight or that either team is not fulfilling their commitment to developing the Hub, a governance proposal can be passed to dissolve the committee, suspend funding for either or both teams, and return unused funds to the Community Pool.
With these things in mind, the committee will:
The committee will not:
The oversight committee will convene for 5 meetings during the 2024 year, with one meeting at the beginning of the year, and one meeting at the end of each quarter. Each meeting will consist of presentations by representatives of Informal and Hypha, followed by questions and discussion from committee members. Committee members and core team members will communicate asynchronously to iterate on feedback given during these quarterly meetings.
Meeting minutes and summaries will be produced by the core teams and made available to the community after being approved by the committee, along with the teams’ quarterly reports, and the committee’s grades on the quarter’s progress. The format of the teams’ reports and the grading system are shown in appendix B and appendix C.
Occurs at the beginning of the year (January 2024) to discuss and formalize roadmaps for each team.
Meeting at the end of each quarter (e.g., Mar 2024, Jun 2024, Sep 2024, Dec 2024) to assess road map progress and undertake the review & reporting process.
The membership of the committee will not change throughout the funding period named in this proposal. The members of the committee are as follows:
A core part of the committee and the community’s oversight over the team rests on their ability to remove the team and end funding. However, this should take place in a smooth framework so that the team has the ability to correct performance issues if any exist, before the “nuclear option” is invoked.
If, for two consecutive quarters, more than half of the committee members submit a grade of “Needs improvement” in the overall performance category, they must also write and submit a signaling proposal calling for removal of the team and suspension of the funding.
Hypha and Informal submit reports separately, are graded separately, and can be removed separately.
As a reminder, any member of the community can submit a proposal to remove funding at any time, without following this procedure.
Our total ask of $5.7 million will cover the following broad areas listed below, and this is roughly how we expect to distribute effort within these areas. While these line items are not exact, they represent the approximate time investment from our teams on each of these areas.
Budget | Notes | |
R&D | $1,700,000.00 | Opt-in Security, Atomic IBC, Mesh Security, IBC Routing |
Interchain Security | $1,500,000.00 | Read-only protocol, epochs, general maintenance and support |
Gaia maintenance | $1,500,000.00 | Upgrades, release management & incident response |
Testing and testnets | $1,000,000.00 | Simulated testing, Cometmock, testnets validator outreach |
Total | $5,700,000.00 |
This implementation involves several compromises because of the inflexibility of the classic SDK 0.45 vesting multisig currently running on the Cosmos Hub.
Informal Systems came up with the original protocol design and specification for Interchain Security in 2021, and led the implementation of Replicated Security shortly thereafter. Replicated Security is a complex piece of software which includes the core provider and consumer modules, a protocol to allow consumer chains to have their own unbonding periods, key assignment, and the sovereign-to-consumer migration feature which allows a transition to Replicated Security without disrupting IBC connections.
Interchain Security has become the Cosmos Hub’s flagship product and we continue to lead its development through six releases of Gaia (v8 through v12) this year. Informal has also published research on Shared Security and comparisons of different security options for the Hub.
Together, Hypha and Informal have implemented several testing frameworks for Replicated Security, making QA and verification much easier for builders using this technology.
In 2022, Hypha re-established the Cosmos Hub Testnet program and has since built it into a thriving testing ground for every consumer chain on the Hub. We coordinated Cosmos Hub’s third incentivized testnet, Game of Chains, and continued that work into the Replicated Security Testnet. This testnet has been the home of ten launch rehearsals, four synchronized upgrades, and one re-launch; it is an instrumental part of the onboarding process for Hub consumer chains.
Hypha has continued to publish educational content and analysis of Replicated Security (Swiss Booklet: Neutron, Preparing for Replicated Security, Conditional Basic Income) throughout 2023.
For a detailed breakdown of our teams’ past work, see here: Appendix of past work
Hypha and Informal will submit and present separate quarterly reports, and will be graded separately.
This is an overview of work done by the team in the previous quarter, and a review of how it matched up to the plan for the quarter.
Completed tasks and successes.
Tasks that took longer than expected, or turned out to be more complicated. Unexpected events such as security incidents, etc.
Places where we deviated from the quarter’s plan, due to better understanding of tasks involved, unforeseen circumstances, and strategic adjustments.
Taking into account the work and events of the previous quarter, what is planned for the next quarter? This is also guided by the overall roadmap and long term strategic goals from the roadmap.
What are the major focuses this quarter? What are some individual tasks or deliverables that will be worked on?
Some tasks involve a higher level of uncertainty than others. This is a place to highlight expected complications in the work.
One of our core roles is to build what the Cosmos Hub community wants. We also take the role of leading through research and development, but ultimately, everything deployed on the Hub is subject to community approval.
In which ways has community feedback been solicited in the past quarter, and on what? What is the feedback?
How has this feedback modified the work executed this quarter, next quarter’s plan, and the year’s roadmap?
Baseline maintenance and regular processes are a significant part of the teams’ work. Operational tasks are not necessarily dependent on the roadmap or subject to shifting over quarters, but reflect the consistency and reliability of core operations.
What Hub upgrades, emergency or planned, have happened? How did they go?
What consumer chain testnets, launches, and upgrades have happened?
Were there any security events that happened? What was the outcome and how were they fixed?
How much was spent this quarter? (of course, the vesting account puts an upper limit on this)
How much remains for the year?
The committee will assess the work of the previous quarter in four categories, and give a grade for each one, along with an overall grade. There are three grades:
These are the four categories, which roughly correspond to the areas covered by each report and result in an overall performance grade:
How well did the team execute against the plan set at the last meeting?
How well is last quarter’s execution and this quarter’s plan fulfill the high level roadmap?
How well did the team do in considering input from the community, and putting processes in place to analyze and integrate this input?
How well have testnet, upgrades, and general operation been going on the Cosmos Hub and consumer chains?
How well is the team doing overall? Grades for each of the four categories should factor into this (i.e., If all four categories are ‘needs improvement’ then the overall performance should logically not be ‘exceeds expectations’). Two consecutive quarters of ‘needs improvement’ should result in a signaling proposal to remove the team.