Risk Scores for DeFi— Alpha Release
Note: Please read the disclaimer at the end of this post. This is not investment advice.
It is easy to forget — especially as people “ape” into unannounced projects on a moment’s notice — that for many people outside the DeFi community, something as safe as Compound or Aave still seems like the riskiest investment they’ll ever make. Even for the risk-tolerant folks who have been using these products for years, it still can be hard to compare risks across protocols. If I lend USDC on Cream.finance for 4% interest, is it worth the risk versus just lending on Aave for 2%?
Many people don’t even know what risk factors exist with decentralized lending, not to mention how to quantify those risks. Some lending protocol users do not even know about liquidations, and few understand that they are far and away the most important component in preventing depositor losses. As we mention in a recent blog post, Gauntlet is building a platform to manage risk in DeFi. We’ll use this platform to help set protocol safety parameters using Automated Governance, allowing protocols to make tradeoffs between risk and capital efficiency. But this raises a natural question — how do we communicate these risks to users?
Gauntlet is launching a risk score for DeFi Protocols. Starting with collateralized lending, we have created a standard for quantifying the market risks investors face when earning yields across these products. Utilizing data from centralized and decentralized exchanges combined with on-chain user data, we are able to run simulations directly against protocol smart contracts to estimate market risk. This will allow users to make decisions on risk across protocols using a consistent framework. We’re partnering with DeFi Pulse to publish these scores as “Economic Safety Grades”. DeFiPulse has been a leader in educating users about DeFi protocols and we’re excited to work together to help people better understand risk.
Risk in On-chain Lending
We’ll cover how we measure risk in collateralized lending at a high-level in this post, but if you want the in-the-weeds details, check out our Compound risk assessment. The main risk to depositors in on-chain lending is insolvency, i.e. that the protocol might not have enough collateral to pay back depositors. This is not the only risk to participants — for instance borrowers might care about rapid changes to interest rates. However, the score is focused on this risk of insolvency. There are a few key factors that contribute to this risk:
- User behavior
- Collateral Volatility
- Relative Collateral Liquidity
- Protocol parameters
- Smart contract risk
Our score does not purport to be able to model smart contract risk, such as reentrancy attacks. We believe that auditors and formal verification tools are best suited for assessing this form of risk. However, these other four factors greatly determine the chance of insolvency in audited on-chain lending protocols.
When users borrow on Compound, they choose how much collateral they provide and how much debt they take out. It is this user behavior that affects how often liquidations occur in the protocol. If borrowers are risk-averse and only take out small positions against their collateral (e.g. $3 of debt on $10 of collateral), it becomes very unlikely that a position becomes eligible for liquidation, let alone creates a risk of insolvency. Another thing that clearly affects risk here is the volatility of the collateral. If the value of the collateral is changing wildly, a 25% drop wouldn’t be out of the ordinary, but if it isn’t, the chance of insolvency could be negligible. The protocol can also restrict borrowers from taking out very risky positions, which would allow protocol parameters to lower risk.
The success of the protocol also hinges on the ability of liquidators to liquidate profitably. If positions aren’t liquidated, depositors have no guarantee that the protocol stays solvent. It’s probably not immediately clear, given the discount on collateral given to liquidators, how they could ever lose money. Right now, there are billions of dollars deposited in on-chain lending contracts. Liquidators might have to purchase hundreds of millions of dollars of collateral in a short period of time. The discount provides a large profit opportunity, but that comes with substantial risk. What if the price of the collateral falls while trying to sell it? What if exiting the huge collateral position moves the market and creates slippage? It’s hard to imagine 5% slippage on a $750 position, but not on $75mm of assets, especially for collateral types with poor liquidity. Large liquidations can add up, driving the price of the collateral asset down further and triggering a cascade of liquidations on other borrow positions. Some supported collateral types on Compound, Aave, and Maker have had days where the daily volume traded on exchanges is less than the total amount of debt held in the smart contract. It is the relative liquidity, or the amount of liquidity relative to the amount of collateral used, that creates this risk for liquidators, and as a result, the protocol.
Economic Safety Score Model— Alpha
For the first version of the score, we have built a model that covers these four risk factors. Our simulation environment allows agents to interact directly with the protocol’s actual smart contracts. In order to construct the borrowing strategies that agents use, we look at on-chain state to determine how risky users are (in this case, by looking at on-chain collateral ratios). To capture the relative liquidity risk, we pull data from exchanges to model price volatility and collateral liquidity. We use observed volatility to generate synthetic price trajectories that match market conditions. We then train a regression model against the order book data to understand the slippage that liquidators might face given the current in-market liquidity. In our simulation model, we have actual liquidation bots running that call the smart contract directly to liquidate positions. By running simulations against thousands of price trajectories, we can estimate the likelihood of insolvency, and therefore the risk in the protocol. We finally normalize that score 1 to 100 to create the risk score that you see on DeFi Pulse.
At launch, we will be providing risk scores for two on-chain lending protocols, Aave and Compound. We also make a couple of simplifying assumptions that we hope to build on in future versions of the risk score:
- We pick the “riskiest collateral” — we analyze historical liquidity and volatility data to identify the type of collateral most likely to cause issues. This is often the most volatile collateral type, but can also simply be the most-used collateral type. This asset has the greatest requirement for liquidity since there is so much of it to liquidate.
- We then estimate the risk of the system for users borrowing a stablecoin against this collateral type. Stablecoins currently account for more than 70% of borrows on Aave and over 90% on Compound. The next protocol we hope to support, Maker, has 100% stablecoin borrows.
Safety Score Beta and beyond
In our alpha release, we don’t cover every risk that users of lending protocols encounter. An astute observer might have noticed we omit the case where the protocol is illiquid. We hope to model this as well as a few other things as we build towards a beta release:
- Extreme network congestion (we incorporate gas costs currently, but can do a lot to improve here)
- Support liquidation analysis on multi-asset collaterals and borrows, taking into account historical asset price correlations (and shocks to those relationships) akin to portfolio stress testing in traditional finance
- Protocol reserves — Aave and Maker both use their token to back deposits (or in Maker’s case, issued DAI). Compound has a reserve that governance could use to pay back depositors — those all reduce risk and we want to include these in the model
- Other risk factors, e.g. the number of observed liquidator addresses on mainnet. If a protocol doesn’t have a competitive ecosystem of liquidators, there is a greater chance of failure.
We’ll also strive to support more protocols, and hope to release public documentation soon to let protocols that would like a score know what they can do to be included.
The risk score (“this report”) is for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation for any security, nor does it constitute an offer to provide investment advisory or other services by Gauntlet Networks Inc. No reference to any specific security constitutes a recommendation to buy, sell or hold that security or any other security. Nothing in this report shall be considered a solicitation or offer to buy or sell any security, future, option or other financial instrument or to offer or provide any investment advice or service to any person in any jurisdiction. Nothing contained in this report constitutes investment advice or offers any opinion with respect to the suitability of any security, and the views expressed in this report should not be taken as advice to buy, sell or hold any security. The information in this report should not be relied upon for the purpose of investing. In preparing the information contained in this report, we have not taken into account the investment needs, objectives and financial circumstances of any particular investor. This information has no regard to the specific investment objectives, financial situation and particular needs of any specific recipient of this information and investments discussed may not be suitable for all investors. Any views expressed in this report by us were prepared based upon the information available to us at the time such views were written. Changed or additional information could cause such views to change. All information is subject to possible correction. Information may quickly become unreliable for various reasons, including changes in market conditions or economic circumstances.
