ERC-7512: Onchain Audit Representation - EIPs - Fellowship of Ethereum Magicians Fellowship of Ethereum Magicians ERC-7512: Onchain Audit Representation EIPs erc rmeissner September 5, 2023, 8:40pm 1 Discussion for Add EIP: Onchain Representation for Audits by rmeissner · Pull Request #7652 · ethereum/EIPs · GitHub The purpose of the ERC is to propose a standard to represent audits onchain. Any discussion related to this can be facilitated here. 8 Likes Dexaran September 6, 2023, 3:07pm 2 The idea of having on-chain audits is useful. However the implementation proposed in this ERC is overcomplicated significantly. The goal of having on-chain audit registry is to allow contracts to verify if a contract is secure or nobody ever reviewed it (the fact that it was reviewed and assigned a “secure” label does not automatically mean that it is really secure however). Taking this into account lets review audit properties: Auditor - Do we need it at all? name - well, there are better ways of recognizing auditors. For example just keep a track of auditors addresses in an open registry and let contracts match the issuer of the on-chain report against one of the auditors publicly available addresses. uri - it doesn’t tell contract anything. A human can follow the link and read whatever is written there but not the contract. authors - it doesn’t tell contract anything. Audit auditor - ok contract - ok, chainID and address are useful. issuedAt - may be ercs - why is it important at all? I don’t think that a contract may want to conduct this type of standard recognition via audit reports instead of ERC-165 for example. Also, USDT is not compliant with ERC-20 spec, do you expect some auditor like OpenZeppelin or CertiK to say “USDT is not a ERC-20 token”? auditHash - ok auditUri - ok Auditor Verification Why would someone need such a complex system of auditor verification if that same task can be accomplished in a much easier way - just let auditors submit audit reports from a publicly known address and match addresses to the name of the auditing company. Like 0x111111 is OpenZeppelin 0x222222 is CertiK 0x333333 is Callisto Audits etc. You don’t need such a complex structure and the whole load of processes for signing / verifying if it can be done in a way that would allow even a technically inexperienced user to verify who is the auditor. The current ERC does not have any mentions of the findings of an audits. This is the most crucial part honestly. There can be multiple audit reports for one contract and if at least one indicates a problem with the contract - it is more important than all other reports that do not indicate any problems with this exact contract. If you have 3 auditors who have reviewed one contract, two of them found nothing and the third found a critical vulnerability - it’s much more logical to indicate that “the contract might have a critical vulnerability” rather than resort to an assumption “if there is at least one audit report that doesn’t indicate any problems then the contract is most likely safe”. I think that a system that does not allow for findings specification and independent audits submissions for multiple different auditors - will not work or even worse it will deceive users into thinking that some contract is secure while in fact there are problems with it. 2 Likes Dexaran September 6, 2023, 3:32pm 3 I would propose an alternative structure for on-chain audits. Create a “registry” contract that will allow anyone (or a select group of addresses) to issue an “audit report” for another address. This “audit report” should act as a Soulbound Token with configurable properties. I have proposed this type of NFTs in the past (it is easy to turn into SBT by simply removing transferring features): ERC CallistoNFT standard Tokens ERC - CallistoNFT Preamble EIP: Title: Non-fungible Container Token Standard Author: Dexaran Type: Standard Category: ERC Status: Draft Created: 2022-22-02 Simple Summary An interface for non-fungible tokens and minor behavior logic standardisation. Abstract This standard allows for the implementation of a standard API for non-fungible tokens (henceforth referred to as “NFTs”) within smart contracts. This standard introdu… The SBT must contain the following properties: Issuer - the address of the auditor or an auditing company Critical findings: number High severity findings: number Medium severity findings: number Low severity findings: number Audit hash Audit report link Chain ID It is possible to leave severity assignment to auditors I think. In this way it would be possible to ask one contract (registry) and get a list of audits if there are multiple. At the same time if there is already one audit report that says “everything is fine with the contract” but in fact the contract has security problems - there will be a way for other auditors to submit reports that point out security problems of the contract. 2 Likes rmeissner September 6, 2023, 7:05pm 4 Dexaran: Create a “registry” contract that will allow anyone The purpose of the ERC is not to define the registry, but rather a format in which audits can be represented on chain. Dexaran: type of standard recognition via audit reports instead of ERC-165 The security implications are quite different if a contract can self proclaim what they support and if you get an external party “verify” that they follow a standard. Also not all ERCs are interface standards. Dexaran: do you expect some auditor like OpenZeppelin or CertiK to say “USDT is not a ERC-20 token” Yes that would be the correct way and is actually critical for implementations building on top of such standard. Some tokens that claim to be ERC-20 compatible, but because of different behavior of their transfer function require contracts building on top to implement special handling just for these contracts. Dexaran: Auditor - Do we need it at all? That is a good question and it should be considered. The authors fields was meant to provide a indicator which auditor was actually auditing the contracts (as there are differences within audit companies), but there are alternative ways to represent this. Dexaran: The current ERC does not have any mentions of the findings of an audits. This has been discussed with some auditors before and it would be indeed very helpful, but there are some challenges on how to align on the definition for the severity. To not make the ERC more complicated leaving it out would be a first step. I agree that leaving this classification up to the auditors is also a solution. I would rely on the impact of the auditors for this. Dexaran: just let auditors submit audit reports from a publicly known address and match addresses to the name of the auditing company. As the ERC aims to only create a representation of an audit and not define how it is handled on chain, the definition of the verification scheme make it possible to use them independently of specific chain allowing a verification of the representation offchain. Dexaran: Medium severity findings: number Low severity findings: number How useful are these? Normally medium and low severity findings are not security critical and the usage of the auditors of these states might differ. Dexaran: This “audit report” should act as a Soulbound Token with configurable properties. In my opinion this would be an application of an onchain audit representation. I.e. auditors would create the representation, sign it and then anyone could submit it to create a SBT based on it that can be used onchain. PatrickAlphaC September 11, 2023, 4:48am 5 I’ve thought a decent bit about this, and every time, I conclude: “What would this accomplish?” At the end of the day, audits (henceforth, “security reviews”) are for the protocols, not the community. Having a security review hosted on-chain suggests, “You can trust this code because there is a security review here.” However, the security review was paid for, scoped by, and conducted by the protocol. It’s meant for the protocol. In the context of feedback for the protocol. At the same time, if a project wanted to show off its audits for people to learn and grow from, that seems fine. But I’m just nervous that this is a standard for projects to say “Look, we are safe. You can see our on-chain security review” when the security review wasn’t done with the community in mind. My main question is, “Why would we even care to have this standard?” 3 Likes lukasschor September 15, 2023, 1:01am 6 Not sure security reviews are in practice used „for the protocol“. Pretty much every landing page features security reviews as a signal to the community that there has been some measures being taken and that there is a baseline of security focus in the project. Yet the connection of these PDFs on a landing page and the actual smart contract code is very loose. So this ERC is at least an incremental improvement by bringing security reviews closer to the actual code they were covering. Ideally it‘s also going to be the basis of much more significant improvements such as reputation systems being built on top of this standard or new incentive mechanisms where it‘s actually not the protocol team scoping and paying for the security review. 2 Likes chendatony31 September 19, 2023, 1:58pm 7 I have discussed this issue with some security companies before, and the solution is similar to @Dexaran’s. Security companies can issue audit SBT to their audited contracts. which means that each security company or person has their own corresponding SBT contract. We can verify whether the contract belongs to this company by using the url metadata of the contract and the /.well-know/contract.json file of the official company domain name As a wallet (or an explorer), you can enumerate these contracts to display which people/companies have audited the contracts that your user is interacting with and whether they are relatively secure. thezluf September 20, 2023, 5:35am 8 As @rmeissner mentioned, this ERC focuses on standardizing what auditors should sign, rather than defining the registry. The goal is to ensure consistent verification across the ecosystem. blackbigswan September 20, 2023, 10:38am 9 PatrickAlphaC: I’ve thought a decent bit about this, and every time, I conclude: “What would this accomplish?” I totally get where you’re coming from as I share same sentiment. That said, there’s a solid reason to standardize the data format for posting audits on-chain, and it aligns with the SEC’s recent thinking. While the SEC is primarily concerned with reliable financial information, you could argue that audits indirectly fall under this umbrella. Take a look at this document: F5bvlrbWgAEVC0m630×900 169 KB Notice: […] in practice the source code implemented on blockchains are in machine-readable format, may not conform to public descriptions of the code So, creating a standardized, public, and immutable registry for audits of a given protocol makes sense. But a standard is only as good as the infrastructure built around it (gh2source2bytecode compare, documentation2implementation compare, does audit audit actual bytecode or github etc.). That’s mostly going to be off-chain though. Personally I call it “GenslerProtocol”. So yeah, this EIP does have value, especially since there’s at least one clear use case that would benefit from a robust data format. 1 Like lefuncq September 20, 2023, 12:40pm 10 Many people suggest going with a registry instead of the current proposed implementation. That’s precisely what we’ve been working on for the past couple of years at Trustblock. Compared to what we’ve built, the current proposal has its strengths and weaknesses. The main problems I see so far: Expect protocols to add this to their codebase, which is far from trivial Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage Handling upgrades Findings are missing but still valuable information for protocols to act upon. rmeissner September 20, 2023, 1:50pm 11 lefuncq: Expect protocols to add this to their codebase, which is far from trivial I would be interested how this problem is differently handled by the 2 approaches. The current ERC proposes a way that could be used in any registry and without any onchain interaction necessary. The challenge we see with registries is that it is still a very centralized approach (which has its benefits). Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself. Synchronicity? Auditors have to submit audits upon each request, which is very limiting in terms of usage Overhead wise the ERC aims to keep it minimal. Many auditors are already signing their audits as part of the process and the additional overhead to create and sign the onchain representation should be quite low (would be interesting where you see this becoming a blocker). When it comes to publishing the representation and pushing it onchain, then this can be done by anyone. Auditors could upload this side-by-side with their audit files (similar how checksums are published) and anyone could make use of these. This way the overhead to interact with any 3rd party is not required. Angler September 20, 2023, 2:24pm 12 lefuncq: Many people suggest going with a registry instead of the current proposed implementation. The benefit of the representation over a registry IMO is that we can have many different registries that utilize the audit representation. This has multiple benefits. No single entity is in control over the audit representations The user can select from different registries one that matches his needs. E.g. the user might select a registry for smart contracts that are audited only by specific auditors. The registry can combine the representation with other criteria thebensams September 20, 2023, 3:11pm 13 How does this work for upgradable contracts? 1 Like lefuncq September 20, 2023, 3:48pm 14 I would be interested how this problem is differently handled by the 2 approaches. In our case, we have one registry per chain, and these registries contain auditors’ wallet addresses that are allowed to publish audits. We whitelist wallets for now, but the best would be to have governance voting auditors on and off. Therefore the ERC aims to create a building block to make audit information available and verifiable onchain, not to build such registries itself I agree with you it’s clearly out of scope for this EIP, but I think it’d still be interesting to actually work together on an audit registry ERC that we could also integrate on Trustblock. The main reason for a registry is that it will create many more use cases for audits on-chain, according to our research through these couple of years, than if the verification has to be made synchronously with the auditor. With the data being directly accessible, protocols can be used for various composability purposes. Moreover, storing on-chain also guarantees the immutability of audits. The challenge we see with registries is that it is still a very centralized approach (which has its benefits). Audits are delivered by a single entity, so they will always be centralized by nature, right? However, if you meant that registries would be controlled by a single entity (like us for example), then I think it depends on the registry functioning, for example in our case only auditors can publish audits so we don’t control the data. Overall Per my knowledge and seeing how many auditors work, I have not seen many auditors sign their audits (not sure exactly what you meant there) nor keep track of contract addresses they have audited. So, if they want to support this, they must change how they do their business. I believe specifying exactly which contract addresses audited is the right thing, but that isn’t the reality right now If they want to upload old reports, they must find the exact addresses they audited. When they do new ones, they have to ask for deployed contract addresses after an audit, verify them, and add them to the report Preparing audit representation and signing. Depending on the tooling used, it can be automated or simplified. Still, it is an additional step Display their wallet addresses or public keys so that others can verify audits So they should have the right incentive behind it. I know it is not exactly the scope of this EIP, but that’s the practical aspect of it. If protocols want to support this and accept only, let’s say, tokens with audits into their system, they have to: Pre-select auditors they trust. It means that they have to get their wallet addresses/public keys to verify the signature belongs to the auditor they trust Implement verifier either on-chain or off-chain lefuncq September 20, 2023, 4:05pm 15 I agree with you. The way you would implement registries is very interesting and different from the way we implemented them in our system so far. In our case, we have one registry per chain, and protocols can preselect which authorized auditors they want to get audits’ data from. Another significant advantage in favor of registries is that if the audits were to be stored on-chain, we could make them immutable, which is super helpful to balance trust relations further between users, auditors & protocols. srw September 20, 2023, 5:10pm 16 We just published our opinion at ERC-7512: A Solution to the Centralization of Security Audits Data?. The TLDR is that solves the issue with sites such as CoinMarketCap, Etherscan, CoinGecko, etc because they don’t publish accurate data about auditors even when you follow all their forms. Hence, they are a point of centralization that filters real information. On the other hand, we think that this problem should not be limited to audits and a way to add metadata in general could be interesting. Dexaran September 21, 2023, 5:02pm 17 The main problem with this ERC is that it only allows for “positive” audits that say the contract is secure. I own a security auditing company (https://audits.callisto.network/), I’m an auditor myself and in many cases it is really important to say “Warning! This contract is not secure” while others are saying it is secure. It is not secure to pretend “if at least one company said that the contract is secure = we label it as secure” . Also audit records must not be immutable by any means. Imagine a contract was audited, then a vulnerability is discovered but there is already a signed audit report that states that the contract is fine. Anything can happen or be discovered AFTER the audit report so this needs to be accounted. JakubLipinski September 22, 2023, 7:38pm 18 Just for the sake of reference: I tried (and failed) to create an on-chain registry of smart contract audits in 2008 as a part of SolidStamp service. Some of my learnings and ideas are still available at: Medium – 19 Jun 18 SolidStamp; Putting Skin in the Blockchain Game. On-chain registry of Ethereum smart contract audits Reading time: 3 min read Medium – 2 Oct 18 SolidStamp — a flight recorder for the Ethereum ecosystem. It’s been 3 months since we launched SolidStamp, an on-chain registry of smart contract audits. It’s time to share some of the learnings we… Reading time: 5 min read Good luck this time. 1 Like Hans September 23, 2023, 11:19am 19 After all, the proposal seems to be to see if a protocol was audited. We can achieve that more simply, in fact: Auditors (personal or firm, whatever) create NFT to represent the audit information. Additional information like the issued date and a report link can be included in ERC721Metadata. Maintain an address registry of auditors on-chain. What else do we need? rmeissner September 23, 2023, 11:41am 20 You need an agreement of the auditors on this metadata and how to issue that nft. What you described is not the purpose of the nft the purpose is to create a basis that allows you to create such nfts. I.e. could you say that you have a nft contract where you can submit an erc-7512 object which will mint the nft with some basic information (like auditor). This then can be used in different protocol to perform security check (or other logic based on the audits). next page → Home Categories FAQ/Guidelines Terms of Service Privacy Policy Powered by Discourse, best viewed with JavaScript enabled