I have spent a lot of time analyzing NFT data and in particular cryptopunks data. Probably more than most. A thread on solving on-chain data availability issues in the CryptoPunks contract with GhostLogs.
One annoyance is when a bid is accepted the event data emitted is wrong. It resets all the data. Now, this little bug doesn’t impact trading and I do find it to be part of the beauty of punks. This little imperfection is just part of what makes them special. But from a data perspective it does become quite bothersome to properly parse this data.
Below is the function that is called when someone accepts a bid for a punk.
Before the rest of logic executes, we need to pass all our checks between line 215–222.
A breakdown of what happens after line 222:
- Line 224: We assign the punk to the bidder.
- Line 225–226: We adjust the balance of the buyer/seller.
- Line 227: Emit the transfer event.
- Line 229: We set the state of the punk and ensure that it is no longer listed for sale.
- Line 230: Grab the bid value from our mapping on line 220.
- Line 231: We reset the punk bid data.
- Line 232: We update the mapping to ensure the seller has the ETH amount of the sale in the pendingWithdrawals.
- Line 233: We finally emit the event that the punk was bought.
Can you see any problems here? Let me show you a quick example of the data that is emitted in a transaction — 0x23517ddbc9c255e5b65a42b20572640a599291c37023096b43095f237aa80976
Let’s look at the two events that are emitted in the transaction above. Starting with the “Transfer”- Everything looks good. The buyer and the seller are appropriately represented in the data. Now on to our problem area. “PunkBought” event shows the seller address accurately, the buyer as “0x000…000” and the amount sold for as 0?
Problematic to say the least. In the TX above the punk 1775 was sold for 75 ETH. But our event data says 0.
Why is this?
Well, look back at our punkBought event data.
PunkBought(punkIndex, bid.value, seller, bid.bidder);
- punkIndex: This is being emitted properly, cool, no problem here.
- Bid.Value: Well this is problematic. Bid.Value is being reset to 0 on line 231
- Seller: This is being emitted properly, cool, no problem .
- Bid.Bidder: Again line 231 is setting our bidder to 0x0000..000 before emitting the event.
This could have all been avoided by simply moving the event before we reset our data. If this event was moved to line 228 then we wouldn’t have this issue. Damn immutable code.
Well, this problem is actively being solved by GhostLogs. GhostLogs allows you to create a mainnet fork of Ethereum and modify a contract’s source code to emit new data, edit existing state variables, add new view functions. This Ghost Fork will then replay all the transactions from your contract, allow you to export all this data into CSV and then provide me with a new RPC endpoint to listen to events from mainnet with new logic.
Let’s go through this example together and what we can do with a modified contract. 🚀
First, I input the contract address that I would like to fork and Ghostlogs returns an IDE for me to edit the contract code.
I can now start to modify my contract. As I mentioned we can simply move the event `PunkBought` in the `acceptBidForPunk` function. But I am going to actually create a new event that is only emitted in the `acceptBidForPunk` function, this will make it easier in the future for me to follow the context of the sales in the future. It will also allow me to validate my data is accurate against the original event.
I added a new event called PunkBoughtRevised with the exact same arguments. Next, I will add my event to the function `acceptBidForPunk`.
Now, I can compile and test out a few transactions to make sure my new event is giving me the accurate data! So cool. Immutable contracts be damned… I will get this data.
Once my contract compiles successfully, I can simulate transactions from mainnet to make sure that my new contract logic is emitting the data that I want. Let’s test this with the tx hash above.
Hey, look at that. My data looks good now. My event logs are emitting my new event name with the CORRECT data.
Once my data is correct, I can deploy my contract and grab my new endpoint to subscribe to my topics. For me, I am using my new contract logic for a few different things. First, I am using it to index the data so I have a complete clean history of all sales data that I can use for analysis purposes.
Second, I have been using my GhostLogs WS endpoint for my PunkProvenance Bot. This bot monitors all new punk sales and constructs a visual of each punk’s journey as they are sold and transferred. The thickness of the line represents the ETH value of the sale.
If I wanted to do this without modifying the source code and replaying the events I would have to do something along the following
- Index all of the events from the original contract both past and ongoing.
- Store all that data in a database.
- Create the logic to find Bids that have been accepted, which would look something like this.
- Monitor `PunkBought` event logs that have been fired in the `AcceptBidForPunk` function.
- Match the event transaction to the appropriate function signature to ensure that I am getting just bids being accepted.
- Pull all `BidEntered` & `BidWithdrawn` events continuously ensure that I am keeping track of all opened and closed bids.
- When the Bid is accepted ensure that I match the event to the to the last highest opened bid for the punk.
- Create an endpoint that I can listen to the logic listed above.
- Finally, I can begin to build the visualization and history of the punk.
This process is time consuming and tedious. Changing one line in my contract and giving me a WS endpoint that I can subscribe to minimizes my time to insight from weeks to hours.
This is just a small example of what you can do with GhostLogs. The ability to edit / manage data at the source is much more convenient than having complex ETL pipelines downstream. I am excited to continue to expand my use of GhostLogs and will continue to document my experience use cases as they evolve.
