Reporting iOS crashes to StatsD
Here at Kickstarter we use Crashlytics for our crash reporting in the iOS app. It serves us well and does a lot of work that we don’t want to have to worry about. However, it also introduces more fragmentation into our toolset for monitoring our services. We already have an extensive set of StatsD metrics that we monitor, so it would be nice to see graphs of crashes right next to graphs of our HTTP 200s and 500s.
Crashlytics provides a delegate method -crashlytics:didDetectCrashDuringPreviousExecution: that is called when the library has detected that the last session of the app ended in a crash. We use this method to hit an endpoint on our servers that will increment a StatsD bucket. The request contains info about the build number and iOS version at the time of the crash, and we include that in the StatsD bucket:
STATSD.increment "ios.crash.#{build}.#{ios_version}"Now we can build a graph that shows the total number of crashes over time, or break out into many graphs for crashes per iOS version or app version. The graphs help us to tell a story about the overall stability of our releases, or what fixes have been effective.
For example, when looking at the total number of crashes over time, we can clearly see an increase in mid September:
This is right around the time iOS 7 came out. In order to confirm this we can look at a graph that breaks out crashes per iOS version:
Now it is very clear that iOS 7 has elevated crash rates, though each minor release has been slightly better. For example, the first release of iOS 7 seemed to have a bug in CFNetwork causing hundreds of crashes a day in a function named mmapFileDeallocate. This crash has not happened since 7.0.2, which is reflected in the red and purple curves.
Since the crash rates are still higher than what we saw in iOS 6 we looked for other ways to workaround the most common crashes. One of the more perplexing iOS 7 specific crashes we see has to do with data detectors in UITextView instances. It happens seemingly randomly, and has occurred in every version of iOS 7. In our most recent update we wrote a UITextView subclass that provided a custom implementation of data detectors in hopes of getting around this crash. The benefits of this work can now be seen by looking at a graph of crashes by build #:
Build #510 (the blue curve) is the first build with the fix, and it has the lowest number of crashes we’ve seen for awhile.
This form of crash tracking has been very useful to us. In fact, it’s become so important that we put a version of these graphs on our deploy dashboard so that we can immediately see if an API or mobile web change affects the crash rate of the app. By leveraging the tools and infrastructure that we are already comfortable with from our web app we allow every engineer to take part in the monitoring and diagnosing of iOS app problems.
Written by Brandon Williams.
