Last week I conducted an O'Reilly webinar with mobile developer Max Firtman. If you missed it you can replay the event here: Become a Mobile Application Performance Genius. It covers how important, and how variable, network requests made by an app are when it comes to app performance. And how even on high speed mobile networks, latency is still much higher than we are used to in the wired or wifi world, and it becomes a big factor in mobile app performance.
One of the most interesting parts of participating in large webinars like this is the opportunity to get some market feedback via polls asked of the audience during the webinar. The two polls we presented were about the level and type of performance testing the attendees were doing, and about the level and type of performance monitoring they were doing after the mobile app was deployed. The results were quite interesting:
Only about 15% were doing performance testing on multiple devices and multiple network types, and only one third of those were testing varying signal strength. But for anyone doing mobile app development, it is probably not a surprise either. Effective performance testing of mobile apps is hard and expensive to do. The permutations of devices, network types, signal strength compound exponentially to the point where you have to make some bounding assumptions to fit into your testing budget and time.
The second poll was about how attendees were measuring performance in the field after the app was released:
Think about that. Over half are using their customers as their app performance monitor, and in a very public way! A total of 65% have no quantitative real user monitoring at all. It is not a surprise that there are a lot of poor performing mobile apps out there!
So if you fall into one of the blue categories listed and are having performance problems with your mobile app, take some solace in these results: you are not alone. But that doesn't solve the problem for you either. More performance testing always helps, but because of the exponential costs, I think performance monitoring typically delivers more value for the time and money invested. Optimally you should use both together. Performance test as much as you can, and then let a mobile APM solution quickly identify any "real world" problems that your tests are not able cover.