Junxian Huang, Feng Qian, Alexandre Gerber, Z. Morley Mao, Subhabrata Sen and Oliver Spatscheck
With the recent advent of 4G LTE networks, there has been increasing interest to better understand the performance and power characteristics, compared with 3G/WiFi networks. In this paper, we take one of the first steps in this direction.
Using a publicly deployed tool we designed for Android called 4GTest attracting more than 3000 users within 2 months and extensive local experiments, we study the network performance of LTE networks and compare with other types of mobile networks. We observe LTE generally has significantly higher downlink and uplink throughput than 3G and even WiFi, with a median value of 13Mbps and 6Mbps, respectively. We develop the first empirically derived comprehensive power model of a commercial LTE network with less than 6% error rate and state transitions matching the specifications. Using a comprehensive data set consisting of 5-month traces of 20 smartphone users, we carefully investigate the energy usage in 3G, LTE, and WiFi networks and evaluate the impact of configuring LTE-related parameters. Despite several new power saving improvements, we find that LTE is as much as 23 times less power efficient compared with WiFi, and even less power efficient than 3G, based on the user traces and the long high power tail is found to be a key contributor. In addition, we perform case studies of several popular applications on Android in LTE and identify that the performance bottleneck for web-based applications lies less in the network, compared to our previous study in 3G. Instead, the device's processing power, despite the significant improvement compared to our analysis two years ago, becomes more of a bottleneck.
PDF download link
Public Review uploaded by SharadAgarwal:
This public review was written by Sharad Agarwal.
A large part of the variation in performance that mobile app users experience is due to the underlying cellular connection. In addition, hidden from the user is the impact of this connectivity on the phone's battery. How can I build robust apps or systems in the face of such variance? I need a detailed understanding of the performance and power characteristics of such connections.
The authors have updated their previous IMC 2010 and MobiSys 2010 papers which studied 3G power and performance, to that for LTE in this paper. This paper studies LTE configuration parameters and performance and energy states from the textbook to the lab to live experiments to crowd-sourced data from 20 users across 5 months. In this paper, you will learn about how operator settings can control the transitions between different power states, what is the battery cost of each state, the throughput and latency that users experience today, and how power models can be used to compare 3G, 4G and Wi-Fi consumption.
There are a few weaknesses that the reviewers pointed out. This is primarily a measurement paper, and it may be a too early -- LTE just came to the US market and the performance numbers and usage could change. The comparison between 3G, 4G and Wi-Fi is not perfect -- replaying traces with TCP sessions in them is tricky; while the evaluation of LTE and 3G includes promotion time, Wi-Fi does not include association time; one configuration of 3G is compared to a different one for 4G.
Beyond these issues, the PC agreed that there's a lot to like about this detailed study. Mobile computing is a rapidly evolving field, and we should continue to update our understanding of important issues as underlying technologies change.
We want to thank Sharad Agarwal and other anonymous reviewers for their thoughtful and helpful comments. Our goal in this work is to understand the network performance characteristics, develop the power model and investigate the interaction of mobile applications with the network for LTE, as well as to provide a comparison with 3G and WiFi. However, this work is not just reapplying the methodology of our previous IMC 2010 and MobiSys 2010 works to the new LTE network. Instead, we have devised new methodology to model the energy usage in LTE network, specifically, we have a fine-grained power model considering the instant data rate and achieves more than 94% accuracy. We also use a comprehensive data set from 20 users to compare the interaction of mobile applications with different network state machines, with a fine-grained network energy usage break down. These are the example novel aspects of this work, going beyond merely applying previous studies for LTE.
LTE is an emerging technology and has recently appeared in the commercial markets. While we agree that the performance and measurement results could change over time, we don't think our study is too early. Our work is intended to guide ISPs and application developers to gain more understanding of the complex interaction between the network, user, and application with energy, CPU and network resources considered. This is very important especially in the early stage of a new technology, and discovering problems earlier is also good for the evolution of LTE.
As a part of this work, we use the same set of network traces and replayed them in LTE/3G/WiFi network and power models. This is indeed not perfect since different networks have different network characteristics. However, this provides us a fair comparison for the three network types for energy efficiency. Even with access to real user traces collected in LTE it is still challenging to carry out the comparison. For example, if we compare the LTE traces in LTE model and WiFi traces in WiFi model, the applications and contents of the two traces are completely different, and the comparison may not be fair.
While we consider 3G/LTE promotions, this is mainly due to the signaling overhead for the ISPs, in addition to some delay and energy overhead to the UE. We do not consider WiFi association time in this work, because we observe that in our experiments, it happens much less often than LTE/3G promotion and its energy overhead is also quite small.
In this study, we use the measured configuration for a commercial ISP for LTE/3G as a representative setting. We also selectively studied some key parameters for LTE network with varying values. 3G and LTE configurations vary across countries and carriers, and it is inherently difficult for us to consider all possible configurations. Instead, in this study, we choose one major carrier in US to present a representative comparison.