Acta Metallurgica Sinica(English letters) ›› 2014, Vol. 21 ›› Issue (1): 60-66.doi: 10.1016/S1005-8885(14)60269-7

• Networks • Previous Articles     Next Articles

Sensor data compression based on MapReduce

Yu Yu1,   

  1. Department of Computer Science and Engineer, Ocean University of China, Qingdao 266100, China
  • Received:2013-08-09 Revised:2014-01-05 Online:2014-02-28 Published:2014-02-28
  • Supported by:

    This work was supported by the National Natural Science Foundation of China (60933011, 61170258).

Abstract:

A compression algorithm is proposed in this paper for reducing the size of sensor data. By using a dictionary-based lossless compression algorithm, sensor data can be compressed efficiently and interpreted without decompressing. The correlation between redundancy of sensor data and compression ratio is explored. Further, a parallel compression algorithm based on MapReduce [1] is proposed. Meanwhile, data partitioner which plays an important role in performance of MapReduce application is discussed along with performance evaluation criteria proposed in this paper. Experiments demonstrate that random sampler is suitable for highly redundant sensor data and the proposed compression algorithms can compress those highly redundant sensor data efficiently.

Key words:

data compression, sensor data, MapReduce, surveillance application, measurement system

CLC Number: