Publication - Streaming versus Batch Processing of Sensor Data in a Hazardous Weather Detection System
| Authors: | Sims, Mark; Kurose, Jim; and Lesser, Victor | ||||
| Title: | Streaming versus Batch Processing of Sensor Data in a Hazardous Weather Detection System | ||||
| Abstract: | We present several discrete-time Markov queuing models to compare the performance of batch versus streaming processing of sensor data in a weather detection and monitoring system architecture. The first model assumes independent arrivals and illustrates the average case behavior of the system. The remaining models assume correlated arrivals and demonstrate how different scan strategies across multiple elevations impact system performance. We also show how the models are useful in dimensioning the computational resources of the system given workload arrival characteristics. We apply the models to several hypothetical scenarios with varying weather feature arrival and processing rates. We also evaluate our models using processing runtime data obtained by running two NEXRAD algorithms on weather data from six airports. Our results show that for this particular application and reasonable system utilizations, delaying processing start time to perform batch processing does not adversely affect system performance or require significant, additional computational resources. More generally, our models show how system performance is dependent on the scan strategy and the burstiness of the sensor data. | ||||
| Publication: | Proceedings of Second Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks (SECON 2005), pp. 185 - 196 | ||||
| Date: | September 2005 | ||||
| Sources: |
PS: /Documents/msims/simsSecon.ps.gz PDF: /Documents/msims/simsSecon.pdf |
||||
| Reference: | Sims, Mark; Kurose, Jim; and Lesser, Victor. Streaming versus Batch Processing of Sensor Data in a Hazardous Weather Detection System. Proceedings of Second Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks (SECON 2005), pp. 185-196. September 2005. | ||||
| bibtex: | @inproceedings{Sims-387,
author = "Mark Sims and Jim Kurose and Victor Lesser",
title = "{Streaming versus Batch Processing of Sensor Data
in a Hazardous Weather Detection System}",
booktitle = "Proceedings of Second Annual IEEE Communications
Society Conference on Sensor and Ad Hoc
Communications and Networks (SECON 2005)",
pages = "185-196",
month = "September",
year = "2005",
url = "http://mas.cs.umass.edu/paper/387",
}
|
||||
