A new methodology for evaluating incident detection algorithms |
| |
Institution: | 1. Bascom Palmer Eye Institute, Miami, Florida, USA;2. University of California San Francisco, San Francisco, California, USA;3. University of Miami Miller School of Medicine, Miami, Florida, USA;4. American Academy of Ophthalmology, San Francisco, California;1. Stuttgart Media University, Nobelstraße 10, Stuttgart 70569, Germany;2. Institute for Natural Language Processing (IMS), University of Stuttgart, Pfaffenwaldring 5b, Stuttgart 70569, Germany |
| |
Abstract: | We present a novel, off-line approach for evaluating incident detection algorithms. Previous evaluations have focused on determining the detection rate versus false alarm rate curve––a process which we argue is inherently fraught with difficulties. Instead, we propose a cost-benefit analysis where cost mimics the real costs of implementing the algorithm and benefit is in terms of reduction in congestion. We argue that these quantities are of more practical interest than the traditional rates. Moreover, these costs, estimated on training data, can be used both as a mechanism to fine tune a single algorithm as well as a meaningful quantity for direct comparisons between different types of incident detection algorithms. We demonstrate our approach with a detailed example. |
| |
Keywords: | |
本文献已被 ScienceDirect 等数据库收录! |
|