Cascaded Classifier for Pareto-Optimal Accuracy-Cost Trade-Off Using Off-the-Shelf ANNs | SpringerLink
Skip to main content
Springer Nature Link
Log in

Cascaded Classifier for Pareto-Optimal Accuracy-Cost Trade-Off Using Off-the-Shelf ANNs

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13164))

  • 1981 Accesses

Abstract

Machine-learning classifiers provide high quality of service in classification tasks. Research now targets cost reduction measured in terms of average processing time or energy per solution. Revisiting the concept of cascaded classifiers, we present a first of its kind analysis of optimal pass-on criteria between the classifier stages. Based on this analysis, we derive a methodology to maximize accuracy and efficiency of cascaded classifiers. On the one hand, our methodology allows cost reduction of 1.32\(\times \) while preserving reference classifier’s accuracy. On the other hand, it allows to scale cost over two orders while gracefully degrading accuracy. Thereby, the final classifier stage sets the top accuracy. Hence, the multi-stage realization can be employed to optimize any state-of-the-art classifier.

C. Latotzke and J. Loh—Contribute equally to the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 9151
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 11439
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Note that the width and depth of the ANNs used for this case study are adjusted for the CIFAR 10 dataset.

References

  1. ImageNet. https://www.image-net.org/. Accessed 01 Feb 2021

  2. Badami, K., Lauwereins, S., Meert, W., Verhelst, M.: Context-aware hierarchical information-sensing in a 6\(\upmu \)W 90nm CMOS voice activity detector. In: ISSCC, vol. 58, pp. 430–432 (2015). https://doi.org/10.1109/ISSCC.2015.7063110

  3. Benbasat, A.Y., Paradiso, J.A.: A framework for the automated generation of power-efficient classifiers for embedded sensor nodes. In: SenSys, vol. 5, pp. 219–232 (2007). https://doi.org/10.1145/1322263.1322285

  4. Bergstra, J., Yamins, D., Cox, D.D.: Hyperopt: a python library for optimizing the hyperparameters of machine learning algorithms. In: Proceedings of the 12th Python in Science Conference, vol. 13, pp. 13–19 (2013)

    Google Scholar 

  5. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. CoRR, abs/1605.07678 (2017)

    Google Scholar 

  6. Cocaña-Fernández, A., Ranilla, J., Gil-Pita, R., Sánchez, L.: Energy-conscious fuzzy rule-based classifiers for battery operated embedded devices. In: FUZZ-IEEE, pp. 1–6 (2017). https://doi.org/10.1109/FUZZ-IEEE.2017.8015483

  7. Courbariaux, M., Bengio, Y.: BinaryNet: training deep neural networks with weights and activations constrained to +1 or -1. CoRR, abs/1602.02830 (2016)

    Google Scholar 

  8. Goens, A., Brauckmann, A., Ertel, S., Cummins, C., Leather, H., Castrillon, J.: A case study on machine learning for synthesizing benchmarks. In: MAPL, pp. 38–46 (2019). https://doi.org/10.1145/3315508.3329976

  9. Goetschalckx, K., Moons, B., Lauwereins, S., Andraud, M., Verhelst, M.: Optimized hierarchical cascaded processing. JETCAS 8, 884–894 (2018). https://doi.org/10.1109/JETCAS.2018.2839347

    Article  Google Scholar 

  10. Horowitz, M.: 1.1 Computing’s energy problem (and what we can do about it). In: ISSCC, pp. 10–14 (2014). https://doi.org/10.1109/ISSCC.2014.6757323

  11. Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: CVPR, pp. 2261–2269 (2016). https://doi.org/10.1109/CVPR.2017.243

  12. Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: CVPR, pp. 2372–2379 (2009). https://doi.org/10.1109/CVPR.2009.5206627

  13. Kouris, A., Venieris, S.I., Bouganis, C.: Cascade CNN: pushing the performance limits of quantisation in convolutional neural networks. In: FPL, pp. 155–1557 (2018). https://doi.org/10.1109/FPL.2018.00034

  14. Krizhevsky, A., Nair, V., Hinton, G.: CIFAR-10 (Canadian Institute for Advanced Research). http://www.cs.toronto.edu/~kriz/cifar.html

  15. Latotzke, C., Gemmeke, T.: Efficiency versus accuracy: a review of design techniques for DNN hardware accelerators. IEEE Access 9, 9785–9799 (2021). https://doi.org/10.1109/ACCESS.2021.3050670

    Article  Google Scholar 

  16. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  17. Lecun, Y., Cortes, C., Burges, C.: The MNIST Database of handwritten digits. http://yann.lecun.com/exdb/mnist/

  18. Li, L., Topkara, U., Coskun, B., Memon, N.: CoCoST: a computational cost efficient classifier. In: ICDM, pp. 268–277 (2009). https://doi.org/10.1109/ICDM.2009.46

  19. Lin, Z., Memisevic, R., Konda, K.R.: How far can we go without convolution: improving fully-connected networks. CoRR, abs/1511.02580 (2015)

    Google Scholar 

  20. Ouali, M., King, R.D.: Cascaded multiple classifiers for secondary structure prediction. Protein Sci., 1162–1176 (2000). https://doi.org/10.1110/ps.9.6.1162

  21. Price, M., Glass, J., Chandrakasan, A.P.: A scalable speech recognizer with deep-neural-network acoustic models and voice-activated power gating. In: ISSCC, pp. 244–245 (2017). https://doi.org/10.1109/ISSCC.2017.7870352

  22. Rossi, D., et al.: 4.4 A 1.3TOPS/W @ 32GOPS fully integrated 10-core SoC for IoT end-nodes with 1.7\(\upmu \)W cognitive wake-up from MRAM-based state-retentive sleep mode. In: ISSCC, vol. 64, pp. 60–62 (2021). https://doi.org/10.1109/ISSCC42613.2021.9365939

  23. Stadtmann, T., Latotzke, C., Gemmeke, T.: From quantitative analysis to synthesis of efficient binary neural networks. In: ICMLA, vol. 19, pp. 93–100 (2020). https://doi.org/10.1109/ICMLA51294.2020.00024

  24. Venkataramani, S., Raghunathan, A., Liu, J., Shoaib, M.: Scalable-effort classifiers for energy-efficient machine learning. In: DAC, vol. 67, pp. 1–6 (2015). https://doi.org/10.1145/2744769.2744904

  25. Xu, Z., Kusner, M., Weinberger, K., Chen, M.: Cost-sensitive tree of classifiers. In: PMLR, vol. 28, no. 1, pp. 133–141 (2013)

    Google Scholar 

  26. Xu, Z., Kusner, M.J., Weinberger, K.Q., Chen, M., Chapelle, O.: Classifier cascades and trees for minimizing feature evaluation cost. JMLR 15(1), 2113–2144 (2014)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgment

This work was partially funded by the German BMBF project NEUROTEC under grant no. 16ES1134.

Author information

Authors and Affiliations

  1. RWTH Aachen University, 52062, Aachen, Germany

    Cecilia Latotzke, Johnson Loh & Tobias Gemmeke

Authors
  1. Cecilia Latotzke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Johnson Loh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tobias Gemmeke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cecilia Latotzke .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. Department of Computer Science, University of Reading, Reading, UK

    Varun Ojha

  3. Department of Computer Science, University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. Cambridge Judge Business School, University of Cambridge, Cambridge, UK

    Gabriele La Malfa

  5. Department of Biochemistry, University of Cambridge, Cambridge, UK

    Giorgio Jansen

  6. Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  7. University of Catania, Catania, Italy

    Giovanni Giuffrida

  8. Department of Informatics, Dana-Farber Cancer Institute, Boston, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Latotzke, C., Loh, J., Gemmeke, T. (2022). Cascaded Classifier for Pareto-Optimal Accuracy-Cost Trade-Off Using Off-the-Shelf ANNs. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2021. Lecture Notes in Computer Science(), vol 13164. Springer, Cham. https://doi.org/10.1007/978-3-030-95470-3_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95470-3_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95469-7

  • Online ISBN: 978-3-030-95470-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation