skip to main content
10.1145/3666025.3699335acmconferencesArticle/Chapter ViewAbstractPublication PagessensysConference Proceedingsconference-collections
research-article
Open access

Fast-Inf: Ultra-Fast Embedded Intelligence on the Batteryless Edge

Published: 04 November 2024 Publication History

Abstract

Batteryless edge devices are extremely resource-constrained compared to traditional mobile platforms. Existing tiny deep neural network (DNN) inference solutions are problematic due to their slow and resource-intensive nature, rendering them unsuitable for batteryless edge devices. To address this problem, we propose a new approach to embedded intelligence, called Fast-Inf, which achieves extremely lightweight computation and minimal latency. Fast-Inf uses binary tree-based neural networks that are ultra-fast and energy-efficient due to their logarithmic time complexity. Additionally, Fast-Inf models can skip the leaf nodes when necessary, further minimizing latency without requiring any modifications to the model or retraining. Moreover, Fast-Inf models have significantly lower backup and runtime memory overhead. Our experiments on an MSP430FR5994 platform showed that Fast-Inf can achieve ultra-fast and energy-efficient inference (up to 700x speedup and reduced energy) compared to a conventional DNN.

References

[1]
Joshua Adkins, Branden Ghena, Neal Jackson, Pat Pannuto, Samuel Rohrer, Bradford Campbell, and Prabal Dutta. 2018. The signpost platform for city-scale sensing. In Proceedings of the 17th ACM/IEEE International Conference on Information Processing in Sensor Networks. IEEE, New York, NY, USA, 188--199.
[2]
Mikhail Afanasov, Naveed Anwar Bhatti, Dennis Campagna, Giacomo Caslini, Fabio Massimo Centonze, Koustabh Dolui, Andrea Maioli, Erica Barone, Muhammad Hamad Alizai, Junaid Haroon Siddiqui, et al. 2020. Battery-less zero-maintenance embedded sensing at the mithræum of circus maximus. In Proceedings of the 18th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 368--381.
[3]
Saad Ahmed, Bashima Islam, Kasim Sinan Yildirim, Marco Zimmerling, Przemysław Pawełczak, Muhammad Hamad Alizai, Brandon Lucia, Luca Mottola, Jacob Sorber, and Josiah Hester. 2024. The Internet of Batteryless Things. Commun. ACM 67, 3 (2024), 64--73.
[4]
Khakim Akhunov and Kasim Sinan Yildirim. 2022. AdaMICA: Adaptive Multicore Intermittent Computing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1--30.
[5]
Abu Bakar, Rishabh Goel, Jasper de Winkel, Jason Huang, Saad Ahmed, Bashima Islam, Przemysław Pawełczak, Kasim Sinan Yildirim, and Josiah Hester. 2022. Protean: An energy-efficient and heterogeneous platform for adaptive and hardware-accelerated battery-free computing. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 207--221.
[6]
Abu Bakar, Tousif Rahman, Alessandro Montanari, Jie Lei, Rishad Shafik, and Fahim Kawsar. 2022. Logic-based intelligence for batteryless sensors. In Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications. ACM, New York, NY, USA, 22--28.
[7]
Abu Bakar, Alexander G Ross, Kasim Sinan Yildirim, and Josiah Hester. 2021. REHASH: A Flexible, Developer Focused, Heuristic Adaptation Platform for Intermittently Powered Computing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1--42.
[8]
Peter Belcak and Roger Wattenhofer. 2023. Fast Feedforward Networks. arXiv preprint arXiv:2308.14711.
[9]
Emmanuel Bengio, Pierre-Luc Bacon, Joelle Pineau, and Doina Precup. 2015. Conditional computation in neural networks for faster models. arXiv preprint arXiv:1511.06297.
[10]
Sourav Bhattacharya and Nicholas D Lane. 2016. Sparsification and separation of deep learning layers for constrained resource inference on wearables. In Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 176--189.
[11]
Naveed Anwar Bhatti and Luca Mottola. 2017. HarvOS: Efficient code instrumentation for transiently-powered embedded sensing. In Proceedings of the 16th ACM/IEEE International Conference on Information Processing in Sensor Networks. IEEE, New York, NY, USA, 209--219.
[12]
Sizhen Bian and Michele Magno. 2023. Evaluating Spiking Neural Network on Neuromorphic Platform For Human Activity Recognition. In Proceedings of the 2023 ACM International Symposium on Wearable Computers. ACM, New York, NY, USA, 82--86.
[13]
Leo Breiman. 2017. Classification and regression trees. Routledge, London, UK.
[14]
Han Cai, Ji Lin, Yujun Lin, Zhijian Liu, Haotian Tang, Hanrui Wang, Ligeng Zhu, and Song Han. 2022. Enable deep learning on mobile devices: Methods, systems, and applications. ACM Transactions on Design Automation of Electronic Systems 27, 3 (2022), 1--50.
[15]
Luca Caronti, Khakim Akhunov, Matteo Nardello, Kasim Sinan Yildinm, and Davide Brunelli. 2023. Fine-grained hardware acceleration for efficient batteryless intermittent inference on the edge. ACM Transactions on Embedded Computing Systems 22, 5 (2023), 1--19.
[16]
Gianmarco Cerutti, Renzo Andri, Lukas Cavigelli, Elisabetta Farella, Michele Magno, and Luca Benini. 2020. Sound event detection with binary neural networks on tightly power-constrained IoT devices. In Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design. IEEE, New York, NY, USA, 19--24.
[17]
Gianmarco Cerutti, Lukas Cavigelli, Renzo Andri, Michele Magno, Elisabetta Farella, and Luca Benini. 2022. Sub-mW keyword spotting on an MCU: Analog binary feature extraction and binary neural networks. IEEE Transactions on Circuits and Systems I: Regular Papers 69, 5 (2022), 2002--2012.
[18]
Jongouk Choi, Larry Kittinger, Qingrui Liu, and Changhee Jung. 2022. Compiler-directed high-performance intermittent computation with power failure immunity. In Proceedings of the IEEE 28th Real-Time and Embedded Technology and Applications Symposium. IEEE, New York, NY, USA, 40--54.
[19]
Alexei Colin and Brandon Lucia. 2016. Chain: tasks and channels for reliable intermittent programs. In Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications. ACM, New York, NY, USA, 514--530.
[20]
Elia Cunegatti, Doina Bucur, and Giovanni Iacca. 2023. Peeking inside Sparse Neural Networks using Multi-Partite Graph Representations. arXiv preprint arXiv:2305.16886.
[21]
Leonardo Lucio Custode, Pietro Farina, Eren Yildiz, Renan Beran Kilic, Kasim Sinan Yildirim, and Giovanni Iacca. 2024. Github Repo. https://github.com/DIOL-UniTN/Fast-Inf-FFF.
[22]
Leonardo Lucio Custode and Giovanni Iacca. 2022. Interpretable pipelines with evolutionary optimized modules for reinforcement learning tasks with visual inputs. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, New York, NY, USA, 224--227.
[23]
Jasper De Winkel, Vito Kortbeek, Josiah Hester, and Przemysław Pawełczak. 2020. Battery-free game boy. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--34.
[24]
Li Deng. 2012. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine 29, 6 (2012), 141--142.
[25]
Harsh Desai, Matteo Nardello, Davide Brunelli, and Brandon Lucia. 2022. Camaroptera: A long-range image sensor with local inference for remote sensing applications. ACM Transactions on Embedded Computing Systems 21, 3 (2022), 1--25.
[26]
Digilent. [n. d.]. Analog Discovery 2 (Legacy). https://digilent.com/reference/test-and-measurement/analog-discovery-2/start
[27]
Ferhat Erata, Eren Yildiz, Arda Goknil, Kasim Sinan Yildirim, Jakub Szefer, Ruzica Piskac, and Gokcin Sezgin. 2023. ETAP: Energy-aware timing analysis of intermittent programs. ACM Transactions on Embedded Computing Systems 22, 2 (2023), 1--31.
[28]
Pietro Farina, Subrata Biswas, Eren Yildiz, Khakim Akhunov, Saad Ahmed, Bashima Islam, and Kasim Sinan Yildirim. 2024. Memory-efficient Energy-adaptive Inference of Pre-Trained Models on Batteryless Embedded Systems. arXiv preprint arXiv:2405.10426.
[29]
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. 2020. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning. PMLR, Vienna, Austria, 3259--3269.
[30]
Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a Neural Network Into a Soft Decision Tree. arXiv preprint arXiv:1711.09784.
[31]
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2022. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision. Chapman and Hall/CRC, Boca Raton, FL, USA, 291--326.
[32]
Graham Gobieski, Brandon Lucia, and Nathan Beckmann. 2019. Intelligence beyond the edge: Inference on intermittent embedded systems. In Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, New York, NY, USA, 199--213.
[33]
Arda Goknil and Kasim Sinan Yildirim. 2022. Toward Sustainable IoT Applications: Unique Challenges for Programming the Batteryless Edge. IEEE Software 39, 5 (2022), 92--100.
[34]
Sridhar Gopinath, Nikhil Ghanathe, Vivek Seshadri, and Rahul Sharma. 2019. Compiling KB-sized machine learning models to tiny IoT devices. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, New York, NY, USA, 79--95.
[35]
Song Han, Huizi Mao, and William J Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv preprint arXiv:1510.00149.
[36]
Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems 28 (2015), 9.
[37]
Josiah Hester and Jacob Sorber. 2017. Flicker: Rapid Prototyping for the Batteryless Internet-of-Things. In Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 1--13.
[38]
Josiah Hester, Kevin Storer, and Jacob Sorber. 2017. Timely execution on intermittently powered batteryless sensors. In Proceedings of the 15th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 1--13.
[39]
Andrey Ignatov. 2017. Real-time human activity recognition from accelerometer data using Convolutional Neural Networks. https://github.com/aiff22/HAR
[40]
Texas Instruments Inc. 2017. MSP430FR59xx Mixed-Signal Microcontrollers (Rev. F). https://www.ti.com/lit/ds/symlink/msp430fr5969.pdf
[41]
Texas Instruments Inc. 2019. EnergyTrace User Guide. https://software-dl.ti.com/simplelink/esd/simplelink_cc13x2_26x2_sdk/4.10.00.78/exports/docs/ble5stack/ble_user_guide/html/energy-trace/energy-trace.html
[42]
Texas Instruments Inc. 2020. FRAM FAQs. https://www.ti.com/lit/wp/slat151/slat151.pdf
[43]
Infineon. 2020. 8MB EXCELON LP Ferroelectric RAM. https://www.infineon.com/dgdl/Infineon-CY15B108QN_CY15V108QN_Excelon(TM)_LP_8-Mbit_(1024K_X_8)_Serial_(SPI)_F-RAM-DataSheet-v10_00-EN.pdf?fileId=8ac78c8c7d0d8da4017d0ee7134b6ff4
[44]
Texas Instruments. 2016. Low-Energy Accelerator (LEA) Frequently Asked Questions (FAQ). https://www.ti.com/lit/an/slaa720/slaa720.pdf
[45]
Bashima Islam and Shahriar Nirjon. 2020. Zygarde: Time-Sensitive On-Device Deep Inference and Adaptation on Intermittently-Powered Systems. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--29.
[46]
Seunghyeok Jeon, Yonghun Choi, Yeonwoo Cho, and Hojung Cha. 2023. HarvNet: Resource-Optimized Operation of Multi-Exit Deep Neural Networks on Energy Harvesting Devices. In Proceedings of the 21st International Conference on Mobile Systems, Applications, and Services. ACM, New York, NY, USA, 42--55.
[47]
Jeff Johnson. 2018. Rethinking floating point for deep learning. arXiv preprint arXiv:1811.01721.
[48]
Kyle Johnson, Zachary Englhardt, Vicente Arroyos, Dennis Yin, Shwetak Patel, and Vikram Iyer. 2023. MilliMobile: An Autonomous Battery-free Wireless Microrobot. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. ACM, New York, NY, USA, 1--16.
[49]
Chih-Kai Kang, Hashan Roshantha Mendis, Chun-Han Lin, Ming-Syan Chen, and Pi-Cheng Hsiu. 2020. Everything leaves footprints: Hardware accelerated intermittent deep inference. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 11 (2020), 3479--3491.
[50]
Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin. 2015. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530.
[51]
Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980.
[52]
Vito Kortbeek, Kasim Sinan Yildirim, Abu Bakar, Jacob Sorber, Josiah Hester, and Przemysław Pawełczak. 2020. Time-sensitive intermittent computing meets legacy software. In Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, New York, NY, USA, 85--99.
[53]
Seulki Lee and Shahriar Nirjon. 2019. Neuro.ZERO: a zero-energy neural network accelerator for embedded sensing and inference systems. In Proceedings of the 17th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 138--152.
[54]
Seulki Lee and Shahriar Nirjon. 2020. Fast and scalable in-memory deep multitask learning via neural weight virtualization. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. ACM, New York, NY, USA, 175--190.
[55]
Seulki Lee and Shahriar Nirjon. 2022. Weight Separation for Memory-Efficient and Accurate Deep Multitask Learning. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications. IEEE, New York, NY, USA, 13--22.
[56]
Ji Lin, Wei-Ming Chen, Yujun Lin, Chuang Gan, Song Han, et al. 2020. MCUNet: Tiny Deep Learning on IoT Devices. Advances in Neural Information Processing Systems 33 (2020), 11711--11722.
[57]
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270.
[58]
Kiwan Maeng, Alexei Colin, and Brandon Lucia. 2017. Alpaca: Intermittent execution without checkpoints. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1--30.
[59]
Amjad Yousef Majid, Carlo Delle Donne, Kiwan Maeng, Alexei Colin, Kasim Sinan Yildirim, Brandon Lucia, and Przemysław Pawełczak. 2020. Dynamic task-based intermittent execution for energy-harvesting devices. ACM Transactions on Sensor Networks 16, 1 (2020), 1--24.
[60]
Mahathir Monjur, Yubo Luo, Zhenyu Wang, and Shahriar Nirjon. 2023. Sound-Sieve: Seconds-Long Audio Event Recognition on Intermittently-Powered Systems. In Proceedings of the 21st International Conference on Mobile Systems, Applications, and Services. ACM, New York, NY, USA, 28--41.
[61]
Alessandro Montanari, Manuja Sharma, Dainius Jenkus, Mohammed Alloulah, Lorena Qendro, and Fahim Kawsar. 2020. ePerceptive: energy reactive embedded intelligence for batteryless sensors. In Proceedings of the 18th ACMConference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 382--394.
[62]
Tushar S Muratkar, Ankit Bhurane, and Ashwin Kothari. 2020. Battery-less internet of things---A survey. Computer Networks 180 (2020), 107385.
[63]
Powercast. 2016. The Powercast P2110B Powerharvester. https://www.powercastco.com/wp-content/uploads/2016/12/P2110B-Datasheet-Rev-3.pdf
[64]
Powercast. 2018. The Powercast TX91501B Powercaster. https://www.powercastco.com/wp-content/uploads/2019/10/User-Manual-TX-915-01B-Rev-A-1.pdf
[65]
Shvetank Prakash, Matthew Stewart, Colby Banbury, Mark Mazumder, Pete Warden, Brian Plancher, and Vijay Janapa Reddi. 2023. Is TinyML Sustainable? Commun. ACM 66, 11 (2023), 68--77.
[66]
Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, and Nicu Sebe. 2020. Binary neural networks: A survey. Pattern Recognition 105 (2020), 107281.
[67]
Benjamin Ransford and Brandon Lucia. 2014. Nonvolatile memory is a broken time machine. In Proceedings of the Workshop on Memory Systems Performance and Correctness. ACM, New York, NY, USA, 1--3.
[68]
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision. Springer, Cham, Switzerland, 525--542.
[69]
Teodora Sanislav, George Dan Mois, Sherali Zeadally, and Silviu Corneliu Folea. 2021. Energy harvesting techniques for internet of things (IoT). IEEE Access 9 (2021), 39530--39549.
[70]
Nathan Serafin, Souradip Ghosh, Harsh Desai, Nathan Beckmann, and Brandon Lucia. 2023. Pipestitch: An energy-minimal dataflow architecture with lightweight threads. In 2023 56th IEEE/ACM International Symposium on Microarchitecture (MICRO). ACM, New York, NY, USA, 14.
[71]
Bharath Sudharsan, Sonu Prasad, Dan Jose, and John G Breslin. 2022. TMM-TinyML: tensor memory mapping (TMM) method for tiny machine learning (TinyML). In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. ACM, New York, NY, USA, 865--867.
[72]
Bharath Sudharsan, Simone Salerno, and Rajiv Ranjan. 2022. TinyML-CAM: 80 FPS image recognition in 1 kB RAM. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. ACM, New York, NY, USA, 862--864.
[73]
Cheng Tai, Tong Xiao, Yi Zhang, Xiaogang Wang, et al. 2015. Convolutional neural networks with low-rank regularization. arXiv preprint arXiv:1511.06067.
[74]
Pete Warden. 2018. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209.
[75]
Pete Warden and Daniel Situnayake. 2019. TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. O'Reilly Media, Sebastopol, CA, USA.
[76]
Harrison Williams and Matthew Hicks. 2024. Energy-adaptive Buffering for Efficient, Responsive, and Persistent Batteryless Systems. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. 268--282.
[77]
Fan Yang, Ashok Samraj Thangarajan, Sam Michiels, Wouter Joosen, and Danny Hughes. 2021. Morphy: Software defined charge storage for the iot. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 248--260.
[78]
Li Yang and Abdallah Shami. 2020. On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing 415 (2020), 295--316.
[79]
Kasim Sinan Yildirim, Amjad Yousef Majid, Dimitris Patoukas, Koen Schaper, Przemysław Pawełczak, and Josiah Hester. 2018. Ink: Reactive kernel for tiny batteryless sensors. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. ACM, New York, NY, USA, 41--53.
[80]
Eren Yildiz, Saad Ahmed, Bashima Islam, Josiah Hester, and Kasim Sinan Yildirim. 2023. Efficient and Safe I/O Operations for Intermittent Systems. In Proceedings of the 18th European Conference on Computer Systems. ACM, New York, NY, USA, 63--78.
[81]
Eren Yildiz, Lijun Chen, and Kasim Sinan Yildirim. 2022. Immortal Threads: Multithreaded Event-driven Intermittent Computing on {Ultra-Low-Power} Microcontrollers. In Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, USA, 339--355.
[82]
Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang. 2019. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE 107, 8 (2019), 1738--1762.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SenSys '24: Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems
November 2024
950 pages
ISBN:9798400706974
DOI:10.1145/3666025
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2024

Check for updates

Author Tags

  1. batteryless embedded systems
  2. fast feedforward networks

Qualifiers

  • Research-article

Funding Sources

Conference

Acceptance Rates

Overall Acceptance Rate 174 of 867 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 432
    Total Downloads
  • Downloads (Last 12 months)432
  • Downloads (Last 6 weeks)166
Reflects downloads up to 05 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media