A 1-D CNN inference engine for constrained platforms

Abstract:

1D-CNNs are used for time series classification in various domains with a high degree of accuracy. Most implementations collect the incoming data samples in a buffer before performing inference on them. On edge devices, which are typically constrained and single-threaded, such an implementation may interfere with time-critical tasks, such as sample acquisition. In this work, we propose TICO, an inference scheme that interleaves the convolution operations between sample intervals, allowing us to reduce the inference latency. Furthermore, our scheme is well-suited for storing data in ring buffers, yielding a small memory footprint. We demonstrate these improvements by comparing TICO to TFLite’s inference method for a 4-layer CNN, giving an 11% reduction in the inference delay while almost halving the memory usage. Our approach is feasible on common consumer devices such as the Arduino Nano BLE 33 and Arduino AVR series, which we show by performing fence intrusion classification on these devices