The rapid proliferation of intelligent edge devices has intensified the demand for real-time artificial intelligence (AI) inference under strict constraints on latency, energy consumption, and hardware resources. Conventional cloud-based inference and general-purpose processors are increasingly inadequate for meeting these requirements, particularly in latency-critical and power-constrained edge environments. This paper presents the design, implementation, and evaluation of a custom framework for hardware acceleration of AI inference at the edge, emphasizing energy efficiency and hardware–software co-design. The proposed system adopts a heterogeneous architecture that integrates a general-purpose host processor with a specialized hardware accelerator optimized for neural network inference. Lightweight convolutional neural networks representative of edge workloads are mapped onto the accelerator using a dataflow-oriented execution model, low-precision arithmetic, and a multi-level memory hierarchy to minimize data movement. A hardware-software co-design approach is used to make sure that model architectures, compilation strategies, and runtime execution all work with the capabilities of the accelerator. The accelerator is prototyped on an FPGA-based edge platform and evaluated using real-time inference benchmarks. Experimental results show that inference latency and energy use are much lower than with CPU- and GPU-based systems. This means that the system works better per watt while staying within tight edge power budgets. The findings further reveal critical trade-offs between inference accuracy, latency, power consumption, and architectural flexibility, and they confirm the effectiveness of quantization and memory optimization techniques in enabling real-time edge AI inference. This work provides a systematic design and evaluation framework for AI-driven hardware acceleration at the edge and offers practical insights into architectural and co-design strategies that address the limitations of general-purpose processors for emerging edge AI applications.
Keywords: Edge AI; Hardware Acceleration; AI Inference; Energy Efficiency; Hardware–Software Co-Design