Attention:NSF PAR will be unavailable due to scheduled facility maintenance July 10th, 5 PM EDT through July 13th. We apologize for any inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


This content will become publicly available on August 6, 2026

Title: Design and Development of a Real-Time Camera-Based Smart Cooking Assistant
Personalized cooking recipe recommendation systems offer the potential to improve dietary choices for unhoused individuals and those transitioning out of homelessness. However, existing systems often neglect the needs of users with minimal cooking experience, providing little guidance during meal preparation. This study proposes the development of an intelligent cooking assistant system designed to offer realtime, step-by-step support throughout the cooking process. The system integrates a Raspberry Pi 5 mini-computer with a Raspberry Pi AI HAT+ (AI HAT+) and Raspberry Pi AI Camera (AI Camera), strategically mounted above the cooking area to continuously monitor culinary activity. At its core, the assistant utilizes a deep learning image classification model built on Ultralytics’ You Only Look Once version 11 (YOLO11) framework, trained on a curated dataset of 1,339 images collected during the preparation of chicken teriyaki and pasta dishes. The model achieved 100% precision and 99% recall of identifying all six cooking states utilized in this work, resulting in an average confidence accuracy of 91% during real-time tests. The system is intended to enable greater culinary independence among individuals with little cooking experience, such as those affected by long-term homelessness.  more » « less
Award ID(s):
2125654
PAR ID:
10647109
Author(s) / Creator(s):
 ;  ;  ;  
Publisher / Repository:
IEEE
Date Published:
Page Range / eLocation ID:
361 to 366
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; ; et al (, IEEE)
    AI-based virtual learning assistants are intelligent systems that revolutionize education by offering personalized learning opportunities that make learning methods more accessible, adaptive, and data-driven. These systems leverage machine learning and natural language processing to offer instant feedback and interactive engagement catering to individual student needs, thereby augmenting human learning assistants. This research compares two implementations of the AI-Enhanced Learning Assistant (AIELA) prototype, addressing limitations in scalability, privacy, and accessibility identified in the original Raspberry Pi-based implementation. The new VLA prototype implements a web application interface to replace the centralized hardware-based prototype in the previous model. The new model enables students to use their own mobile or computer devices to resolve specific hardware constraints. Two exploratory classroom demonstrations introduced the tool, followed by surveys gauging usability, engagement, and overall effectiveness. Students rated the web-based AIELA as user-friendly and moderately effective (~3.8-3.9), though its support for deeper learning and addressing misconceptions was lower (2.96). Despite comfort with using AIELA (4.15) and its usefulness for worksheets (3.73), HLAs consistently received higher marks for conceptual support, although limitations in survey design constrain direct comparisons. Future work should emphasize semester-long trials, improved data collection, and enhanced conversational strategies to balance Socratic prompting with direct guidance better, thereby complementing human instruction. 
    more » « less
  2. ; ; ; ; ; (, IEEE)
    Natural-language interaction between passengers and autonomous vehicles is essential for trust, safety, and user experience, but deploying Large Language Models (LLMs) on automotive edge platforms is constrained by compute, memory, energy, and privacy. We present Pi-talk, an edge-only system that enables real-time passenger–vehicle dialogue using a Small Language Model (SLM) running entirely on embedded hardware. Pi-talk performs multimodal fusion of onboard camera, ultrasonic distance, and navigation context via a lightweight encoder–adapter module that aligns modalities into compact semantic tokens for a pre-trained SLM. The SLM produces context-aware explanations of driving decisions, route options, and situational updates without cloud connectivity. Safety is enforced through a real-time safety envelope that gates responses and actions using distance thresholds and timing constraints. We further adapter-tune the SLM (on-device or offline) and deploy it with INT8 quantization and an Open Neural Network Exchange (ONNX) runtime to achieve efficient batch = 1 inference on Raspberry-Pi–class hardware. We evaluate task quality (evaluation loss), end-to-end latency, CPU utilization, and memory footprint, and include ablations contrasting unimodal vs. fused inputs. Results show that Pi-talk sustains few-second, edge-only inference while meeting stringent resource and latency limits and maintaining the safety envelope required for autonomous operation. To our knowledge, Pi-talk is among the first edgeonly, multimodal passenger–vehicle dialogue systems that both fine-tune and run a small language model entirely on Raspberry Pi–class, CPU-only hardware with an explicit while enforcing a runtime safety envelope. 
    more » « less
  3. ; ; ; ; ; (, EMDL 2021: 5th International Workshop on Embedded and Mobile Deep Learning)
    Occupancy detection systems are commonly equipped with high quality cameras and a processor with high computational power to run detection algorithms. This paper presents a human occupancy detection system that uses battery-free cameras and a deep learning model implemented on a low-cost hub to detect human presence. Our low-resolution camera harvests energy from ambient light and transmits data to the hub using backscatter communication. We implement the state-of-the-art YOLOv5 network detection algorithm that offers high detection accuracy and fast inferencing speed on a Raspberry Pi 4 Model B. We achieve an inferencing speed of ∼100ms per image and an overall detection accuracy of >90% with only 2GB CPU RAM on the Raspberry Pi. In the experimental results, we also demonstrate that the detection is robust to noise, illuminance, occlusion, and angle of depression. 
    more » « less
  4. ; ; ; (, IEEE)
    Flood detection is difficult in rural areas with little or no monitoring infrastructure. Smaller streams and flood-prone regions often remain unmonitored, which leaves communities vulnerable. Commercial systems cost much and use proprietary designs, so many communities cannot use them. This work presents AquaCam, a low-cost and open-source flood detection system that uses a Raspberry Pi and a camera to measure stream water levels automatically. AquaCam captures images and trains a lightweight convolutional neural network (YOLOv8) with the collected data. The model learns to recognize water in natural backgrounds and measure water height. To test whether AquaCam can adapt to new environments, we evaluated the trained model at a different site with no retraining. The system still identified water levels accurately. This shows that the approach is practical and generalizable. AquaCam moves flood detection toward being affordable, accessible, and adaptable for the communities that need it. 
    more » « less
  5. ; ; (, IEEE)
    Large Language Models (LLMs) offer significant potential for enhancing smart home assistants through more natural and responsive automation. However, solely relying on cloud-based LLMs raises concerns about network dependency and user privacy. In this paper, we conduct a comprehensive evaluation to assess the feasibility of deploying LLMs on low-cost devices, enabling smart home automation in terms of latency and energy consumption. In particular, we first build a comprehensive and reproducible testbed that integrates benchmark and well-trained LLMs with the Home Assistant platform on low-cost edge devices, such as Raspberry Pi 4 and Pi 5. Leveraging this testbed, we evaluate on-device LLM inference performance in terms of inference latency, energy consumption, and thermal characteristics, and provide theoretical estimates of remaining runtime in battery-powered settings. We evaluate the on-device performance on two benchmark quantized models (e.g., Home-1B-v3 and Home-3B-v3) for smart homes. Experimental results show that the Raspberry Pi 5 significantly outperforms the Raspberry Pi 4 in terms of low latency and high energy efficiency across both models, due to its superior processing capabilities. In particular, for the Home-1B-v3 model, the mean inference energy decreases from 264.3 J to 133.9 J, representing a 49.3% reduction in consumption, and the mean latency decreases from 40.3 s to 18.7 s, resulting in a 53.4% improvement in delay in Pi 5 compared to Pi 4. The findings of the paper provide valuable empirical insights for energy and latency-aware smart home automation using quantized LLMs on low-cost edge hardware. This work lays the groundwork for future research on the energy efficiency of LLM applications. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Text Encoded Conversion
  • XML
  • Save / Share this Record
  • Send to Email