LLMs for visually impaired

There's Something Fundamentally Wrong With LLMs

LLMs aren't trained on the "vast majority of speech," experts warn, a major blind spot that could have sweeping consequences.

Understanding Multimodal LLMs

An introduction to the main techniques and latest models

How LLMs are and are not like the brain

Hi from buttondown! At the bottom of this newsletter is a bit of administrivia about the new platform How LLMs are and are not like the brain Beneath all the...

Hallucination by proxy in LLM-assisted differential diagnosis

Current evidence suggests that LLM assistance could augment the diagnostic accuracy of clinicians. However, these systems are black boxes, susceptible to hallucinations, and project a potentially...

Hallucination by proxy in LLM-assisted differential diagnosis

Current evidence suggests that LLM assistance could augment the diagnostic accuracy of clinicians. However, these systems are black boxes, susceptible to hallucinations, and project a potentially...

Artificial intelligence-powered smart vision glasses for the visually impaired

Purpose: In India, 4.80 million people are blind, and 4.69 million have severe visual impairment. Globally, the digital era and the advent of artificial intelligence devices offer solutions for daily challenges faced by the visually impaired, but they are often expensive. This study evaluates the acceptance and usability of a cost-effective AI-powered spectacle-mounted assistive device in a multicentric Indian cohort. Methods: Smart vision glass (SVG) is a lightweight tool, mounted on a spectacle frame with braille-coded keys and a real-time voice interface for guiding the visually impaired, and is connected to a smartphone Android App. The four major functions of the device are as follows: “Things Around You,” “Reading,” “Walking Assistance,” and “Face Recognition.” Ninety participants were recruited from five vision rehabilitation centers across India, and feedback was collected after a month’s usage. Results: The mean age of the participants was 23.5 years (range: 6 to 56 years) with 58.9% male; 51% from rural areas; and 64.4% students. As per the World Health Organization (WHO) classification, 90% of the participants were blind (best corrected visual acuity in better eye less than 3/60). A total of 100% of the participants were able to access all the functions of the SVG. Participants reported a positive experience with “Reading” (72.9%), “Things Around You” (44.7%), “Face Recognition” (36.5%), and “Walking Assistance” (22.4%). About 2/3rd of the participants utilized the device for an hour or more per day. Conclusion: SVG is a promising cost-effective device for the blind and severe visually impaired in aiding their day-to-day challenges and becoming self-reliant.

ggml-org/Llama-macOS

A cosy home for your LLMs.

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)

Assisting blind people with AI and audio using smart glasses: system design with YOLOv8 variants comparisons

This paper introduces a novel system design leveraging Vuzix Blade 2 smart glasses to enhance the mobility and independence of visually impaired individuals. The study critically examines existing assistive navigation and object detection technologies, identifying their limitations and gaps. The designed system integrates real-time object detection, distance estimation, and OCR functionalities, providing auditory feedback through a robust and efficient pipeline. The designed application enhances the independence and safety of visually impaired individuals, particularly in navigating university campuses. A dataset comprising 15,951 annotated images from the university campus was used for training and evaluation. A comparative analysis of three YOLOv8 models (YOLOv8-N, YOLOv8-S, and YOLOv8-M) was conducted, balancing accuracy and computational efficiency to optimise system performance. The pipeline also offers a comprehensive framework for developers and researchers to build inclusive systems combining AR, computer vision, and AI. Results show high object detection accuracy (precision: 0.90, recall: 0.83) and reliable distance estimation with a minor error of 0.33 m. Results demonstrate the system’s capability to detect obstacles within one meter, provide precise distance estimation, and convert textual information into speech, validating its potential for real-world applications. This study emphasises the significant role of AI-driven solutions in advancing assistive technologies, paving the way for more accessible and inclusive navigation systems. Compared with recent assistive systems such as Smart Cane (He in CCF Trans. Pervasive Comput. Interact. 5:382–395, 2023), OrCam MyEye (Amore in J. Med. Syst. 47:11, 2023), and IrisVision (Gopalakrishnan in Comparison of visual function analysis of people with low vision using three different models of augmented reality devices, 2024), the proposed system demonstrates superior integration of detection, text recognition, and real-time feedback within a lightweight wearable device.

lukeslp/awesome-accessibility

Curated list of accessibility resources, tools, and best practices for creating inclusive digital experiences

llm-wiki

llm-wiki · GitHub

Smart Glasses for Disabled People

This paper introduces an innovative assistive technology aimed at improving the quality of life for individuals with visual impairments. This project presents a system that combines the capabilities You Only Look Once [6] (YOLO) deep learning algorithm with the ESP32 [7] microcontroller to create Object Detection Glasses. These glasses employ real-time object detection to provide users with auditory feedback about their surroundings, enabling them to navigate and interact with their environment more independently. The first task of the glasses is to take pictures of the object and store it as a snap. The second task is that they compare the captured image to the existing image. In order to convert the text into speech, it used Text to Speech technology (TTS). The picture will be taken by ESP 32 with the perfect size the image will be displayed and then by using the speaker the object will be detected. The integration of YOLO [6] and ESP32 [7] offers a cost-effective and efficient solution for enhancing accessibility and inclusivity, promoting greater autonomy and safety for individuals with visual disabilities.

eSight - Electronic eyewear for the visually impaired

eSight is the most advanced all-in-one device for people with visual impairment. Fill out our online form to learn if eSight could help you.

SpecAssist: Smart Glasses with Real-time speech recognition and transcription for the hearing impaired

Download Reference Data on IJERT | On 01-07-2023 by Noel Jacob, Rajat Mathew, Sathwik P Nair, Prof. Divya Sunny published SpecAssist: Smart Glasses with Real-time speech recognition and transcription for the hearing impaired