According to the World Health Organization (WHO), more than hundreds of million people have some type of visual disability today. Vision impairment has a personal impact with lifelong consequences because more than 80\% of our perception, cognition, learning, and daily activities are mediated through vision. As a result, visually impaired and blind people stagnate at work and in education due to inaccessible technologies. In this regard, we present a speech-based assistive system for visually impaired and blind people to interact with ChatGPT. The system employs automatic speech recognition (ASR), text-to-speech (TTS), and a messenger bot (Telegram) to enable a natural interface to ChatGPT for people with visual disabilities. Our preliminary experiments showed the feasibility of the proposed system as an assistive technology. We have made the source code publicly available at https://github.com/IS2AI/telegram-bot-chatgpt to stimulate the development of assistive systems for the visually impaired and blind.