Certified orientation and mobility specialists (COMS) work with clients who are blind or visually impaired (BVI) to help them travel independently with confidence. Part of this process involves creating a narrative description of a route and using specific techniques to help the client internalize it. We focus on the problem of automatically generating a narrative description of an indoor route based on a recording from a smartphone. These automatically generated narrations could be used in cases where a COMS is not available or to enable clients to independently practice routes that were originally learned with the help of a COMS. Specifically, we introduce Clew3D, a mobile app that leverages LIDAR-equipped iOS devices to identify orientation and mobility (O&M) landmarks and their relative location along a recorded route. The identified landmarks are then used to provide a spoken narration modeled after traditional O&M techniques. Our solution is co-designed with COMS and uses methods and language that they employ when creating route narrations for their clients. In addition to presenting Clew3D, we report the results of an analysis conducted with COMS regarding techniques and terminology used in traditional, in-person O&M instruction. We also discuss challenges posed by vision-based systems to achieve automatic narrations that are reliable. Finally, we provide an example of an automatically generated route description and compare it with the same route provided by a COMS.

Clew3D: Automated Generation of O&M Instructions Using LIDAR-Equipped Smartphones / H. Harriman, D. Ahmetovic, S. Mascetti, D. Moyle, M. Evans, P. Ruvolo - In: ASSETS '21 / [a cura di] J. Lazar, J.H. Feng, F. Hwang. - [s.l] : ACM, 2021. - ISBN 9781450383066. - pp. 1-3 (( Intervento presentato al 23. convegno International ACM SIGACCESS Conference on Computers and Accessibility tenutosi a Virtual event nel 2021 [10.1145/3441852.3476564].

Clew3D: Automated Generation of O&M Instructions Using LIDAR-Equipped Smartphones

D. Ahmetovic
Secondo
;
S. Mascetti;
2021

Abstract

Certified orientation and mobility specialists (COMS) work with clients who are blind or visually impaired (BVI) to help them travel independently with confidence. Part of this process involves creating a narrative description of a route and using specific techniques to help the client internalize it. We focus on the problem of automatically generating a narrative description of an indoor route based on a recording from a smartphone. These automatically generated narrations could be used in cases where a COMS is not available or to enable clients to independently practice routes that were originally learned with the help of a COMS. Specifically, we introduce Clew3D, a mobile app that leverages LIDAR-equipped iOS devices to identify orientation and mobility (O&M) landmarks and their relative location along a recorded route. The identified landmarks are then used to provide a spoken narration modeled after traditional O&M techniques. Our solution is co-designed with COMS and uses methods and language that they employ when creating route narrations for their clients. In addition to presenting Clew3D, we report the results of an analysis conducted with COMS regarding techniques and terminology used in traditional, in-person O&M instruction. We also discuss challenges posed by vision-based systems to achieve automatic narrations that are reliable. Finally, we provide an example of an automatically generated route description and compare it with the same route provided by a COMS.
computer vision; guided navigation; object classification; orientation and mobility
Settore INF/01 - Informatica
Book Part (author)
File in questo prodotto:
File Dimensione Formato  
ASSETS_Poster_Submission.pdf

accesso riservato

Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione 429.52 kB
Formato Adobe PDF
429.52 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
3441852.3476564.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 461.84 kB
Formato Adobe PDF
461.84 kB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/904593
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact