Video The Voice of Art – IBM Watson, Connexis Digital Mentors Channel, posted on YouTube on December 6th, 2018.
Link: https://youtu.be/ogpv984_60A?si=KZTNq6SQ7cLC5KFS
In 2017, IBM celebrated 100 years in Brazil. That year, the company partnered with the Pinacoteca Museum (São Paulo city) to implement The Voice of Art project.
It consisted of an experience that developed an app (cognitive assistant) capable of “talking” with the museum public about seven artworks from the collection. It operated with Watson technology, a cognitive system that uses artificial intelligence to learn content from numerous sources: videos, photos, interviews, audio, etc.
Watson was fed information about Brazilian art history, the selected paintings, and their contexts for six months. Upon arriving at the Pinacoteca, the visitor received a smartphone with the app and a headset.
Along the way, sensors indicated the proximity of the artworks available. Then, through audio, the visitor could ask the questions they wanted, how they wanted, getting the answers on the headset. It is, therefore, an individualised and interactive experience with Brazilian art.
People with hearing impairment, who were literate in Portuguese, could use the technology through written texts via chat. However, most deaf people in Brazil do not understand or write well in Portuguese, communicating using Libras, the Brazilian Sign Language.
In fact, according to data from the World Federation of the Deaf (WFD), about 80% of deaf people in the world have insufficient education and/or problems with writing.1
The solution is more suitable for the visually impaired public. In these cases, it works as an interactive assistive technology.
In this post, we have the official video of the project. Despite being an exciting video, I have some reviews regarding the approach chosen to justify the absence of Brazilians in museums.
We know the problem is much more complex than simply the need to interact with artworks or matters of taste. This absence is just a symptom of more serious aspects, such as social inequality or educational deficiencies in my country.
I don’t think the video intended to simplify the problem. However, an approach is always a choice and reveals a specific curatorship, a decision on what to prioritise in the discourse. I believe there would be other approaches with more significant potential for contribution to justify the importance of this experience, like its capacity for inclusion and educational mediation through digital entertainment.
However, this does not detract from the project’s or video merits. It was beautiful, in my opinion. The Voice of Art seemed to be a very innovative and exciting proposal with great potential to become something commonplace in museums in the near future.
The implications that this will bring us regarding unemployment caused by automation, reduction of social relations and human mediation in museums, and the issues arising from biases caused by algorithms are for discussion on another occasion.
Notes
1 – Improving multimodal web accessibility for deaf people: sign language interpreter module, by Matjaž Debevc & Primož Kosec & Andreas Holzinger, DOI 10.1007/s11042-010-0529-8
Video The Voice of Art – IBM Watson, Connexis Digital Mentors Channel, posted on YouTube on December 6th, 2018.
Acknowledgements: Thanks to Alberto Nogueira Veiga and all who gave me their precious feedback.
Image: Screenshot from The Voice of Art video.
Please help me improve my English by sending me your suggestions through this contact form. Thanks!
Published on my blog in Portuguese on October 15, 2022: A Voz da Arte: Inteligência Artificial Watson da IBM na Pinacoteca de São Paulo
