Status: AI is making a visual podcast for deaf children

Podcast for deaf and hard of hearing children using images generated by artificial intelligence. For KRO-NCRV Willem Wever the Elfstedentocht, the lunar landing and the flood disaster were carried over.


The bottom line is that inclusivity plays an important role. Roseanne van de Lemkolk, digital creative at KRO-NCRV, and her team investigated how deaf and hard of hearing children consume podcasts. “Now this is done via transcript, but soon we find out that children will not read such a large text. Moreover, it is important to convey a feeling and the interpreter is very good at it.” Thus began the ambition to “translate” the podcast “Toen ik was 12” from the children’s show Willem Wever.

The focus has been on artificial intelligence, especially the open AI system VQGAN + CLIP. this a tool It can convert text into an image. Research among deaf and hard-of-hearing children by trainer Koninkliki Kentalis shows that it’s important to make sure podcasts come alive in three ways: you have to ‘translate’, use a sign interpreter and create (atmospheric) images.

  • Appreciation by the target group
  • The core of transportation

This was a project from NPO Innovation and the goals have always been substantive: the broadcaster wants to know that the target group will rate the podcast enough. In the test set of two different schools, the final podcast was rated 7.5. Van de Lemkolk: “We did it in two stages so that we could improve a little. For example, it is important that the translator does not stand too much in front of the picture. For example, the translation should be at the bottom of the picture.” It also turns out that the “interpretation” of music is important for conveying the essence.

  • Image consistency
  • Synchronization with translation

Serge Cornelissen van Cornelistoles, who is responsible for this technique, received texts When I was 12 With the task of creating appropriate images so that it becomes a running story. One of the challenges was to combine the story and AI visuals and keep it consistent. “If a kid is wearing a blue coat, you should see that coat later in the story. It makes sense, but you need to pay attention to it during the manufacturing process.”

Another challenge is the zeitgeist of the story. Astronaut Andre Kuipers spent his childhood in his room watching television. This is of course not possible flat screenTelevision, as it was in the seventies. The system uses the database “ImageNet_16384” which contains fourteen million images. “He knows a lot,” Cornelissen says, “but not everything.” “You see, the laws of physics are tough. We introduced the task ‘Draw a child watching the moon go down on TV.’ Then you get a moon with a child on it and there is still a TV floating in the air somewhere. He understands the concepts, but merging is hard. To help a little with that.”

Other pictures work very well: “It contained Andre Kuipers’ dream transforming his bed into a spaceship. It is precisely this picture that could make such a computer so beautiful. We couldn’t draw that ourselves. Sometimes not understanding the world is a good thing in reality. This makes it a fun game between photo and computer.” There was also extensive cooperation with interpreters, says van de Lemkolk. “This is important when it comes to speed. You have to make sure that the frequency of the images is in sync with the translation. Sign language is the key, not the sound.”

  • Views 2600
  • Good social mediaReach

Three issues of Willem Weaver’s podcast appeared online: about the Elfstedentocht, the flood disaster, and the moon landing. So the final score was rated 7.5 and the three series were watched 2,600 times. This may not sound like much, but with about five hundred deaf and hard of hearing children between the ages of 9 and 12 in the Netherlands, that’s a good percentage. The series has also been well watched on LinkedIn, also because well-known sign language interpreter Irma Slues shared it. Van de Lemkolk: “We wanted to make this content accessible to everyone.”


technology open source It’s not very expensive, but you have to invest the time to select the images and possibly edit them to make it a running story. In terms of cost and speed, according to the participants, it is not comparable to making a movie all on your own.

*This article originally appeared in the June issue of Emerce magazine (#190).

About the author: Bas Hakker is an editor at Emerce.

Leave a Comment