Modules: Multimedia Toolkit

Be a friend to your interviewee. They will certainly share their stories with you.
Can you be a good friend of your interviewees? 1

Half of the world’s languages are dying in less than a century’s time. Languages that contain indigenous knowledge of many human communities around us will be permanently lost unless we start documenting them now in a medium that is universally accessible. OpenSpeaks Audio-visual toolkit is a four-day series of modules that are designed to help you learn how to digitally-document native languages and publish them in open and accessible standards.

Goal of this page is to help you understand the process of audio-visual recording a native speaker speaking in their own language, editing, and publishing the recorded content with the vital information for others to find and use it.

Use this resource page more like a handbook to learn about the overall documentation process. It can be roughly broken down into four major modules that you can learn in four relaxed days or in four hours if you are hell of a genius!

Module Content Duration
1 Basics of audio-visual recording, and the interview process 1 day
2 Hardware and software for recording, and recording process 1 day
3 Curating and publishing metadata 1 day
4 Editing audio and video, and publishing them 1 day

 

1. Module 1: Basics of audio-visual recording:

An overview of what are aimed from the recording and how to go about it.

Audio-visual recordings play the first step of the entire documentation process that you will learn in this toolkit. The idea of this documentation is to have you as an archivist and record an interview/conversation of native language speakers. There are a few pre-requisites before you start. Take a look at them below:

  • Make your imagination your friend: Imagine yourself as an archivist who is out for recording a language by interviewing a native speaker. Think of the challenges that you might face while talking to them—a loss of translation may be. Are you ready to listen to a totally alien language for over half hour when you are expected to smile, nod and record it?
  • Plan in advance and practice them: Planning for a documentation starts with knowing your interviewee(s). Do some research about their language, culture, and may be a few most used phrases (check some basic Greek phrases here) in their language that you can use and amaze them when you talk to them. Many people love it when someone attempts to speak in their language with a genuine interest. Use a spreadsheet or even an app to plan everything thoroughly. And have a plan B in case anything fails. Several different challenges can work against you or you can win over by a little planning. Write them down, and even practice with a family member or friend in advance, or even in front of a mirror. You will learn more about what questions to ask later in this module.
  • Know your hardware and software: As you are going to rely on a bunch of gears and software (you will learn about them in the next module), it is important that you know well about them. But how well is well? Well, as long as you know how to operate for your need and troubleshoot in emergency. You don’t need to know everything in details, but know the basics. For instance, if you are going to use your phone for the audio and video recording, check what apps are good. It is advisable to use apps that show the audio and video levels while recording so you know you are actually recording.
  • Take some physical/digital notes if required: You can always listen to the recording and in general, there is no need for any separate note-taking. But, just in case you want to keep a separate but parallel note of something important, keep a notebook or phone handy. But please keep in mind, any noise you make during the recording might get recorded and will reduce the quality of the recording.
  • Ensure you record in a quiet place: The most challenging aspect of the recording is getting a clean audio and video. Check the list below to know about what to avoid:
    Noise sources Possible solutions
    Noise from outside the recording place 1. Have a conversation before you record and ask the interviewee what is the least noisy place in their house, and move their.
    2. If you have access to a good light source, avoid recording in outdoors
    Interference of phones and other electrical devices Keep your own phone in “plane” mode and try to request your interviewee to do the same or keep their phones away. Most digital recorders have interference during incoming calls which unfortunately get recorded with distortions.

Interview process:

During a digital documentation, the best emotion is captured when your interviewee trusts you the most. When you become a friend of theirs, they would open up to share something that they care about. However, trust is built over time. How do you bring it in a short interview?

Here’s something for you—you ask the most trivial questions in the beginning and slowly move towards a more personal conversation that will push the interviewee to become a little more emotional.

Below is a sample interview format which is set around a face-to-face conversation that is being recorded. It would vary slightly different when done over a phone/voice/video call. In a physical interview like this, your body language matters even more and a good one can entirely set the mood of the subject. So a thumb rule is be empathetic, warm, and show genuine interest to learn about the person you are talking to.

1. Introduce yourself, tell them the purpose of the interview, and clearly seek their permission to record the interview
Hi, My name is Monica. I love to learn about the beautiful stories that are hidden in the languages around us. I would love to talk to you today, ask a few questions and listen your stories in your language.I would like to record this interview as the purpose is to share these stories with the outside world. Can I do that? (proceed with a yes from them) Thank you!This interview will take about half hour to 45 mins. After editing, I will upload this publicly under a free license called Creative Commons Share-Alike license called CC-BY-SA 4.0. It allows anyone to use, share, and modify the content even for commercial reproduction. Alright?

2. So, what did you have for breakfast today?
3. What’s your favorite color?
4. Can you pronounce your name for me in your own language?
These three questions above are kind of icebreakers. You should ideally be monitoring the recording. Check your recorder and ensure everything is recorded properly. It might take a few seconds and you might want to have a casual conversation with the interviewee  telling them how you are ensuring that the recording is done properly.
5. You mind telling me which city/town you were born?
(skip if they’re not comfortable)6. So, did you play any local or traditional games as a child? Can you share little about those games and tell me how they used to be played back then?
7. Do you remember any bedtime stories that your parents or grandparents would have told you? Can you narrate one to me?
(ask them to first narrate in their language and then give you a quick summary in English or whatever the link language they and you speak in)
8. And, did you enjoy any of the the traditional songs that your community might be singing during any special occasion like festivals. It would be great if I can hear one from you.
9. Can you share how you spend your entire day detailing all activities?
10. What’s that traditional food that reminds you of your childhood? How it is prepared?
11. Imagine I come to your house as a guest. How you’d greet me and make me feel home?
 (be a good sport and go with the flow here by acting as a guest)Make sure you react to their talk appropriately during the recording without making any noise as you are recording, but smile when they say something humorous and keep nodding. If they go overboard, you should stop them politely.
12. (optional) Video-record how some of the unique phonemes or words are pronounced in their language in a close-up shot so that others can learn. This might help others to dive deeper into the linguistic nuances.

The next step is setting up your gadgets—your camera (if doing a video recording), and your digital recorder. Learn more about the audio and video recording process here.

Module 2. Hardware and software for recording, and recording process:

a. Audio recording:

A home studio setup consisting of a computer installed with a free and open source audio recording/editing software like Audacity, a professional microphone, and a monitoring headphone.
A home studio setup consisting of a computer installed with a free and open source audio recording/editing software like Audacity, a professional microphone, and a monitoring headphone. Read more in our Pronunciation Toolkit.
  1. Home studio: You need a microphone to be able to record the audio. If you can, I would suggest to record in a small home studio setup like the picture above (consists of a USB microphone, a computer, and a monitor headphone).
A digital audio recorder is used to record audio during field recording. (Marie-Lan Nguyen, CC-BY 2.5)

2: Field recording with a recorder or phone: The recording setup will largely vary if you are meeting someone outside your home for a field recording. In that case you will need to carry an audio recorder or a smartphone (some sort of recording app installed in it) with earphones. If you’re using a portable recorder make sure you cover the top of the mic with a soft cotton cloth or fake fur to a) avoid dust going inside, and b) the sound of the wind during outdoor recording. Use a rubber band to tighten the base and never touch the cloth/fur while recording. Mics can capture small little movements and completely distort the audio.

  1. Recording from phone: Earphones that come with the phones generally work both for phones and computers as compared to the default microphone provided along with . However, avoid sitting in an open space as there is a high probability of a lot of noise being captured unless if you are using a shotgun microphone.

4: Audio editing software: If editing from a computer, Audacity, a free and open source audio editing software is the first choice for many seasoned recording artists. It is robust, easy to use and can be used in multiple platforms. If you are using your phone or tablet to record and edit the audio, then, use your native recording app or try to find a good free alternative in your respective app store. Ideally the recording/editing app should be allowing you to record in a decent losless quality (minimum requirement is 44100 Hz, above 16 bit PCM i.e. 24 or 32 bit, above 220 kbps; check your settings to find these). Save the audio in .WAV or .FLAC (Audacity supports both). If your recorder/phone does not support these formats, try to use an app/online converter like this (MP3→FLAC or M4A→FLAC) to convert the audio into .FLAC.

b. Video recording:

  1. Which camera to use: Frankly speaking, the video is less important here as compared to the audio. With low quality video, viewers would still be able to manage if the audio is loud and clear. So if you are keen on investing, invest on a good quality microphone that can either be connected with the camera or can be used as a secondary recorder. But do not trust your camera’s default microphone. They can literally jeopardize your hard work. As far as the camera goes, you can literally use any camera that allows you to record in a decent quality i.e. above 720p (1280×720 px)—from your phone to a point and shoot camera to a dSLR.
a) Using a camera: Use a shotgun microphone that can be connected directly into your camera so that you don’t need to invest much on audio syncing during post production.
b) Using a phone for recording video: These days most phones come with high quality hardware that are capable of recording good video. But the real key to recording quality video in a phone lies in stabilizing the shot while recording. You can only do that by investing in a small tripod (they are generally really cheap and do the job) that can hold your phone. For this particular project, tripods will be the best.
  1. How to edit the videos: You need to compress the video using a free software like Handbrake, and upload that into YouTube or something similar without making it public. We will download it and ask you to delete so that you don’t have to worry about the amount of space it will take in your hard drive.

Module 3. Curating and publishing metadata

Annotation, subtitling of audio/video, translation of transcription and other content

Annotation is the process of collecting certain metadata that are not necessarily transcriptions. Audio/video content will surely need subtitles in largely spoken languages like English for a wider coverage. Amara is an Open Source video subtitling platform (learn how to use it from here). YouTube also allows video subtitling and Closed Captioning. Transcriptions are generally created to have a verbatim version of the interview. Ideally, you need to work post-interview with a native speaker to create the transcription to ensure there is no loss of information in the process. However, transcription is not a easily digestible. So you need to create summaries for each section of the interview which will capture the highlights and sometimes details (for instance a game play or story).

Curating metadata:

Collecting metadata is a very crucial part of the documentation process. When you document something in a less-known language and publish it online, you also need to share some of the most vital information about the documentation. See a sample below that is taken from our Karbi-language documentation page.

Language details Recording details
Language Karbi Recording content Narration of a folklore, a folk song, a local festival, traditional games, Meaning of “Karbi”, speaker’s daily activities
Dialect N/A Recording location Remote, Speaker at Karbi Anglong district, Assam, India
Alternate name(s) Mikir Recording date Sunday, May 21, 2017 at 8:55 AM
SIL Code mjw Recordist(s) Subhashish Panigrahi
Current state Living, endangered Hardware Zoom H1
Language group Tibeto-Burman languages Software Audacity
Possible influences Assamese, Naga #files 10
Open Language Archives Community (OLAC) here File format(s) wav, flac
Swadesh word list here Bit rate 16 bit PCM
Ethnologue here Total audio length (HH:MM:SS) 00:53:37
SIL International here Copyright CC-BY-SA 4.0
Wikipedia here Video N/A
Wikipedia in respective language N/A Image N/A
Scolarly citations on Google Scholar here
Internet Archive resources here
Speaker details
Speaker Gender Male
Speaker Age 50–60
Speaker Origin Karbi Anglong district, Assam, India
Speaker’s Name D.S. Teron
Speaker’s Pictures N/A

Module 4: Editing audio and video, and publishing them

Editing:

There are many ways of editing audio files. We would generally recommend to first record in a lossless format like .wav and 16-bit PCM. and then edit the audio rather than recording in mp3. Some phones do not record in .wav and you might end up recording in a different format if you are using the phone’s native app. You might want to do some research and find a good alternatives for both Android and iOS (examples of some Android apps, and some for iOS).

Audacity, a Free Software, is overall a swift, professional-standard, and swift software for audio editing. It accepts most industry standards for input and output. Below is a tutorial to learn how to make your vocals clear and loud using Audacity. You can find many more by doing a mere YouTube search. We have elaborated more about the audio recording workflow specifically for building a talking primer or dictionary in any language in our Pronunciation toolkit where you can also download Kathabhidhana, a Linux/Mac recording tool, open datasets, and read in details about how to record.

Video editing is rather a complex process. Here is an article for the free and open source alternatives for Linux. Of all of these, Kdenlive is probably the best and is also supported on most major Linux distributions, Windows and Mac.

Publishing:

We would always suggest to publish the audio and video files in the most open and the best quality file format. However, if you need to choose between open standards and quality, then go for the highest quality.
Wikimedia Commons: Our most recommended platform for uploading both audios and videos would be Wikimedia Commons. Please note, you would need an account registered in any of the Wikimedia projects like Wikipedia. If you don’t have one, create one from here. Wikimedia Commons only allows open standards. So the best option for audio would be .WAV or .FLAC, and video would be .webm (unfortunately, a lossy format).
YouTube (only video or audio with a static still photo): YouTube allows a wide range of file formats, and most importantly an option to upload your videos under a Creative Commons license. Vimeo also supports Creative Commons licenses. Between, YouTube and Vimeo, you might want to make a judgement which one to choose depending on your volume of videos produced per month as a normal account on Vimeo limits the monthly upload to only 500mb where YouTube allows unlimited uploading.

While publishing your audio/video online, try to maximize the metadata so that your content is discoverabe and searchable. If you add decent amount of information in the description and other metadata fields, it becomes easier for the readers to locate them when search for relevant keywords. If you know multiple languages, feel free to add metadata in multiple languages as well.

Additional section: Social media

Social media platforms are great ways to promote endangered, indigenous and other marginalized languages, mainly for two reasons: a) As most social media hubs like Facebook, Twitter, Snapchat already have a good user base, one does not need to invest too much to promote their content, b) Most of the popular social media are managed by big corporations or at least by startups that are innovating every now and then to make their user experience better. It is important to make use of their great features optimally but securely. Check out this toolkit — that consists of mostly open practices and methodologies, and some open educational resources on configuration settings — to experiment with social media platforms as a production and promotion tool for your language.

Check out some of the frequently asked questions.

A version of this Open Educational Resource is also published and cataloged on OER Commons , and can also be accessed from the Peer To Peer University (P2PU) .

Attribution:

  1. Characters designed by Dooder / Freepik. CCBY)
  2. Akshay Roongta, Mozilla Open Leadership mentor, who brought a wide range of design thinking to this project