Is it possible to play sound in the browser with a matlab web app?

I'm a researcher studying hearing and we use MATLAB to generate and play sounds to research participants. In light of the social distancing measures in response to COVID-19 our lab has stopped testing participants in person. I'd like to be able to convert our existing experiments to web apps using the matlab compiler, but have been unable to verify whether or not its possible to play sound on the client computer. (i.e. in the web browser running the app, not the host/server)

Does anyone know whether this is possible or supported?

[Mohammad Sami]

Which version of matlab are you working with ?

Starting from R2019b, you can use the uihtml element to do almost anything you can do in a webpage.

[Mohammad Sami]

I don't know if the other audio playback functionality is supported in the compiled webapp. Maybe someone else knows.

[Mohammad Sami]

Example of audio playback

https://www.mathworks.com/help/matlab/ref/uihtml.html

[rob mcleod]

Jordan I would be very interested to know if you managed to get this working? I am also researching hearing and having exactly the same problem.

[Jordan Beim]

Hi Rob,

I ended up using the uihtml function that Mohammad Sami linked to embed an html audio module in my web app. My matlab code generates an audio file on the fly and saves it with a unique file name that is passed to the uithml module through the data structure detailed in the uithml documentation. My html code tries to wait for the file to fully buffer and then starts playback and reports its status changes back to the matlab code. This allowed me to render the pre-generated audio on the clients computers instead of the server, but likely wouldnt work as well for continuous audio that needs to be changed in real time. Let me know if there's anything I could provide more detail about.

[Benjamin Lentz]

Hello Jordan,

I have the same problem as you did in April last year. I also tried to solve it with uithml. I managed to play sounds in the browser (with javascript and Audio objects, because I wanted it to play directly and not pressing any additional buttons), but:

1. When I do not change the audio file name and overwrite the same audio to play different sounds, Firefox keeps playing the first one. I guess because the audio information is still in the cache. I tried many ways to delete the Audio object but it did not work.
2. When I try to change the audio file names they cannot be played. I guess because as you say, one somehow has to wait before the file is buffered?

I have also tried to solve the second point but it did not work out so far. How did you do it? Could you maybe provide some code?

[Jordan Beim]

Hi Benjamin,

I'm not terribly experienced in html or javascript so my solution may be somewhat crude, but it has worked well enough for our purposes. My uihtml module loads javascript code that establishes a few event listeners for several status changes of an html audio object. The statuses I listen for are:

1. canplaythrough - this gives only an estimate of a large enough buffer, but for relatively stable internet connections it is accurate enough for our purposes. I have not found a better way to ensure that the file is completely buffered.

2. play - so the matlab app can have an approximate start time of audio playback.

3. ended - so that if audio playback needs to block code execution it can be released once the audio has finished.

Each of these events results in a different code being saved in htmlComponent.Data, and MATLAB's uihtml module listens for changes in that data structure. I use waitfor() to force the MATLAB app to await appropriate status codes from the audio object event listeners to control the buffering and output of sound.

In addition to the audio object event listeners, the javascript htmlComponent has a DataChanged listener to recieve codes from the MATLAB app. The commands the javascript listens for are:

1. initialize - create the audio event listeners and return a success message.

2. load - set the src property of the audio object to the filename specified in the load message.

3. play - run the play() method to start playback.

From the standpoint of my web app the playback control flow goes as follows:

On app startup: initialize the html audio code to add the previously mentioned audio event listeners.

When ready to play sounds:

0. Generate stimulus audio and save to a .wav file using a unique file name.

1. Save a numeric code in the Data field of the uihtml object in MATLAB to trigger the DataChanged listener to update the src property of the audio object with the unique filename that is also passed with the command.

2. MATLAB waits for htmlComponent.Data code generated by the html audio status 'canplaythrough'

3. Upon recieving 'canplaythough' message, MATLAB app sends a status code to trigger the DataChanged listener to run the play() method of the audio object. In some experiments, we give the listener control over when playback starts using a uibutton in the MATLAB web app gui, in other cases playback happens immediately after the canplaythrough status is detected.

4. MATLAB listens for the html data to indicate playback has started.

5. MATLAB listens for the html data to indicate playback has ended.

6. MATLAB deletes the audio file that was previously sent/played and prepares to generate/write a new audio file with a unique name.

That loop is repeated on a per trial basis for the purposes of the experiments that we're running on the web using the web app server. I hope this paints a decent picture, but I'd be happy to share more specifics if there are still missing pieces. I'm not sure how generalizable the actual HTML code will be, so let me know if you still want to take a look.

[Benjamin Lentz]

Hi Jordan,

thanks for your quick and detailed answer!

That's even more complicated than I expected. That must have been a lot work to figure out.

So as I understand, one keypoint is that MATLAB and the HTML (or Javascript) code have to interact frequently. They are checking a (re)action of the respective other side by waiting on changes of the Data field. I guess for this communication there should be only one HTML object created in the MATLAB Web App which is saved and reused?

I already found a nice documentation about the communication of MATLAB and HTML ( mathworks.com/help/matlab/creating_guis/create-an-html-file-that-sets-data-or-responds-to-data-changes-from-matlab.html ) and your explanations are quite helpful so I'm optimistic that it will work. Thanks a lot!