Local Audio Share

Native file change

Add FOREGROUND_SERVICE, FOREGROUND_SERVICE_MEDIA_PROJECTION permission and HMSAudioshareActivity activity in the android/app/src/main/AndroidManifest.xml file -

<manifest {...} >
  <uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
  <uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION" />
  ...

  <application {...} >
    ...

    <activity
      android:name="com.reactnativehmssdk.HMSAudioshareActivity"
      android:label="@string/app_name" />

    ...
  </application>
</manifest>

Start Audio Share

You can call startAudioshare method available on HMSSDK instance with the mode in which you want to start the audio stream.

Audio stream mode is of type HMSAudioMixingMode enum. Lets take a look at values of HMSAudioMixingMode enum -

1. HMSAudioMixingMode.TALK_ONLY: Only data captured by the mic will be streamed in the room.
2. HMSAudioMixingMode.TALK_AND_MUSIC: Data captured by the mic and playback audio being captured from the device will be streamed in the room.
3. HMSAudioMixingMode.MUSIC_ONLY: Only the playback audio being captured from the device will be streamed in the room.

Following is the code snippet of using startAudioshare method -

import { HMSAudioMixingMode } from '@100mslive/react-native-hms';

try {
    await hmsInstance.startAudioshare(HMSAudioMixingMode.MUSIC_ONLY);

    console.log('Start Audio Share Success');
} catch (error) {
    console.log('Start Audio Share Error: ', error);
}

File Player and Mic Nodes

First you need to create instances of HMSAudioFilePlayerNode and HMSMicNode classes.

HMSAudioFilePlayerNode requires a parameter of type string which will be used at control music player in room.

import { HMSAudioFilePlayerNode, HMSMicNode } from '@100mslive/react-native-hms';

// creating File Player Node, which will be used as an audio source
const audioFilePlayerNode = new HMSAudioFilePlayerNode('audio_file_player_node');

// creating Mic Node, which will be used as an audio source
const micNode = new HMSMicNode();

Audio Mixer Sources

Next, you create an instance of HMSAudioMixerSource class using list of nodes that you created in above step -

import { HMSAudioMixerSource } from '@100mslive/react-native-hms';

const audioMixerSource = new HMSAudioMixerSource({
    node: [audioFilePlayerNode, micNode]
});

SDK Initialization with Audio Mixer Sources

Now you have custom audio sources to setup audio share in your app.

Next, you set "custom audio source" on audioSource property while creating an instance of HMSAudioTrackSettings class.

Lets take a look at the SDK Initialization with custom audio sources -

import { HMSSDK, HMSAudioTrackSettings, HMSTrackSettings } from '@100mslive/react-native-hms';

// creating "Custom Audio Sources" from the "Audio Mixer Source" we created in above step
const customAudioSources = audioMixerSource.toList();

// Creating Audio Track Settings with "Custom Audio Sources"
const audioSettings = new HMSAudioTrackSettings({
    audioSource: customAudioSources
});

// Creating Track Settings
const trackSettings = new HMSTrackSettings({
    audio: audioSettings
});

// Creating instance of `HMSSDK` with custom track settings
const hmsInstance = await HMSSDK.build({ trackSettings });

Play and Stream Audio File(s)

You can use play method available on HMSAudioFilePlayerNode instance (that we created in 1st step) to play a file on device in meeting room.

play method accepts three parameters:

1. fileUrl: string local URL of the file on device. You can get the local URL of a file on device using library such as react-native-document-picker.
2. loops: boolean[optional] whether to play file in a loop.
3. interrupts: boolean[optional] whether to interrupt the already playing file. If this is set to false then current file will be played after already playing file.

try {
  const audioFilePlayerNode = ... // `HMSAudioFilePlayerNode` instance that we created in 1st step

  await audioFilePlayerNode.play('...file_url...');
  console.log('Start Audioshare Success');
} catch(error) {
  console.log('Start Audioshare Error: ', error);
}

Play multiple files "back-to-back"

You can use interrupts property (3rd parameter) of play method available on HMSAudioFilePlayerNode instance (that we created in 1st step) to play multiple files back-to-back. When you set interrupts property to false while calling play method, the SDK does not interrupts the already playing audio file, if any is playing. Current Audio file will be played after the already playing file.

The default value of interrupts property is false. So, You don't have to explicitly pass this value to play method. You can simply call play method multiple times with different files URLS without passing interrupts property. This way each audio file will be played back-to-back.

try {
  const audioFilePlayerNode = ... // `HMSAudioFilePlayerNode` instance that we created in 1st step

  audioFilePlayerNode.play('...1st_file_url...');

  // `interrupts` property (3rd parameter) usage example
  // Below statement is same as above
  audioFilePlayerNode.play('...2nd_file_url...', undefined, false); // 2nd file will be played after 1st one

  audioFilePlayerNode.play('...3rd_file_url...'); // 3rd file will be played after 2nd one

  console.log('Start Audioshare Success');
} catch(error) {
  console.log('Start Audioshare Error: ', error);
}

Play multiple files "concurrently"

You can create instance of HMSAudioMixerSource class (refer to step 2 for more info) with multiple instances of HMSAudioFilePlayerNode class to play multiple files concurrently.

import {
    HMSAudioFilePlayerNode,
    HMSMicNode,
    HMSAudioMixerSource
} from '@100mslive/react-native-hms';

// Multiple `HMSAudioFilePlayerNode` class instances to play multiple files concurrently
const backgroundMusicNode = new HMSAudioFilePlayerNode('background_music_node');
const audioFilePlayerNode = new HMSAudioFilePlayerNode('audio_file_player_node');

// Mic node
const micNode = new HMSMicNode();

// Creating Audio Mixer Source with "Mic" node and multiple "Audio File Player" node
const audioMixerSource = new HMSAudioMixerSource({
    node: [backgroundMusicNode, audioFilePlayerNode, micNode]
});

// Initialize SDK with `audioMixerSource` [refer to step 3]

After SDK initialization with multiple file player nodes, you can play a looping background music at low volume and an audio file at the same time by passing second parameter loops in play function as true -

// Playing background music file in loop and setting value for `interrupts` to `true`
// so that it can be played with other files
backgroundMusicNode.play(fileUrl, true, true);

// Playing normal music file and setting value for `interrupts` to `true`
// so that it can be played with background and other files
audioFilePlayerNode.play(fileUrl, false, true);

Share Audio playing on your iPhone

Note: iOS allows to get access to audio playing on iOS device (for example, from another app like spotify) only while broadcating your entire iPhone screen. So, you should implement screen sharing in your app for this to work. You should follow Screen Share setup docs for iOS to set it up.

After you have completed the screenshare setup for iOS, You can follow below steps to enable device audio broadcasting while sharing your screen:

1. Create instance of HMSAudioMixerSource class (refer to step 2 for more info) with instance of HMSScreenBroadcastAudioReceiverNode class.

import {
    HMSScreenBroadcastAudioReceiverNode,
    HMSAudioFilePlayerNode,
    HMSMicNode,
    HMSAudioMixerSource
} from '@100mslive/react-native-hms';

// creating Node to share audio playing on iOS
const screenBroadcastAudioReceiverNode = new HMSScreenBroadcastAudioReceiverNode();

// creating File Player Node, which will be used as an audio source
const audioFilePlayerNode = new HMSAudioFilePlayerNode('audio_file_player_node');

// creating Mic Node, which will be used as an audio source
const micNode = new HMSMicNode();

// Creating Audio Mixer Source with "Mic" node and multiple "Audio File Player" node
const audioMixerSource = new HMSAudioMixerSource({
    node: [audioFilePlayerNode, micNode, screenBroadcastAudioReceiverNode]
});

// Initialize SDK with `audioMixerSource` [refer to step 3]

Now your audio mixer source is set to receive audio from your broadcast extension.

2. Next, you need to setup broadcast extension to send audio to the main app

Broadcast extension receives audio that's playing on your iOS device in processSampleBuffer function in your RPBroadcastSampleHandler class. To send audio from broadcast extension to main app, you call process(audioSampleBuffer) function on HMSScreenRenderer:

override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
    ...

    case RPSampleBufferType.audioApp:
        _ = self.screenRenderer.process(audioSampleBuffer: sampleBuffer)
        break
    ...
}

Now your broadcast extension is set to send audio to the main app.

And that's it. Now your custom mixer source in the main app can receive the audio from broadcast extension as well.