Render Video - SurfaceView (Deprecated)

At the very least for rendering video you'll need an instance of org.WebRTC.SurfaceViewRenderer and a HMSVideoTrack instance from a peer.

For the SurfaceViewRenderer, as of 2.0.9 of the 100ms Android SDK, only adding the SDK is enough to receive WebRTC objects including the SurfaceViewRenderer. In versions prior to this, add the dependency implementation 'org.WebRTC:google-WebRTC:1.0.32006' to receive SurfaceViewRenderer.

In XML layouts it would look like

    <org.WebRTC.SurfaceViewRenderer
        android:id="@+id/peerVideo"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        />

The peer's video is in hmsPeer.videoTrack and their ScreenShare video will be an instance of HMSVideoTrack in the list hmsPeer.auxiliaryTracks.

You would get them like so:

val hmsVideoTrack : HMSVideoTrack? = hmsPeer.videoTrack

Getting and Preparing a SurfaceView

Get an instance of the SurfaceView.

val surfaceView : SurfaceViewRenderer = findViewById(R.id.surface_view)

Set the scaling to whatever you prefer. Aspect Fit is recommended for common use cases. See more scale types here.

surfaceView.setEnableHardwareScaler(true)
surfaceView.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT)

Adding a video to the SurfaceView

Each time you want to add a video to the surfaceview, it needs to be initialized again.

surfaceView.init(SharedEglContext.context, null)

val hmsVideoTrack : HMSVideotrack = hmsPeer.videoTrack
hmsVideoTrack?.addSink(surfaceView)

It's also important to remove it when done.

Removing a video from the SurfaceView

It's very important to remove the video when you're done showing it, or before you show a different video on the same SurfaceViewRenderer.

hmsVideoTrack.removeSink(this)
surfaceView.release()

There is a limit to the number of SharedEglContext's that can be bound per mobile device. It's normally high enough to easily accomodate all the videos that will reasonably fit on a screen. If the video's aren't released however, it becomes very easy to exceed this limit and the app will crash.

Until you call removeSink the video is being streamed over the device's data so it's also important remove sinks.

Special cases for videos in RecyclerViews

If you're going to be using a RecyclerView to display multiple videos in a grid there are specific places where you should do these operations.

Currently the best known way is in the ViewHolder to have a method for binding items, a method to start the surfaceview and a method to release the surfaceview along a boolean check.

The Adapter calls bind item as usual but also stops the surfaceview when it binds. Two additional Adapter methods are overloaded. onViewAttachedToWindow and onViewDetachedFromWindow which will call SurfaceView binding and releasing respectively but while checking the sinkAdded property.

sinkAdded keeps track of whether a track was rendered to a Peer's surface view or not. Since someone might have turned off their video but still be speaking and so a tile might be shown with no video.

Complete source code is available here.

PeerViewHolder

init Scaler types can be set here and don't need to be reset per bind call.

startSurfaceView will init the SurfaceViewRenderer and binding the track if one exists, this also sets the sinkAdded to true.

stopSurfaceView will release the SurfaceViewRenderer if there was a video attached and also remove the video sink.

class PeerViewHolder(view: View, private val getItem: (Int) -> TrackPeerMap) :
    RecyclerView.ViewHolder(view) {
    private val TAG = PeerViewHolder::class.java.simpleName
    private var sinkAdded = false

    init {
        itemView.findViewById<SurfaceViewRenderer>(R.id.videoSurfaceView).apply {
            setEnableHardwareScaler(true)
            setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT)
        }
    }

    fun startSurfaceView() {
        if (!sinkAdded) {
            itemView.findViewById<SurfaceViewRenderer>(R.id.videoSurfaceView).apply {

                getItem(adapterPosition).videoTrack?.let { hmsVideoTrack ->
                    init(SharedEglContext.context, null)
                    hmsVideoTrack.addSink(this)
                    sinkAdded = true
                }
            }
        }
    }

    fun stopSurfaceView() {
        itemView.findViewById<SurfaceViewRenderer>(R.id.videoSurfaceView).apply {

            if (sinkAdded && adapterPosition != -1) {
                getItem(adapterPosition).videoTrack?.let {
                    it.removeSink(this)
                    release()
                    sinkAdded = false
                }
            }
        }
    }

    fun bind(peer: TrackPeerMap) {

        if (!sinkAdded) {
            itemView.findViewById<SurfaceViewRenderer>(R.id.videoSurfaceView).apply {
                setEnableHardwareScaler(true)
                setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT)
                sinkAdded = false
            }
        }
        itemView.findViewById<TextView>(R.id.peerName).text = peer.peer.name
    }

}

PeerAdapter

The adapter stops the surface every time it binds. Starts it when the view is attached to the window, and stops it when the view is detached from the window. Also it passes its own getItem method to the PeerViewHolder in onCreateViewHolder.

    override fun onBindViewHolder(holder: PeerViewHolder, position: Int) {
        getItem(position)?.let {
            holder.stopSurfaceView()
            holder.bind(it)
        }
    }

    override fun onViewAttachedToWindow(holder: PeerViewHolder) {
        super.onViewAttachedToWindow(holder)
        holder.startSurfaceView()
    }

    override fun onViewDetachedFromWindow(holder: PeerViewHolder) {
        super.onViewDetachedFromWindow(holder)
        holder.stopSurfaceView()
    }

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): PeerViewHolder {
        val view =
            LayoutInflater.from(parent.context).inflate(R.layout.layout_peer_item, parent, false)
        return PeerViewHolder(view, ::getItem)
    }