仅从WebRTC的offer侧,来看建立连接的完整过程

  1. GUM:navigator.mediaDevices.getUserMedia()

  2. PeerConnection:new RTCPeerConnection(peerConfiguration) 其中peerConfiguration就是STUN或者TURN servers。它的目的是生成ICECandidates

  3. Add tracks: 把GUM中获取的tracks,循环通过peerConnection.addTrack(track,localStream)进行添加

  4. Create an Offer: offer = await peerConnection.createOffer() 将会生成一个RTC Session Desc (Type: offer or answer+ SDP:Session Description Protocol 用于在网络上描述多媒体会话的参数,时间、媒体格式、协议、带宽)

  5. set LocalDesc:peerConnection.setLocalDescription(offer);

  6. set RemoteDesc:peerConnection.setRemoteDescription(answer)

  7. 添加IceCandidates:获取对方的IceCandidate后,将其添加至peerConnection.addIceCandidate();

发送给信令服务器的最基本的内容

为了实现WebRTC双向的通信,至少需要实现如下信息的交换。假如要实现更加复杂的逻辑,则需要类似用户ID,room等的传递,这个结合业务实际进行signaling server的设计,但是不管如何,下边这两个是建立连接的基础。

  • IceCandidate:Trickle ICE的实现所需。

  • RTC Session Desc (Type: offer or answer+ SDP:Session Description Protocol 用于在网络上描述多媒体会话的参数,时间、媒体格式、协议、带宽)

事例代码中offer及answer实现的完整逻辑

offer:

  1. GUM:get user media

  2. create P.C.

  3. add STUN to P.C. to gen ICECandidate

  4. add tracks

  5. Create an offer :RTC Session Desc (Type(offer or answer) + SDP)

  6. send offer & ICECandidates to signaling server (同时:peerConnection.addEventListener('icecandidate',e=>{}) 这个监听事件实现ICECandidates发生变化时,及时推送给对端。)

  7. get answer

  • setLocalDesc:offer

  • setRemoteDesc:answer

  • add ICECandidates via addEventListener

answer:

  1. GUM:get user media

  2. create P.C.

  3. add STUN to P.C. to gen ICECandidate

  4. add tracks

  5. get offer

  6. Create an answer :RTC Session Desc (Type(offer or answer) + SDP)

  7. send answer & ICECandidates to signaling server (同时:peerConnection.addEventListener('icecandidate',e=>{}) 这个监听事件实现ICECandidates发生变化时,及时推送给对端。)

  • LocalDesc:answer

  • RemoteDesc:offer

  • add ICECandidates via addEventListener

上述过程的原生JS代码 便于理解对照逻辑

服务端


const fs = require('fs');
const https = require('https')
const express = require('express');
const app = express();
const socketio = require('socket.io');
app.use(express.static(__dirname))

//we need a key and cert to run https
//we generated them with mkcert
// $ mkcert create-ca
// $ mkcert create-cert
const key = fs.readFileSync('cert.key');
const cert = fs.readFileSync('cert.crt');

//we changed our express setup so we can use https
//pass the key and cert to createServer on https
const expressServer = https.createServer({key, cert}, app);
//create our socket.io server... it will listen to our express port
const io = socketio(expressServer,{
    cors: {
        origin: [
            // "https://localhost",
            "https://192.168.137.90"
            // 'https://LOCAL-DEV-IP-HERE' //if using a phone or another computer
        ],
        methods: ["GET", "POST"]
    }
});
expressServer.listen(8181);

//offers will contain {}
const offers = [
    // offererUserName
    // offer
    // offerIceCandidates
    // answererUserName
    // answer
    // answererIceCandidates
];
const connectedSockets = [
    //username, socketId
]



io.on('connection',(socket)=>{
    // console.log("Someone has connected");
    const userName = socket.handshake.auth.userName;
    const password = socket.handshake.auth.password;

    if(password !== "x"){
        socket.disconnect(true);
        return;
    }
    connectedSockets.push({
        socketId: socket.id,
        userName
    })

    //a new client has joined. If there are any offers available,
    //emit them out
    if(offers.length){
        socket.emit('availableOffers',offers);
    }
    
    socket.on('newOffer',newOffer=>{
        offers.push({
            offererUserName: userName,
            offer: newOffer,
            offerIceCandidates: [],
            answererUserName: null,
            answer: null,
            answererIceCandidates: []
        })
        // console.log(newOffer.sdp.slice(50))
        //send out to all connected sockets EXCEPT the caller
        socket.broadcast.emit('newOfferAwaiting',offers.slice(-1))
    })

    socket.on('newAnswer',(offerObj,ackFunction)=>{ //ackFunction是需要返回给客户端的数据
        console.log(offerObj);
        //emit this answer (offerObj) back to CLIENT1
        //in order to do that, we need CLIENT1's socketid
        const socketToAnswer = connectedSockets.find(s=>s.userName === offerObj.offererUserName)  //这个部分是为了找到offer的socketId,好将offerToUpdate发给它,而不发给其他人。
        if(!socketToAnswer){
            console.log("No matching socket")
            return;
        }
        //we found the matching socket, so we can emit to it!
        const socketIdToAnswer = socketToAnswer.socketId;
        //we find the offer to update so we can emit it
        const offerToUpdate = offers.find(o=>o.offererUserName === offerObj.offererUserName)
        if(!offerToUpdate){
            console.log("No OfferToUpdate")
            return;
        }
        //send back to the answerer all the iceCandidates we have already collected
        ackFunction(offerToUpdate.offerIceCandidates);
        offerToUpdate.answer = offerObj.answer
        offerToUpdate.answererUserName = userName
        //socket has a .to() which allows emiting to a "room"
        //every socket has it's own room
        socket.to(socketIdToAnswer).emit('answerResponse',offerToUpdate)
    })

    socket.on('sendIceCandidateToSignalingServer',iceCandidateObj=>{
        const { didIOffer, iceUserName, iceCandidate } = iceCandidateObj;
        // console.log(iceCandidate);
        if(didIOffer){
            //this ice is coming from the offerer. Send to the answerer
            const offerInOffers = offers.find(o=>o.offererUserName === iceUserName);
            if(offerInOffers){
                offerInOffers.offerIceCandidates.push(iceCandidate)
                // 1. When the answerer answers, all existing ice candidates are sent
                // 2. Any candidates that come in after the offer has been answered, will be passed through
                if(offerInOffers.answererUserName){
                    //pass it through to the other socket
                    const socketToSendTo = connectedSockets.find(s=>s.userName === offerInOffers.answererUserName);
                    if(socketToSendTo){
                        socket.to(socketToSendTo.socketId).emit('receivedIceCandidateFromServer',iceCandidate)
                    }else{
                        console.log("Ice candidate recieved but could not find answere")
                    }
                }
            }
        }else{
            //this ice is coming from the answerer. Send to the offerer
            //pass it through to the other socket
            const offerInOffers = offers.find(o=>o.answererUserName === iceUserName);
            const socketToSendTo = connectedSockets.find(s=>s.userName === offerInOffers.offererUserName);
            if(socketToSendTo){
                socket.to(socketToSendTo.socketId).emit('receivedIceCandidateFromServer',iceCandidate)
            }else{
                console.log("Ice candidate recieved but could not find offerer")
            }
        }
        // console.log(offers)
    })

})

客户端

const userName = "Rob-"+Math.floor(Math.random() * 100000)
const password = "x";
document.querySelector('#user-name').innerHTML = userName;

//if trying it on a phone, use this instead...
const socket = io.connect('https://192.168.137.90:8181/',{
// const socket = io.connect('https://localhost:8181/',{
    auth: {
        userName,password
    }
})

const localVideoEl = document.querySelector('#local-video');
const remoteVideoEl = document.querySelector('#remote-video');

let localStream; //a var to hold the local video stream
let remoteStream; //a var to hold the remote video stream   
let peerConnection; //the peerConnection that the two clients use to talk
let didIOffer = false;

let peerConfiguration = {
    iceServers:[
        {
            urls:[
              'stun:stun.l.google.com:19302',
              'stun:stun1.l.google.com:19302'
            ]
        }
    ]
}

//when a client initiates a call
const call = async e=>{
    await fetchUserMedia();

    //peerConnection is all set with our STUN servers sent over
    await createPeerConnection();

    //create offer time!
    try{
        console.log("Creating offer...")
        const offer = await peerConnection.createOffer();
        console.log(offer);
        peerConnection.setLocalDescription(offer);
        didIOffer = true;
        socket.emit('newOffer',offer); //send offer to signalingServer
    }catch(err){
        console.log(err)
    }

}

const answerOffer = async(offerObj)=>{
    await fetchUserMedia()
    await createPeerConnection(offerObj);
    const answer = await peerConnection.createAnswer({}); //just to make the docs happy
    await peerConnection.setLocalDescription(answer); //this is CLIENT2, and CLIENT2 uses the answer as the localDesc
    console.log(offerObj)
    console.log(answer)
    // console.log(peerConnection.signalingState) //should be have-local-pranswer because CLIENT2 has set its local desc to it's answer (but it won't be)
    //add the answer to the offerObj so the server knows which offer this is related to
    offerObj.answer = answer 
    //emit the answer to the signaling server, so it can emit to CLIENT1
    //expect a response from the server with the already existing ICE candidates
    const offerIceCandidates = await socket.emitWithAck('newAnswer',offerObj)
    offerIceCandidates.forEach(c=>{
        peerConnection.addIceCandidate(c);
        console.log("======Added Ice Candidate======")
    })
    console.log(offerIceCandidates)
}

const addAnswer = async(offerObj)=>{
    //addAnswer is called in socketListeners when an answerResponse is emitted.
    //at this point, the offer and answer have been exchanged!
    //now CLIENT1 needs to set the remote
    await peerConnection.setRemoteDescription(offerObj.answer)
    // console.log(peerConnection.signalingState)
}

const fetchUserMedia = ()=>{
    return new Promise(async(resolve, reject)=>{
        try{
            const stream = await navigator.mediaDevices.getUserMedia({
                video: true,
                // audio: true,
            });
            localVideoEl.srcObject = stream;
            localStream = stream;    
            resolve();    
        }catch(err){
            console.log(err);
            reject()
        }
    })
}

const createPeerConnection = (offerObj)=>{
    return new Promise(async(resolve, reject)=>{
        //RTCPeerConnection is the thing that creates the connection
        //we can pass a config object, and that config object can contain stun servers
        //which will fetch us ICE candidates
        peerConnection = await new RTCPeerConnection(peerConfiguration)
        remoteStream = new MediaStream()
        remoteVideoEl.srcObject = remoteStream;

        localStream.getTracks().forEach(track=>{
            //add localtracks so that they can be sent once the connection is established
            peerConnection.addTrack(track,localStream);
        })

        peerConnection.addEventListener("signalingstatechange", (event) => {
            console.log(event);
            console.log(peerConnection.signalingState)
        });

        peerConnection.addEventListener('icecandidate',e=>{  
            console.log(e)
            if(e.candidate){
                socket.emit('sendIceCandidateToSignalingServer',{
                    iceCandidate: e.candidate,
                    iceUserName: userName,
                    didIOffer,
                })    
            }
        })
        
        peerConnection.addEventListener('track',e=>{  
            console.log("Got a track from the other peer!! How excting")
            console.log(e)
            e.streams[0].getTracks().forEach(track=>{
                remoteStream.addTrack(track,remoteStream);
                console.log("Here's an exciting moment... fingers cross")
            })
        })

        if(offerObj){
            //this won't be set when called from call();
            //will be set when we call from answerOffer()
            // console.log(peerConnection.signalingState) //should be stable because no setDesc has been run yet
            await peerConnection.setRemoteDescription(offerObj.offer)
            // console.log(peerConnection.signalingState) //should be have-remote-offer, because client2 has setRemoteDesc on the offer
        }
        resolve();
    })
}

const addNewIceCandidate = iceCandidate=>{
    peerConnection.addIceCandidate(iceCandidate)
    console.log("======Added Ice Candidate======")
}


document.querySelector('#call').addEventListener('click',call)


//on connection get all available offers and call createOfferEls
socket.on('availableOffers',offers=>{
    console.log(offers)
    createOfferEls(offers)
})

//someone just made a new offer and we're already here - call createOfferEls
socket.on('newOfferAwaiting',offers=>{
    createOfferEls(offers)
})

socket.on('answerResponse',offerObj=>{
    console.log(offerObj)
    addAnswer(offerObj)
})

socket.on('receivedIceCandidateFromServer',iceCandidate=>{
    addNewIceCandidate(iceCandidate)
    console.log(iceCandidate)
})
 
function createOfferEls(offers){
    //make green answer button for this new offer
    const answerEl = document.querySelector('#answer');
    offers.forEach(o=>{
        console.log(o);
        const newOfferEl = document.createElement('div');
        newOfferEl.innerHTML = `<button class="btn btn-success col-1">Answer ${o.offererUserName}</button>`
        newOfferEl.addEventListener('click',()=>answerOffer(o))
        answerEl.appendChild(newOfferEl);
    })
}

关于WebRTC基础的一些概念点

网络通信基础

WebRTC 实现点对点通信时,需要解决几个关键问题:NAT穿越、媒体协商和网络协商。这些机制共同确保了不同网络环境下的设备能够建立直接通信。

NAT(网络地址转换)

网络地址转换(NAT)是一种通过修改数据包IP标头中的网络地址信息,将一个IP地址空间映射到另一个IP地址空间的方法。NAT主要有两种类型:

  • 一对一:提供IP地址的一对一转换。

  • 一对多:将多个私有主机映射到一个公开的IP地址,这也是IPv4地址耗尽的实用解决方案。

某些NAT和防火墙可能限制外部连接,即使获得了公网IP地址,也可能无法建立直接连接。这种情况下,需要使用TURN服务。

媒体协商

媒体协商是为了解决通信双方能力不对等的问题。例如,如果Peer-A支持VP8和H264编码,而Peer-B支持VP9和H264,它们需要协商使用共同支持的H264编码。这个过程通过交换SDP(Session Description Protocol)信息来实现。SDP描述了会话的详细信息,包括支持的媒体格式、编解码器等。

网络协商

网络协商主要是探测双方的网络类型,以找到可能的通信路径。在WebRTC中,这通过STUN/TURN服务器来确定网络类型,然后收集ICE候选项(candidates)来确定建立哪种类型的连接。

STUN(Session Traversal Utilities for NAT)

STUN是一种允许NAT后的客户端发现自己公网地址和NAT类型的协议。它帮助位于NAT后的设备找出自己的公网地址和端口,这些信息用于在NAT后的两台主机之间建立UDP通信。

TURN(Traversal Using Relays around NAT)

TURN用于在直接连接(即使使用STUN)不可能的情况下。它通过中继服务器转发数据,允许NAT或防火墙后的客户端接收来自外部的数据。客户端通过Allocate请求从TURN服务器获得一个公网中继端口。

ICE Candidates

ICE(Interactive Connectivity Establishment)候选项表示WebRTC与远端通信时可能使用的协议、IP地址和端口。主要有三种类型:

  • host:表示本地网络地址,用于内网P2P连接。

  • srflx(server reflexive):表示通过STUN服务器发现的公网地址,用于非对称NAT环境下的外网P2P连接。

  • relay:表示TURN服务器分配的中继地址,用于对称NAT环境或无法建立P2P连接的情况。

ICE过程就是收集这些不同类型候选项的过程:在本机收集host候选项,通过STUN协议收集srflx候选项,使用TURN协议收集relay候选项。

Trickle ICE

Trickle ICE 是一种优化的 ICE(Interactive Connectivity Establishment)过程,旨在加速 WebRTC 连接的建立。它的核心工作原理是:当创建 offer 或 answer 时,立即发送可能只包含主机候选项的 SDP,而不是等待所有候选项收集完毕。随后,每当发现新的候选项(如通过 STUN 或 TURN 服务器获得的候选项),就立即通过信令通道发送给对方。对方收到新的候选项后,使用 addIceCandidate 方法将其立即添加到 ICE 代理中。这种逐步收集和交换候选项的方法显著缩短了连接建立时间,提高了在复杂网络环境中的连接成功率,并改善了用户体验。Trickle ICE 使得应用能够更快地开始初始连接,同时持续优化连接质量,特别适用于对实时性要求高的应用,如视频通话。尽管实现 Trickle ICE 需要信令层的额外支持,但其带来的性能提升和用户体验改善使其成为现代 WebRTC 应用中不可或缺的技术。

Signaling Server

信令服务器在WebRTC应用中扮演着关键角色,主要负责三个核心功能:实现ICE Candidate和RTC Session Description的交换、房间管理、以及处理人员进出房间的逻辑。首先,信令服务器作为中介,促进参与通信的对等端之间交换ICE Candidate和RTC Session Description信息。RTC Session Description包含了SDP(会话描述协议)数据,描述了媒体格式、编解码器等会话细节。这个交换过程对于建立点对点连接至关重要,使得参与者能够相互发现并协商最佳的通信路径。其次,信令服务器管理虚拟"房间"的概念,这些房间代表不同的通信会话或群组。它维护房间的状态,包括当前参与者的列表和房间的配置信息。最后,信令服务器处理用户进入和离开这些虚拟房间的逻辑。当用户加入房间时,服务器通知房间内的其他参与者,并可能触发必要的连接建立过程;当用户离开时,它会更新房间状态并通知其他参与者。通过这些功能,信令服务器确保了WebRTC应用中实时通信的顺畅进行,尽管它不直接参与媒体数据的传输。

附录及参考:

original YouTube:https://www.youtube.com/watch?v=g42yNO_dxWQ

GitHub:https://github.com/robertbunch/webrtc-starter

Socket IO: https://socket.io/docs/v4/

Mozilla WebRTC API:https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API

Wikipedia:https://en.wikipedia.org/wiki/WebRTC

WebRTC调试地址:

chrome://webrtc-internals/ :这是最重要的WebRTC调试工具。它显示了当前和过去的WebRTC连接的详细信息,包括ICE候选、媒体轨道、统计信息、错误日志等。你可以使用这个工具来查看和分析WebRTC连接的各个方面。

chrome://webrtc-logs/ :这个页面提供了与WebRTC相关的日志文件。你可以下载这些日志文件并用于进一步分析和调试WebRTC问题。

chrome://net-internals/#webrtc :这个页面提供了关于WebRTC网络活动的详细信息,包括网络事件日志、ICE连接状态、TURN和STUN服务器使用情况等。