How to capture tab audio from Zoom/Meet in Chrome Extension and transcribe with Deepgram?

[MUHAMMAD-ALI-BILAL]

Hi, I'm building a Chrome Extension that should capture the remote participant’s audio during web meetings (Zoom, Google Meet, Skype) and transcribe it in real time using Deepgram's STT API.

What I need:

1. Capture tab or output audio (not my microphone).
2. Stream audio live to Deepgram WebSocket API.
3. Show the live transcription in the extension.

Guidelines

• I have read the above statement and can confirm my post is relevant to the GitHub feature areas Issues and/or Projects.

[AreebEhsan]

You can use Chrome’s tabCapture API to grab the audio from the meeting tab itself like Zoom or Google Meet. That way you get the actual audio coming from the tab, not your mic. Just keep in mind it only works for audio playing in Chrome tabs.

Once you have the audio stream, you’ll need to use the Web Audio API to grab small chunks of the audio data and send those to Deepgram’s WebSocket API in real time. This usually means setting up an AudioContext and using something like an AudioWorklet or ScriptProcessorNode to process the audio.

For showing live transcriptions, you can listen to the messages coming back from Deepgram and display them right in your extension popup or even inject the text into the page if you want.

However, capturing system-wide audio for some desktop apps can be much harder and usually not doable directly through Chrome extensions because of security restrictions.

[ahmedbenkrarayc]

Hey! That sounds like an awesome project, building a Chrome Extension that captures and transcribes remote audio in real time is a pretty ambitious task, but definitely doable.

To address what you’re trying to achieve:

1. Capturing remote participant audio (not mic input):
   Unfortunately, Chrome Extensions can’t natively access the system audio or the raw output audio from a tab due to security restrictions. However, you can try using the tabCapture API to capture the audio from the active tab (like Google Meet or Zoom in the browser). Note that:

   • For this to work, the meeting needs to be in a browser tab (not in the Zoom desktop app).
   • You’ll need "tabCapture" permission and a background script to manage it.

2. Streaming audio to Deepgram WebSocket API:
   Once you get the audio stream via tabCapture, you'll likely get a MediaStream. You can use the Web Audio API (e.g., AudioContext.createMediaStreamSource) to process it and convert it to raw PCM, then chunk and stream that to Deepgram’s WebSocket. Deepgram’s API docs are pretty solid on this, so you’ll want to follow their guidance for real-time transcription.

3. Displaying transcription in the extension:
   You can receive the real-time transcript from the Deepgram WebSocket response and update the extension popup or content script UI accordingly. A simple innerText update loop should work, just make sure to handle things like disconnections or transcript formatting.

Let me know if you want code snippets. I’ve tinkered with similar audio processing stuff in browser extensions. Good luck, and curious to see how it turns out!

[MUHAMMAD-ALI-BILAL]

I have written the below code please check it.

Error i am getting is:
Error in event handler: TypeError: chrome.tabCapture.capture is not a function at chrome-extension://hcnhnenogfikagihiglflcgommeeoocf/background.js:7:23
Context
Unknown

Manifest.json file code

{
  "manifest_version": 3,
  "name": "Tab Audio Capture",
  "version": "1.0",
  "description": "Capture audio from current tab.",
  "permissions": ["tabCapture", "tabs", "activeTab"],
  "action": {
    "default_popup": "Popup/Popup.html"
  },
  "background": {
    "service_worker": "background.js"
  },
  "host_permissions": ["<all_urls>"]
}

Background.js file code

let recorder;
let stream;
let socket;

chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
  if (request.action === "start-capture") {
    chrome.tabCapture.capture({ audio: true, video: false }, async (capturedStream) => {
      if (!capturedStream) {
        console.error("Failed to capture tab audio.");
        return;
      }

      stream = capturedStream;

      // Optional: Setup Deepgram WebSocket here

      recorder = new MediaRecorder(stream, {
        mimeType: "audio/webm;codecs=opus"
      });

      recorder.ondataavailable = async (event) => {
        if (event.data.size > 0) {
          console.log("Audio chunk captured:", event.data);
          // Later: send event.data to Deepgram via WebSocket
        }
      };

      recorder.start(250); // Capture chunks every 250ms
      console.log("Started recording.");
    });
  }

  if (request.action === "stop-capture") {
    if (recorder && recorder.state !== "inactive") {
      recorder.stop();
      stream.getTracks().forEach(track => track.stop());
      console.log("Stopped recording.");
    }
  }
});

Popup.html File Code
There is just two button, When i click on the Start Capture button then it will show the error.

<!DOCTYPE html>
<html>
<head>
  <title>Audio Capture</title>
</head>
<body>
  <h3>Capture Tab Audio</h3>
  <button id="start">Start Capture</button>
  <button id="stop">Stop Capture</button>

  <script src="popup.js"></script>
</body>
</html>

Popup.js File Code

let mediaRecorder;
let socket; // For Deepgram later
let stream;

document.getElementById('start').addEventListener('click', async () => {
  chrome.runtime.sendMessage({ action: "start-capture" });
});

document.getElementById('stop').addEventListener('click', async () => {
  chrome.runtime.sendMessage({ action: "stop-capture" });
});

[Tushar051]

Capture Tab Audio:

Use Chrome’s chrome.tabCapture API with "audio": true to capture the audio stream from the tab running Zoom or Meet.
Process Audio Stream:

Use the Web Audio API or MediaRecorder to capture audio chunks from the stream.

Convert chunks into a suitable audio format (e.g., WAV or raw PCM) for Deepgram.

Send Audio to Deepgram:

Use Deepgram’s real-time transcription API by opening a WebSocket connection.

Stream the audio chunks to Deepgram’s API in real time.

Deepgram responds with transcription results as they arrive.

Display Transcription:

Handle Deepgram’s transcription events and display text live in your extension UI.

[MUHAMMAD-ALI-BILAL]

I have written the below code please check it.

Error i am getting is:
Error in event handler: TypeError: chrome.tabCapture.capture is not a function at chrome-extension://hcnhnenogfikagihiglflcgommeeoocf/background.js:7:23
Context
Unknown

Manifest.json file code

{
  "manifest_version": 3,
  "name": "Tab Audio Capture",
  "version": "1.0",
  "description": "Capture audio from current tab.",
  "permissions": ["tabCapture", "tabs", "activeTab"],
  "action": {
    "default_popup": "Popup/Popup.html"
  },
  "background": {
    "service_worker": "background.js"
  },
  "host_permissions": ["<all_urls>"]
}

Background.js file code

let recorder;
let stream;
let socket;

chrome.runtime.onMessage.addListener((request, sender, sendResponse) => {
  if (request.action === "start-capture") {
    chrome.tabCapture.capture({ audio: true, video: false }, async (capturedStream) => {
      if (!capturedStream) {
        console.error("Failed to capture tab audio.");
        return;
      }

      stream = capturedStream;

      // Optional: Setup Deepgram WebSocket here

      recorder = new MediaRecorder(stream, {
        mimeType: "audio/webm;codecs=opus"
      });

      recorder.ondataavailable = async (event) => {
        if (event.data.size > 0) {
          console.log("Audio chunk captured:", event.data);
          // Later: send event.data to Deepgram via WebSocket
        }
      };

      recorder.start(250); // Capture chunks every 250ms
      console.log("Started recording.");
    });
  }

  if (request.action === "stop-capture") {
    if (recorder && recorder.state !== "inactive") {
      recorder.stop();
      stream.getTracks().forEach(track => track.stop());
      console.log("Stopped recording.");
    }
  }
});

Popup.html File Code
There is just two button, When i click on the ** Start Capture ** button then it will show the error.

<!DOCTYPE html>
<html>
<head>
  <title>Audio Capture</title>
</head>
<body>
  <h3>Capture Tab Audio</h3>
  <button id="start">Start Capture</button>
  <button id="stop">Stop Capture</button>

  <script src="popup.js"></script>
</body>
</html>

Popup.js File Code

let mediaRecorder;
let socket; // For Deepgram later
let stream;

document.getElementById('start').addEventListener('click', async () => {
  chrome.runtime.sendMessage({ action: "start-capture" });
});

document.getElementById('stop').addEventListener('click', async () => {
  chrome.runtime.sendMessage({ action: "stop-capture" });
});

[mecodeatlas]

Welcome to the GitHub Community, @MUHAMMAD-ALI-BILAL, we're happy you're here! You are more likely to get a useful response if you are posting your questions in the applicable category and are explicit about what your project entails--giving a few more details might help someone give you a nudge in the right direction. I've gone ahead and moved it for you. Good luck!