How to use automatic captions for Twitch streams with OBS
by Saralene
In this document, I will instruct you on how to install the OBS captions plugin by ratwithacompiler, starting with the shortest and simplest way to get it working, and then I will introduce its other features and how you can utilize it to its fullest. I’ll try to be as straightforward as possible, and will use images to help make it clear.
Table of Contents
How captions appear to your audience
Why captions matter
You might be asking yourself, first of all, Saralene, why do you want more streamers to use this plugin? What are the benefits of using Closed Captions on my stream? You might be saying, I know how inaccurate automatic captions can sometimes be, and this just doesn’t seem worth the trouble.
I totally get that. However, there are a lot of compelling scenarios that make captions worth having, and I personally believe that even if captions were to be only 50% accurate, it is highly worth having them for the reasons I outline below.
- People who are deaf cannot get anything at all out of your streams under normal circumstances without using external software. While they might already have such software, it will show your willingness to offer accessibility if you offer captions on your own channel. Allowing them to understand even some of your stream is significantly better than if they could not watch your stream meaningfully at all.
- People who are hard of hearing but otherwise can hear, will often enable captions or subtitles for any and all content they watch, as much as possible. Even if the captions are a bit wrong, they can use this to make it easier for them to understand what is being said overall, and put less of a strain on them. This could make them more likely to watch your stream more often.
- For various reasons, people may be watching your stream with the audio muted or in scenarios where they cannot hear it clearly due to outside factors, without headphones to help them out. This is super common in multitasking scenarios or when people are watching streams while they are at work. In this situation, captions will allow them to at least have some idea of what is being said on a stream that they like to watch even during times where they otherwise can’t hear it clearly.
With all of this in mind, I believe that having captions at all allows a wider audience, allows the existing audience to attend streams they would otherwise be unable to, and generally improves conversion rate of short-time viewership to long-term viewership among otherwise marginalized groups of people.
This is really important, and it doesn’t take much effort at all to actually implement automatic Closed Captions via Google Cloud on your stream! It’s free and requires only a few basic setup steps on your part! If you want to go the extra mile to set it up to your exact liking, such as adding text replacement filters for usernames or other major terms, you can, but the basic setup takes only a couple minutes and a few very short steps!
How captions appear to your audience
Twitch’s player natively supports embedded Closed Captions in all streamed content. They will not show up in Clips, but you can find them anywhere else that they are featured.
(An example of the default appearance of closed captions on a stream)
When Captions are enabled for a stream, they will default to Off for all viewers who do not ordinarily use Captions, so you don’t have to worry about this new feature suddenly surprising your existing viewers. Instead, a small “CC” button will appear on their player next to the other standard options that appear.
(How the closed captions toggle button appears on a typical stream player, disabled and enabled)
Users can customize the appearance of their captions to their liking out of a variety of different colors, background settings, opacity settings, and locations, through the settings menu provided by Twitch as part of the player settings.
(Location of the Closed Caption Options in Twitch’s player settings menu)
(Example of the Closed Caption Settings dialog box)
All of this allows viewers to customize how their captions look to best suit their own situation, while making it a completely optional feature that will go completely unused by the users who don’t need it, or find it too distracting.
Having said that, I realize some of you might be considering adding captions directly to your stream, baked into the video. That’s okay to do if you want to, but I suggest that you enable embedded captions as well if you do that.
The reasoning for this is simple: User needs can vary more wildly than we expect. Different users will have different usage scenarios, and customization will allow it to better suit their specific needs.
Requirements
-Must have OBS Studio version 28.0 or later.
NOTE: Streamlabs Desktop or other similar programs, even though they are based off of OBS, might not work. SE.Live works fine since it is also just an add-on to OBS.
-Must NOT be using the AMD hardware encoder on Windows (Mac/Linux are fine). The reason it doesn’t work when the AMD hardware encoder is used on Windows is unclear. You should typically probably be using software encoding instead anyway, if you have a relatively modern CPU.
Installation
- Open https://github.com/ratwithacompiler/OBS-captions-plugin/releases in your browser
- Download the latest release for your OS (Windows, Linux, or Mac)
- Copy the DLL file from the release’s zip file to your OBS’ 64-bit plugins folder. (Not compatible with 32-bit versions of OBS, but most people aren’t using them anymore anyway.)
On Windows, this should be: (OBS folder, defaults to C:\Program Files\obs-studio)\obs-plugins\64bit\
On Linux, this should work: $HOME/.config/obs-studio/plugins
On MacOS, you should find this folder by selecting “File” and then “Show Settings Folder” in OBS, then navigating to the “plugins” subfolder. - Congratulations! It’s installed and will now appear as a dock in your OBS window!
- If you do not see the dock when opening OBS, please select “Docks” in OBS and then check “Captions”.
Basic setup instructions
To get started with the captions plugin, we have to select a sound source and enable it. This should ideally be a separated microphone source. If your mic is separate from the rest of your sound, you will get better results than YouTube auto-captioning. Otherwise, it will be about the same as YouTube auto-captioning in quality.
- Click on the cog/gear icon on the Captions dock to open the Captions settings.
- Check the box marked “Captioning Enabled”.
- Set the “Caption Source” to “Mic/Aux”, or to your other voice audio track if using multiple voice sources (see “Using multiple audio tracks” below).
- Click “Save”. You now have Captions!
- To tell if Captions are properly enabled, please check the Dock. It will tell you the current status of your captions. “CC Off” will be displayed when not streaming or recording. When it is enabled and working, you will see a red circle 🔴 as well as information on what audio device is being used and whether it is for streaming, recording, or both.
Captions disabled entirely:
Captions enabled, but not running:
Captions enabled, and running:
Testing your captions
Testing your captions before going live is a good idea. There’s an easy way to do this, because the captions will display live in real time when they are active, and you can set them to be active while recording, not only while streaming. I’ll detail the process here.
- First, you want to set the captions to output to local recordings as well. Do this by changing the “Output To” option in the Captions Settings. If you don’t want the captions in your recordings, you can just do this temporarily for the sake of easier testing.
BE ADVISED: If you output to recordings, and then upload them to YouTube, this might mess up the rate captions are displayed on YouTube unless you delete them manually. So, in the long term, if your local recordings are going to be uploaded to YouTube, consider only outputting to streams.
- Now, you want to start a test recording. This is done by pressing Start Recording in OBS.
- Speak into your microphone, and observe the Captions Dock. You should see something resembling the words that you have said into the microphone. The Captions Dock will constantly monitor anything you say, so you can try it out and see if it’s to your liking.
- Stop the recording, having confirmed that your captions are working.
If they did NOT work, the things to look at are:
- Have you set the correct Caption Source?
- Is your microphone moving the volume bars in OBS?
- Does the Captions Dock correctly show “🔴CC (Mic/Aux) for Recording” during the recording?
If all of these look fine, it should be working – there’s a chance that the Google Cloud API used for this is currently down or that your voice was just too quiet for it to understand. Adjust accordingly.
Using multiple audio tracks
One particularly advanced usage scenario involves using multiple audio tracks to capture multiple voices or sources for captioning. For example, if you are doing a collaboration with another streamer, you may need to do this if you want both of your voices to be captioned.
This is probably the most complicated and advanced part of setting up the captions plugin, but it serves an important purpose, so I want people to know how they can make use of it.
In OBS, there is a section for “Advanced Audio Properties” that you can access from the three-dot menu next to any of your audio sources in the Audio Mixer.
In this settings section, you can see that all audio is set to one or all of 6 audio tracks by default.
Typically, most streamers never need to mess with these tracks, unless they are separating out licensed music from their VOD recordings. However, this setting can also be used to decide which audio tracks will be sent to the Captions plugin, and which will not.
Here is an example from my own stream audio settings, showing that Audio Track 6 is disabled for all of the sources except the ones that have some actual microphone audio being sent to them.
I do recommend using Track 6 for this. Do not disable the other tracks on the microphone sources, or you will most likely make yourself inaudible to your viewers.
You can then set the Caption Source in the Caption Settings to the audio track that you have separated away from the others, as seen below.
As a side note, if you absolutely cannot separate your microphone audio from the rest of your audio, this will all still work even in situations where all of your audio tracks are being included, even ones that have a mix of microphone audio and other audio, i.e. music or gameplay audio.
In that particular situation, the quality of the captions will mostly just be reduced to something closer to that of the automatic captions you see on your average YouTube video. Certainly there are worse fates than that; it’s still much better than nothing.
Using text filtering
The last tab of the Caption Settings is the Text Filtering section.
In short, Text Filtering allows you to define specific words that will be replaced with other words whenever they are caught on your stream. For example, you might find that your unique streamer name is not well understood by the captions, and you might want to fix that.
The text filters allow for scenarios like this or other common misunderstandings you want to correct.
You don’t need to use text filters in any way. They are only there for you to use if you want to go the extra mile in improving its recognition of specific terms.
Text filtering also supports regular expressions (Regex). If you don’t already know what those are, you probably don’t need to use the Regex filter setting. Regular expressions are for detecting specific text patterns so that you can replace them in any situation.
Given how captions work and because they are captioning your own trusted stream hosts, you probably do not need any sort of advanced text filtering at all.
Other options and settings
Profanity Filter
The profanity filter will censor profanity into asterisks to keep your captions clean.
If the option is enabled, Google Cloud will determine which words require censorship, but this includes pretty much any profanity you would worry about.
While the drop-down box says that the Profanity Filter is unreliable, it has proven extremely reliable in the majority of my tests!
Please note that the most common and undeniable slurs are always censored, regardless of whether the Profanity Filter is enabled or not. You do not necessarily need to enable the Profanity Filter if your only goal is to keep your stream safe.
You might consider experimenting with this in your own time to determine if you want to enable the Profanity Filter, or otherwise set up any text filtering of your own.
Caption When
This option simply controls if captions capture when the Caption Source is audible (“Caption Source is heard on stream”, default) or if captions capture even when it is not (“Mute Source is heard on stream”).
This is mainly useful in very complex scenarios where you are capturing captions from a different source than the ones you are actually recording or sending to your viewers. Most users will never need to change this option.
Transcripts
Will fill in information on saving Transcripts at a later date. Sorry about that! It’s pretty straightforward though.
Open Captions
Will fill in information on using Open Captions at a later date. Sorry about that!
I’ll try to fill these sections in sometime soon!