Streaming in Depth: A Technical Exploration of Twitch’s Protocols

10 min readOct 5, 2023

Photo by Caspar Camille Rubin on Unsplash

Welcome to our journey into the inner workings of Twitch.tv! In this article, we’ll dig deep into how Twitch makes its main services work behind the scenes. An interesting element is the variety of protocols used within the same website, due to the need to offer services of a heterogeneous nature. Special attention will be paid to the Login Phase, Chat, and Video Streaming.

What is Twitch?

Twitch is a video streaming platform owned by Twitch Interactive, a subsidiary of Amazon. The site focuses primarily on video games, including playthroughs and broadcasts of Esport. The site’s content can be viewed live or on demand.

The site allows you to filter your search by type of game and then choose the stream you like from the ones suggested. Each stream has a chat which, if you have an account, allows you to interact with all the users watching the stream. It is also possible to donate to the streamer to show your support, as the platform is completely free and does not require any subscription from users or streamers.

Decrypting Twitch Traffic

Although streaming is accessible to anyone, the connections to the various services offered by Twitch are — of course — completely encrypted. For this reason, we had to decrypt all generated traffic to have a better overview of the traffic and data we wanted to analyze.

For this purpose, we exported the environment variable SSLKEYLOGFILE to a user-accessible file, after which we started the browser in the same terminal.

In addition, we set the file path in Wireshark where the environment variable will produce its output, so Wireshark can see all generated traffic in plain text.

In Wireshark: Edit -> Preferences -> Protocols -> SSL

Behind the Login Process

After visiting Twitch's home page, the login procedure was carried out by entering one’s credentials. Using Wireshark and analyzing the captured traffic and the generated log file respectively, it was possible to obtain some useful information.

To visualize the conversation between the client and the login server, we have filtered the packets captured (sniffed) by Wireshark to show only those packets where the IP address 52.27.1.153 was involved.

The IP address in question, located in Wilmington (USA), is one of the servers Twitch uses to authenticate its users.

TCP and TLS Handshake

The connection was opened using a three-way handshake, a procedure used to reliably establish a TCP connection between two hosts that involves the exchange of three messages between client and server.

Next, there is an exchange of packets using the TLS protocol. This protocol enables secure communication between client and server by allowing the exchange of various information.

The Client Hello package contains attributes that the client sends to the server, including:

Session ID: contains the session identification that the client would like to use during the connection.
Chiper Suite: contains the list of encryption algorithms supported by the client in order of preference.

The server responds with Server Hello communicating the ID for the connection and the algorithm chosen from those proposed by the server.

The Client Key Exchange and Server Jey Exchange packages generate the keys for encryption and authentication of the data coming from the application level.

The final message is Server Hello Done, which indicates the end of the “Hello” phase to the client. After that, the Change Cipher Spec is sent to inform that the connection is encrypted and the data can be transmitted securely.

The method just described does not only take place during the login phase but whenever a client establishes a connection with a server using the HTTPS protocol. From now on, to avoid redundancies, the TLS protocol will no longer be described, since the procedure is the same for each server contacted.

User Login

Analyzing the HTTP traffic instead, it can be seen that the client requests the authentication page via the GET method. The server, after checking the validity of the request (pages available or not), responds with a 200 OK and returns the HTML code of the page in question.

A cookie test was also carried out to ensure that the requests came from the same browser.

After the client had received the HTML code of the login page, a deliberately incorrect username and password were entered. More precisely, the username was entered correctly (i.e. "Nindo7"), while the password entered was "prvmellia" instead of "provamellia". In addition, thanks to the Wireshark settings described in the section above (i.e. Decrypting Twitch Traffic), it was possible to display the data in clear.

As is visible from the image below, in fact, the server replies with 400 Bad Request because the authentication procedure was not successful.

When both username and password are entered correctly, the client responds with 200 OK, indicating that the procedure was successful.

Understanding Twitch.tv’s Chat

In each channel, there is a chat that allows messages to be exchanged between streaming viewers. The application layer protocol used for chat is IRC defined in several RFCs, each proposing different implementations. Twitch follows the protocol defined in RFC 1459, with a few modifications. Wanting to provide an encrypted transmission, the lower-level protocol used will be TLS, which is why communication takes place via port 443.

It is possible to filter all traffic generated by Twitch to get only Chat traffic. The IP address of the server in this case is 52.42.192.125 located in Portland, USA.

That’s a lot of information, let’s go through it one by one.

From TCP to WebSocket

A TCP connection is established between the client and the IRC server.

TCP | [SYN]
TCP | [SYN,ACK] 
TCP | [ACK]

An encrypted connection is established between the client and the server, following various steps defined in RFC 5346.

TLSv1.2 | Client Hello
TLSv1.2 | Server Hello
TLSv1.2 | Server Key Exc TLSv1.2 
TLSv1.2 | Client Key exchange TLSv1.2
TLSv1.2 | Change Cipher Spec

A WebSocket connection is established, following the steps defined in RFC 6455. The client sends a message that it wants to open a WebSocket connection with the higher-level protocol IRC and the server announces that the connection has been accepted.

HTTP | GET /HTTP /1.1 
HTTP | HTTP/1.1 101 Switching Protocols

Messages are exchanged via the WebSocket connection to set the IRC chat parameters, after which the actual exchange of chat messages takes place.

WebSocket

Unpacking WebSocket Messages

First of all, the client enables the use of Twitch chat commands.

CAP REQ :twitch.tv/tags twitch.tv/commands\r\n

An example could be CLEARCHAT which allows Twitch to ban a user permanently or temporarily.

Then the client sends the token provided by the API and the nickname of the Twitch account to connect to the service.

PASS oauth:kwm7uhhx4n4i5wmrrhwem882ppto32\r\n
NICK nindo7\r\n

At this point, the server informs the client that the use of commands has been correctly enabled.

:tmi.twitch.tv CAP * ACK :twitch.tv/tags twitch.tv/commands\r\n

And also that the authentication was successful.

:tmi.twitch.tv 001 nindo7 :Welcome, GLHF!\r\n
:tmi.twitch.tv 002 nindo7 :Your host is tmi.twitch.tv\r\n
:tmi.twitch.tv 003 nindo7 :This server is rather new\r\n
:tmi.twitch.tv 004 nindo7 :-\r\n
:tmi.twitch.tv 375 nindo7 :-\r\n
:tmi.twitch.tv 372 nindo7 :You are in a maze of twisty passages,all alike.\r\n :tmi.twitch.tv 376 nindo7 :>\r\n

The client requests to connect to the "gosu" channel.

JOIN #gosu\r\n

The server sends information needed by the user for the application, such as the user’s ID and the set of emoticons he is authorized to use in any channel.

@badges=;color=;display-name=Nindo7;emote-sets=0,19151;user- id=85756562;user-type= :tmi.twitch.tv GLOBALUSERSTATE\r\n

The server reports that the request for access to the “gosu” channel has been accepted.

:nindo7!nindo7@nindo7.tmi.twitch.tv JOIN #gosu\r\n

The server communicates your account information, i.e. whether you are subscribed to the channel, the name to be displayed, and the code of the emote set that can be used in this channel.

@badges=;color=;display-name=Nindo7;emote- sets=0,19151;mod=0;subscriber=0;user-type= :tmi.twitch.tv USERSTATE #gosu\r\n

The server communicates information about the chat room, i.e. who is allowed to write, whether emote can be used, and the language of the broadcaster.

@broadcaster-lang=en;emote-only=0;followers-only=-1;r9k=0;room- id=41939266;slow=3;subs-only=0 :tmi.twitch.tv ROOMSTATE #gosu\r\n

The server communicates the list of users connected to the chat, however, the chat room only provides broadcast messages in the channel.

:nindo7.tmi.twitch.tv 353 nindo7 = #gosu :nindo7\r\n :nindo7.tmi.twitch.tv 366 nindo7 #gosu :End of /NAMES list\r\n

What happens instead when we try to send a message? This is a message sent from the client to the server, where “pasta” is the message sent and @sent-ts=%number indicates the timestamp of the message.

@sent-ts=1496049101666 PRIVMSG #gosu :pasta

Closing Thoughts on Chat

It is important to note that traffic without the procedure outlined in the Decrypting Twitch Traffic section above would result from the TLS protocol the only messages displayed by Wireshark as TLS are the packets used to establish an encrypted connection.

The protocol used above TLS is WebSocket, an Internet technology developed by W3C standardized in RFC 6455. This protocol makes it easier to handle data in real time, as the server is allowed to send data to the client without having to be prompted by the client in any way. In this way, the client does not need to update the HTML page every time the server has to send new data.

The complexity of the IRC protocol makes it hard to cover all of its functionalities in brief, which is why we decided to report only the chat setup and the sending of a few messages, however, it is fair to point out that the IRC protocol has a vast range of functionalities not covered in this report. In fact, through the traffic analyzed, we noticed that in the case where the user connects to a stream without having a Twitch account, a random nickname is provided for the chat, which allows them to see the messages sent but not to send any.

Twitch.tv’s Streaming Ecosystem

In addition to Wireshark, we also used the "Inspect" command element of the Chromium web browser to analyze the traffic generated by the streaming. With the “Network” option, it is indeed possible to view all the traffic captured at the application level.

Analyzing the packets, we noted that the streaming protocol used by Twitch is HTTP Live Streaming (HLS).

HLS is a protocol created by Apple Inc. which, in addition to providing a reliable streaming service, allows for adaptation of the bit rate of media files sent from the server to the client’s network conditions. This is of great utility for the user as it allows them to continue watching streaming at a lower resolution even when network conditions are not optimal.

HLS is based on the standard HTTP protocol, which, being the same used in classic web browsing, appears to be well tolerated by firewalls and proxies that regulate normal traffic on the standard port for HTTP. In contrast, other more specific protocols, such as RTP, need dedicated ports and run a greater risk of being blocked.

The operating principle is based on dividing the content to be exchanged into small fragments (the video in this case). You can see how each segment of the movie is a file with the extension .ts or an audio/video container in MPEG-2 Transport Stream format. When streaming begins, the client will first request the playlist file (in .m3u8 format), which shows it. The order in which the movie files (.ts precisely) must be downloaded and played, and then the various fragments of video.

Important information can be gleaned from the contents of the file regarding the type of file that will be received from the client.

#EXT-X-TARGETDURATION:5 → Specifies that the
multimedia files will have a maximum length of 5 seconds.
#EXT-X-MEDIA-SEQUENCE:X → Specifies that X is the number
of the sequence of the first segment.
#EXTINF:2.000 → Specifies the actual duration of the segment.

In this case, we filtered captured packets on the IP address 52.223.195.23, where the IP address in question, located in Seattle (United States), is one of the servers that Twitch uses to receive content multimedia mentioned above.

From the capture, you can see an alteration of requests from the client.

HTTP GET /v1/playlist/…/*.m3u8
HTTP GET /v1/segment/…/*.ts

And the related server responses

HTTP/1.1 200 OK (application/vnd.apple.mpegurl)
HTTP/1.1 200 OK (video/mp2t)

By checking the size of the captured packets, you can see how they have noticeably different sizes. The responses respectively contain a “playlist” file and a multimedia file, which justifies their larger size compared to requests.

Conclusions

Twitch is a platform that has revolutionized the world of video game entertainment. Until then the world of video game entertainment consisted of videos uploaded to YouTube which were displayed at times after their creation. With the advent of Twitch.tv pro-players and entertainers had the chance to play their favorite games live, showing off their emotional reactions in real-time. This change in modality allowed for greater contact between the content creator and their fans.

By inserting advertising within your channel it is possible to make economic profit from transmission. Being a Streamer has become a real job for some.

This article wouldn’t have been possible without the invaluable input and hard work of Damiano Franzò and Carlo Negri. My friends, former roommates, and course colleagues in Computer Engineering at Politecnico di Torino.