DEEPTIKTOK________ _THREE METHODS _______FOR TRACING
___VIDEO MEMES
ELENA PILIPETS / UNIVERSITY OF SIEGEN
UNDERSTANDING TIKTOK VIDEOS
1. TikTok videos are short. 2. TikTok is all about vertical video. 3. Many videos tend to be music focused and use original sounds. 4. TikTok has an extensive library of sound templates. 5. Many videos also apply searchable sound and video effects in combination with personalized text/emoji stickers.
-> TikToks are variously networked/searchable/templatable
Engaging logic of TikTok video memes
Platform Metadata
interactive features
(likes, views, shares, comments,
challenges, duets etc.)
+
timestamps, accounts, hashtags, sounds, captions, stickers, effects, etc.
+
moving images/audio content
TikToks are variously networked/searchable/templatable. The main networker is often the sound, not the hashtag.
The usual frame rate is 30 frames per second -> 1 second of video generates 30 images.
Tiktok video length can range from 3 seconds to 3 minutes.
TIKTOK: THE SOCIAL MOVING IMAGE
On sampling and visualization techniques see e.g., Manovich 2013, on video analysis with AI see e.g., Arnold et al. 2019)
Google Video Intelligence API, for example, extracts and annotates the middle frame of each shot based on its visual content. The method is very powerful when it comes to complex story arcs and multiple shot transitions. Limitation: Web detection is not included.
Many TikToks are filmed in one shot.
15 seconds of this video generate 450 frames. Frame rate = 30 fps.
Google Video AI label detection provides the labels of the individual objects present in a given video. However it does not contain additional web information, which is crucial for understanding the social aspects of video distribution and TikTok virality.
Google Video AI - Video Content Analysis
“Google Vision API labels detect and extract information about entities in an image, across a broad group of categories. (...) Web detection detects web references to an image based on content from pages with similar images.” (Google Cloud Vision API + Memespector GUI (Chao 2021) + see e.g., Hagen 2017; Omena et al. 2021; Pilipets 2021; Rogers et al. 2021; Pearce et al. 2022; Burkhardt & Rogers 2022; Tucci 2022; on GV labels see e.g., Geboers & Van de Wiele 2020).
15 sec/15 frames
Extracted video frames and audio tracks can be analyzed with different AI features.
How can we analyze a collection of videos?
001_mp4_frame0001.jpg
001_mp4_frame0002.jpg
001_mp4_frame0003.jpg
…
002_mp4_frame0001.jpg
002_mp4_frame0002.jpg
002_mp4_frame0003.jpg
…
001_mp4_audio.flac
002_mp4_audio.flac
…
Video Extractor (Chao 2023) offers different sampling techniques for extracting frames (stills) from a folder of input video files. It saves extracted images in a new folder, which then can be analyzed with different computer vision techniques. It also extracts audio tracks for speech-to-text recognition.
Speech-to-text converter (Chao 2023) transcribes audio content as captions using Google Cloud AI. It saves recognized text in a spreadsheet file, which then can be merged with TikTok metadata. The main affordance of this tool for TikTok research is that it (at least partially) helps to work around the problem of the “original sound”.
Audio content: TikTok creators often upload their videos with “original sound” instead of using TikTok’s built-in music library. If the video is linked with a song taken from the TikTok library, the song and author name is listed. However, as soon as content involves audio editing, the sound will be listed as “original sound” and provided with a unique numeric ID. —> The challenge of sound recognition/sense making.
Spreadsheet output with TikTok metadata scraped using Zeeschuimer (Peeters 2021)
Networked soundscapes
Resonant sounds and co-hashtags
1.
For more on TikTok sounds see e.g., Abidin & Kaye 2021; Bainotti et al 2022, Geboers et al. 2022, Pilipets 2023
Networked soundscapes
#deeptiktok
co-hashtags
listed sounds
original sounds
🎵WOAH
AND THE DEEPFRIED
ANIMAL
DANCE
MEME
Platform Metadata
engagement metrics
(likes, reblogs, comments, etc.)
+
hashtags, timestamps, account names,
captions, sounds, stickers, effects, etc.
Computer Vision Data
e.g., Google Vision API Labels
(objects, colors, bodies, actions, etc. )
+
detection of web entities and pages with matching images; speech detection
digital (moving) images as data multiplicities
How to contextualize based on research questions and using different visualization layouts (plot, stack, grid, network, montage, matrix, etc)?
.jpg .png .gif, .mp4, etc.
#deeptiktok | 951 videos | 18199 frames
Deep TikTok is a branch of Alternative or Elite TikTok. While Alt TikTok highlights the independent nature of diverse categories,
DeepTok sticks with a handful of ultra-original, niche topics, and is known for its deepfried aesthetic.
Effect templates
video content and co-hashtags
2.
#dreamcore
🌳
Effect templates
video content and co-hashtags
areyoucomingtothetree?
🌳 TOP 10 WEIRDCORE TIKTOKS PUBLISHED WITH #🌳
Cross-platform riffing
References across the web
3.
TIKTOK METHODOLOGIES><ETHICS?
1. include close-up visualizations of body images only after careful evaluation that content creators had intended for their posts to be widely circulated 2. account for potential bias that come with AI analysis; look at the tonality/stance of post captions 3. consider alternative methods of data visualization/ethical fabrication, involving creative, bricolage-style transfiguration of original data
ethical fabrication (Markham 2012) | situated data analysis (Rettberg 2020) | data feminism (D’Ignazio & Klein 2021) | TikTok Cultures Research Network