1 of 13

Anime4K Meets WebGPU

Video Upscaling for WebGPU

Ruijun(Daniel) Zhong Yuanqi Wang

Tong Hu

​

12/11/2023�

2 of 13

Our team

Ruijun(Daniel) Zhong

​

CGGT 1st Year

GitHub: DanielZhong

LinkedIn: Daniel Zhong

email: dzhong@seas.upenn.edu

Yuanqi Wang

​

MCIT+MSE/CIS 3rd Year

GitHub: plasmas

LinkedIn:Yuanqi Wang

email: wyq@seas.upenn.edu

Tong Hu

​

MCIT+MSE/CIS 3rd Year

GitHub: TongHuoAo

LinkedIn: Tong Hu

tonghu@seas.upenn.edu

Live Demo: https://bit.ly/aniwgpu

3 of 13

Why Important?

Problems with Anime Upscaling

  • Inefficient Real-Time Application
  • Suboptimal Image Quality
  • Limited Cross-Platform Compatibility

​

Benefits of Anime4K-WebGPU

  • Cross-Platform
  • Fast and Efficient
  • Client-Side Only
  • Future-Proofing
  • Developer Friendly
    • Integrate into Any Pipeline

Fig 1. Anime4K CNN x2 Upscale [10].

Live Demo: https://bit.ly/aniwgpu

4 of 13

Implementation

Pipelines

  • Deblur
  • Denoise
  • Restore CNN / GAN
  • Upscale CNN (x2) / GAN (x3, x4)

WGSL Shader Auto Generation

  • MPV Specialized GLSL → WGSL Compute

Package https://bit.ly/ani4kpkg (Github)

  • Fully Encapsulated
  • Lightweight, No Dependency
  • Combination of Pipelines Permitted

Real-Time Live Demo https://bit.ly/ani4kdemo (Github)

  • https://bit.ly/aniwgpu (Deployed Web App)

WEBGPU

Live Demo: https://bit.ly/aniwgpu

5 of 13

Different Effects

Live Demo: https://bit.ly/aniwgpu

Fig 2. Showcase of Anime4K Denoise, Deblur, Restore, and Upscale Effects [9].

6 of 13

Performance Analysis - Image Quality

Fig 3. Anime4K Upscale+Restore Combination vs. Real-ESRGAN x4 [8].

Live Demo: https://bit.ly/aniwgpu

Anime4K runs 800-1000x faster

Large Buffer Needed (Max 400 x 400px Resolution)

VS

7 of 13

Performance Analysis - GPUs

Live Demo: https://bit.ly/aniwgpu

Fig 4a. Performance Analysis - GPU Time with 720p input video [7] on RTX 3070Ti and 4090

Real-ESRGAN takes 737 ms to render a 200 x 133 image (RTX 3070Ti)

8 of 13

Performance Analysis - Input Video Size

Live Demo: https://bit.ly/aniwgpu

Fig 4b. Performance Analysis - GPU Time with 360p, 720p, and 1080p input video [7].

Too Heavy for Real-Time 1080p Upscale

9 of 13

Live Demo - https://bit.ly/aniwgpu

Live Demo: https://bit.ly/aniwgpu

Fig 5. Live Demo Web App.

10 of 13

References & Base Code

​

  1. Repo: Anime4K. https://github.com/bloc97/Anime4K
  2. Repo: Anime4K for Unity. https://github.com/keijiro/UnityAnime4K
  3. Repo: WebGPU Super Resolution. https://github.com/sona1111/webgpu-super-resolution
  4. Base Code: WebGPU Samples. https://github.com/webgpu/webgpu-samples
  5. Tutorial: Your First WebGPU App. https://codelabs.developers.google.com/your-first-webgpu-app
  6. Demo Video: One Punch Man. https://www.youtube.com/watch?v=km2OPUctni4
  7. Demo Video: Miss Kobayashi's Dragon Maid. https://www.youtube.com/watch?v=NQF3a6A7kcQ
  8. Demo Picture: Violet Evergarden. https://www.flickr.com/photos/sunshinetoday168/50383058796
  9. Demo Picture: Higurashi Series. https://myanimelist.net/anime/934/Higurashi_no_Naku_Koro_ni
  10. Demo Video: Uncle from Another World. https://myanimelist.net/anime/49220/Isekai_Ojisan

Live Demo: https://bit.ly/aniwgpu

11 of 13

Acknowledgement

  • Liam Dugan

​

  • Shadow Team
    • Jason
    • Keyu Lu
    • Crystal Zou

​

  • Real-ESRGAN WebGPU Team (CIS 5650 Fall 2021)
    • Paul Everett Jewell
    • Yuxuan Zhu

Live Demo: https://bit.ly/aniwgpu

12 of 13

Thank You!

Ruijun(Daniel) Zhong

danielzhong2000@gmail.com

​

Yuanqi Wang

wangyuanqi1996@gmail.com

​

Tong Hu

tonghu218@gmail.com

Contact us!

13 of 13