2022-07-14 –, Liffey Hall 2
By using Streamlit and streamlit-webrtc, we can create web-based real-time computer vision apps only with ~10 or 20 additional lines of Python code.
To turn computer vision models into real-time demos, we have conventionally used OpenCV modules such as cv2.VideoCapture and cv2.imshow(). However, such apps are difficult or impossible to share with friends, run on smartphones, or integrate with modern interactive widgets and other data views and inputs.
Web-based apps don't have such problems.
Streamlit provides an easy way to build web apps quickly, and streamlit-webrtc allows to use real-time video streams.
You can create real-time video apps with modern interactive views and inputs, and host these apps on the cloud to use from any devices with browsers.
In this talk, I will demonstrate the development process using these libraries and show a variety of examples so that we see how easy and useful they are and can make use of them in daily development and research.streamlit-webrtc extends Streamlit to be capable of dealing with real-time video and audio streams.
With a combination of these libraries, developers can rapidly create real-time computer vision and audio processing apps for which OpenCV has typically been used.
I am the author of streamlit-webrtc and a member of the Streamlit Creators program (selected community members).
The repository of streamlit-webrtc is here: https://github.com/whitphx/streamlit-webrtc
My lightning talk about streamlit-webrtc at PyCon JP 2021 is available: https://youtu.be/_LuLs8H1gJc
Articles about this library:
- Developing Web-Based Real-Time Video/Audio Processing Apps Quickly with Streamlit
- Real-Time Video Streams With Streamlit-WebRTC
As linked from the repo, demo apps I have developed are available online:
- Demo showcase including real-time object detection: https://share.streamlit.io/whitphx/streamlit-webrtc-example/main/app.py
- Source code: https://github.com/whitphx/streamlit-webrtc-example/blob/main/app.py
- Real-time Speech-to-Text: https://share.streamlit.io/whitphx/streamlit-stt-app/main/app_deepspeech.py
- Source code: https://github.com/whitphx/streamlit-stt-app
- Real-time style transfer: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py
- Source code: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py
- Real-time Tokyo 2020 Pictogram: https://share.streamlit.io/whitphx/tokyo2020-pictogram-using-mediapipe/streamlit-app
- Source code: https://github.com/whitphx/Tokyo2020-Pictogram-using-MediaPipe
- Video chat: online demo is not available because it does not have an auth mechanism and is only for private use.
- Source code: https://github.com/whitphx/streamlit-video-chat-example
Streamlit and streamlit-webrtc provide a new easy way to turn computer vision models into real-time web-based apps. They have some advantages over the conventional ways which use OpenCV.
Expected audience expertise: Domain:none
Expected audience expertise: Python:some
Yuichiro works as a professional software developer and also loves contributing to OSS projects.
As a Pythonista, he has participated in various projects including web development, multimedia streaming, data management, computer vision, and machine learning.