How to build a realtime ASR with opensource Whisper

Building a real-time Automatic Speech Recognition (ASR) system using OpenAI's Whisper involves several steps, including setting up your environment, downloading Whisper models, and creating a real-time audio processing pipeline. Here's a general approach to get you started:

Step 1: Install Whisper

First, you need to ensure that Python is installed on your system. Whisper can be installed via pip. In your terminal, run:

bash
pip install git+https://github.com/openai/whisper.git

Make sure your Python environment has the necessary dependencies for Whisper.

Step 2: Real-Time Audio Processing Setup

For real-time ASR, you need to capture audio from your microphone, process it in chunks, and then feed these chunks to Whisper for transcription. Python libraries such as sounddevice and numpy can be helpful for capturing and processing audio in real-time.

bash
pip install sounddevice numpy

Step 3: Implementing Real-Time Transcription

The following Python script provides a basic structure for implementing real-time ASR with Whisper. This script captures audio from the microphone, processes it in real-time, and prints the transcriptions:

```

import sounddevice as sd

import numpy as np

import whisper

# Load Whisper model

model = whisper.load_model("base")  # You can choose different model sizes

def audio_callback(indata, frames, time, status):

    # This function will be called for each audio block captured by the microphone

    audio_np = np.array(indata[:, 0], dtype=np.float32)  # Convert to numpy array

    # Here, you could buffer the audio and then process it with Whisper

    

    # Example: Passing the audio to Whisper directly for real-time transcription

    # might not be efficient or feasible due to the model's expectations for input length and format.

    # Instead, consider buffering the audio and processing it in larger chunks.

    

    # Print the status for debugging

    print("Capturing audio...", status)

# Start streaming from microphone

with sd.InputStream(callback=audio_callback):

    sd.sleep(10000)  # Keep capturing for a certain time or implement a loop for continuous capture

```

Notes:

• This script sets up a basic framework. You need to adapt the audio_callback function to buffer audio data and then periodically send this data to the Whisper model for transcription. Whisper typically requires a longer audio segment (several seconds) to produce accurate transcriptions, so real-time performance may vary based on how you implement buffering and processing.
• Consider the limitations and performance of the Whisper model you choose (base, small, medium, large). Larger models offer better accuracy but require more computational resources.

Step 4: Post-Processing and Displaying Transcriptions

After receiving the transcription from Whisper, you can display it in real-time or use it as needed in your application. Be mindful of the latency involved in processing and transcription when designing your real-time ASR system.

Final Thoughts:

• Real-time ASR with Whisper involves handling streaming audio data, which can be complex. Performance and accuracy will depend on your implementation details and the computational resources available.
• Test with different Whisper models and audio processing strategies to find the best balance between latency and transcription accuracy for your use case.

Regular Expressions in Python

Regular Expressions in Python || Python Tutorial || Learn Python Programming

import re

string = 'Bruce Lee'

pattern = '^(\w+)\s+(\w+)$' #match firstname lastname

match = re.search(pattern, string)

if match: 

    print(string)        #Bruce Lee

    print(match.span()) #(0,9)

    print(match.group()) #Bruce Lee

pattern = '^(?P<fn>\w+)\s+(?P<ln>\w+)$' #match firstname lastname

match = re.search(pattern, string)

if match: 

    print(string)            #Bruce Lee

    print(match.span())      #(0,9)

    print(match.group('fn')) #Bruce

    print(match.group('ln')) #Lee

pattern = '^[a-zA-Z!]+$'

string='m!sha'

match = re.search(pattern, string)

if match: 

    print(string)            #m!sha

re.search() # return the first match

re.findall(pattern, string, flags=0) # return all non-overlapping matched of pattern in string, as a list of strings or tuples

string='Bruce Lee'

pattern = '[a-z]+' #match firstname lastname

match = re.findall(pattern, string)

if match: 

    print(match)            #['ruce','ee']    

match = re.finditer(pattern, string) #iterator version

re.match(pattern, string, flags=0) #

match(pattern, string, flags=0)

    Try to apply the pattern at the start of the string, returning

    a Match object, or None if no match was found.

re.fullmatch(pattern, string, flags=0) #

fullmatch(pattern, string, flags=0)

    Try to apply the pattern to all of the string, returning

    a Match object, or None if no match was found.

values = ['https://www.socratica.com', 'http://www.socratica.org','not_a_www_https://']

regex = 'https?://w{3}.\w+.(org|com)'

for value in values:

    if re.fullmatch(regex, value):print(value)

https://www.youtube.com/watch?v=K8L6KVGG-7o

https://github.com/CoreyMSchafer/code_snippets/tree/master/Python-Regular-Expressions

import re

emails = '''

CoreyMSchafer@gmail.com

corey.schafer@university.edu

corey-321-schafer@my-work.net

'''

pattern = re.compile(r'[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+')

matches = pattern.finditer(emails)

for match in matches:

    print(match)

#https://github.com/CoreyMSchafer/code_snippets/blob/master/Python-Regular-Expressions/simple.py

import re

text_to_search = '''

abcdefghijklmnopqurtuvwxyz

ABCDEFGHIJKLMNOPQRSTUVWXYZ

1234567890

Ha HaHa

MetaCharacters (Need to be escaped):

. ^ $ * + ? { } [ ] \ | ( )

coreyms.com

321-555-4321

123.555.1234

123*555*1234

800-555-1234

900-555-1234

Mr. Schafer

Mr Smith

Ms Davis

Mrs. Robinson

Mr. T

'''

sentence = 'Start a sentence and then bring it to an end'

pattern = re.compile(r'start', re.I)

matches = pattern.search(sentence)

print(matches)

##

# https://github.com/CoreyMSchafer/code_snippets/blob/master/Python-Regular-Expressions/urls.py

import re

urls = '''

https://www.google.com

http://coreyms.com

https://youtube.com

https://www.nasa.gov

'''

pattern = re.compile(r'https?://(www\.)?(\w+)(\.\w+)')

subbed_urls = pattern.sub(r'\2\3', urls) #back reference

print(subbed_urls)

# matches = pattern.finditer(urls)

# for match in matches:

#     print(match.group(3))

Example of callable in re.sub()

s='abcdeABCDE'

def star_ord(m):

    return f'*<{m.group(0)}>*'

re.sub('[ae]',star_ord,s)

#'*<a>*bcd*<e>*ABCDE'

reference:

https://docs.python.org/3/library/re.html

 https://www.youtube.com/watch?v=nxjwB8up2gI

https://github.com/CoreyMSchafer/code_snippets/tree/master/Python-Regular-Expressions

https://www.python-engineer.com/posts/regular-expressions/

https://youtu.be/AEE9ecgLgdQ?si=2XZpygF-cFoAL0-i

[Tutorial & Examples] try block in Python

1) Motivation:

Why we need it?

open('demo_read.txt')

If the 'demo_read.txt' is not exist yet,  we will encounter 2 issues:

a) The script will exit immediately 

b) The script will print a bit verbose and messy error message:

Traceback (most recent call last):

  File "c:\Users\liuga\git_downloads\study\python\pickle_notes.py", line 1, in <module>

    open('demo_read.txt')

FileNotFoundError: [Errno 2] No such file or directory: 'demo_read.txt'

Can we continue run the script without exiting? 

Can we print out a clean or better customized error messages?

Yes, yes, Let's move on.

2) Better way!

try:

    open('demo_read.txt')

except FileNotFoundError as e:

    print(e)

The terminal will display:

[Errno 2] No such file or directory: 'demo_read.txt'

3) What is more?

Example 1:

try:

    open('demo_read.txt')

    x = 0

except FileNotFoundError as e:

    print(e)

except Exception as e:

    print(e)

else:

    print(f'Everthing is fine, no error!')

finally:

    print(f'The try-block is done no matter what happens. You will always see this line')

When running it, terminal will display:

Everthing is fine, no error!

The try-block is done no matter what happens. You will always see this line

Example 2:

try:

    open('demo_read.txt')

    x = y

except FileNotFoundError as e:

    print(e)

except Exception as e:

    print(e)

else:

    print(f'Everthing is fine, no error!')

finally:

    print(f'The try-block is done no matter what happens. You will always see this line')

When running it, terminal will display:

name 'y' is not defined

The try-block is done no matter what happens. You will always see this line

Example 3:

files = ['demo_read0.txt', 'demo_read.txt']

for file in files:    

    print(f'processing {file}')

    try:        

        f = open(file)                

    except FileNotFoundError as e:

        print(e)

    except Exception as e:

        print(e)

    else:

        print(f'Everthing is fine, no error for {file}')

        f.close()

    finally:

        print(f'The try-block is done no matter what happens. You will always see this line')

When running it, terminal will display:

processing demo_read0.txt

[Errno 2] No such file or directory: 'demo_read0.txt'

The try-block is done no matter what happens. You will always see this line

processing demo_read.txt

Everthing is fine, no error for demo_read.txt

The try-block is done no matter what happens. You will always see this line 

Conclusion:

Now I would like to sum up the try-block with my own language as this: it is a mechanism to handle error with more choices. 

This article is inspired by [1]

Reference:

[1] https://www.youtube.com/watch?v=NIWwJbo-9_8

Git 快速入门 以及 常用基本命令

《Git 快速入门 以及 常用基本命令》

                   2023/2/17

Lesson01:什么是Git？

Git是一套程序源代码的分布式版本管理系统，最初用于管理Linux核心代码的开发，后来被多个开源工程采用，现在已经成为互联网协作开发的标准的源代码管理软件。

官方网站

https://git-scm.com/

Git安装

https://git-scm.com/downloads

Git初运行

$ git version

Git在线体验

https://try.github.io/

采用Git的云服务

• GitHub
• gitee(oschina)
• 等等

 

Lesson2: 代码开发流程

1. 系统开发，编写代码
2. 提交代码给Git本地库
3. 将代码提交到Git远程库，分享给团队其他人
4. 从远程库获取最新代码
5. 继续修改编写代码
6. 重复第二步及以后的操作

Git基础概念

这是本期重点，这些基础概念必须掌握。

1. 本地工作文件夹
2. Git索引区（Stage）
3. Git库（Repository）

• local:本地库
• remote:远程库（服务器端）

 

Lesson 3: 建立一个Git库

$ git init

设置基础信息

$ git init

$ git config -l

$ git config --global user.name "koma"

$ git config --global user.email "koma@komavideo.com"

$ git config --global color.ui true

$ git config -l

方便的命令

$ git config --help

$ git help config

 

 

Lesson04: 第一次提交(commit)

知识点

• 建立文件（本地工作文件夹）
• 追加文件（索引区）
• 提交文件（本地库）

实战演习

$ mkdir myweb

$ cd myweb

#建立本地Git库

$ git init

#编辑(修改)本地文件

$ nano index.htm

...

...

#本地工作文件夹状态确认

$ git status

$ git add index.htm

#本地工作文件夹状态再次确认

$ git status

#将索引区内容提交本地库

$ git commit -m "created index.htm"

#查看提交历史

$ git log

 

Lesson05: 查看提交履历

知识点

• git log命令的使用

实战演习

$ git log

$ nano index.htm

...

...

$ git add index.htm

$ git commit -m "added code."

$ git log

$ git log –oneline #用一行扼要现实git log

$ git log -p

$ git log --stat

$ git log --help

 

Lesson06:

把握Git状态

知识点

• git status
• git checkout -- [file]

实战演习

$ nano index.htm

...

...

$ git status

$ git checkout -- index.htm

$ git status

$ nano index.htm

...

...

$ git add .

$ git reset HEAD index.htm

$ git checkout -- index.htm

$ git status

...

$ nano index.htm #modify the file, action2

$ git add .

$ git commit -m ‘add new’

$ git reset HEAD index.htm

$ git checkout -- index.htm #This will change it back before action2

$ git status

 

Lesson07

比较修改内容

知识点

• git diff [--cached]

实战演习

$ nano index.htm

...

...

#工作文件夹比较

$ git diff

#把修改文件追加到索引区

$ git add index.htm

#无法比较工作文件夹的修改文件

$ git diff

#索引区比较

$ git diff --cached

 

Lesson08

Git文件操作

知识点

• git add [file1 file2 ...]
• git add .
• git rm
• git mv

实战演习

$ nano index.htm

...

$ nano style.css

...

$ git add .

$ git status

$ git mv index.htm index.html

$ git status

$ git rm --cached style.css

$ git status

 

Lesson09:

Git忽略管理

设置Git忽略的文件，这些文件不参与Git库的提交和管理。（例如：Node.js的[node_modules]文件夹）

知识点

• .gitignore

帮助网页:

https://git-scm.com/docs/gitignore

实战演习

$ nano test.tmp

...

$ git status

$ nano .gitignore

...

*.tmp

...

$ git status

$ mkdir subdir

$ nano subdir/my.css

...

$ git status

$ nano subdir/my.tmp

...

$ git status

 

Lesson10:

更新最后的提交

知识点

• git commit -m "commit message"
• git commit --amend
• git commit -am "commit message" [--amend]

实战演习

$ nano index.htm

...

$ git add .

$ git commit -m "modified."

$ git log

# remove debug info.

$ nano index.htm

...

$ git add .

$ git commit --amend

$ git status

$ git log

 

 

Note:

git commit -am 'detailed message for -am'

https://stackoverflow.com/questions/19877818/git-commit-m-vs-git-commit-am

Using the option -am allows you to add and create a message for the commit in one command.

 

啥时候会用到amend呢，有时候疏忽，把用户名密码提交了，这肯定不能留痕，所以 用amend 把以前的修改覆盖掉，这样就没事了

 

Lesson11:

返回过去1

知识点

• git reset --hard HEAD       # 返回到最后1次提交
• git reset --hard HEAD~     # 返回到倒数第2次提交
• git reset --hard HEAD~n   # 返回到倒数第n次提交
•  

实战演习

$ git status

$ nano main.html

...

#建立多个提交履历(5个以上)

...

$ git add .

$ git commit -m "1...5"

$ git status

$ git add .

$ git log

$ git reset --hard HEAD #z这个例子中是把git 管理的代码库自从最后一次提交后的所有修改全部删除

$ git log

$ git reset --hard HEAD~

$ git log

$ git reset --hard HEAD~2

$ git reset --hard <SOME_COMMIT> # can move the specific commit

 

https://stackoverflow.com/questions/9529078/how-do-i-use-git-reset-hard-head-to-revert-to-a-previous-commit

 

Lesson12

返回过去2

知识点

• git reflog [-n num]
• git reset --hard [commit_id]

实战演习

返回过去之后，通过git reflog命令找到现在的位置（commit_id），再从过去返回回来。

$ git log

$ git reset --hard HEAD~2

$ git reflog

* git reset --hard [commit_id]

 

Reference logs, or "reflogs", record when the tips of branches and other references were updated in the local repository. Reflogs are useful in various Git commands, to specify the old value of a reference. For example, HEAD@{2} means "where HEAD used to be two moves ago", master@{one.week.ago} means "where master used to point to one week ago in this local repository", and so on. See gitrevisions(7) for more details.

 

Lesson13

使用分支

Git分支功能对于项目开发中的团队合作有着非常重要的作用，同时对于生产环境的更新管理也起着不可替代的作用，是Git最重要的功能。

在项目开始前，应该首先对Git分支的管理有一个明确地规划，明确每个分支的功能和担当者，这样才会保证项目正常推进，不至于陷入混乱。

知识点

• git branch [name]
• git checkout branch_name

实战演习

$ git branch

$ git branch dev

$ git checkout dev

$ nano style.css

...

$ git add .

$ git commit -m "modified style.css."

$ git log

$ git checkout master

$ git log

 

Lesson14

合并分支

知识点

• git merge
• git branch -d [name]

实战演习

$ git branch

$ git checkout dev

$ nano index.htm

...

$ git add .

$ git commit -m "modified1."

$ git log

$ git branch

$ git checkout master

$ git log

$ git branch

$ git merge dev

$ git log

$ git branch -d dev #删除dev分支

$ git branch

 

 

Lesson15

制造分支冲突

当团队中多人同时编辑一个文件的时候，难免会出现源代码的编辑合并冲突的问题，那么我们该怎么解决呢？

知识点

• 做一个源代码冲突的场景
• git checkout -b [branch_name]

实战演习

$ git checkout -b dev

$ nano index.htm

...

$ git add .

$ git commit m "modified by dev."

$ git checkout master

$ nano index.htm

...

$ git add .

$ git commit -m "modified by master."

$ git branch

$ git merge dev

#出现源代码版本冲突，需要手动进行合并解决

 

 

Lesson16

解决分支冲突

知识点

• git merge [branch_name]

实战演习

$ git branch

$ git checkout master

$ git merge dev

$ nano index.htm

...

$ git add index.htm

$ git commit -m "merged by leader."

$ git log

$ git branch

 

Lesson17  使用Tag标签

知识点

• 系统版本号管理
• git tag [tag_name] [commit_id]
• git show [tag_name]
• git tag -d [tag_name]

系统版本号管理

任何软件系统，应用程序在发布时都应该给一个版本号，来管理每次发布的内容，便于今后的管理。

例如：

1.1.4

NNN.abc.xxx

• NNN:大版本号
• abc:每次做出的小更新时，发布的版本号
• xxx:每次bug修正时发布的版本号

实战演习

$ git tag

$ git tag v1.0.0

$ git tag

$ nano index.htm

...

#修正bug1

...

$ git add .

$ git commit -m "fixed bug1."

$ git tag v1.0.1

$ git tag

$ nano index.htm

...

#修正bug2

...

$ git add .

$ git commit -m "fixed bug2."

$ git tag v1.0.2

$ git tag

$ nano index.htm

...

#新功能追加

...

$ git add .

$ git commit -m "added feature1."

$ git tag v1.1.0

$ git tag

$ git show v1.0.1

$ git show v1.0.2

 

Lesson18 使用别名

在Git中可以将经常使用的命令以别名缩写的方式简化使用。

知识点

• git config --global alias.[name] [command_name]

实战演习

#将checkout命令简化为co

$ git config --global alias.co checkout

#将branch命令简化为br

$ git config --global alias.br branch

#将commit命令简化为cm

$ git config --global alias.cm commit

#将status命令简化为st

$ git config --global alias.st status

$ git br

$ git co dev

$ git st

$ git config -l

当然这种别名的定义根据每个人的使用习惯不同而不同，也可以在项目开始前作为项目的统一规则制定下来，使每个项目开发成员都统一使用一套大家都认可的别名，这样提高项目组内部的沟通效率。

 

Lesson19 玩转开源中国Gitee

本期课程为您讲解如何使用Git命令连接远程服务器，如果编写提交代码到服务器端，完成团队的协同开发。

知识点

• git clone [url]
• git remote -v
• git push [origin] [master]

操作步骤

1. git clone 把远程库克隆到本地文件夹
2. 编辑本地文件夹的文件
3. 提交编辑的文件到本地信息库(git add ./git commit)
4. 将本地库同步(sync)到远程Git服务器

实战演习

通过操作开源中国LearnOSC库，为您展示如何使用分支，如果提交代码，让您了解整个Git远程库的使用过程。

$ git clone https://gitee.com/komavideo/LearnOSC

$ cd LearnOSC

$ git status

$ git branch

$ git branch -a

$ git branch dev remotes/origin/dev

$ git branch

$ git checkout dev

$ git status

$ nano README.md

...

git config --global color.ui true

...

$ git add .

$ git commit -m "为初期设定增加color.ui选项"

$ git status

$ git remote -v # 检查远端服务器设置

$ git push origin dev

 

  参考文献：

【1】 https://gitee.com/komavideo/LearnGit