Pitch shifting User manual
v1.0 Chu Bu Wan Shan pitch shiftingDe Yuan Li Jie Shao Ji Shi Xian Fang Shi
Introduction
Pitch shifting effect: Bian Diao ,Yi Xiang Xiao Guo Qi ,Gai Xiao Guo Qi Ke Yi Shi Xian Dui Yuan Shi Yin Pin Yin Diao De Diao Zheng ,Jin Xing Sheng Diao Huo Zhe Jiang Diao ,Sheng Jiang De Cheng Du Tong Guo Ban Yin Jie ^[1]^ (semetone)Lai Heng Liang . Ke Yong Yu Dian Zi Bian Sheng Qi , Ji Ta Bian Diao Qi Yi Ji Qia La OkYing Yong Zhong ^[2]^.
Shi Xian Fang Shi
Shi Xian pitch shifting effectDe Fang Shi You Yi Xia Ji Chong :
-
Time scaling and resampling
Zui Jian Dan De Fang Shi ,Tong Guo Diao Zheng Cai Yang Lu Gai Bian Zui Zhong Bo Fang De Shi Chang (duration),Cong Er Gai Bian Yin Diao ,Shi Xian Bian Diao Gong Neng .
-
Pitch Synchronous Overlap and Add ,PSOLA^[3]^ ,Ji Yin Tong Bu Die Jia
Jiang Xin Hao Fen Ge Cheng Zhong Die Zheng overlap segment ,Zai Die Jia Qian Xu Yao Que Ding Ji Yin Zhou Qi Yi Ji Qi Shi Dian (Pitch control,Yi Ban Zai Shi Yu Chu Li ),Gen Ju Xu Yao Diao Zheng Die Jia Qu Yu De Da Xiao ,Da Dao Sheng Jiang Diao De Mu De ,Chu Li Hou De Xin Hao Jiang Hui Gai Bian Yin Diao He Shi Chang . Ji Yin Pin Lu Ji Qi Shi Dian Pan Bie De Zhun Que Yu Fou Dui Chu Li Hou De Xin Hao Shi Fou Ping Wen Zhi Guan Zhong Yao . Duo Yong Yu Hua Yin Xin Hao Zhong .
-
Phase vocoder
Jiang Xin Hao Fen Ge Cheng Jiao Duan De Hu Xiang Zhong Die De Zhong Die Zheng ,Tong Guo FFTChu Li (STFT),Zai Pin Yu Zhong An Zhao Sheng Jiang Bi Li Diao Zheng Xin Hao De Xiang Jiao $\phi$,Ran Hou Tong Guo IFFTHuan Yuan Zhi Shi Yu Xin Hao ,Bing Die Jia Shu Chu . You Yu Pin Yu Zhong Xiang Wei De Bian Hua Deng Jie Yu Shi Yu Zhong Pin Lu De Bian Hua ,Yin Ci Ke Yi Shi Xian Xin Hao Yin Diao De Sheng Jiang . Phase vocoderDe Chu Li Si Lu Yu PSOLALei Si ,You Yu Shi Zai Pin Yu Zhong Diao Zheng Xin Hao Yin Diao ,Yin Ci Geng Wei Jing Que Ke Kong ,You Ci Yan Sheng Chu Bu Shao Ji Yu phase vocoderDe Chu Li Fang An .
Ben Wen Jiang Hui Zhong Dian Jiang Jie Phase vocoder.
Yu Bei Zhi Shi
Zai Jin Ru Xia Yi Jie Duan Jiang Jie Ji Yu Phase vocoderDe Pitch shifting effectZhi Qian ,Xian Bu Chong Yi Xie Qian Zhi Li Lun :Sheng Le Li Lun .
Sheng Le Li Lun
Pitch,Ke Yi Yong Lai Heng Liang Sheng Diao Gao Di De Wu Li Liang ,You Bie Yu Yin Liang . Tong Chang Qing Kuang Xia ,Ke Yi Jiang pitchYu Pin Lu Lian Xi Qi Lai ,Dui Yu Te Ding De pitch,Te Ding Pin Lu De Xin Hao Dui Qi Ying Xiang Jiao Da . Zai Sheng Le Ling Yu ,Standard PitchDe Pin Lu Wei A440,Sheng Gao Yi Ge octaveWei A880,Jiang Di Wei A220^[6]^.
Ban Yin Jie (semitone),Sheng Le Xue Zhong Heng Liang Yin Diao Gao Di De Dan Wei Zhi Yi ,Yi Ge Ba Du (octave)De Yin Cheng An 12Ping Jun Lu Fen Wei 12Ge Ban Yin ,Ti Sheng Yi Ge Ban Yin Xiang Dang Yu Zai Yin Diao Pin Lu De Ji Chu Shang Cheng Shang 2^1/12^
Yi Xia Wei Ge Ge Yin Jie Dui Ying De Pin Lu :
Geng Duo Nei Rong Qing Can Yue [6].
Jian Dan Ying Yong :Time scaling and resampling
Yuan Shi Xin Hao Chang Du Wei L ,Cai Yang Lu Wei fs ,Yao Shi Xian Sheng Yi Ge Ban Yin Gao (semeton),Ke Yi Xian Jiang Yuan Shi Xin Hao Cong L Cha Yang Zhi L * 2^1/12^ ,Ran Hou Yi 2^1/12^ *fs De Cai Yang Lu Bo Fang Ji Ke Sheng Yi Ge Ban Yin Gao . Sheng Liang Ge Ban Yin Gao Shi Xu Yao Cha Yang De Xi Shu Wei 2^1/12^,Yi Ci Lei Tui . Dui Yu Jiang Diao Lai Shuo Dui Ying De Xi Shu rateZe Wei 2^1/12^ (one semetone),Zui Hou Bo Fang Shi Ye Xu Yi Dui Ying Xi Shu De Cai Yang Lu Jin Xing Bo Fang .
[y_in,fs] = audioread('test.wav');
rate = 2^(6/12); % raise 6 semetones
y_out_len = y_in_len * rate;
y_out = interp1( linspace(0,1,y_in_len),y_in,linspace(0,1,y_out_len));
p = audioplayer(y_out,fs*rate);
p.play
Phase Vocoder
Yuan Li Jie Shao
Vocoder Shi Yi Lei Zhi Jie Dui Te Ding Pin Lu Xin Hao De Fu Zhi He Xiang Wei Jin Xing Bian Ji Xiu Gai De Gong Ju De Zong Cheng ,Phase vocoderShu Yu Qi Zhong De Yi Chong ,Zhu Yao Yong Yu Diao Zheng Xin Hao Te Ding Pin Lu De Xiang Wei Xin Xi . Phase vocoderYou Liang Chong Shi Xian Mo Xing ,Qi Zhong Yi Chong Shi Lu Bo Zu Zong He Mo Xing (Filter Bank Summation Model),Tong Guo Yi Cu Deng Kuan , Zhong Xin Pin Lu Cong 0~fs/2De Dai Tong Lu Bo Qi Jiang Xin Hao Fen Ge Cheng Yi Ge Ge Pin Duan De "Dan Pin "Xin Hao ,Ran Hou Jin Xing Chu Li ,Zui Hou Die Jia Huan Yuan .
Ling Yi Chong Wei Zhu Kuai Fen Xi /Zong He Mo Xing (Block by Block Analysis / Synthesis Model),Tong Guo Zhu Bu Bu Jin De Fang Shi ,Zhong Die Di Cong Yuan Shi Xin Hao Zhong Qu Chu Analysis_block,Bu Jin Chang Du Wei analysis len. Zhi Hou Dui Analysis blockZhong De Xin Hao Jin Xing FFTBian Huan Zhi Pin Yu ,Zai Pin Yu Zhong Dui Pin Yu Xin Xi (Fu Zhi , Xiang Wei )An Xu Qiu Jin Xing Chu Li (Processing),Zui Hou Tong Guo IFFTHuan Yuan Pin Yu Xin Hao Zhi Shi Yu ,Bing Jin Xing Zong He Xiang Jia (Synthesis and overlap),De Chu Synthesis block. Tong Yang De ,Zong He Xiang Jia Ye You Bu Jin Chang Du ,Wei Synthesis len.
Shi Ji Shang Gen Ju FFTJi Suan Gong Shi ,Ke Yi Jiang FFTKan Cheng Shi Yi Xi Lie De Dai Tong Lu Bo Qi De Zu He ,Zhe Xie Dai Tong Lu Bo Qi De Zhong Xin Pin Lu Wei
Shi Xian Guo Cheng
Zhong Dian Jie Shao Shi Xian De Guo Cheng ,Yan Shi De Shi Hou Ke Yi Shi Yong matlab
Shang Yi Jie Ti Dao ,Ji Yu Block by Block Analysis / Synthesis Model De Phase vocoderZhu Yao Fen Wei San Ge Bu Zou ,analysis-->processing-->synthesis. Ben Jie Jiang Hui Ju Ti Chai Jie Zhe Xie Bu Zou ,Jie Shao Ru He Tong Guo Phase vocoderShi Xian Pitch shiftingDe Xiao Guo .
A(Analysis) -->B(Processing)
B --> C(Synthesis)
Analysis
Analysis Jie Duan Zhu Yao Jin Xing De Shi STFT(Short time fouria transform),Ji Zhong Die Di Xiang Qian Qu Gu Ding Chang Du De Dian ,Bing Jiang Dian Cun Chu Zai Frame De bufferZhong Jin Xing FFTChu Li ,Chu Li Hou De Shu Ju Jiang Hui Zai ProcessingZhong Shi Yong . Ju Ti Bu Zou Ru Xia :
-
Zhong Die Fen Zheng . Yi analysis hopWei Bu Jin Chang Du ,Xiang Qian Qu Zhi Fen Zheng ,Suo Qu De Dian Shu Wei win len,Cun Zai Frame De Ji Cun Qi Zhong ,Yong Yu Hou Xu De FFTBian Huan .
-
Jia Chuang ,FFT. Dui FrameZhong De Shu Ju Cheng Shang Yi Ge win lenChang Du De hanning window,Jia Chuang Hou De Xin Hao Ying Yong FFT.
-
Ji Suan Fu Zhi , Xiang Jiao . Ji Suan FFTHou Xin Hao De Fu Zhi mag He Xiang Jiao angle .
Qi Zhong analysis hopDe Qu Zhi Yu win lenYou Jiao Da De Lian Xi ,You Yu Xiang Qian Qu Dian Fen Zheng Xu Yao Zhong Die 75%De Shu Ju Yi Huo De Jiao Da De Pin Lu Fen Bian Lu He Liang Hao De Chu Li Xiao Guo ^[4]^,Yin Ci analysis hop = (1-0.75)win len*,Dang win lenQu 256Shi ,analysis hop = 64.
Yi Xia Wei Ci Bu Fen Dui Ying De matlabDai Ma :
if n == step_num-1 % Fen Zheng ,An Zhao analysis_len Yi Dong
frame = x( end - win_len + 1:end);
else
frame = x(analysis*n +1 : analysis*n + win_len);
end
frame_fft = fft((win.*frame) ) ; % Jia Chuang , Hou fft
angle_pre = frame_angle;
frame_angle = angle(frame_fft);
frame_mag = abs(frame_fft);
...
Xu Yao Zhu Yi De Di Fang :
Zai matlab/pythonZhong ,mag Ke Yi Tong Guo abs()Han Shu Qiu Jie , angle Ke Yi Tong Guo angle()Han
Shu Qiu Jie ,Shu Ru Jun Wei FFTHou De Fu Shu . Er Zai C/C++Zhong You Yu Wu Fa Diao Yong
Processing
Ci Bu Fen Shi Phase vocoderDe Zhong Dian ,Jiang Hui Diao Zheng Xin Hao De Yin Diao ,Zhu Yao De Chu Li Wei Phase unwrap. Wei Liao Bian Yu Li Jie ,Xian Hui Gu Yi Xia Jiao Pin Lu $\omega$Yu Xiang Wei (Xiang Jiao )$\varphi$De Guan Xi : $$ \omega \times t = \varphi $$ Er $2\pi\omega=f$,Yin Ci Shun Shi Pin Lu Yu Xiang Jiao De Guan Xi Wei : $$ f_i=2\pi\varphi/t $$ Er analysisJie Duan Xiang Qian Bu Jin Chang Du analysis len Ke Yi Kan Cheng Shi Shi Jian Jian Ge $t$.
unwrapDe Zhu Yao Bu Zou
-
Ji Suan Qian Hou Liang Zheng Jiao Pin Lu De Bian Hua : $$ \Delta = (angle_{cur}-angle_{pre})-\Omega $$ Qi Zhong $\Omega = 2\pi[0:win_{len}]/win_{len}$,Dui Ying FFTHou Mei Yi Ge Dian De Biao Zhun Jiao Pin Lu ,Jiao Pin Lu Cheng Shang Shi Jian Jian Ge Ji Wei Xiang Wei Jiao
$\omega \times t = \varphi $ -
Ji Suan Zai analysis hopDe Shi Jian Jian Ge Zhong De Xiang Wei Chai Yi : $$ \Delta \varphi = \Delta angle -Round(\Delta /2\pi)*2\pi $$ Qi Zhong ,Round()Wei Qu Zheng Han Shu ,$\Delta \varphi$Jiang Hui Zai $(-\pi,\pi)$Zhi Jian ,
-
Gen Ju Xiang Wei Chai Yi (Bian Hua Lu )$\Delta \varphi$Ji Suan Shun Shi Jiao Pin Lu : $$ \omega_i = \Omega + \Delta \varphi/analysis_{hop} $$
-
Ji Suan Tong Bu Xiang Wei $$ angle_{syn} = angle_{syn} + \omega_i *Synthesis_{hop} $$ Zhu Yi ,Zhe Li Ji Suan Chu De Xiang Wei Shi Lei Ji Xiang Wei ,Bing Gen Ju Bi Zhi Dui Xiang Wei Jin Xing Liao Zong He Tong Bu
-
Shu Chu Tong Bu Xiang Wei
$angle_{syn}$ Ji Tong Bu Zhen Fu mag
Ben Jie Can Kao Dai Ma Ru Xia :
omega = 2*pi*analysis.* ( 0:win_len-1 )' ./win_len;
delta = frame_angle - (angle_pre + omega);
phase_unwrap = delta - round(delta/(2*pi))*2*pi;
% caculate the principal argument of the nomial inital phase
% also called delta_phi
phase_inc = (phase_unwrap + omega) / analysis;
% phase(omega)increment
if first_time
first_time = false;
y_angle = frame_angle;
else
y_angle = y_angle + phase_inc * synthesis;
% synthesis phase caculation
% Yong Qian Yi Zheng De angle!!!
end
Synthesis
Zai Shang Yi Jie Li Wo Men Yi Jing Zai Shi Yu Chu Li Zhong Qiu Jie Chu Liao Tong Bu Xiang Wei Yi Ji Tong Bu Zhen Fu ,Wei Liao Jin Yi Bu De Chu Li ,Wo Men Xu Yao Jiang Xin Hao Tong Guo IFFTHuan Yuan Zhi Shi Yu ,Yi Xia Shi Ju Ti De Bu Zou :
-
Gen Ju Fu Shu De Fu Zhi Shu Xing Shi $y = Ke^{\phi j}$,Jiang Xin Hao Huan Yuan Cheng Fu Shu : $$ y_{complex} = mage^{angle_{syn}*j} $$
-
IFFT,Hou Jia Chuang Shu Chu Dai Hun Die Xin Hao $$ y_{toSyn} = real(IFFT(y_{comples}))*HanningWindow $$
-
Hun He Die Jia Shu Chu ,Die Jia Bu Chang Wei Synthesis hop
Ci Bu Fen matlabCan Kao Dai Ma Ru Xia :
h = frame_mag.*exp(y_angle*1j);
y_ifft = (real(ifft(h)) ).*win;
overlap = temp(synthesis+1:win_len) + y_ifft(1:win_len-synthesis); % Die Jia
temp = [overlap ; y_ifft(win_len-synthesis+1:win_len) ]; % Zu He
toSyn = temp(1:synthesis);
Xu Yao Zhu Yi De Di Fang :
Zai C/C++De Ji Chu Ku Zhong Bu Zhi Chi Fu Shu Yun Suan ,Yin Ci Gen Ju Gong Shi $y = Ke^{\phi j}$Huan Yuan De Fang Shi Xing Bu Tong ,Xu Yao Tong Guo San Jiao Han Shu Fen Bie Huan Yuan Chu Shi Bu He Xu Bu ,Gong Shi Ru Xia : $$ real = magcos(angle_{syn}),imag = mag*sin(angle_{syn}) $$
Shi Xian Xiao Guo
Ci Chu Ji Lu Shi Yong phase vocoder Shi Xian pitch shifting Shi De Shi Ji Shi Yong Xiao Guo .
Ce Shi Yong Li 1: Shi Yong matlabSheng Cheng 1k Hz Ji 4k HzDe Zheng Xian Xin Hao Zuo Wei Ce Shi Xin Hao ,Tong Guo matlab Bian Xie De Phase vocoder Cheng Xu ,Dui Bi Chu Li Qian Hou Xin Hao De Pin Yu Xin Xi .
Shu Ru De Ce Shi Xin Hao :
f1 = 1000;f2 = 4000;
t = linspace(0,1,fs);
x = sin(2*pi*f1*t)+cos(2*pi*f2*t);
ra = rand(length(x),1);
x = x + 0.2.*ra';% Xia Tu Zhong Wei x=x + *ra';
x = x'/max(x);
Cong Shang Tu Zhong Ke Yi Kan Dao Shi Yong 1kHz,4kHzDie Jia De Zheng Xian Xin Hao Zuo Wei Ce Shi Xin Hao ,pitch shifting De Bi Li Wei 1.406(Yue Deng Yu 2^6/12^),Li Lun Shang Jing Guo pitch shifting Hou 1K Yu 4kFen Bie Bian Cheng :1000*1.406 = 1406, 4000*1.406=5624,Qian Hou Dui Bi Ke Yi Fa Xian Shi Ji Chu Li Chu Lai De Xin Hao Zhu Pin Shi Fen Jie Jin Li Lun Pin Lu ,You Ci Shuo Ming Phase vocoderXiao Guo Jiao Wei Li Xiang .
Ce Shi Yong Li 2: Shi Yong Chang Gui De Yin Le Xin Hao Zuo Wei Ce Shi Xin Hao ,Zai Chu Li Hou (Ru Sheng Gao 6 semetones)Yu Shi Yong Adobe Audition Jin Xing Tong Yang Chu Li Hou De Xin Hao Jin Xing Dui Bi ,Bi Jiao Liang Zhe Chai Yi .
Ce Shi Wen Jian Wei Long Live.wav,Shi Yong matlabBian Xie De Phase vocoderJin Xing Ce Shi ,Cheng Xu Jian Fu Lu . Chu Li Hou De Wen Jian Wei test_matlab.wav. Er Zai Adobe AuditionZhong ,Shi Yong De She Zhi Ru Xia :
Chu Li Hou De Wen Jian Wei test_AU.wav,Ting Yin Ce Shi .
Matlab/PythonShi Xian
Phase vocoderDe matlabShi Xian Qian Mian De Zhang Jie Yi Jing You Ti Ji ,Zai Ci Bu Zai Zhui Shu ,Ke Can Jian Fu Jian . PythonShi Xian Yu matlabLei Si ,Fu Lu Tong Yang Zhai Lu You Qi Shi Xian De Dai Ma ,Bu Guo Xu Yao Zhu Yi De Shi Shi Yong pythonShi ,Wu Lun Shi Xiang Liang Huan Shi Shu Zu De Xing Shi ,Qi Di Yi Wei De Xia Biao Shi 0Er Bu Shi 1 .
C/C++Shi Xian
Kao Lu Dao Matlab/PythonShi Xian Shi Shi Yong De Shi Xiang Liang De Xing Shi ,Yu Shi Ji Sheng Chan Huan Jing Zhong Shi Yong C/C++De Liu Shi Xian Fang Shi You Suo Bu Tong ,Zhe Li Gei Chu C/C++Shi Xian Phase vocoderDe Can Kao Dai Ma ,Zhu Yao Cheng Xu Ru Xia :
for (int n = 0; n < win_len; n++) {
double input_temp = wav.data[offset*ana_len +n];
fft_in[n][0] = input_temp * hanning_win[n]; // Jia Chuang
fft_in[n][1] = 0;
original_input.push_back(input_temp); // test Ji Lu Ji Jiang Jin Xing Chu Li De Yuan Shi Shu Ju
}
fftw_execute(fft); // Dui Shu Ru Xin Hao Shi Yong fft
for (int n = 0; n < win_len; n++) {
real[n] = fft_out[n][0]; // Bao Cun Shi Bu
imag[n] = fft_out[n][1]; // Bao Cun Xu Bu
angle_pre[n] = angle[n];
// Notice the different between real() and abs() func
mag[n] = sqrt(real[n] * real[n] + imag[n] * imag[n]);
// Notice:angle(x+yj) = actan2(y,x);
angle[n] = atan2(imag[n],real[n]);
// phase unwrap
double delta = (angle[n] - angle_pre[n]) - omega[n] * ana_len; // phi Shu Zu !
double delta_phi = delta - round( delta / (2 * PI) ) * (2 * PI);// phi
double y_unwrap = delta_phi/ana_len + omega[n];
if (is_firstTime) {
angle_syn[n] = angle[n];
}
else {
angle_syn[n] = angle_syn[n] + y_unwrap * syn_len; // phi,shifted angle
}
syn_angle.push_back(angle_syn[n]); // Ji Lu Tong Bu Xiang Wei test
// prepare for ifft
ifft_in[n][0] = mag[n] * cos(angle_syn[n]); // Huan Yuan Shi Bu
ifft_in[n][1] = mag[n] * sin(angle_syn[n]); // Huan Yuan Xu Bu
}
is_firstTime = false;
// Notice:fftwKu Zhong ifftShu Chu De Zhi Mei You Jin Xing Gui Yi Hua ,Xu Yao Chu Yi ifft_points Cai Yu matlabZhong De Jie Guo Xiang Deng
fftw_execute(ifft);
// Shi Yong ifft_outZhong De Shi Shu Bu Fen Jin Xing synthesis and overlap add
double toSOA[win_len];
double overlap[win_len - syn_len];
for (int n = 0; n < win_len; n++) {
toSOA[n]=hanning_win[n]*ifft_out[n][0] / double(win_len); // Gui Yi Hua ,Jia Chuang
if (n < win_len - syn_len) {
overlap[n] = overlap_buff[syn_len + n] + toSOA[n]; // overlap add
overlap_buff[n] = overlap[n];
}
else
{
overlap_buff[n] = toSOA[n];
}
}
// output is in overlap_buff[0:syn_len]
for (int output_n = 0; output_n < syn_len; output_n++) { // output the phase shifting signal ,test
audio_out << overlap_buff[output_n] << endl;
}
}
Reference
[1] Can Kao Lai Yuan :http://blog.sina.com.cn/s/blog_7ae6c3a00101hqwa.html
[2] https://en.wikipedia.org/wiki/Pitch_shift
[3] https://en.wikipedia.org/wiki/PSOLA
[4] Bernardini.pdf Trational implementations of a phase-vocoder: The tricks of the trade.
[5] https://en.wikipedia.org/wiki/Filter_bank#FFT_filter_banks
[6] https://en.wikipedia.org/wiki/Pitch_(music)#Pitch_and_frequency