Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing / Edition 1

Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing / Edition 1

by Tong Zhang, C.C. Jay Kuo
     
 

ISBN-10: 0792372875

ISBN-13: 9780792372875

Pub. Date: 01/31/2001

Publisher: Springer US

Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing is an up-to-date overview of audio and video content analysis. Included is extensive treatment of audiovisual data segmentation, indexing and retrieval based on multimodal media content analysis, and content-based management of audio data. In addition to the commonly…  See more details below

Overview

Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing is an up-to-date overview of audio and video content analysis. Included is extensive treatment of audiovisual data segmentation, indexing and retrieval based on multimodal media content analysis, and content-based management of audio data. In addition to the commonly studied audio types such as speech and music, the authors have included hybrid types of sounds that contain more than one kind of audio component such as speech or environmental sound with music in the background. Emphasis is also placed on semantic-level identification and classification of environmental sounds. The authors introduce a new generic audio retrieval system on top of the audio archiving schemes. Both theoretical analysis and implementation issues are presented. The developing MPEG-7 standards are explored.
Content-Based Audio Classification and Retrieval for Audiovisual Data Parsing will be especially useful to researchers and graduate level students designing and developing fully functional audiovisual systems for audio/video content parsing of multimedia streams.

Read More

Product Details

ISBN-13:
9780792372875
Publisher:
Springer US
Publication date:
01/31/2001
Series:
Springer International Series in Engineering and Computer Science, #606
Edition description:
2001
Pages:
136
Product dimensions:
9.21(w) x 6.14(h) x 0.44(d)

Table of Contents

List of Figuresix
List of Tablesxiii
Prefacexv
Acknowledgmentsxix
Part IIntroduction
1.Introduction3
1.Significance of Proposed Research3
1.1Video Segmentation and Annotation3
1.2Audio and Visual Content Analysis4
1.3MPEG-7 Standard Development6
2.Review of Previous Work8
2.1Work on Video Indexing and Retrieval8
2.2Work on Audio Content Analysis9
2.3Work on Visual Content Analysis12
3.Summary of the Proposed System12
3.1Framework for Video Segmentation and Indexing12
3.2Content Analysis of the Audio Stream13
3.3Content Analysis of Image Sequences17
4.Contribution of the Research17
5.Outline of the Monograph19
Part IIVideo Content Modeling
2.Video Content Modeling23
1.Common Model for Video Content23
2.Models for Different Video Types24
2.1News Bulletin24
2.2Variety Show Video24
2.3Sports Video25
2.4Documentaries26
2.5Feature Movies and TV Series27
3.Proposed Scheme for Video Content Parsing28
4.Design of Index Table for Non-linear Access30
4.1The Primary Index Table30
4.2The Secondary Index Tree30
Part IIIAudio Content Analysis
3.Audio Feature Analysis35
1.Audio Features for Coarse-Level Segmentation and Indexing of Generic Data35
1.1Short-Time Energy Function35
1.2Short-Time Average Zero Crossing Rate37
1.3Short-Time Fundamental Frequency38
1.4Spectral Peak Track42
2.Audio Features for Fine-Level Classification and Retrieval of Sound Effects48
2.1Timbre Features48
2.2Rhythm Features53
4.Generic Audio Data Segmentation and Indexing55
1.Detection of Segment Boundaries55
2.Classification of Each Segment56
2.1Detecting Silence56
2.2Separating Sounds into with and without Music Components58
2.3Detecting Harmonic Environmental Sounds61
2.4Distinguishing Pure Music61
2.5Distinguishing Songs62
2.6Separating Speech with Music Background and Environmental Sound with Music Background62
2.7Distinguishing Pure Speech63
2.8Classifying Non-harmonic Environmental Sounds65
3.Post-Processing65
5.Sound Effects Classification and Retrieval69
1.Hidden Markov Model and Gaussian Mixture Model69
1.1The Gaussian Mixture Model70
1.2The Hidden Markov Model71
1.3Hidden Markov Model with Continuous Observation Density72
1.4Hidden Markov Model with Explicit State Duration Density73
2.Clustering of Feature Vectors74
3.Training of HMM Parameter Sets75
3.1The Training Process75
3.2Implementational Issues77
3.3Comparison with the Baum-Welch Method79
3.4Incorporation of the Viterbi Algorithm79
4.Classification of Environmental Sound80
5.Query-by-Example Retrieval of Environmental Sound81
Part IVImage Sequence Analysis
6.Image Sequence Analysis85
1.Histogram Difference value in Image Sequences85
1.1Definition of the Metrics85
1.2Histogram Difference of the Y-Component86
1.3Histogram Difference of the U and V Components88
1.4Histogram Difference of the Combined Code89
2.The Twin-Comparison Approach91
2.1The Original Algorithm91
2.2Experimental Results and Modifications92
3.Shot Change Detection Based on Combined Y- and V-Components97
3.1Determination of the Lower Threshold97
3.2Determination of the Higher Threshold98
3.3Framework of the Proposed Scheme99
4.Adaptive Keyframe Extraction and Associated Feature Analysis102
4.1Adaptive Keyframe Extraction102
4.2Feature Analysis of Keyframes103
Part VExperimental Results
7.Experimental Results107
1.Generic Audio Data Segmentation and Indexing107
1.1Audio Database107
1.2Coarse-Level Classification Results108
1.3Segmentation and Indexing Results109
2.Environmental Sound Classification and Retrieval112
2.1Timbre Retrieval with GMM112
2.2Sound Effects Classification Results112
2.3Sound Effects Retrieval Results114
3.Shot Change Detection and Keyframe Extraction115
3.1Shot Change Detection Results115
3.2Keyframe Extraction Results116
4.Index Table Generation117
4.1Index Table for News Bulletin117
4.2Index Table for Documentary119
Part VIConclusion
8.Conclusion and Extensions123
1.Conclusion123
2.Feature Extraction in the Compression Domain124
3.System Integration and Applications125
4.Contributions to MPEG-7126
References129
Index135

Read More

Customer Reviews

Average Review:

Write a Review

and post it to your social network

     

Most Helpful Customer Reviews

See all customer reviews >