hello and welcome back today we are going to be discussing speech perception so we do have quite a lot of ground to cover in this week's lecture though I promise it shorter than last week's lecture uh we'll start out by discussing how we perceive speech goal were some of the physics of sound some real Basics here to you know help get everyone on the same footing cover some era anatomy and then discuss in more details specifically how we can examine speech sound using spectrograms we're then going to um for part two here connect more closely with what's in the textbook by examining what's known as the decoding problem and how we go about perceiving the complex acoustic qualities of speech so we'll discuss co-articulation categorical perception and the canal effect we'll discuss the development of speech perception and we're going to close out today's lecture by discussing some different examples um methodologies and theories surrounding speech perception uh before we jump into the lecture content um a couple of housekeeping items so you will buy now probably seen a couple of the quiz Recaps um posted on each week's lesson page so you can look at those to go over what the correct answers were for the quiz responses uh that's going to be really your best preparation for the midterm exam the quiz questions are the same format as the final exam questions I'm writing all the questions so you know the same kind of voice and uh the same generally the same format if you ever have any questions about the quiz or about your answer or about your specific submission feel free to get in touch with Kara or myself and we're happy to answer any follow-up questions that maybe aren't addressed in the quiz recap to help you better prepare for the exams or um you know improve your quiz marks throughout the term if that's something you're looking to do uh and also apologies to anyone who sent me emails um you know certainly uh through January it was maybe a little bit slow you can probably hear that I'm still sick uh so I've been sick for quite a while now and still slowly getting over it I will say today's probably one of the first days that I've really started to feel better I managed to pull myself together for about two hours to record lecture but beyond that I feel like too little butter spread over too much bread but starting to come out of that now so um here we're going to you know with that behind us we're going to jump into our lecture content so speech perception is a very special thing for humans we've talked about the human specificity for language and other lectures what's really interesting for humans is that we can understand all speakers of our language even people we've never heard before our auditory system is able to accommodate differences in people's speech be it or differences due to Accents uh sex or gender different speaking conditions whether they have a cold or not whether they're wearing a mask or not and this when you think about it is a fairly incredible process because it's something that computers and computers are quite good at a number of other things computers are quite bad at perceiving speech or until very recently they were quite bad uh if you are training a computer for speech you can train it to your voice and at best they can usually recognize very general speech or the speech of one person um but often they misunderstand what you're asking if you've ever said you know hey Alexa hey Siri hey Google insert you know AI here they often misunderstand what you're asking and they're much less effective at recognizing speech than humans are speech is different than other kinds of sounds we produce speech in a way that is variable and speech contains different kinds of information that need to be put together to create what we think is the perception of spoken language in this section of lecture we're going to discuss some of the physical properties of sound because of course sound makes up speech sound is all about vibration to make a sound something has to vibrate whether it's the string of a guitar your larynx that is your voice box uh or the loudspeakers on your radio sound waves consist of vibrating particles which knock into other particles causing those particles then to vibrate and knock into more particles and so on and so forth and this is how sound waves travel away from their Source like the ripples on the surface of water we hear sounds because the vibrations in the air cause our eardrums to vibrate and these vibrations in our eardrums are converted into nerve signals that are sent to our brains so we're going to talk in more detail now about that whole process everything I've just said to you so if you didn't get it on the first pass we're talking about it again here so sound is energy that travels through the air as waves the particles that vibrate as part of a sound wave move back and forth along an axis so the axis is what the sound wave is traveling across it's actually not something that's modeled you know in the real world it's not in 2D space but we model it in 2D space it's you know part of our work is scientists and so we can look at a graph here of a sound wave here's our sound wave curving up and down so it's vibrating or traveling through space and it's moving across time so the time is our x-axis in this little graph so this sound wave represents um a sound particle passing through a particular point in space and this is the most commonly used type of graph to graph sound we can extract two important features from this graph we can extract The Sounds pitch and the sounds volume so we'll talk about each of these now in turn we're going to talk first about pitch or frequency so this is the frequency of occurrence of the vibration how often do the vibrations happen this is measured in cycles per second so that is the number of wavelengths that pass by a given point in a given amount of time so giving you a practical example here we've got low frequency sound on the left hand side so we've got this sound particle how many times does it pass through a given point in space within one second compared to a high frequency sound how many times does a sound particle pass through a given point in space within one second how many cycles per second we've got lower frequency and higher frequency so low frequency fewer cycles per second high frequency is more cycles per second so a low frequency sound is also known as a low pitch sound so it's going to be a lower sound imagine the lower keys on a piano keyboard like I have here a high frequency sound is going to have a higher pitch a higher sound so imagine the higher keys on a piano the physiological perception of sound wave frequency is pitch so the acoustic variable the actual physical property is frequency our perception of it is pitch so that's frequency will now move on to amplitude amplitude um is the amount of change that a wave undergoes during one cycle so you can imagine this you know if our x-axis is time and y-axis is the amplitude or the height of the waveform so that's going to be the distance or displacement by a point on a waveform from its equilibrium point that is how far is the particle moving along its axis so the changing and amplitude of a sound wave um along its axis and it can either be wide a higher amplitude that's a higher volume or it can be narrow which is a lower amplitude or a lower volume here the physiological perception of amplitude is loudness or volume so amplitude is the physical property our perception of that physical property of sound is volume so some very basic properties uh you know a brief intro to physics lesson for you on sound will now move into a brief intro uh to the anatomy of the auditory system and of course our auditory system as users of a spoken language is what's used to perceive speech so our ear is really exquisitely designed to pick up on frequency and amplitude of sounds in a very sophisticated way so looking at the diagram of the ear here we've got the outer ear which is the part of the ear you can actually see and we have the middle ear within the middle ear we've got the tympanic membrane or the eardrum so pressure waves in the air caused by sound waves caused the eardrum to vibrate so of course those vibrating particles eventually hit your eardrum it causes then your eardrum to vibrate these vibrations of the eardrum are mechanically Amplified within the middle ear by the ossicles so the malice the incus and the stapes so there are three small bones that transmit the sound from the eardrum to the inner ear in the inner ear now the sound transmits from the ossicles to the cochlea the cochlea is the key organ of auditory sensation so think about sensation versus perception the cochlea is the key organ of auditory sensation it's this snail-shaped organ within your inner ear inside the cochlea is the basilar membrane which is a structure extending inside the length of the cochlea and it divides the cochlea into so as vibrations enter the cochlea they're carried as waves through the fluid within the coconut so there's um there's fluid within the cochlea that's how the sound waves travel through it so they travel all the way through the Cocula and then they go down the vascular membrane and they come back out the other side down the other side of the basicular membrane and so these vibrations are the sound waves coming into the ear the vibrations within the cochlea that are moving the fluid within the cochlea are picked up by hair cells on the surface of the vascular membrane the surface of the vascular membrane is covered in hair cells which are specialized cells that are sensitive to movements in the cochlear fluid the cochlear nerve then so the sound has now traveled all the way through the Cocula and is sent to the cochlear nerve is sending auditory information from the cochlea directly to the brain directly to primary auditory cortex so a general overview of what's happening when we're perceiving sound how the different anatomical um anatomical pieces of the ear contribute to hearing sound this is something that would maybe be a really good exam question asking you you know how is sound transmitted through the ear what are the key organs of the ear or um the sensation of sound so the tympanic membrane the ossicles so the three bones that transmit the sound from the tympanic membrane or the eardrum and then the Cocula within the Cocula you've got the vascular membrane and then following the cockle you've got the information sent to the cochlear nerve so thinking about that keeping that in the back of your mind as you're preparing for your midterm um this next little part I won't be asking about um but you also have the semicircular canals and so you can see those at the junction between the middle ear and the inner ear um and those and the vestibular nerve work together to carry spatial orientation information to the brain and the eustucian tube helps to regulate pressure in the ear and this is connected to the nasal cavity so you could imagine that with the eardrum vibrating you know that would create a buildup of pressure inside of the ear and so these Fusion tube helps to regulate that pressure buildup again this is also background information something you won't be asked about um but this is just from one of my anatomy textbooks that I have here at the house um just showing anatomy of the cochlea uh in panel D so this bottom panel here you could zoom in on your own device I can't zoom in in my presenter review um but we've got the hair cells on the inside of the cochlea and then we've got a cross section of La calcula um and then going into a cross section of one turn of the calculus so you can read through the description here if this is something that you're interested in um and just giving you more of a visual representation again of that beautiful design of this primary auditory organ okay now moving back to important information and information you need to know so the cochlea is tonotopically organized what does that mean well there's a progressive arrangement of hair cells within the Cocula that are sensitive to different frequencies so sounds come in at different frequencies and what's interesting is this organization principle of the cochulon of the vascular membrane extends to the auditory cortex so it's mirrored within auditory cortex so the copula is designed in such a way that the base of the cochlear response responds to high frequency sounds and the Apex so going all the way in to the inside of that snail curve responds to low frequency sounds because low frequency sounds can travel farther than high frequency sounds and so this organization this organization from high frequency to low frequency on marcocula is extended to primary auditory cortex so our primary auditory cortex is located on the superior temporal gyrus so here we've got temporal gyrus which I'm tracing with my cursor and you can see that scalpel kind of pulling out the superior portion of temporal gyrus um so this is the primary part of the brain responsible for processing auditory information and this suggests that the brain is set up to be sensitive to different frequencies inside of the hearable spectrum and that we are set up to discriminate between those different um those different frequencies and so you can actually see um on this image here that you've got um a portion of primary auditory cortex which is responsible for perceiving um low frequency sounds so what is perceived at the Apex are what's sensed at the apex of the Cocula and a portion of primary auditory cortex here responsible for perceiving high frequency sounds and that's what's sense sensed at the base of the copula so you can see low frequency to high frequency within primary auditory cortex and similarly low frequency to high frequency within the cochlea so that is what it's meant by tonatopic organization we'll recall what we've talked about last week that information from the left ear is sent to the right hemisphere to the primary auditory cortex on the right Hemisphere and information from the right ear is sent to primary auditory cortex in the left hemisphere and then language processing happens a little bit after that so first you perceive that it is sound and then you'll perceive that's language and processing happens in different regions of the brain for that and so this is also demonstrating our hearing range uh so going from about 200 Hertz um up to about 20 000 Hertz other animals have different hearing ranges um so this is just uh the organization of the human um hearing range and let's see if I've got through everything on my notes yeah okay that's everything for Tata topic organization um so if you do have any questions um about what we've talked about so far with physical properties of sound um or the anatomical setup of our auditory system feel free to ask us I know this is something that's not covered in the textbook but I still think it's important for us to know about um so I'm happy to answer any questions on this now moving into the Acoustics of speech so we've talked about sound up to this point what makes perceiving speech a little bit different we've so far given examples with just frequency and amplitude but speech is in fact made up of many complex sound waves that are stacked on top of each other and we can actually kind of see here um Pitch and amplitude so we can see you know some lower amplitude sounds some higher amplitude sounds um we can also see you know different changes um in the overall frequency as well so how tight are the lines together or how loose are the lines this is actually um a clip of someone saying the phrase sound example and so this is how that bit of speech unfolds over time so the pitch and amplitude qualities of that utterance we see high peaks of energy we see low peaks of energy but the graph doesn't actually give us characteristics that we think are important for actually decoding speech to look at other more complex properties of speech we look at something called a spectrogram and so that's our graph here on the bottom of the slide spectrograms break down the Acoustics of speech into the amplitude of a waveform at different frequencies of that waveform so we can analyze it so our top graph here is just amplitude over time the spectrogram allows us to see Pitch volume and Timbre over time so camber is something we're not really talking about here but we can view all of those properties within a spectrogram spectrograms allow us to look at signal properties at specific points in time it helps us to see Pitch volume and Timbre so we've got our graph of sound example just amplitude over time at the top and it's the same utterance within our spectrogram sound example within the spectrogram now it might be they might seem like you're looking at a total mass here uh so let's break down this example into its component parts we have three different dimensions on this graph the x-axis is time so we're familiar with that the y-axis is frequency and the color intensity is amplitude uh so on this graph the purple and black um means the least amount of energy um so the lowest amplitude and the yellow means the most energy or the heights amplitude and you can see that color bar on the right hand side of the graph the vertical line so the breaks that you're seeing in the spectrogram the breaks of the dark um the dark portions so where my cursor is um at this spot um relate to the opening and closing of the vocal Folds the horizontal lines that you're seeing so here um just at the beginning of the utterance you see these three yellow horizontal lines those are called formants we're going to talk about performance on the next slide this spectrogram is showing us a digital production of a voice saying 19th century and so this formant that you're seeing is that I the I vowel in 19th so that first vowel that you're hearing in 19th um we're gonna break this down again more in the next few slides so here I've colored different formants I'm using different colors so we've got four months and the White and the orange and in the blue performance change over time as the Acoustics of that speech signal change so speech is made up of areas of concentrated acoustic energy called formants performance are the results of resonance within the human vocal tract so here's another important term resonance occurs when a system like our vocal tracts vibrate at a certain frequency the exact frequency at which an object resonates is determined by its size its shape its material resonance is the sound wave hitting an object and that object vibrating the frequency of that vibration is what's determined by its eye shape and material performance are distinct frequency components of the acoustic signal produced by speech different hormones differ in their frequency we can represent those differences on a spectrogram and hear those differences when hearing speech and produce those differences when producing speech so for example the vowel sounds a and ah have subtly different frequency bands and so we're able to decode speech using spectrograms and can figure out what vowel was a person uttering by looking at the different formant qualities so again those formants are bands of high amplitude energy so high amplitude frequencies in speech performance um actually before I get into this part so um maybe giving you a bit of information about what do you need to know about all of this um I'll just jump back for a quick second so what do you need to know about spectrograms you would need to know the three properties that we can extract from a spectrogram so time frequency and amplitude um so the three properties that I have represented here so a spectrogram can tell us about time frequency and amplitude and I need you to know about formant Transitions and steady state formats which I'll talk about now um I'm never going to give you a spectrogram and ask you to decode speech there are other classes at Western where you will have to do that um I just really want you to know the basics um you know this is a psychology of language course I'm more interested in the psychology of all of this which is why formant Transitions and study state formants are important so formants can be used to describe consonants and vowels a formant transition indicates a consonant so that's where you have a shift in the formant um so looking at the do example here on the right and so you can see the shift this little dip that formant transition that's the D portion of the do uh the Dew morpheme um and for the boo example you also have that little shift that's the portion of this syllable steady state performance or vowels so it's that steady kind of flat line that we're seeing in these formats that as well and do and we have those steady state form it's those kind of flat lines what we see here in this spectrogram that the formants for vowels are roughly the same we see three different formants for the u vowel um we see that they're roughly the same position the formant transitions uh the and the D look different from them um so we need to know that form of transitions indicate a consonant on a spectrogram and that a steady state formant indicates that there's a vowel when we want to bring this all together now so when looking at speech in a spectrogram like the figure on the right here depicting the sentence sound example there are a lot of different features of the signal that we can clearly observe and so we'll take what we've learned in these last few slides and apply it here so the horizontal lines in what's kind of like a comb-like structure are the vowels and so at about 0. 5 seconds there's the vowel e for sound example I think sound yeah sound example um at 1. 3 seconds that ah or ample example yeah there we go sorry I'm making a little bit of a mess of this um but anyways these comb shape structure are the vowels these are the steady state formats that we've talked about on the last Slide the kind of straight lines that you're seeing these high the bands of high acoustic energy those you know more white lines that we're seeing across the vertical lines that we see within the spectrogram correspond to abrupt sounds which are often characterized as transients um so a transient in speech is usually um there's typically a stop consonant we're going to talk about stop consonants more next week but a stop consonants where you stop your airflow uh so like in the syllable D in sound um and so okay yeah so it's my first battle example was wrong um so the duh at 0.
8 seconds so right about here where you see more black so that's stopping airflow there is not acoustic energy at that point um and the um at about one second that's right before that we've got again another stop um for a sample and so stopping your flow there as well areas which have a lot of energy in the high frequencies are going to appear as lighter colors so like at about . 3 seconds um is going to correspond to um noisier sounds um so we can see the S at both 0. 3 seconds and 1.
1 seconds so at about 0. 3 seconds the sound example that's s fricative we see a lot of white energy up here at the top of the graph um and the sound example the sample um at about 1.
