1978 to 1986: Multichannel, Temporally-Interlaced, Pulsatile Speech Processors

1. May 1985: UCSF (Lindsay Vurek) finished the implementation of a CIS proceesor but did NOT test it on a patient: It was capable of stimulating each of its 4 channels at a rate of 610 PPS -- for a combined rate of 2440 PPS.

2. June-July 1985: At UCSF, RTI tested a CIS processor, it was a CIS processor except that its pulse-rate was "much too low" (only 313 PPS/channel for the 2 'peak-picked' channels out of 6).

Time-Line of Cochlear Implant Research During this Period:

• In 1978 or before, the Paris group (Chouard's group) stimulated multiple electrode channels with simultaneous pulsatile stimuli.  Information source:  Immediately after the 1978 Paris cochlear implant conference, I visited Chouard's engineering lab and the audiologist in charge of patient testing and rehabilitation.

• Sometime in the early or mid-1980's, the same group used base-to-apex-sequenced, non-simultaneous pulsatile stimulation.  Each channel was stimulated at a pulse-rate of ~330 PPS because it was sequenced to approximate "the 3 msec traveling wave's traversal-time across the normal cochlea."  The authors did not describe or suggest any other purpose for the new stimulus paradigm (e.g., there was no indication that they used the interleaved-pulses method in order to reduce channel interactions).  It was not clear if the channels were voltage- or current-controlled, nor if the electrode channels were isolated or unisolated from each other.  Information source: published articles from the 1980's by the Paris group (publications //TBD).

• Interestingly, it's quite possible that the very first low-pulse-rate interleaved-pulses (IP) processors may have been built and tested by the Melbourne research group in the early 1980's: A brief description with references.

• The first UCSF real-time research processor was a 4-channel analog processor completed in 1981. See pp. 166 of Merzenich, 1983 for a diagram of one commonly-used configuration of UCSF's table-top real-time 4-channel analog processor. See pp. 355-359 of White, 1983 for an example of how the take-home version of the analog real-time processor was used for testing cochlear implant patients in 1981-1985.  More information about (1) UCSF's first real-time processor and (2) UCSF's other early research processor can be found here. Information about other researchers' earlier implementations of analog multi-channel processors can be found here (see the content after paragraph 2)..

• In 1980-82, the second UCSF real-time processor was specified at the architectural and processing-parameter level. Lindsay Vurek began implementing it. It was probably the simplest interleaved-pulses processor that could be conceived. But that does not imply that it was simple to implement in the early 1980's.  This was UCSF's second real-time processor:   

  • Summary of Processor Specifications: The pulse-rate of the channels was adjustable prior to the initiation of any given test, but was held constant during the real-time stimulation.  The maximum pulse-rate-per-channel was to be no lower than 1000 PPS.  [The completed system met all specifications except for this one: When the system was completed, the maximum pulse-rate was 610 PPS per channel if all four channels were stimulated (610 PPS can be calculated directly from the code on page 1 of Software for the UCSF CIS real-time processor).  The maximum pulse-rate per channel was inversely proportional to the number of stimulated channels:  For example, if only 2 channels were stimulated, the system could produce 1220 PPS per channel.]  The processor did not contain any form of F0 extraction.  The software did not incorporate a "peak-picker" algorithm.  With the exception of the adjustable-cutoff-frequency-envelope-detectors for each channel, the analog components of UCSF's CIS system were essentially identical to those in UCSF's analog real-time system.  The following link contains documentation for: [1] the analog hardware, [2] the IBM-PC-compatible computer system, and [3] the associated digital & analog computer-interfaces.  As in UCSF's analog real-time system, each component (e.g., mic preamp, pre-emphasis filter, bandpass filters, compressors, safety-clippers) had easily-accessed input and output connectors (i.e., bnc connectors) and could be easily interconnected in different configurations for testing.  These analog components were very flexible, with adjustable processing parameters for most of the modules.  UCSF's interleaved-pulses real-time system was only 4-channels.  However, it was specifically designed for "expansion by duplication" to 8-channels if the 4-channel system showed significant value for patients. -- By incorporating an external digital synchronization signal, or 'trigger' signal into the 4-channel system, and then duplicating this 4-channel system, we could easily create an 8-channel system. UCSF's 4-channel real-time IP processor was the first CIS processor to be implemented -- a good many years after it was first proposed, and an additional 4 years before RTI finally implemented and tested a CIS processor.

  • Temporarily skipping forward on this "time-line:" At long last, in May 1985, UCSF's 4-channel interleaved-pulse system was finally completed. --> just at the time that UCSF's last research patient was ready for testing.  This processor became UCSF's 2nd real-time, table-top speech-processor.  A listing of the source code and CPU interfaces used for the processor is available here: Software and CPU interfaces within the UCSF CIS real-time processor.  Page 1 of this document contains the interface code for specifying the pulse-rate, pulse-widths, inter-pulse-delays, etc.  The pages that follow contain primarily Intel assembler code that drives/controls the D/A and A/D operations that implement the CIS.  Page 4 contains a list of the hardware interfaces in the 'PC-compatible' part of the system.  Some interesting details about this UCSF real-time processor can be found here and a very brief published description can be found in this excerpt from p. 626 of Bruce, et al. 1999.  The following 1991 letter from Dr. David Morledge may be of interest.  In the mid-80's David was a Speech and Hearing Science grad student working in our UCSF group, helping us conduct speech perception tests.
    

  Now moving back on the time-line to 1980-1982: It became apparent that implementing and testing the 2nd UCSF real-time processor within a reasonable amount of time required more engineering support than was available to the UCSF research group at that time. -- This was the case even for the relatively simple interleaved-pulses processor described above, -- largely because the small number of engineers and technicians had been moved primarily into production of an analog 4-channel, 4-coil RF "crystal-set" transmission system that was to deliver 4 channels of bandpassed, compressed-analog speech to patients.  This "clinical and 'research'" system was to be distributed to a significant number otolaryngologists around the US and Britain!  See this page for more information about this reallocation of engineering resources.
  

• As a consequence, the UCSF researchers requested that the NIH neural prosthesis contract office support RTI for the purpose of implementing speech processors for cochlear implant research.  Fortunately, NIH funded the UCSF - RTI collaboration that started in the 4th quarter of 1983.  Shortly thereafter, the collaboration expanded into a collaboration for both implementing and testing speech processing systems.  Shortly thereafter, RTI also started conceiving of, and designing, speech processors.

• In 1983, when RTI started their NIH contract to implement and test processors with UCSF, RTI was asked by UCSF to implement, in RTI's soon-to-be-built highly-flexible block-diagram simulation system, an interleaved pulses (IP) processor with specifications at least as good as those already defined for the UCSF real-time IP processor. Interestingly, this was the only new processor specified to RTI. UCSF, at that point in time, decided to focus their work on this new processing strategy. Once it was implemented and tested, we believed we could then design even more useful processors with our newly acquired knowledge. It's important to note that the UCSF 4-channel, real-time, compressed-analog processor had already been constructed and used to test research patients! Some of the results of these tests at UCSF had already been published by 1983. In addition, a great deal had been learned by the other groups at Utah and Melbourne using multichannel systems with patients. UCSF was very anxious to reduce channel interactions and convey as much usable temporal and spatial information to patients as possible.
  

• 1983-1985: Lindsay Vurek continued building UCSF's real-time, high-rate IP processor and, in addition, had to devote time to other engineering work at UCSF. Remarkably, during this same time period, Lindsay Vurek initiated, and with the help and support of Mike Merzenich, wrote an NIH grant proposal to build an even faster real-time, 8-channel, bench-top, speech processor. Unfortunately, that grant was not funded. Under their NIH contract, RTI implemented and validated the safety of their highly-flexible block-diagram laboratory system. They got a significant amount of that job done by June-July 1985.
  

• Around March, 1985, RTI proposed testing two processors to UCSF (see excerpt 1 and excerpt 2).  They proposed to (1) implement the UCSF analog processor in their non-real-time, highly-flexible system and to (2) implement a new type of processor that RTI had invented, the "multipulse excitation" processor -- that RTI thought held promise.  UCSF was positive about RTI's proposal, but reminded RTI of the importance of implementing the high-rate IP processor which had been a top goal of UCSF for a number of years. -- the same processor that had been specified to RTI at the beginning of RTI's contract.  Indeed, at the time, Lindsay Vurek was still trying to construct the real-time version of this IP processor!  It's a good thing that UCSF continued to "encourage" RTI in this direction:  Judging from RTI's proposals for processors in its May 15, 1985 NIH contract proposal,  RTI did not indicate any interest in Interleaved Pulses (IP) processors.  RTI's interest level certainly changed significantly after they tested the IP processor.  And it changed even more in 4 years after they finally implemented the high-rate version of that IP processor!

•  As a consequence of this communications between UCSF and RTI, the following three processors were implemented and tested by RTI in June-July, 1985 at UCSF with the help of UCSF's last research patient:

  1. The Analog Processor:  Both RTI and UCSF wanted RTI to implement the analog processor in its block-diagram, non-real-time stimulator in order to compare and "calibrate" it with the UCSF analog processor. -- to make sure the RTI non-real-time system performed similarly to the UCSF real-time analog system -- which had already been successfully tested on several patients.

  2. The Interleaved Pules (IP) processor:  As previously stated, UCSF wanted RTI to implement the interleaved pulses processor that UCSF had been trying to implement for so long!  Indeed, RTI partially complied with this request, but failed to deliver the high pulse rates requested.  Far worse, was that RTI failed to inform UCSF that it could not, or had decided not to, implement and test pulse-rates higher than 300 PPS per channel.  It was not until many months after the patient had completed testing that UCSF learned that RTI had not tested at the higher rates requested. 

  3. The Multipulse Excitation Processor:  RTI invented and designed this new processing system and believed it had significant potential.

• As documented and described above, in May, 1985 UCSF's relatively high-rate IP finally was ready for testing.  Amazingly, this IP processor was not used to test UCSF's last research patient.  Unfortunately, we (UCSF/RTI) used RTI's implementation to "test-out" this processing strategy.  Unbeknownst to UCSF, RTI had only been able to, or simply had decided to only, implement a low-pulse-rate version of the interleaved-pulse system that UCSF had specified to them.  RTI had not implemented, or perhaps had not been able to implement, the requested high pulse-rate stimulation (1000 PPS/channel).  More importantly, RTI had failed to inform UCSF of that deficiency before, or during, the time of patient testing.  Perhaps RTI thought it was unimportant!

  • Why had UCSF decided to use RTI's implementation of UCSF's simple "high-rate" IP processor?"  In addition to UCSF's (incorrect) belief that RTI's implementation better met UCSF's high-channel-pulse-rate specification, it was believed that RTI's block-diagram system would be more likely to be used in the near future. -- because UCSF's grant was not continued.  As a consequence, UCSF thought it best to concentrate efforts on using/testing the RTI implementation.  UCSF would NOT have chosen to use the RTI implementation had UCSF been informed of the RTI system's severe deficiency.

• The first interleaved pulses (IP) processor to actually be tested in the United States was implemented by RTI and "tested-on" UCSF's last "research patient in June-July, 1985: block diagram of processors on page 7,  excerpt from page 13 and continuation from the top of page 16, of QPR 7 from RTI (1985).   "Strategy 3" was the first Interleaved-Pulses (IP) processor ever tested by the UCSF/RTI group.  It was a CIS processor except that that channel pulse-rate was far too low!  Pulses were interleaved so that no electrode channel was stimulated simultaneously with any other channel.  The multi-channel system operated at 312 PPS per channel.  No F0 extractor was used.  Each channels' "instantaneous compression function" was the same as that used in subsequent CIS implementations.  In addition, a peak-picker was included in this processor (see "The Peak-Picker");  it was later renamed by RTI as the "N-of-M" method).  The preliminary speech perception tests indicated that the system performed better than the other processors tested: excerpt from pages 24 & 25 of QPR 7.  This temporally-interlaced system was later named by RTI as: (1) the "maximum-rate IP" processor, or the "max-rate IP" processor.  After the pulse-rate was finally increased sufficiently in 1989, and the performance improved dramatically, the processor was renamed (2) the "Super-Sampler," and shortly thereafter it was re-branded again as (3) the CIS processor!