2) United States Patent Harris (ADAPTIVE HIGH FIDELITY REPRODUCTION 8 (76) Inventor: Scott (2). Notice: EM Rancho Santa F patent is USC. 154(6) by 269 days. (21) Appl. No. 102848,993 (22) Filed: May 18, 2004 (6) jor Publication Data US 2ongio213414.A1 Get. 28, 2008 Related U.S. Application Data " Harris, PO, Box 927649, cA (US) 92192 Subject to any disclaimer, the term ofthis ded or adjusted under 35, (@2) Division oF application No, 09/799,460, filed on Ma 5, 2001, now Pat, No, 6,738,318, (1) Ine a, Hose 120 (2006.01) a. 36902; 369447.15 «us 100 110: 105 USOO71S4819B2 (io) Patent Noz US 7,154,819 B2 (4s) Date of Patent Dee. 26, 2006 (58) Field of Classification Search None See application fle for complete seach history. 66) References Cited US. PATENT DOCUMENTS 4018448 A © 41977 Di Mateo son2783 4828391 4198) Schwvarte 4892178 A T1989 Tre, Sawos02 A 71995 Page 5910990 A 61999 Jang 6195435 BI 22001. Kimura 6.686.531 BI 22004 Pennock wt a suis * cited by examiner Primary Examiner Paul Huber on ABSTRACT Audio is adaptively associated with speakers, depending on the speaker configuration tht is present. Each speaker it receives an audio assignment based on its individual spectral cbarocteristes. As more speakers are added, content is adaptively associated with that you speaker, and taken away from the previous. 13 Claims, 2 Drawing Sheets 150 PLAYER MEMORY |-170 AMPLIFIER 155 160U.S. Patent Dec. 26, 2006 Sheet 1 of 2 US 7,154,819 B2 100 7150 110: 05) PLAYER 1017” [7 170 AMPLIFIER 165-J MEMORY Ls 7 160 FIG. 1 250 260 255.268: MEM| 165 FIG. 2 350 300 ca MelU| | 65 31 FG. 3U.S. Patent Dec. 26, 2006 Sheet 2 of 2 US 7,154,819 B2 POLLS ‘SPEAKERS DETERMINE SETTING DISPLAY CONTENTS ADAPTIVELY ASSOCIATED FIG. 4 LR NORMAL MULTI-TRACK ENHANCED ALL MULTI TRACK & INFO IN PQ HEADERS TELLING CD PLAYER IGNORE PARTS OF 505 THE STREAM a FIG, 5B BROADBAND PIPE ” 525 (SATELITE RADIO, SIDE 1 CABLE, ETC) NORMAL NORMAL — ENHANCED FIG. 5C FIG. 5DUS 7,154,819 B2 1 ADAPTIVE HIGH FIDELITY REPRODUCTION SYSTEM ‘This application is a divisional of US. application Ser. No, 09/799,460 filed Mar, 5, 2001 now U.S. Pat. No, 6.738318, BACKGROUND High fidtity systems attempt to simulate the sound that comes from actual sound-producing objects. Real music is produced when each of a plurality of different instruments, ata dillereat location, produces is own unigne sound. Fach ‘nsrument also as unigue soe tuning characterises. The real music is produced fom these instruments, at diferent. Jocations, producing sounds, Producing a simulation of this real music is the objective of a high fidelity musie repro duction system. ‘Movies, in contrast, actually have a diferent objective for ‘heir sound production. Inthe 1980s, movie sound became format with multiple channels proving the sound output “This format, called surround sound, produced five or more channel of sound The chanaelsinluded lft and right main channels for stereo music. A center channel was used for ‘mono paris ofthe reproduction such as the voice. In ad tion, left and right surround channels; were provided for special ellets. In addition, adltional channels may be rovided fr sound having special characteristics such as sub ‘woos. This sound sytem attempts to proce the feling of actually being par ofthe action depicted by the movie. SUMMARY. ‘The present inventor believes tht an ideal musical repro duction, like real music, should produce the sound fiom a plurality of instruments, each coming from is own tuned source that has tuning/music reproduction characteristics that is most closely representative of the instrument. The current system of stewo reproduction reproduces most, if ‘ot al, instruments, fom to different sources (speakers), both of which are tuned the same. According to the present system, information is produced for reprouction by music reproduction hardware. The infor- mation 2s produced has a numberof separated parts, Thats, cach steam of audio information, suchas a song, may bave separated parts that form that steam. In one embodiment, ‘those parts may be tacks onthe audio reproduction medium, The separated parts are adaptively associated with dtfr- cent sie reproduction hardware based oa the actual char- acteristics ofthe hardware producing the musi. Thats, for ‘example, the violin sounds may be produced by the speaker ‘most closely tuned to violins. Another speaker, eg, most closely tuned to horns, may reproduce te hors, Another aspect automatically determines specific charac teristics ofthe hardware, and forms file indicative of those specific characteristics of the hardware. The contents ofthat file i used to adaptively associate the content ofthe media, eg, the music, withthe hardware, BRIEF DESCRIPTION OF THE DRAWINGS “These and other aspects ofthe invention will e desribed in detail with reference 10 the accompanying. drawings, wherein: FIG. 1 shows a block diagram of th system: FIG. 2 shows @ wied connection between amplifier and speakers x“ » % « 2 FIG. 3 shows shows a wireless connection; FIG. 4 shows a flowchart of operation: FIGS. 54-SD show diferent ways of storing information for use in the present application, DETAILED DESCRIPTION “Modern audi reproduction media may have mote space for storing the data that is indicative of the audi. For ‘example, the so-called DVD may have 100 times the storage capability ofa standard CD, also known as 2 “Redbook” format CD. Various enhanced CD formats have also been suggested which provide more data on the media and that can be used to store more information. In addition, modern ‘compression formats, such asthe MP3 fom, allows reduc- ing the size that is occupied by information placed on the smesia, Again, this has the effect of allowing the media to Sore more information, Broadband channels are also available. For example, satelite radio channels are proposed. Broadband Interaet channels have been used. In addition, audio content may also be produced over a cable and te like ‘The present system will be described with refereace 10 information being siored on the audio medium. A plurality of tracks are provided on the medium, The tracks each include information shout a different aspeet of the audio stream that is recorded on the medium. The audio medium is shown here a being a disk, but itcould be understood that ‘any different kindof audio containing medium could be used with the system of the present invention, ach rack may representa specified kind of information, In one aspect, each track includes information about the same kind of instruments. The instruments included on a single track may be of the same time, ell violin, or may hhave the same spectral characteristics, that is all sting instrumeots, oF all producing output within a specified spectral region, or primarily within a specified spectral region. Sounds may be grouped based on spectral charae~ teristics, eg, by using a Last Fourier trasform on recorded sound from the instrument. Each instrument may be char- acterized on the spectrum, e.g, by fomning a histogram indicating that amount of energy in each special bucket. Altematively, instruments or sounds which may effectively ‘compress may be grouped together, The insteaments which are suflicienly similar may be grouped togetber a a track. ‘This has a numberof advantages inthe context ofthe present system. Fist of all, it makes the information on the track more compressible by certain compression systems such as MP3, since each instrument on the track has similar chat- acteristics. In dition, on readout, the tack ean be accu rnitely repeoduced by the sume kind of reproduction equip “Multiple eacks are placed on the medium for diferent purposes. For example, a single medium may iachude movie siyle tracks suchas lf, igh, center, ft surtound and right surround, and also subwoofer setting, The left and right. ‘racks on the medium represen the stereo information. The remaining information in the tracks may represen informa tion ftom different individual instruments of instrument |ypes. This information may include separate acs for each of voices, strings, winds, guitar, percussion, bas strings, and bass winds, with the understanding that’a each diferent ‘stumeat aay also be broken up based on is charactris- ties eg. bas or toble. The abave has described 13 tracks for cach stored item of information. It should be understood, ‘however that there may be fewer or more tracks, up 10US 7,154,819 B2 3 20 tracks, Since each track may represen information ofthe specified instrument type, the information in the track may be highly compressible, ‘As canbe soen fom te above, the medium will typically include more information that is nocessary to actually plays bok the audio on any system. For example, the medium ‘may include stereo left and stereo right channels. However, ‘on some systems, 10 speakers may be provided for diferent instrument types, and this information includes pats ofthe information thai also within the left and right stereo. Ifthe 10 separated channels are used, the audio left and audio right {information might not be used. Therefore, the audio medium, ‘may include redundant information. Adaptive decisions are ‘made during playback indicating which speakers and or Which music reproduction equipment gels which conteat ‘Aa embodiment is shown ia FIG, 1. disk 100 includes a plurality of tracks of information. For example, if the twocks above are used, the stream, shown as 110, may include 13 different channels. The medium may also include control trek 108 which may be a data track: including information about which tacks oa the medium include ‘which informatio. ‘The medium is read out by a player. The contents ofthe ‘medium is interpreted by the adaptive elemeat that s either in the player, or ina controller or amplifier associated with 2 the player. The adaptive element i shown herein as 150, nd as beiag pat of the amplite. “The amplifier is connected to a plurality of diferent speakers or dilleren amplified speakers, Each speaker sys- tem, such as 155, has specified spectral andlor othe sound producing charicerisies. Ia an embodiment, each speaker may also be ative, inthe sense that it includes an electronic ‘module associated withthe speaker. That electonie module allows communication with the speaker, and may include information about the speaker's characterises. In another ‘embodiment, characteristics ofthe speakers may be obtained ina different way ‘The charectersis ofthe speaker may be communicated to the memory 165 over the speaker wire using serial formats such as universal serial bus, of RS 232 for example Allematively, the amplifier 150 may inede medium reading capability shown as 170. This reading capability ‘may read storage medium, suchas a loppy disk, memory stick, CD, or mini CD which is inserted therein, The medium includes information about the speakers, which is then read fromthe medium, and stored in the memory. Another way of ‘communicating information iso obtain characteristics from, 4 public network such a the Interaet. Tn another aspect, each speaker that is purchased is provided with an audio medium such as a CD or DVD. That. andio medium is intended to be played in the CD player ‘associated with the stereo, The contents ofthe CD ae payed asnormal CD audio, However, electronic infomation about the speakers is encoded inthe CD audio, Thus, this includes ‘specified code that ean be read by the amplifier 180, and. inicates that speaker infomnation follows. The following information includes speaker information. The main amplifier 150 includes also a processor 170 which makes adaptive decisions about which speakers will bye soleted to play each track of channel on the medium, “This adaptive decison will be based on the specific char- acteristics ofthe speakers, and the specific characteristics of the audio. The decision is basod on, of course, the specific hardware which is present in the system. More hardware, actually more speakers, in the system, will enable better 6s sound, When fewer speakers are present, racks will eed to bbe combined. In the miaimum conliguration, only wo 8 4 speakers are presen, and the standard stereo is played. Bach time a speaker is aed, i receives multiple tracks assigned tw be played toi, based on its speceal characteristics. Tis enables the user to make purchases based on their prefer- cenees, The user who likes the sound of strings, for example, ‘may purchase a speaker that is tied to strings. Whea this speaker is added to the amplifier system 150, i charater- istics are stored in memory 168, Playing of media will thereafter assign information ftom the media 100 to those speakers, base onthe speakers characterises. Conversely, other speakers for hors, and her instruments may also be purchased. Each speaker is adaptively associated with the content for those speakers. Fach extra speaker is assigned ‘with sound, and that sound is hence not played by the other speakers. Therefore, more speakers allow better reprodue~ tion ofthe sound, Different ways of getting the information nto the memory are also considered. FIG. 2 illustrates up plug and play type operation of doing this. In FIG. 2, the amplifier 250 is ‘connected vi astandacd Line connection to the speaker 260 ‘The speakee 260 includes an electronics module therein 268, ‘The module 268 communicates with a corresponding mosl- ‘ule inthe amplifier, using any serial protocol but preferably themet, USB, or RS 232. Any protocol that may commu nicate over a2 wireline may be used. In this embodiment, ‘the ampli may poll the speaker using a low voltage level signa. Since the signal is at a low voltage level, it will produce litle if any sound out ofthe speaker. Hever, the electronics module 265 within the speaker may stil recog nize this as control signals. The speaker responds with information indicative of its individual spetral charactris- tics, Ths information is then stored in the memory 165 ‘within the amplifier. The information may also be used in the playback mode, to determine channel allocations forthe information from the media. A wieless alternative is shown in FIG. 3. This may use wireless formats such as bluetooth, wireless LAN, or some other wireless fomat. FIG. 3 shows a bluctooth module 310, in the speaker 300. The amplifer 350 also includes a bluetooth module showa ab 388. Again, this system may ‘operate by polling. The speaker may respond toa poll with, information indiative of the speaker's individual charac teristics. This information is then store inthe memory 168, Inany of these embodiments the user can purchase more speakers at any time, Setings foe the music are automat cally defermined by the characteristics ofthe speaker. ‘The above-described operations may operate according 0 the flowchart of FIG. 4, which may mun inthe processor 170 AC400, che system poll all speakers. This may be eat ‘outa each time of power on ormay be carried ot only once for example on initial connection. The speakers may also include the capability of determining room acoustics, ia ‘which case it may be desirable to poll the speakers at each power up, ora time intervals, AC 40S, the system determines settings based on the polling. These settings may optionally be displayed at 410. ‘At 415, the content of the tracks is adaptively associated ‘with the user’ individual stere setup. The above has described the information stored on the ‘medium. This “enhanced” information may be stored onthe ‘media in a number of diferent ways. FIG. 8A shows the medium being» disk with a first portion that has nonnal CD stereo 500, that can be read by ‘any CD player, and reproduced through nomual stereo equipment, A second, enhanced portion of the disk 505 includes multack eahanced information, Since the firs. portion is then typical CD form, this setup will require thatUS 7,154,819 B2 5 the medium hove addtional space available. An advantage ‘ofthis system is thatthe medium can he read on any’standard CD player. FIG. 5B shows another system in which the entire ‘medium is stored in multitrack format, In this system, the anda stereo information is iterleaved with other tacks ff additional information. Standard CD format includes headers that are spevifiod by the standard, These headers include information such as P and Q parts. These headers include signals that instruct standard CD player to ignore certain pats of the data steam that is stored on the disk ‘those parts being reproduced only by enuanced players. For example, CDs may inchudeeapabiity of quad reproduction, and the enanced information tracks could be labeled as (quad, so that a standard player ignores this information, FIG, 8C shows another alteratve ia which a dual cited is has fist side 520 representing nonnal information and 4 second side 525 which is an eahanced disk While the above has described the information being present onthe disk, it should be understood that other forms of reproduction and obtaining of information are possible All such forms are intended t9 be encompassed, Other embodiments are within the disclosed invention. What is claimed is: 1. A method, comprising forming infomation indicative of reconded sound incl ing a stream of infomation including multiple pars, Which multiple paris collectively form a steam of audio; reading back said multiple parts, and adaptively deter- ‘ining an association between said mullite pars, and audio reprodetion hardware that wil play back said snipe parts 2. Amethod asin claim 1, futher comprising storing said information on a medium which stores said stream of audio. 3. Amethod asin lai 1, futher comprising transmitting said information over a channel 4.4 method as in claim 1, wherein said adaptively determining comprises determining characteristics of said 8 6 avdio reproduction hardware, and associating parts from ssid multiple pars with each item of audio reprodetion hardware, based on said characteristics 8. A method asin claim 4, whorein said characteristics include spectral characteristics, 6. A method a in claim 4, further comprising storing said charveteristes in a memory associated with said audio reproduction hardware, "TA method as in claim 2, wherein said medium stores ‘wo kinds of information, a first kind of information asso ciated with standard recorded sound, and a second kind of information including seid multiple parts 8. A method, comprising Forming a file indicative of spectral characteristics ofeach of a plurality of speakers, where at least one of the speakers hus dilleent speaker hardware, forming dif- ferent spectral characteristics than another of sid speakers: and storing said file in a controller associated with controling audio playback from said each of said plurality of speakers, 9. A method sin claim 8, wherein sid storing comprises avtomatially communicating sad file from said speaker to suid conteller, over a wire which connects said speaker to said controle, and which normally carries audio infomma- tion 10. A method as in claim 8, wherein said storing com- rises automatically communicating said file from said speaker to said controller, over a wireless chanel 1, A method as in claim 10, wherein said wicless channel i via blue tooth, 12, A method as in claim 8, farher comprising detemnin- ing spetal characterises of said speaker, and storing said spectral characteristics in said fle 13. A method as in claim 12, farther comprising deter- ‘mining room acoustics, and storing information indieaiveot said rom acoustics in said file