Jump to content

Frederick Jelinek

From Wikipedia, the free encyclopedia

Frederick Jelinek
Born
Bedřich Jelínek

(1932-11-18)November 18, 1932
Kladno, Czechoslovakia (now Czech Republic)
DiedSeptember 14, 2010(2010-09-14) (aged 77)
Baltimore, Maryland, U.S.
CitizenshipAmerican
Alma materMassachusetts Institute of Technology
Known forAdvancement of natural language processing techniques
SpouseMilena Jelinek
Scientific career
FieldsInformation theory, natural language processing
WorkplacesCornell University, IBM Research, Johns Hopkins University
Robert Fano
Notable students
Neil Sloane

Frederick Jelinek (November 18, 1932 – September 14, 2010) was a Czech-American researcher in information theory, automatic speech recognition, and natural language processing. He was a leading figure in the development of statistical approaches to speech and language processing. At IBM Research, where he led the Continuous Speech Recognition Group, Jelinek applied information-theoretic and probabilistic methods to speech recognition and later statistical machine translation.[1]

Jelinek was the director of the Center for Language and Speech Processing at Johns Hopkins University from 1993 till 2010. He was elected to the National Academy of Engineering in 2006 for his contributions to statistical language processing and automatic speech recognition, and received the Association for Computational Linguistics Lifetime Achievement Award in 2009.[2][3]

Early life and education

[edit]

It is generally believed that scientific talent reveals itself in early youth. ... This was certainly not my case. I somehow slid into my scientific profession. My mother wished for me to become a physician, just like my father. ... I myself wanted to be a lawyer, defender of the unjustly accused. But my career is the result of political circumstances, academic possibilities, and lucky accidents.

— Talking about his life in a 2001 speech.[4]

Jelinek was born Bedřich Jelínek on November 18, 1932, in Kladno to Vilém and Trude Jelínek.[5] His father was a Jewish dentist. His mother, was born in Switzerland to Czech Catholic parents and later converted to Judaism.[6][7] Following the German occupation of Czechoslovakia, Vilém planned to emigrate but ultimately remained in the country. He was later deported to the Theresienstadt concentration camp, where he died in 1945.[6][7] The family was forced to move to Prague, while Jelinek, his mother and sister avoided deportation.[8][7]

After the war, Jelinek attended a gymnasium, having missed several years of schooling because of anti-Jewish restrictions during the occupation.[7] After the communist takeover in 1948, his mother decided to emigrate when Jelinek was prevented from taking the high-school graduation examination.[7] The family settled in New York, where he studied engineering in evening classes at the City College of New York.

He later received a stipend from the National Committee for a Free Europe that enabled him to attend the Massachusetts Institute of Technology, where he studied information theory under Robert Fano.[7][8] He obtained his Ph.D. in 1962.[9]

Career

[edit]

After receiving his doctorate in 1962, Jelinek joined the faculty of Cornell University, where he taught and conducted research in information theory. He had hoped to collaborate with linguist Charles F. Hockett on applying information theory to language, but the proposed research did not materialize. Jelinek nevertheless helped establish the prominence of Cornell's information-theory program.[10]

In 1972, Jelinek joined IBM Research at the Thomas J. Watson Research Center, initially while on leave from Cornell; he left Cornell permanently in 1974.[7] He became head of IBM's Continuous Speech Recognition Group and remained at IBM until 1993.[11] The group applied information-theoretic and statistical methods to speech recognition and later to machine translation.[11][7]

Following the fall of communism in Czechoslovakia in 1989, Jelinek renewed ties with the Czech academic community. He invited Czech researchers to work with his IBM group and taught at Charles University and the Czech Technical University in Prague.[7] He continued these collaborations after moving to Johns Hopkins, and in 2001 spent a sabbatical at the Institute of Formal and Applied Linguistics at Charles University, where he also received an honorary doctorate.[7]

In 1993, Jelinek retired from IBM and joined Johns Hopkins University as director of the Center for Language and Speech Processing and Julian S. Smith Professor of Electrical Engineering.[12] Soon after joining Johns Hopkins, he began organizing annual summer research workshops in speech and language processing, which brought together researchers and students from academia, government, and industry.[10][12]

In 2010, Tomáš Mikolov spent a research internship at Johns Hopkins University in Jelinek's group, where he worked on recurrent neural-network language models.[13]

Research and legacy

[edit]

In the 1950s, information theory attracted growing interest as a way to study language and communication, although Claude Shannon warned against applying it too broadly. During the 1960s, probabilistic approaches to language became less influential as formal linguistics, associated with Noam Chomsky, gained prominence. Research in machine translation was also curtailed after the 1966 ALPAC report led to reduced U.S. government funding.[14]

When Jelinek began working on speech recognition at IBM in 1972, statistical and information-theoretic approaches to speech and language processing were no longer widely pursued. His work helped re-establish these approaches as an important direction.[14][15]

At IBM, Jelinek led an effort to develop general-purpose speech recognition for dictation, a broader goal than the limited tasks pursued by many contemporary U.S. research groups.[14] His group treated speech recognition as a probabilistic decoding problem: given an acoustic signal, a system would identify the most likely sequence of words using statistical models of both the speech signal and the language.[15][16] The models were estimated from speech data rather than relying primarily on manually specified linguistic rules, providing the framework for a series of statistical speech-recognition systems developed by the IBM group during the 1970s and 1980s.[16][15]

He was not a pioneer of speech recognition, he was the pioneer of speech recognition.

— Steve Young (2010)[16]

The group's early experiments used the New Raleigh grammar, a finite-state hidden Markov model that generated artificial English sentences. During this work, IBM researchers introduced the terms acoustic model and language model, as well as perplexity as an information-theoretic measure of the difficulty of a recognition task.[15] The group's early statistical approach was described in several papers in 1975 and 1976, including Jelinek's Continuous Speech Recognition by Statistical Methods in Proceedings of the IEEE.[15][17]

Around 1978, the group began developing the Tangora system for more realistic dictation tasks. It used a 5,000-word vocabulary, statistical language models based on trigrams, and hidden Markov models for acoustic modeling.[15] The group subsequently developed methods for smoothing language-model probabilities, modeling phones according to their phonetic context, and clustering acoustic parameters, techniques that became important components of statistical speech recognition.[15][16] These statistical methods helped lay the foundations for later advances in speech recognition and language technology.[12]

Beyond its technical work, Jelinek's group also influenced the methodology used to evaluate speech and language systems. At IBM, competing algorithms were compared on fixed training and test sets using predefined quantitative measures, and Jelinek advocated sharing datasets, evaluation methods, and research results outside the company.[14] When DARPA established a new speech-recognition program in 1986, it adopted a similar common-task approach, in which participating research groups worked on shared tasks and data and were evaluated using common criteria.[14] IBM subsequently donated its aligned English–French Canadian parliamentary corpus to the Association for Computational Linguistics' Data Collection Initiative in 1989, and Jelinek's advocacy of shared language resources contributed to efforts that led to the establishment of the Linguistic Data Consortium in 1992.[14][15][18]

The IBM group also extended its probabilistic approach beyond speech recognition. In 1987, it began research on statistical machine translation, developing models for translation from French into English using parallel text from the proceedings of the Parliament of Canada.[15] The work resulted in the 1990 paper A Statistical Approach to Machine Translation, co-authored by Jelinek and seven IBM colleagues, and helped establish statistical methods as a major approach to machine translation.[19][14]

Linguist quotation

[edit]

Jelinek became widely associated with a remark often paraphrased as "Every time I fire a linguist, the performance of the speech recognizer goes up." The exact wording and date vary among sources.[20][21]

According to Steve Young, "the story goes that one day one of his linguists resigned, and Fred decided to replace him not by another linguist but by an engineer. A little while later, Fred noticed that the performance of his system improved significantly. So he encouraged another linguist to find alternative employment, and sure enough performance improved again."[16]

In a 2004 invited talk, later published as Some of My Best Friends are Linguists, Jelinek said that he had long hoped the remark was apocryphal, but accepted that he had made a similar statement. He emphasized that his IBM group had not been hostile to linguistics; rather, the researchers had struggled to incorporate linguistic knowledge effectively into their statistical systems.[22]

Honors and awards

[edit]

Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM.[16][23] He received the Society Award for "outstanding technical contributions and leadership" from the IEEE Signal Processing Society for 1997,[24] and the ESCA Medal for Scientific Achievement in 1999.[25] He was a recipient of an IEEE Third Millennium Medal in 2000, the European Language Resources Association's first Antonio Zampolli Prize in 2004,[26] the 2005 James L. Flanagan Speech and Audio Processing Award,[27] and the 2009 Lifetime Achievement Award from the Association for Computational Linguistics.[23][15] He received an honoris causa Ph.D. from Charles University in 2001,[28] was elected to the National Academy of Engineering in 2006 and was made one of twelve inaugural fellows of the International Speech Communication Association in 2008.[16]

Personal life

[edit]

In 1957, while visiting Prague, Jelinek met film student Milena Tobolová through his friend Miloš Forman. Tobolová had written the screenplay on which the film Easy Life (Snadný život) was based.[29][30] Considered a dissident by the Czechoslovak authorities, she was initially prevented from emigrating. Jelinek sought assistance from Jerome Wiesner and Cyrus Eaton, with Eaton lobbying Nikita Khrushchev on her behalf.[31] Tobolová was permitted to leave Czechoslovakia in January 1961, and she and Jelinek married later that year.[31][32] They had a daughter and a son.[32]

Jelinek died of a heart attack on September 14, 2010, aged 77.[33] He was survived by his wife, two children, and three grandchildren.[34]

Selected publications

[edit]
  • Jelinek, Frederick (1968). Probabilistic Information Theory: Discrete and memoryless models. McGraw-Hill series in systems science. New York: McGraw-Hill. 689p. LCCN 68-11611 (review)
  • ———————- (1969). "Fast sequential decoding algorithm using a stack". IBM Journal of Research and Development 13(6):675–685. doi:10.1147/rd.136.0675.
  • ———————- (1969). "Tree encoding of memoryless time-discrete sources with a fidelity criterion". IEEE Transactions on Information Theory 15(5):584–590. doi:10.1109/TIT.1969.1054355. (received 1971 "Best Paper" award)
  • Bahl, Lalit R.; John Cocke, Frederick Jelinek, Josef Raviv (1974). "Optimal decoding of linear codes for minimizing symbol error rate". IEEE Transactions on Information Theory 20(2):284–287. doi:10.1109/TIT.1974.1055186. (received Information Theory Society Golden Jubilee paper award)
  • ———————- (1976). "Continuous speech recognition by statistical methods". Proceedings of the IEEE 64(4):532–556. doi:10.1109/PROC.1976.10159.
  • Brown, P.; J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R, Mercer and P. Roossin (1988). "A statistical approach to language translation" Archived August 7, 2011, at the Wayback Machine. In Dénes Vargha, ed. Coling 88: Proceedings of the 12th conference on Computational linguistics, volume 1. Budapest: John Von Neumann society for computing sciences. pp. 71–76. doi:10.3115/991635.991651. ISBN 963-8431-56-3.
  • ———————- (1990). "Self-Organized Language Modeling for Speech Recognition". In Alex Waibel & Kai-Fu Lee, eds. Readings in speech recognition. San Mateo: Morgan Kaufmann. 629p. ISBN 1-55860-124-4.
  • ———————-; John D. Lafferty and Robert L. Mercer. (1990) "Basic methods of probabilistic context free grammars". Technical Report RC 16374 (72684), IBM.
    • Reprinted in Laface, Pietro; Renato De Mori (1992). Speech Recognition and Understanding: Recent advances, trends, and applications. NATO ASI series. Series F, Computer and systems sciences, 75. New York: Springer-Verlag. pp. 345–360. ISBN 0-387-54032-6.
  • ———————- (1997). Statistical Methods for Speech Recognition. Cambridge, Mass.: MIT Press. 283p. ISBN 0-262-10066-5. (review) (review 2)
  • Chelba, Ciprian; Frederick Jelinek (2000). "Structured Language Modeling". Computer Speech & Language 14(4):283–332. doi:10.1006/csla.2000.0147 (received 2002 "Best Paper" award).
    • Expanded version of a presentation at NLDB'99. Klagenfurt, Austria, June 17–19, 1999 (arXiv:cs/0001023).
  • Xu, Peng; Ahmad Emami and Frederick Jelinek (2003). "Training Connectionist Models for the Structured Language Model". In Michael Collins and Mark Steedman, eds. EMNLP '03 Proceedings of the 2003 conference on Empirical methods in natural language processing. East Stroudsburg, Penn.: Association for Computational Linguistics. pp. 160–167. ISBN 1-932432-13-2. doi:10.3115/1119355.1119376. (won "best paper" award)

References

[edit]
Notes
References
  1. ↑ Liberman, Mark (2010). "Obituary: Fred Jelinek". Computational Linguistics. 36 (4): 595–599. doi:10.1162/coli_a_00032.
  2. ↑ "Frederick Jelinek, 77, pioneer in speech and text understanding technology". The Gazette. Johns Hopkins University. September 20, 2010. Retrieved August 17, 2026.
  3. ↑ Jelinek, Frederick (2009). "ACL Lifetime Achievement Award: The Dawn of Statistical ASR and MT". Computational Linguistics. 35 (4): 483–494. doi:10.1162/coli.2009.35.4.35401.
  4. ↑ Jelinek, Frederick (November 22, 2001). How I Got Here (Speech). Prague. Archived from the original on March 16, 2008. Retrieved December 17, 2010. Honoris causa degree acceptance speech.
  5. ↑ "V USA zemřel vědec českého původu, jenž naučil stroje mluvit". Lidové noviny (in Czech). ČTK. September 25, 2010. Retrieved August 17, 2026.
  6. 1 2 Juang, Biing-Hwang; Rabiner, Lawrence (2011). "Frederick Jelinek 1932–2010". Memorial Tributes: Volume 15. National Academies Press. pp. 214–217. doi:10.17226/13160.
  7. 1 2 3 4 5 6 7 8 9 10 Hajic, Jan (November 2010). "Prof. Frederick Jelinek, 1932–2010". EACL Newsletter. Vol. 12. Retrieved December 19, 2010.
  8. 1 2 Juang, Biing-Hwang; Rabiner, Lawrence (2011). "Frederick Jelinek 1932–2010". Memorial Tributes: Volume 15. National Academies Press. pp. 214–217. doi:10.17226/13160.
  9. ↑ "Frederick Jelinek, 77, pioneer in speech and text understanding technology". The Gazette. Johns Hopkins University. September 20, 2010. Retrieved August 17, 2026.
  10. 1 2 Juang, Biing-Hwang; Rabiner, Lawrence (2011). "Frederick Jelinek 1932–2010". Memorial Tributes: Volume 15. National Academies Press. pp. 214–217. doi:10.17226/13160.
  11. 1 2 Liberman, Mark (2010). "Obituary: Fred Jelinek". Computational Linguistics. 36 (4): 595–599. doi:10.1162/coli_a_00032.
  12. 1 2 3 Sneiderman, Phil (September 20, 2010). "Frederick Jelinek, 77, pioneer in speech and text understanding technology". The JHU Gazette. Johns Hopkins University. Retrieved December 16, 2010.
  13. ↑ Rijcken, Emil (January 31, 2023). "Uncovering the Pioneering Journey of Word2Vec and the State of AI science – an in-depth interview with Dr. Tomas Mikolov". BNVKI. Retrieved August 17, 2026.
  14. 1 2 3 4 5 6 7 Liberman, Mark (December 2010). "Obituary: Fred Jelinek". Computational Linguistics. 36 (4): 595–599. doi:10.1162/coli_a_00032.
  15. 1 2 3 4 5 6 7 8 9 10 Jelinek, Fred (December 2009). "The Dawn of Statistical ASR and MT". Computational Linguistics. 35 (4): 483–494. doi:10.1162/coli.2009.35.4.35401. S2CID 1486422.
  16. 1 2 3 4 5 6 7 Young, Steve (November 2010). "Frederick Jelinek 1932–2010: The Pioneer of Speech Recognition Technology". Speech and Language Processing Technical Committee Newsletter. IEEE Signal Processing Society. Archived from the original on July 28, 2011. Retrieved December 16, 2010. Adapted from a speech given in 2006.
  17. ↑ Jelinek, Frederick (1976). "Continuous Speech Recognition by Statistical Methods". Proceedings of the IEEE. 64 (4): 532–556. doi:10.1109/PROC.1976.10159.
  18. ↑ Cieri, Christopher; Liberman, Mark; Cho, Sunghye; Strassel, Stephanie; Fiumara, James; Wright, Jonathan (2022). "Reflections on 30 Years of Language Resource Development and Sharing". Proceedings of the Thirteenth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association. pp. 543–550.
  19. ↑ Brown, Peter F.; Cocke, John; Della Pietra, Stephen A.; Della Pietra, Vincent J.; Jelinek, Frederick; Lafferty, John D.; Mercer, Robert L.; Roossin, Paul S. (1990). "A Statistical Approach to Machine Translation". Computational Linguistics. 16 (2): 79–85.
  20. ↑ Jurafsky, Daniel; James H. Martin (2009). Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. Prentice Hall series in artificial intelligence (2nd ed.). Upper Saddle, New Jersey: Prentice Hall. p. 83. ISBN 978-0-13-187321-6.
  21. ↑ Moore, Roger K. (2005). Results from a Survey of Attendees at ASRU 1997 and 2003 (PDF). INTERSPEECH-2005. Lisbon, September 4–8, 2005. Archived from the original (PDF) on July 20, 2011.{{cite conference}}: CS1 maint: location (link)
  22. ↑ Jelinek, Frederick (2005). "Some of my Best Friends are Linguists". Language Resources and Evaluation. 39 (1): 25–34. doi:10.1007/s10579-005-2693-4.
  23. 1 2 Jan, Jelinek (June 13, 2006). "Curriculum Vitae". Archived from the original on September 3, 2006. Retrieved December 17, 2010.
  24. ↑ "Society Award". IEEE Signal Processing Society. Retrieved December 21, 2010.
  25. ↑ "1999 ESCA Medal for Scientific Achievement". International Speech Communication Association. 1999. Archived from the original on August 2, 2009. Retrieved December 21, 2010.
  26. ↑ "In Honour of Prof. Antonio Zampolli". European Language Resources Association. Archived from the original on July 21, 2011. Retrieved December 21, 2010.
  27. ↑ "IEEE James L. Flanagan Speech and Audio Processing Award recipients". IEEE. Retrieved December 21, 2010.{{cite web}}: CS1 maint: deprecated archival service (link)
  28. ↑ "Dr. h. c. prof. F. Jelinek" (Press release). Charles University in Prague. November 22, 2001. Retrieved December 17, 2010.
  29. ↑ hershenson, Roberta (December 31, 1989). "Czech Couple Keep Eye on Homeland". The New York Times. Retrieved August 17, 2026.
  30. ↑ Willoughby, Ian (June 9, 2008). "Milena Jelinek – member of golden generation of Czech filmmakers now teaching screenwriting at Columbia". One on One. Radio Prague. Retrieved February 1, 2014.
  31. 1 2 Hershenson, Roberta (December 31, 1989). "Czech Couple Keep Eye on Homeland". New York Times. Retrieved December 17, 2010.
  32. 1 2 Lohr, Steve (September 24, 2010). "Frederick Jelinek, Who Gave Machines the Key to Human Speech, Dies at 77". New York Times. p. B10. Retrieved December 16, 2010.
  33. ↑ Dresser, Michael (September 19, 2010). "Frederick Jelinek, speech recognition pioneer, dies". The Baltimore Sun. Retrieved December 16, 2010.
  34. ↑ Lohr, Steve (September 24, 2010). "Frederick Jelinek, Who Gave Machines the Key to Human Speech, Dies at 77". The New York Times. p. B10. Retrieved August 17, 2026.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Frederick_Jelinek&oldid=1369900403"