Mark Johnson's selected publications

• Kristina Toutanova and Mark Johnson (2007) A Bayesian LDA-based model for semi-supervised part-of-speech tagging, to appear in Proceedings of NIPS 20
• Mark Johnson (2007) Why Doesnt EM Find Good HMM POS-Taggers? Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 296-305. Note: There are (at least) three mistakes in the paper:
  • There are two mistakes in the formula for Digamma in equation (5). The correct recurrence is: Ψ(x) = Ψ(x+1) - 1/x. (Thanks to Kevin Gimpel for pointing this out).
  • The sign in front of the x^{- 4} term is incorrect in the approximation for g(x) in equation (5). The correct approximation is log(x) + 0.04167 x^{- 2} - 0.00729 x^{- 4} +0.00384 x^{- 6} - 0.00413 x^{- 8}. For more details on how this was computed, see the comments in the C implementation of the Digamma function. (Thanks to Jason Baldridge for pointing this out).
  • There's a mistake in the Gibbs sampler formula in Figure 4 on page 302. The last denominator is missing a term "+ s α_y" (Thanks to David Chiang for pointing this out).
  The code for this research was written while I was a Visiting Researcher at Microsoft Research, so unfortunately I can't release it. However, you can also download a C++ implementation of a Gibbs sampler for estimating PCFGs, a C implementation of the Digamma function and an implementation of the Inside-Outside algorithm that can optionally perform Variation Bayes.
• Mark Johnson (2007) Transforming Projective Bilexical Dependency Grammars into efficiently-parsable CFGs with Unfold-Fold , Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 168-175. After the final version of this paper was accepted for publication I learnt of very similar work by Jason Eisner and John Blatz, which predates this paper and takes a very similar perspective on this issue.
• Jianfeng Gao, Galen Andrew, Mark Johnson and Kristina Toutanova (2007) A Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 824-831.
• Mark Johnson, Thomas L. Griffiths and Sharon Goldwater (2007) Bayesian Inference for PCFGs via Markov Chain Monte Carlo, Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 139-146.
• Mark Johnson, Thomas L. Griffiths and Sharon Goldwater (2007) Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models, in B. Schoelkopf, J. Platt and T. Hoffman, eds., Advances in Neural Information Processing Systems 19, The MIT Press.
  • The published version (in the NIPS proceedings) has a typo in equation (4) that defines adaptor grammars; it's fixed in the version here. (Thanks to Julia Hockenmaier for pointing this out).
• Sharon Goldwater and Thomas L. Griffiths and Mark Johnson (2006) Contextual Dependencies in Unsupervised Word Segmentation, Proceedings of ACL/COLING 2006.
• Matthew Lease, Eugene Charniak, Mark Johnson, and David McClosky (2006) A Look At Parsing and Its Applications, Proceedings of AAAI 2006.
• Matt Lease and Mark Johnson (2006) Early Deletion of Fillers In Processing Conversational Speech, Proceedings of the North American Conference on Computational Linguistics (NAACL'06)
• David McClosky, Eugene Charniak, and Mark Johnson (2006) Effective Self-Training for Parsing, Proceedings of the North American Conference on Computational Linguistics (NAACL'06)
• Sharon Goldwater, Tom Griffiths and Mark Johnson (2005) Interpolating between types and tokens by estimating power-law generators (draft), to appear in NIPS 2005.
• Eugene Charniak and Mark Johnson (2005) Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking, Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005)
• Sharon Goldwater and Mark Johnson (2005) Representational Bias in Unsupervised Learning of Syllable Structure, Proceedings of the Workshop on Psychocomputational Models of Human Language Acquisition, ACL 2005.
• Matthew Lease, Eugene Charniak, and Mark Johnson (2005) Parsing and its Applications for Conversational Speech. 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'05).
• M. Johnson, E. Charniak and M. Lease (2004) An Improved Model for Recognizing Disfluencies in Conversational Speech Rich Transcription Fall Workshop.
• Michelle Gregory, Mark Johnson and Eugene Charniak (2004) Sentence-Internal Prosody Does not Help Parsing the Way Punctuation Does Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004
• Massimiliano Ciaramita and Mark Johnson. Multi-Component Word Sense Disambiguation. Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text (Senseval 3/ACL 2004), 97-100.
• Sharon Goldwater and Mark Johnson (2004) Priors in Bayesian Learning of Phonolgical Rules. 7th Annual Meeting of the ACL Special Interest Group on Computational Phonology (SIGPHON'04), 35-42.
• Keith Hall and Mark Johnson (2004) Attention Shifting For Parsing Speech. ACL'04, 40-46
• Mark Johnson and Eugene Charniak (2004) A Tag-Based Noisy Channel Model of Speech Repairs. ACL'04, 33-39.
• Brian Roark, Murat Saraclar, Michael Collins, and Mark Johnson (2004) Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm. ACL'04, 47-54.
• M. Johnson (2003) Learning and parsing stochastic unification-based grammars, in Schoelkopf and Warmuth, "Learning theory and Kernel Machines", Springer.
• M. Ciaramita and M. Johnson (2003) Supersense Tagging of Unknown Nouns in WordNet. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2003).
• M.Ciaramita, T. Hofmann, M.Johnson (2003) Hierarchical Semantic Classification: Word Sense Disambiguation with World Knowledge. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-03).
• Yasemin Altun, Mark Johnson, Thomas Hofmann (2003) Loss Functions and Optimization Methods for Discriminative Learning of Label Sequences, In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2003).
• Mark Johnson and Stefan Riezler (2002) Statistical models of language learning and use Cognitive Science 26, pages 239-253.
• Yasemin Altun, Thomas Hofmann and Mark Johnson (2002) Discriminative Learning for Label Sequences via Boosting, in Advances in Neural Information Processing Systems (NIPS*15), 2003.
• Keith Hall and Mark Johnson (2003) Language modeling using efficient best-first bottom-up parsing ASRU 2003
• Goldwater, S. and M. Johnson (2003) ``Learning OT Constraint Rankings Using a Maximum Entropy Model'', In Proceedings of the Stockholm Workshop on 'Variation within Optimality Theory. April 26-27, 2003 at Stockholm Univ. Sweden. Eds: Jennifer Spenader, Anders Eriksson, and Östen Dahl. pp. 111-120. (also available in gzipped postscript)
• Geman, S. and M. Johnson (2002) ``Dynamic programming for parsing and estimation of stochastic unification-based grammars'', Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
• Johnson, M. (2002) ``A simple pattern-matching algorithm for recovering empty nodes and their antecedents'', Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
• Riezler, S., T. King, R. Kaplan, R. Crouch, J. Maxwell and M. Johnson (2002) ``Parsing the Wall Street Journal using a Lexical-Functional Grammar and Discriminative Estimation Techniques'', Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
• Johnson, M (2002) ``The DOP estimation method is biased and inconsistent''. Computational Linguistics 28(1), pages 71-76. Available in Adobe Acrobat format.
• Donald Engel and Eugene Charniak and Mark Johnson (2002) ``Parsing and Disfluency Placement'', EMNLP. Available in Adobe Acrobat format.
• Don Blaheta and Mark Johnson (2001) ``Unsupervised learning of multi-word verbs.'' Proceedings of the ACL 2001 Workshop on Collocation. Available in either gzipped postscript and Adobe PDF.
• Eugene Charniak and Mark Johnson. ``Edit Detection and Parsing for Transcribed Speech.'' Proceedings of NAACL 2001. Available in either gzipped postscript and Adobe PDF.
• Mark Johnson. ``Joint and Conditional Estimation of Tagging and Parsing Models.'' Proceedings of ACL 2001. Available as gzipped postscript or Adobe PDF.
• Geman, S. and M. Johnson (2001) Probability and statistics in Computational Linguistics: A brief review. Available in Adobe Acrobat or gzipped Postscript formats.
• Geman, S. and M. Johnson (2001) Probabilistic Grammars and their Applications. Available in Adobe Acrobat or gzipped Postscript formats.
• Stefan Riezler, Detlef Prescher, Jonas Kuhn and Mark Johnson (2000) ``Lexicalized Stochastic Modeling of Constraint-Based Grammars using Log-Linear Measures and EM'', in Proceedings of the 38th Annual Meeting of the ACL, 2000.
• M. Johnson and S. Riezler (2000) ``Exploiting auxiliary distributions in stochastic unification-based grammars'', Proceedings of the 1st NAACL conference. in Adobe PDF and in gzipped postscript
• M. Johnson (2000) ``Stochastic Lexical-Functional Grammar'', slides from talk presented at the LFG 2000 conference, Berkeley. in Adobe PDF and in gzipped postscript
• Mark Johnson and Brian Roark (2000) ``Compact non-left-recursive grammars using the selective left-corner transform and factoring'', in Proceedings of the 18th International Conference on Computational Linguistics (COLING), 2000, pages 355-361.
• Massimiliano Ciaramita and Mark Johnson (2000) ``Explaining away ambiguity: Learning verb selectional preference with Bayesian networks'', in Proceedings of the 18th International Conference on Computational Linguistics (COLING), Vol.1, p.187.
• M. Johnson, S. Geman, S. Canon, Z. Chi and S. Riezler (1999) ``Estimators for Stochastic ``Unification-based'' Grammars'' in The Proceedings of the ACL 1999 in Adobe PDF and in gzipped postscript
• M. Johnson (1999) ``PCFG models of linguistic tree representations'' Computational Linguistics, available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (1999) ``Type-driven semantic interpretation and Feature dependencies in R-LFG'', in The syntax-semantics interface in LFG, M. Dalrymple, ed., available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (1999) ``A Resource-sensitive Interpretation of Lexical Functional Grammar'', JoLLI, available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (to appear) ``Optimality-theoretic Lexical Functional Grammar'' (this is a commentary on Joan Bresnan's presentation at the 1998 CUNY conference) available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (1998) ``Finite State Approximation of Constraint-based Grammars using Left-corner Grammar Transforms'' in 1998 Proceedings of COLING/ACL, available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (1996) ``Left Corner Transforms and Finite State Approximations'', manuscript (a longer but less polished version of the COLING/ACL 98 paper above) available in Gzipped Postscript format or Adobe PDF format
• E. Charniak, S. Goldwater and M. Johnson (1998) ``Edge-based Best-first Chart Parsing'', in 1998 Proceedings of the Workshop on Very Large Corpora, available in Gzipped Postscript format or Adobe PDF format
• M. Johnson (1998) ``Proof Nets and the Complexity of Processing Center-Embedded Constructions'', The Journal of Logic, Language and Information. 7(4), pages 433-447. Preprint available in Adobe PDF format.
• M. Johnson and M. Kay (1997) Copies of slides used in ESSLLI 1997 summer school course ``Topics in Parsing and Generation'' in Adobe Acrobat format or Gzipped Postscript format. A gzipped tar file of the Prolog code used in this class is also available.
• M. Johnson (1997) ``Features as Resources in R-LFG'' Proceedings of the 1997 LFG Conference, CSLI Press. in PDF,
• M. Johnson (1996) Resource-sensitivity in Lexical-Functional Grammar The Proceedings of the 1996 Roma Workshop.
• M. Johnson and S. Bayer (1995) Features and Agreement in Lambek Categorial Grammar Proceedings of the 1995 Formal Grammar Workshop.
• Mark Johnson (1995) Memoization in top-down parsing Computational Linguistics 21:3, pages 405-417
• S. Bayer and M. Johnson (1995) Features and Agreement Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics.
• Johnson, M. and J. Doerre (1995) Memoization of Coroutined Constraints Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics.
• Johnson, M. (1994) Computing with Features and Formulae. Computational Linguistics, 20.1.
• Johnson, M. (1993) The Complexity of Inducing a Rule from Data, J. Mead, ed., The Proceedings of The Eleventh West Coast Conference on Formal Linguistics, Stanford Linguistics Association, CSLI Press.
• Johnson, M. (1988). Attribute Value Logic and Theory of Grammar. CSLI Lecture Notes Series, Chicago University Press.
• Johnson, M. (1985) Parsing with Discontinuous Constituents, in the Proceedings of the 23rd Annual Meeting of the Association for Computational Linguistics
• Johnson, M. (1984) A Discovery Procedure for Certain Phonological Rules, in the Proceedings of the 10th International Conference on Computational Linguistics and 22nd Annual Meeting of the Association for Computational Linguistics.