Publications, Mark Johnson, Cognitive and Linguistic Sciences, Brown
University
To remove any frames surrounding this page,
click here
(Last updated 23rd June, 2009)
- Sharon Goldwater, Tom Griffiths and Mark Johnson (2009)
A Bayesian Framework for Word Segmentation: Exploring the Effects of Context,
Cognition 112:1, pp. 21-54. You can download a preprint
here.
- William P. Headden III, Mark Johnson and David McClosky (2009)
Improving Unsupervised Dependency Parsing with Richer Contexts and Smoothing
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics,
pp. 101-109.
(bib)
-
Micha Elsner, Eugene Charniak and Mark Johnson (2009)
Structured Generative Models for Unsupervised Named-Entity Clustering
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics,
pp. 164-172.
(bib)
- Mark Johnson and Sharon Goldwater (2009)
Improving nonparameteric Bayesian inference: experiments on
unsupervised word segmentation with adaptor grammars,
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics,
pp. 317-325.
(bib)
- Jianfeng Gao and Mark Johnson (2008)
A comparison of Bayesian estimators for unsupervised Hidden Markov Model POS taggers,
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 344-352.
(bib)
- David McClosky, Eugene Charniak, and Mark Johnson (2008)
When is Self-Training Effective for Parsing?
Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008),
pp. 561-568.
(bib)
- Mark Johnson (2008)
Using Adaptor Grammars to Identify Synergies in the Unsupervised Acquisition of Linguistic Structure,
Proceedings of the 46th
Annual Meeting
of the Association for
Computational Linguistics:
Human Language
Technologies.
pp. 398-406.
(bib)
- Mark Johnson (2008)
Unsupervised Word Segmentation for Sesotho Using Adaptor Grammars,
Proceedings of the Tenth Meeting of ACL Special Interest Group on Computational Morphology and Phonology.
pp. 20-27.
(bib)
- Kristina Toutanova and Mark Johnson (2007)
A Bayesian LDA-based model for semi-supervised part-of-speech tagging,
to appear in
Proceedings of NIPS 20
(bib)
- Noah Smith and Mark Johnson (2007)
Weighted and Probabilistic Context-Free Grammars Are Equally Expressive
Computational Linguistics 33:4, pages 477-491.
- Mark Johnson (2007)
Why Doesnt EM Find Good HMM POS-Taggers?
Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL),
pages 296-305. Note: There are (at least) three mistakes in the paper:
- There are two mistakes in the formula for Digamma in equation (5).
The correct recurrence is: Ψ(x) = Ψ(x+1) - 1/x.
(Thanks to Kevin Gimpel for pointing this out).
- The sign in front of the x- 4 term is incorrect in the approximation for g(x) in equation (5).
The correct approximation is log(x) + 0.04167 x- 2 - 0.00729 x- 4 +0.00384 x- 6 - 0.00413 x- 8.
For more details on how this was computed,
see the comments in the C implementation of the Digamma function. (Thanks to Jason Baldridge for pointing this out).
- There's a mistake in the Gibbs sampler formula in Figure 4 on page 302.
The last denominator is missing a term "+ s αy" (Thanks to David Chiang for
pointing this out).
The code for this research was written while I was a Visiting Researcher
at Microsoft Research, so unfortunately I can't release it.
However, you can also download
a C++ implementation of a Gibbs sampler
for estimating PCFGs,
a C implementation of
the Digamma function and
an implementation of the Inside-Outside algorithm that can optionally
perform Variation Bayes.
- Mark Johnson (2007)
Transforming Projective Bilexical Dependency Grammars into efficiently-parsable CFGs with Unfold-Fold ,
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics,
pages 168-175.
After the final version of this paper was accepted for publication I learnt of very similar work by
Jason Eisner and John Blatz, which predates this paper and takes a very similar perspective
on this issue.
- Jianfeng Gao, Galen Andrew, Mark Johnson and Kristina Toutanova (2007)
A Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics,
pages 824-831.
- Mark Johnson, Thomas L. Griffiths and Sharon Goldwater (2007)
Bayesian Inference for PCFGs via Markov Chain Monte Carlo,
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 139-146.
- Mark Johnson, Thomas L. Griffiths and Sharon Goldwater (2007)
Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models, in
B. Schoelkopf, J. Platt and T. Hoffman, eds.,
Advances in Neural Information Processing Systems 19,
The MIT Press.
- The published version (in the NIPS proceedings) has
a typo in equation (4) that defines adaptor grammars; it's fixed in the version here.
(Thanks to Julia Hockenmaier for pointing this out).
- Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson (2007)
Distributional Cues toWord Boundaries: Context is Important,
Proceedings of the 31st Boston University Conference on Language Development.
- Sharon Goldwater and Thomas L. Griffiths and Mark Johnson (2006)
Contextual Dependencies in Unsupervised Word Segmentation, Proceedings of ACL/COLING 2006.
- Matthew Lease, Eugene Charniak, Mark Johnson, and David McClosky (2006)
A Look At Parsing and Its Applications,
Proceedings of AAAI 2006.
- Matt Lease and Mark Johnson (2006)
Early Deletion of Fillers In Processing Conversational Speech, Proceedings of the North American Conference on Computational Linguistics (NAACL'06)
- David McClosky, Eugene Charniak, and Mark Johnson (2006)
Effective Self-Training for Parsing, Proceedings of the North American Conference on Computational Linguistics (NAACL'06)
- Sharon Goldwater, Tom Griffiths and Mark Johnson (2005)
Interpolating between types and tokens by estimating
power-law generators (draft), to appear in NIPS 2005.
- Eugene Charniak and Mark Johnson (2005)
Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking,
Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005)
- Sharon Goldwater and Mark Johnson (2005)
Representational Bias in Unsupervised Learning of Syllable Structure,
Proceedings of the Workshop on Psychocomputational Models of Human Language Acquisition, ACL 2005.
- Matthew Lease, Eugene Charniak, and Mark Johnson (2005)
Parsing and its Applications for Conversational Speech.
2005 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'05).
- M. Johnson, E. Charniak and M. Lease (2004)
An Improved Model for Recognizing Disfluencies in Conversational Speech
Rich Transcription Fall Workshop.
- Michelle Gregory, Mark Johnson and Eugene Charniak (2004)
Sentence-Internal Prosody Does not Help Parsing the Way Punctuation Does
Proceedings of the Human Language Technology Conference of the
North American Chapter of the Association for Computational Linguistics:
HLT-NAACL 2004
- Massimiliano Ciaramita and Mark Johnson.
Multi-Component Word Sense Disambiguation.
Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text (Senseval 3/ACL 2004), 97-100.
- Sharon Goldwater and Mark Johnson (2004)
Priors in Bayesian Learning of Phonolgical Rules.
7th Annual Meeting of the ACL Special Interest Group on Computational Phonology (SIGPHON'04), 35-42.
- Keith Hall and Mark Johnson (2004)
Attention Shifting For Parsing Speech. ACL'04, 40-46
- Mark Johnson and Eugene Charniak (2004)
A Tag-Based Noisy Channel Model of Speech Repairs. ACL'04, 33-39.
- Brian Roark, Murat Saraclar, Michael Collins, and Mark Johnson (2004)
Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm. ACL'04, 47-54.
- M. Johnson (2003)
Learning and parsing stochastic unification-based grammars,
in Schoelkopf and Warmuth, "Learning theory and Kernel Machines",
Springer.
- M. Ciaramita and M. Johnson (2003)
Supersense Tagging of Unknown Nouns in WordNet.
In Proceedings of the Conference on Empirical Methods in
Natural Language Processing (EMNLP 2003).
- M.Ciaramita, T. Hofmann, M.Johnson (2003)
Hierarchical Semantic
Classification: Word Sense Disambiguation with World Knowledge. In
Proceedings of the 18th International Joint Conference on Artificial
Intelligence (IJCAI-03).
- Yasemin Altun, Mark Johnson, Thomas Hofmann (2003)
Loss Functions and Optimization Methods for
Discriminative Learning of Label Sequences,
In Proceedings of the Conference on Empirical Methods in
Natural Language Processing (EMNLP 2003).
- Mark Johnson and Stefan Riezler (2002)
Statistical models of language learning and use
Cognitive Science 26, pages 239-253.
- Yasemin Altun, Thomas Hofmann and Mark Johnson (2002)
Discriminative Learning for Label
Sequences via Boosting, in Advances in Neural Information Processing Systems
(NIPS*15), 2003.
- Keith Hall and Mark Johnson (2003)
Language modeling using efficient best-first bottom-up parsing
ASRU 2003
- Goldwater, S. and M. Johnson (2003)
``Learning
OT Constraint Rankings Using a Maximum Entropy Model'',
In Proceedings of the Stockholm Workshop on
'Variation within Optimality Theory. April 26-27, 2003 at Stockholm Univ.
Sweden. Eds: Jennifer Spenader, Anders Eriksson, and Östen Dahl. pp. 111-120.
(also available in gzipped postscript)
- Geman, S. and M. Johnson (2002) ``Dynamic programming
for parsing and estimation of stochastic unification-based grammars'',
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
- Johnson, M. (2002) ``A simple pattern-matching
algorithm for recovering empty nodes and their antecedents'',
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
- Riezler, S., T. King, R. Kaplan, R. Crouch, J. Maxwell and M. Johnson (2002)
``Parsing the Wall Street Journal using a Lexical-Functional
Grammar and Discriminative Estimation Techniques'',
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.
- Johnson, M (2002) ``The DOP estimation method is biased and inconsistent''.
Computational Linguistics 28(1), pages 71-76.
Available in Adobe Acrobat format.
- Donald Engel and Eugene Charniak and Mark Johnson (2002) ``Parsing and Disfluency Placement'',
EMNLP. Available in Adobe Acrobat format.
- Don Blaheta and Mark Johnson (2001) ``Unsupervised learning of multi-word verbs.'' Proceedings of the ACL 2001 Workshop on Collocation. Available in either gzipped postscript and Adobe PDF.
- Eugene Charniak and Mark Johnson. ``Edit Detection and Parsing for Transcribed Speech.'' Proceedings of NAACL 2001. Available in either gzipped postscript and Adobe PDF.
- Mark Johnson. ``Joint and Conditional Estimation of Tagging and Parsing Models.'' Proceedings of ACL 2001. Available as gzipped postscript or Adobe PDF.
- Geman, S. and M. Johnson (2001) Probability and statistics in Computational Linguistics: A brief review.
Available in Adobe Acrobat or
gzipped Postscript formats.
- Geman, S. and M. Johnson (2001) Probabilistic Grammars and their Applications.
Available in Adobe Acrobat or
gzipped Postscript formats.
- Stefan Riezler, Detlef Prescher, Jonas Kuhn and Mark Johnson (2000)
``Lexicalized Stochastic Modeling of Constraint-Based Grammars using Log-Linear Measures and EM'',
in Proceedings of the 38th Annual Meeting of the ACL, 2000.
- M. Johnson and S. Riezler (2000) ``Exploiting auxiliary distributions
in stochastic unification-based grammars'',
Proceedings of the 1st NAACL conference.
in Adobe PDF and in
gzipped postscript
- M. Johnson (2000) ``Stochastic Lexical-Functional Grammar'',
slides from talk presented at the LFG 2000 conference, Berkeley.
in Adobe PDF and in
gzipped postscript
- Mark Johnson and Brian Roark (2000)
``Compact non-left-recursive grammars using the selective left-corner transform and factoring'', in
Proceedings of the 18th International Conference on Computational Linguistics (COLING), 2000, pages 355-361.
- Massimiliano Ciaramita and Mark Johnson (2000)
``Explaining away ambiguity: Learning verb selectional preference with Bayesian networks'', in
Proceedings of the 18th International Conference on Computational Linguistics (COLING), Vol.1, p.187.
- M. Johnson, S. Geman, S. Canon, Z. Chi and S. Riezler (1999)
``Estimators for Stochastic ``Unification-based'' Grammars''
in The Proceedings of the ACL 1999
in Adobe PDF and in
gzipped postscript
- M. Johnson (1999) ``PCFG models of linguistic tree representations''
Computational Linguistics, available in
Gzipped Postscript format or
Adobe PDF format
- M. Johnson (1999) ``Type-driven semantic interpretation and
Feature dependencies in R-LFG'', in The syntax-semantics interface
in LFG, M. Dalrymple, ed.,
available in Gzipped Postscript format or
Adobe PDF format
- M. Johnson (1999) ``A Resource-sensitive Interpretation of Lexical Functional Grammar'', JoLLI,
available in Gzipped Postscript format or
Adobe PDF format
- M. Johnson (to appear) ``Optimality-theoretic Lexical Functional Grammar''
(this is a commentary on Joan Bresnan's presentation at the 1998 CUNY conference)
available in Gzipped Postscript format or
Adobe PDF format
- M. Johnson (1998) ``Finite State Approximation of Constraint-based
Grammars using Left-corner Grammar Transforms''
in 1998 Proceedings of COLING/ACL,
available in Gzipped Postscript format or
Adobe PDF format
- M. Johnson (1996) ``Left Corner Transforms and Finite State Approximations'',
manuscript (a longer but less polished version of the COLING/ACL 98 paper
above) available in
Gzipped Postscript format or
Adobe PDF format
- E. Charniak, S. Goldwater and M. Johnson (1998)
``Edge-based Best-first Chart Parsing'', in
1998 Proceedings of the Workshop on Very Large Corpora,
available in Gzipped Postscript format or
Adobe PDF format
- M. Johnson (1998) ``Proof Nets and the Complexity of Processing Center-Embedded Constructions'',
The Journal of Logic, Language and Information. 7(4),
pages 433-447.
Preprint available in Adobe PDF format.
- M. Johnson and M. Kay (1997)
Copies of slides used in ESSLLI 1997 summer school course
``Topics in Parsing and Generation''
in Adobe Acrobat format
or
Gzipped Postscript format.
A gzipped
tar file of the Prolog code used in this class is also available.
- M. Johnson (1997) ``Features as Resources in R-LFG''
Proceedings of the 1997 LFG Conference, CSLI Press.
in PDF,
- M. Johnson (1996)
Resource-sensitivity in Lexical-Functional Grammar
The Proceedings of the 1996 Roma Workshop.
- M. Johnson and S. Bayer (1995)
Features and Agreement in Lambek Categorial Grammar
Proceedings of the 1995 Formal Grammar Workshop, pages 123-137.
- Mark Johnson (1995)
Memoization in top-down parsing
Computational Linguistics 21:3, pages 405-417
- S. Bayer and M. Johnson (1995)
Features and Agreement
Proceedings of the 33rd Annual Meeting of the Association for
Computational Linguistics.
- Johnson, M. and J. Doerre (1995)
Memoization of Coroutined Constraints
Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics.
- Johnson, M. (1994) Computing with Features and Formulae.
Computational Linguistics, 20.1.
- Johnson, M. (1993)
The Complexity of Inducing a Rule from Data,
J. Mead, ed.,
The Proceedings of The Eleventh West Coast Conference on Formal Linguistics,
Stanford Linguistics Association, CSLI Press.
- Johnson, M. (1988). Attribute Value Logic and Theory of Grammar.
CSLI Lecture Notes Series, Chicago University Press.
- Johnson, M. (1985) Parsing with Discontinuous Constituents, in the Proceedings of the 23rd Annual Meeting of the Association for Computational Linguistics
- Johnson, M. (1984) A Discovery Procedure for Certain Phonological Rules,
in the Proceedings of the 10th International Conference on Computational Linguistics and 22nd Annual Meeting of the Association for Computational Linguistics.