Statistical Challenges
Fernando Pereira of the University of Pennsylvania has fitted a simple statistical Markov model to a body of newspaper text, and shown that under this model, "Furiously sleep ideas green colorless" is about 200,000 times less probable than "Colorless green ideas sleep furiously".
This statistical model defines a similarity metric, whereby sentences which are more like those within a corpus in certain respects are assigned higher values than sentences less alike. Pereira's model assigns an ungrammatical version of the same sentence a lower probability than the syntactically correct form demonstrating that statistical models can learn grammaticality distinctions with minimal linguistic assumptions. However, it is not clear that the model assigns every ungrammatical sentence a lower probability than every grammatical sentence. That is, "colorless green ideas sleep furiously" may still be statistically more "remote" from English than some ungrammatical sentences. To this, it may be argued that no current theory of grammar is capable of distinguishing all grammatical English sentences from ungrammatical ones.
Read more about this topic: Colorless Green Ideas Sleep Furiously
Famous quotes containing the word challenges:
“The approval of the public is to be avoided like the plague. It is absolutely essential to keep the public from entering if one wishes to avoid confusion. I must add that the public must be kept panting in expectation at the gate by a system of challenges and provocations.”
—André Breton (1896–1966)