The question https://scicomp.stackexchange.com/questions/1458/markov-chain-n-gram-make-more-natural-sounding has been flagged as out of scope. I am not an expert in computer language generation (natural language processing? I'm not sure what the field is...), so I don't have a good grasp of whether or not it's in scope. Could people who know about the topic weigh in?


My close vote wasn't on the basis of topicality but for vagueness. I'd say that the topicality of the question depends in part on what the OP want to achieve: research on spam filtering is probably on-topic, but generating a cognitive dissonance moment in the reader is perhaps less so.

It's a big topic, with several well known almost-but-not-quite trivial steps that will get the OP to "you can read it aloud without much difficulty", but after that things get complicated.

As long as we're writing about it I will elaborate on my objection.

Things I found to be unspecified include.

  1. What it the purpose for which it should "sound natural"? Generating text that can be read aloud is different from generating text which might fool a spam filter.
  2. What metric will be used to measure the "naturalness" of the text? Just opinion? (And yes, this is intimately related to (1).
  3. Does the OP have an appropriate corpus to use as the basis? Garbage in, garbage out, right?
  4. Has the OP tried simple things like using a 3- or 4- or more-chain and making punctuation part of the chain? (These steps are suggested in most (almost every?) introduction to the topic, usually with a discussion of how the complexity scales.)

We need to be given a little help here

  • $\begingroup$ Excellent points. Like I said, I really know nothing about the topic, which is why when there were flags and close votes, I was scratching my head, unsure of what would be a good course of action. This answer is very helpful for my decision-making process. $\endgroup$ Mar 1 '12 at 1:47
  • $\begingroup$ @dmckee: Thanks for your notes. I think your comments would have been a reasonably good 'answer' to the question in the sense of "here are some things you should be considering". I agree that it wasn't a great question as phrased, shall we close it? $\endgroup$ Mar 1 '12 at 6:42
  • $\begingroup$ I responded to your points on my post, but it seems like I could use another resource other than this. @dmckee, you said that 3-chains etc. are part of most tutorials on this. Could you point me to some? Thanks $\endgroup$
    – CoffeeRain
    Mar 6 '12 at 14:22
  • $\begingroup$ @CoffeeRain There is a nice section in Kernighan & Pike, and the source code is available on-line. $\endgroup$ Mar 6 '12 at 17:46
  • $\begingroup$ @dmckee Thanks! $\endgroup$
    – CoffeeRain
    Mar 6 '12 at 17:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .