next up previous
Next: Completion with null productions Up: Null productions Previous: Computing -expansion probabilities

Prediction with null productions

Prediction is mediated by the left-corner relation. For each X occurring to the right of a dot, we generate states for all Y that are reachable from X by way of the tex2html_wrap_inline8519 relation. This reachability criterion has to be extended in the presence of null productions. Specifically, if X has a production tex2html_wrap_inline8523 then tex2html_wrap_inline8525 is a left corner of X iff tex2html_wrap_inline8529 all have a non-zero probability of expanding to tex2html_wrap_inline7691 . The contribution of such a production to the left-corner probability tex2html_wrap_inline8533 is

displaymath8511

The old prediction procedure can now be modified in two steps. First, replace the old tex2html_wrap_inline8423 relation by the one that takes into account null productions, as sketched above. From the resulting tex2html_wrap_inline8423 compute the reflexive transitive closure tex2html_wrap_inline7701 , and use it to generate predictions as before.

Second, when predicting a left corner Y with a production tex2html_wrap_inline8543 , add states for all dot positions up to the first RHS nonterminal that cannot expand to tex2html_wrap_inline7691 , say from tex2html_wrap_inline8547 through tex2html_wrap_inline8549 . We will call this procedure ``spontaneous dot shifting.'' It accounts precisely for those derivations that expand the RHS prefix tex2html_wrap_inline8551 without consuming any of the input symbols.

The forward and inner probabilities of the states thus created are those of the first state tex2html_wrap_inline8547 , multiplied by factors that account for the implied tex2html_wrap_inline7691 -expansions. This factor is just the product tex2html_wrap_inline8557 , where j is the dot position.



Andreas Stolcke
Sat Jun 29 21:49:02 PDT 1996