next up previous
Next: Efficient parsing with large Up: Appendix: Implementation Notes Previous: Prediction

Completion

 

Unlike prediction, the completion step still involves iteration. Each complete state derived by completion can potentially feed other completions. An important detail here is to ensure that all contributions to a state's tex2html_wrap_inline7695 and tex2html_wrap_inline7697 are summed before proceeding with using that state as input to further completion steps.

One approach to this problem is to insert complete states into a prioritized queue. The queue orders states by their start indices, highest first. This is because states corresponding to later expansions always have to be completed first before they can lead to the completion of expansions earlier on in the derivation. For each start index, the entries are managed as a first-in-first-out queue, ensuring that the dependency graph formed by the states is traversed in breadth-first order.

The completion pass can now be implemented as follows. Initially, all complete states from the previous scanning step are inserted in the queue. States are then removed from the front of the queue, and used to complete other states. Among the new states thus produced, complete ones are again added to the queue. The process iterates until no more states remain in the queue. Because the computation of probabilities already includes chains of unit productions, states derived from such productions need not be queued, which also ensures that the iteration terminates.

A similar queuing scheme, with the start index order reversed, can be used for the reverse completion step needed in the computation of outer probabilities (Section 5.2).



Andreas Stolcke
Sat Jun 29 21:49:02 PDT 1996