Unsupported Affix Items and Spell Checker Design in Vim

Each language property must be specified separately. LEMMA_PRESENT (Hunspell) *spell-LEMMA_PRESENT* Only needed for morphological analysis. MAXNGRAMSUGS (Hunspell) *spell-MAXNGRAMSUGS* Set number of n-gram suggestions. Not supported. PSEUDOROOT (Hunspell) *spell-PSEUDOROOT* Use NEEDAFFIX instead. |spell-NEEDAFFIX| SUGSWITHDOTS (Hunspell) *spell-SUGSWITHDOTS* Adds dots to suggestions. Vim doesn't need this. SYLLABLENUM (Hunspell) *spell-SYLLABLENUM* Not supported. TRY (Myspell, Hunspell, others) *spell-TRY* Vim does not use the TRY item, it is ignored. For making suggestions the actual characters in the words are used, that is much more efficient. WORDCHARS (Hunspell) *spell-WORDCHARS* Used to recognize words. Vim doesn't need it, because there is no need to separate words before checking them (using a trie instead of a hashtable). ============================================================================== 5. Spell checker design *develop-spell* When spell checking was going to be added to Vim a survey was done over the available spell checking libraries and programs. Unfortunately, the result was that none of them provided sufficient capabilities to be used as the spell checking engine in Vim, for various reasons: - Missing support for multi-byte encodings. At least UTF-8 must be supported, so that more than one language can be used in the same file. Doing on-the-fly conversion is not always possible (would require iconv support). - For the programs and libraries: Using them as-is would require installing them separately from Vim. That's mostly not impossible, but a drawback. - Performance: A few tests showed that it's possible to check spelling on the fly (while redrawing), just like syntax highlighting. But the mechanisms used by other code are much slower. Myspell uses a hashtable, for example. The affix compression that most spell checkers use makes it slower too. - For using an external program like aspell a communication mechanism would have to be setup. That's complicated to do in a portable way (Unix-only would be relatively simple, but that's not good enough). And performance will become a problem (lots of process switching involved). - Missing support for words with non-word characters, such as "Etten-Leur" and "et al.", would require marking the pieces of them OK, lowering the reliability. - Missing support for regions or dialects. Makes it difficult to accept all English words and highlight non-Canadian words differently. - Missing support for rare words. Many words are correct but hardly ever used and could be a misspelled often-used word. - For making suggestions the speed is less important and requiring to install another program or library would be acceptable. But the word lists probably differ, the suggestions may be wrong words. Spelling suggestions *develop-spell-suggestions* For making suggestions there are two basic mechanisms: 1. Try changing the bad word a little bit and check for a match with a good word. Or go through the list of good words, change them a little bit and check for a match with the bad word. The changes are deleting a character, inserting a character, swapping two characters, etc. 2. Perform soundfolding on both the bad word and the good words and then find matches, possibly with a few changes like with the first mechanism. The first is good for finding typing mistakes. After experimenting with hashtables and looking at solutions from other spell checkers the conclusion was that a trie (a kind

This section continues listing unsupported affix items in Vim, including LEMMA_PRESENT, MAXNGRAMSUGS, PSEUDOROOT, SUGSWITHDOTS, SYLLABLENUM, TRY, and WORDCHARS, detailing why they are not supported. It then discusses the design considerations for Vim's spell checker, explaining why existing spell-checking libraries and programs were unsuitable due to limitations like lack of UTF-8 support, performance issues, and difficulties with non-word characters. Finally, the section introduces two mechanisms for spelling suggestions: modifying the bad word and checking for matches, and performing soundfolding.