[pylucene-dev] other tokenizers ?

[Joh N.]

Hello,

(sorry, i forgot to add this question in my previous
mail)

is PyLucene able to handle a custom tokenization
without any stemming process ?

 actually i would like to feed the index myself with
words from different languages (thus inconsistant
tokenization), but also sgml tags, and maybe even some
numbers,

will it be possible ? where can i found hints on where
to look after that ?

best regards,

J.


	

	
		
___________________________________________________________________________ 
Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo! Messenger 
Téléchargez cette version sur http://fr.messenger.yahoo.com