Good luck

Aug 28, 2008 at 12:50 PM
Interesting project. Looking forward to it.

One question:

Will the lexer support Unicode?

Cheers

leppie
Coordinator
Sep 2, 2008 at 1:02 AM
Hi Leppie,

The project is moving along really well. We have some pretty unmoveable client deadlines which depend upon this project, so the schedules are all important.

Yes the lexer will have full support for Unicode and I mean FULL support. So you will be able to specify the regular expression source in unicode drawing from any of the 1mill plus codepoints and a compact lex table will be created that can be executed against any unicode text stream. This was one of the major drivers for the project, we have not found a lexer generator that fully supports Unicode to date. The full support of Unicode has had negligible impact on Lexer table size and runtime performance. It has put on about 30% time on table generation - which we think is not an issue at all.

We will keep the status updated and are will be publishing the entire lexer source September 15 with first round of testing complete.

Thanks for you enquiry,

Vic Diamond