I`d like to share some tweaks for spirit which may be useful to others(see letter attachment). Some of them are modifications(non-intrusive) of spirit/wave sources, some are my own development.
The sources are in the /headers/ folder. The example of usage is in /trees/ folder. Documentation is in /SpiritAdditions.doc
There is a pretty complicated grammar implementation ( ~120 rules and ~151 lexemes, may be i`ll publish sources later, runs only under Windows ) in /ThorSL/ folder alongside with a bunch of input files in /ThorSL/Shaders/ folder.
You can try the perfomance difference of spirit and my file_position impl by running ThorSL_new_file_pos.exe and ThorSL_old_file_pos.exe, ( use ":dir path" console command to parse all files from the path folder ) for me the new approach is twice faster than old. Although my file_pos and lexer use static data,
i think that with a little tweaking they can be made thread safe.
ThorSL_no_trees.exe is the same grammar but without tree generation, as you will probably see it is 3-5 times faster than the grammar with ast/pt (according to my benchmarks), and trees require a great amount of memory for not-so-big input files (6mb input requires >1gb of ram). So my next intention is to make spirit tree generation faster and less memory consuming. I have an idea to compute the "length" of the rule for example a= b >> c >> ( d >> e >> f | g >> h >> i >> j |
has "length" = 6 ( the longest path to success ) and create a storage of "lenght" units(which hold first/last iters and maybe anything else) for every rule when grammar instantiates, so as the rule starts we begin to use units from it`s storage and consume it as we go, when the rule is hit we create a tree node using the info from the storage, and mark the storage as reset. This will require much fewer allocations, and a tree node can be a user supplied base type for his ast tree.
I would appreciete if you could give me some detailed description/information sources of how current tree generation system works.