Boost Spirit X3: Customizing Internal Memory Allocations?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Boost Spirit X3: Customizing Internal Memory Allocations?

id.zeta
Hi. New boost::spirit user here.


I am wondering if there is a way to use custom memory allocations in X3, for the internal std::vector, tst, etc. thingies, a way other than to replace the global new/delete.

Regards.

------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Joel de Guzman
On 12/26/15 7:29 PM, [hidden email] wrote:
> Hi. New boost::spirit user here.
>
>
> I am wondering if there is a way to use custom memory allocations in X3, for the internal
> std::vector, tst, etc. thingies, a way other than to replace the global new/delete.

X3 should not be using std::vectors directly. It is up to you to supply it with
anything as long as it conforms to the std container interface. If you can spot
somewhere it is used directly, pleas point it out. We can use some kind of traits
or policies to avoid direct use.

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Mario Lang
Joel de Guzman <[hidden email]> writes:

> On 12/26/15 7:29 PM, [hidden email] wrote:
>> Hi. New boost::spirit user here.
>>
>>
>> I am wondering if there is a way to use custom memory allocations in X3, for the internal
>> std::vector, tst, etc. thingies, a way other than to replace the global new/delete.
>
> X3 should not be using std::vectors directly. It is up to you to supply it with
> anything as long as it conforms to the std container interface. If you can spot
> somewhere it is used directly, pleas point it out.

position_cache comes to mind.  It uses std::vector to store iterators.

> We can use some kind of traits or policies to avoid direct use.

--
CYa,
  ⡍⠁⠗⠊⠕

------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

sehe
On 28-12-15 11:47, Mario Lang wrote:
> position_cache comes to mind.  It uses std::vector to store iterators.
I didn't consider position_cache a core X3 feature. It struck me as a
convenience/utility type to aid in implementing the recommended AST
annotation pattern.

I suppose that it, too, could be parameterized, but in reality there is
nothing that forces users to use it in the first place (unless I'm
somehow misinformed?).

The trickier bits come with stateful allocators/scoped allocators and I
would personally not expect the library to take the complexity hit for them.

Seth

------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

id.zeta
Sorry I didn't make it clear. I have two prolems:

1. It appears to mw that x3::parse() create a whole new container (in my case, the said 'std::vector') before assigingn to my supplied one. Could it be possible to use a supplied 'stateful allocator' (C++11) for the temporary container?

2. Could x3::symbols<> be customizable with user-supplied 'stateful allocators'?
I think both could be done with traits/policies and additional ctor/function parameters.

Yeah I am all for stateful.

BTW, headers in spirit/home/x3/operator seems to be using the same header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp as those old in the boost/spirit/home/qi/operator. That could cause compile errors.


On Monday, December 28, 2015 7:42 PM, Seth <[hidden email]> wrote:


On 28-12-15 11:47, Mario Lang wrote:
> position_cache comes to mind.  It uses std::vector to store iterators.
I didn't consider position_cache a core X3 feature. It struck me as a
convenience/utility type to aid in implementing the recommended AST
annotation pattern.

I suppose that it, too, could be parameterized, but in reality there is
nothing that forces users to use it in the first place (unless I'm
somehow misinformed?).

The trickier bits come with stateful allocators/scoped allocators and I
would personally not expect the library to take the complexity hit for them.

Seth


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general



------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

sehe
On 28-12-15 12:59, [hidden email] wrote:
> BTW, headers in spirit/home/x3/operator seems to be using the same
> header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp
> as those old in the boost/spirit/home/qi/operator. That could cause
> compile errors.
Great spot that. I'd simply create a PR for it (shall I do it?)

------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Joel de Guzman
On 12/29/15 6:40 AM, Seth wrote:
> On 28-12-15 12:59, [hidden email] wrote:
>> BTW, headers in spirit/home/x3/operator seems to be using the same
>> header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp
>> as those old in the boost/spirit/home/qi/operator. That could cause
>> compile errors.
> Great spot that. I'd simply create a PR for it (shall I do it?)

Good catch indeed. Yeah, I'd appreciate a PR, if you will :-)

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

id.zeta
In reply to this post by sehe
There might be more eg. SPIRIT_QI_BOOL_POLICIES_SEP_29_2009_0710AM (yeah, BOOST_QI) in bool_policies.hpp in boost/spirit/home/x3/numeric. Maybe an automatic tool could be made to help finding such errors.


On Tuesday, December 29, 2015 6:40 AM, Seth <[hidden email]> wrote:


On 28-12-15 12:59, [hidden email] wrote:

> BTW, headers in spirit/home/x3/operator seems to be using the same
> header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp
> as those old in the boost/spirit/home/qi/operator. That could cause
> compile errors.

Great spot that. I'd simply create a PR for it (shall I do it?)




------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

sehe
On 29-12-15 12:27, [hidden email] wrote:
There might be more eg. SPIRIT_QI_BOOL_POLICIES_SEP_29_2009_0710AM (yeah, BOOST_QI) in bool_policies.hpp in boost/spirit/home/x3/numeric. Maybe an automatic tool could be made to help finding such errors.

I'm on it



On Tuesday, December 29, 2015 6:40 AM, Seth [hidden email] wrote:


On 28-12-15 12:59, [hidden email] wrote:

> BTW, headers in spirit/home/x3/operator seems to be using the same
> header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp
> as those old in the boost/spirit/home/qi/operator. That could cause
> compile errors.

Great spot that. I'd simply create a PR for it (shall I do it?)





------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Joel de Guzman
On 12/29/15 7:28 PM, Seth wrote:
> On 29-12-15 12:27, [hidden email] wrote:
>> There might be more eg. SPIRIT_QI_BOOL_POLICIES_SEP_29_2009_0710AM (yeah, BOOST_QI) in
>> bool_policies.hpp in boost/spirit/home/x3/numeric. Maybe an automatic tool could be made
>> to help finding such errors.
>
> I'm on it

Wonderful!

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

id.zeta
We should return to the topic... stateful allocators aren't so complex and C++11 added them because they could be useful. I wish they were supported everywhere in boost.


On Wednesday, December 30, 2015 10:35 AM, Joel de Guzman <[hidden email]> wrote:


On 12/29/15 7:28 PM, Seth wrote:
> On 29-12-15 12:27, [hidden email] wrote:
>> There might be more eg. SPIRIT_QI_BOOL_POLICIES_SEP_29_2009_0710AM (yeah, BOOST_QI) in
>> bool_policies.hpp in boost/spirit/home/x3/numeric. Maybe an automatic tool could be made
>> to help finding such errors.
>
> I'm on it

Wonderful!

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/



------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general



------------------------------------------------------------------------------

_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Orient
In reply to this post by Joel de Guzman
Joel de Guzman wrote
On 12/29/15 6:40 AM, Seth wrote:
> On 28-12-15 12:59, [hidden email] wrote:
>> BTW, headers in spirit/home/x3/operator seems to be using the same
>> header guards eg. SPIRIT_KLEENE_JANUARY_07_2007_0818AM in kleene.hpp
>> as those old in the boost/spirit/home/qi/operator. That could cause
>> compile errors.
> Great spot that. I'd simply create a PR for it (shall I do it?)

Good catch indeed. Yeah, I'd appreciate a PR, if you will :-)

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
I think it is time to use #pragma once now. Not so long ago there (or on github) was the discussion about "include guard versus pragma once". I sure currently all C++14 compilers (needed for x3) supports pragma once.
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Joel de Guzman
On 12/31/15 12:26 AM, Orient wrote:

> I think it is time to use #pragma once now. Not so long ago there (or on
> github) was the discussion about "include guard versus pragma once". I sure
> currently all C++14 compilers (needed for x3) supports pragma once.

Good point. is that a fact? Do you have a link? I'd gladly use pragma once
if all C++14 compilers support it.

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

sehe
In reply to this post by id.zeta
On 30-12-15 16:41, [hidden email] wrote:
> We should return to the topic... stateful allocators aren't so complex
> and C++11 added them because they could be useful. I wish they were
> supported everywhere in boost.
>
Firstly, it's your topic as well as change, we have thread trees.

Secondly, stateful allocators are complex.

Boost supports them in many places (much better than any standard
library I have the pleasure of dealing with).

The question is what allocator support is appropriate for X3. I'll defer
to its designers.

Seth


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Orient
In reply to this post by Joel de Guzman
Joel de Guzman wrote
On 12/31/15 12:26 AM, Orient wrote:

> I think it is time to use #pragma once now. Not so long ago there (or on
> github) was the discussion about "include guard versus pragma once". I sure
> currently all C++14 compilers (needed for x3) supports pragma once.

Good point. is that a fact? Do you have a link? I'd gladly use pragma once
if all C++14 compilers support it.

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/


------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Almost all compilers I never ever heard before supports the`#pragma once`: https://en.wikipedia.org/wiki/Pragma_once#Portability
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Joel de Guzman
On 1/3/16 11:43 PM, Orient wrote:

> Joel de Guzman wrote
>> On 12/31/15 12:26 AM, Orient wrote:
>>
>>> I think it is time to use #pragma once now. Not so long ago there (or on
>>> github) was the discussion about "include guard versus pragma once". I
>>> sure
>>> currently all C++14 compilers (needed for x3) supports pragma once.
>>
>> Good point. is that a fact? Do you have a link? I'd gladly use pragma once
>> if all C++14 compilers support it.
>>
> Almost all compilers I never ever heard before supports the`#pragma once`:
> https://en.wikipedia.org/wiki/Pragma_once#Portability

Yet, it is still non-standard. And Boost is about standard conformance.
After thinking about it for some time, I think I'll stick with include guards.

Regards,
--
Joel de Guzman
http://www.ciere.com
http://boost-spirit.com
http://www.cycfi.com/

------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general
Reply | Threaded
Open this post in threaded view
|

Re: Boost Spirit X3: Customizing Internal Memory Allocations?

Michael Powell-2
On Sun, Jan 3, 2016 at 4:14 PM, Joel de Guzman <[hidden email]> wrote:

> On 1/3/16 11:43 PM, Orient wrote:
>> Joel de Guzman wrote
>>> On 12/31/15 12:26 AM, Orient wrote:
>>>
>>>> I think it is time to use #pragma once now. Not so long ago there (or on
>>>> github) was the discussion about "include guard versus pragma once". I
>>>> sure
>>>> currently all C++14 compilers (needed for x3) supports pragma once.
>>>
>>> Good point. is that a fact? Do you have a link? I'd gladly use pragma once
>>> if all C++14 compilers support it.
>>>
>> Almost all compilers I never ever heard before supports the`#pragma once`:
>> https://en.wikipedia.org/wiki/Pragma_once#Portability
>
> Yet, it is still non-standard. And Boost is about standard conformance.
> After thinking about it for some time, I think I'll stick with include guards.

Almost all is still not all. Non-standard.

> Regards,
> --
> Joel de Guzman
> http://www.ciere.com
> http://boost-spirit.com
> http://www.cycfi.com/
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Spirit-general mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/spirit-general

------------------------------------------------------------------------------
_______________________________________________
Spirit-general mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/spirit-general