[interprocess] [bimap] Using raw pointers

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[interprocess] [bimap] Using raw pointers

Matias Capeletto
Hello,

Currently Boost.Bimap does not work with interprocess allocators
because it uses references to connect the views and the core. I want
to change this but avoid any performance hit and I am wondering what
is the best way to do it.
It will be easy if Bimap interface will use .left() and .right()
functions to access the map views (then I could just init the views at
each call like Boost.MultiIndex get() index function does) but I am
stuck with the left and right members (and I actually like them).
So, what I want to be able to do is the following:

template<class Bimap>
class left_type {
  Bimap* core;
  public:
  left_type(Bimap* c) : core(c) {};
  ...
};

template<class Bimap> class right_type {...};

template<class LeftKey,class RightKey>
class bimap {
  public:
    left_type<bimap>  left;
  right_type<bimap> right;
  bimap() : left(this), right(this) {}
};

This will not work because I am not using the proper offset_pointer
abstraction, but I would love to be able to avoid it. In the case of
Bimap, these pointers are very easy to maintain.

Will it be possible to add an extension point to Boost.Interprocess to
allow this idiom?

Something like adding a call to the end of priv_find_impl of
segment_manager so user classes can rewire themselves when they are
loaded from shared memory. For example:

template<class T>
void rewire_after_loading_from_shared_memory(T* t) { }

  ...
  template <class T>
  std::pair<T*, size_type> priv_find_impl (const CharType* name, bool lock) {
      // find ret pointer
      T* t_ret = static_cast<T*>(ret) ;
      if( t_ret ) rewire_after_loading_from_shared_memory( t_ret );
      return std::pair<T*, size_type>(t_ret ,sz);
   }
  ...

Or maybe a better place. Then I will be able to rewire the map views
pointers to the core if they are different from it:

template<class L, class R>
void rewire_after_loading_from_shared_memory(bimap<L,R>* t) { t->rewire(); }

If this is not possible because inherent limitations of the
interprocess mechanisms, do you know a good way to make Boost.Bimap
compatible with its allocators?

Thanks,
Best regards
Matias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Ion Gaztañaga
El 25/01/2013 20:05, Matias Capeletto escribió:

> Hello,
>
> Currently Boost.Bimap does not work with interprocess allocators
> because it uses references to connect the views and the core. I want
> to change this but avoid any performance hit and I am wondering what
> is the best way to do it.
> It will be easy if Bimap interface will use .left() and .right()
> functions to access the map views (then I could just init the views at
> each call like Boost.MultiIndex get() index function does) but I am
> stuck with the left and right members (and I actually like them).
> So, what I want to be able to do is the following:
>
> template<class Bimap>
> class left_type {
>    Bimap* core;
>    public:
>    left_type(Bimap* c) : core(c) {};
>    ...
> };

If Bimap exports a pointer type (say Bimap::pointer or similar) that can
be offset_ptr or raw pointer (I think multiindex exports it), then you
just need to use:

typedef boost::pointer_to_other
    <typename Bimap::pointer, Bimap>::type bimap_ptr;

typedef boost::intrusive::pointer_traits
   <typename Bimap::pointer>::
     rebind_pointer<Bimap>::type bimap_ptr;

and use it

template<class Bimap>
class left_type {
   typedef boost::pointer_to_other
      <typename Bimap::pointer, Bimap>::type bimap_ptr;
    bimap_ptr core;
    public:
    left_type(const bimap_ptr &c) : core(c) {};

};

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <[hidden email]> wrote:

> If Bimap exports a pointer type (say Bimap::pointer or similar) that can be
> offset_ptr or raw pointer (I think multiindex exports it), then you just
> need to use:
>
> typedef boost::pointer_to_other
>    <typename Bimap::pointer, Bimap>::type bimap_ptr;
>
> typedef boost::intrusive::pointer_traits
>   <typename Bimap::pointer>::
>     rebind_pointer<Bimap>::type bimap_ptr;

Thanks for the answer Ion.
Ok, so doing this I will only pay for the offset_ptr in case the
allocator needs it. That is good.

But still when users will select an interprocess allocator they will
be paying for every call to the views:

  left_type& left = bm.left;
  left.size(); // needs to use offset_ptr

What MultiIndex is doing is creating the index views with a pointer to
(this) every time the user ask them. I could do the same if I
deprecate .left and .right and include get_left() and get_right()
functions. But as I said I will like to keep the current interface...
is something like the rewire extension impossible? (I am afraid it is
if you are pointing to read only memory).

Or maybe some another trick to point the members to the core.

Best
Matias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Ion Gaztañaga
El 25/01/2013 21:47, Matias Capeletto escribió:

> On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <[hidden email]> wrote:
>> If Bimap exports a pointer type (say Bimap::pointer or similar) that can be
>> offset_ptr or raw pointer (I think multiindex exports it), then you just
>> need to use:
>>
>> typedef boost::pointer_to_other
>>     <typename Bimap::pointer, Bimap>::type bimap_ptr;
>>
>> typedef boost::intrusive::pointer_traits
>>    <typename Bimap::pointer>::
>>      rebind_pointer<Bimap>::type bimap_ptr;
>
> Thanks for the answer Ion.
> Ok, so doing this I will only pay for the offset_ptr in case the
> allocator needs it. That is good.
>
> But still when users will select an interprocess allocator they will
> be paying for every call to the views:
>
>    left_type& left = bm.left;
>    left.size(); // needs to use offset_ptr

Have this problem also with iterators. Containers with interprocess
allocators will return an iterator that can be placed in shared memory
(contains offset_ptr) but when you use it to iterate in your process,
you pay the price of offset_ptr. I think multiindex also pays this price.

> What MultiIndex is doing is creating the index views with a pointer to
> (this) every time the user ask them. I could do the same if I
> deprecate .left and .right and include get_left() and get_right()
> functions. But as I said I will like to keep the current interface...
> is something like the rewire extension impossible? (I am afraid it is
> if you are pointing to read only memory).

I don't fully understand what do you achieve with "rewire", can you
elaborate, please?

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

TONGARI J
In reply to this post by Matias Capeletto
2013/1/26 Matias Capeletto <[hidden email]>

> On Fri, Jan 25, 2013 at 9:30 PM, Ion Gaztañaga <[hidden email]>
> wrote:
> > If Bimap exports a pointer type (say Bimap::pointer or similar) that can
> be
> > offset_ptr or raw pointer (I think multiindex exports it), then you just
> > need to use:
> >
> > typedef boost::pointer_to_other
> >    <typename Bimap::pointer, Bimap>::type bimap_ptr;
> >
> > typedef boost::intrusive::pointer_traits
> >   <typename Bimap::pointer>::
> >     rebind_pointer<Bimap>::type bimap_ptr;
>
> Thanks for the answer Ion.
> Ok, so doing this I will only pay for the offset_ptr in case the
> allocator needs it. That is good.
>
> But still when users will select an interprocess allocator they will
> be paying for every call to the views:
>
>   left_type& left = bm.left;
>   left.size(); // needs to use offset_ptr
>
> What MultiIndex is doing is creating the index views with a pointer to
> (this) every time the user ask them. I could do the same if I
> deprecate .left and .right and include get_left() and get_right()
> functions. But as I said I will like to keep the current interface...
> is something like the rewire extension impossible? (I am afraid it is
> if you are pointing to read only memory).
>
> Or maybe some another trick to point the members to the core.
>

Is it possible for you to make left & right in a union and put it in the
front of bimap (as 1st base or member) so you can cast (this) to bimap?

Not sure about strict-aliasing though...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
In reply to this post by Ion Gaztañaga
On Fri, Jan 25, 2013 at 11:49 PM, Ion Gaztañaga <[hidden email]> wrote:

> El 25/01/2013 21:47, Matias Capeletto escribió:
>> But still when users will select an interprocess allocator they will
>> be paying for every call to the views:
>>
>>    left_type& left = bm.left;
>>    left.size(); // needs to use offset_ptr
>
> Have this problem also with iterators. Containers with interprocess
> allocators will return an iterator that can be placed in shared memory
> (contains offset_ptr) but when you use it to iterate in your process, you
> pay the price of offset_ptr. I think multiindex also pays this price.

Ok, that is interesting. Maybe I should just use offset_ptr then, if
every time I iterate over the container I am using them anyways.

>> What MultiIndex is doing is creating the index views with a pointer to
>> (this) every time the user ask them. I could do the same if I
>> deprecate .left and .right and include get_left() and get_right()
>> functions. But as I said I will like to keep the current interface...
>> is something like the rewire extension impossible? (I am afraid it is
>> if you are pointing to read only memory).
>
> I don't fully understand what do you achieve with "rewire", can you
> elaborate, please?

Maybe you do not understand this because is a crazy idea in the
context of Interprocess. I looked at your code and in the
segment_manager, before you return the user the found Bimap* I wanted
a user extension point to be called on the pointer:

    if( t_ret ) rewire_after_loading_from_shared_memory( t_ret );
      return std::pair<T*, size_type>(t_ret ,sz);

And I will do something like (this function being a friend of the views):

template<class L, class R>
void rewire_after_loading_from_shared_memory(bimap<L,R>* t) {
  t->left.rewire(t); t->right.rewire(t);
}

In left_type and right_type private section:
void rewire(Core* c) { if(core!=c) core=c; }

Event if this could work, the question will be if we want to allow
users to place views in shared memory. For Bimap, I think it doesn't
make sense... you should always stored the bimap and then get the
views.

But now I looked better at MultiIndex implementation and it is
actually pushing the index as part of the main container hierarchy
using CRTP to access the core, something I thought about yesterday
also. I think I will end up doing the same for Bimap then.

Thanks,
Best
Matias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
In reply to this post by TONGARI J
On Sat, Jan 26, 2013 at 4:46 AM, TONGARI <[hidden email]> wrote:
> 2013/1/26 Matias Capeletto <[hidden email]>
>> Or maybe some another trick to point the members to the core.
>
> Is it possible for you to make left & right in a union and put it in the
> front of bimap (as 1st base or member) so you can cast (this) to bimap?
>
> Not sure about strict-aliasing though...

Look at my mail to Ion, it seems that MultiIndex was putting the
indexes as part of the hierarchy to achieve this.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Ion Gaztañaga
In reply to this post by Matias Capeletto
El 26/01/2013 7:32, Matias Capeletto escribió:

> But now I looked better at MultiIndex implementation and it is
> actually pushing the index as part of the main container hierarchy
> using CRTP to access the core, something I thought about yesterday
> also. I think I will end up doing the same for Bimap then.

That's quite clever ;-) I think it's a good solution for your problem.

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
On Sat, Jan 26, 2013 at 8:50 AM, Ion Gaztañaga <[hidden email]> wrote:
> El 26/01/2013 7:32, Matias Capeletto escribió:
>> But now I looked better at MultiIndex implementation and it is
>> actually pushing the index as part of the main container hierarchy
>> using CRTP to access the core, something I thought about yesterday
>> also. I think I will end up doing the same for Bimap then.
>
> That's quite clever ;-) I think it's a good solution for your problem.

Well, unfortunately, I am back to square one. Again, I will not have
any problem if the views will be accessed from a left() and right()
functions but I can not find a way to get from the view data member to
the core that is not undefined in C++.  For reference, I have to get
from the left data member to the bimap parent:

  class bimap { ... left_map_view left; ... } ;

Any attempt to use the offset of the member to get to the bimap class
seems to be forbidden by the standard.
So, I am back to using offset_ptr for the map_view data members and
maybe at one point think about migrating to function-accessed views to
avoid it :(

Best regards
Matias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Ion Gaztañaga
El 27/01/2013 9:02, Matias Capeletto escribió:
> Any attempt to use the offset of the member to get to the bimap class
> seems to be forbidden by the standard.

Yes, i'ts undefined behaviour in theory but not in practice.
Boost.Intrusive has a useful function implemented for many compilers:

http://www.boost.org/doc/libs/1_52_0/boost/intrusive/parent_from_member.hpp

It's similar to a "downcast" but using members. Like "static_cast", it
does not work with virtual inheritance.

In your case the pointer to member value can be a compile-time constant
(so &Bimap::left can be passed as a template parameter). This constant
can be part of the type "left_map" or the compile-time pointer to member
could be obtained by left_map by a metafunction, call parent_from_member
passing "this" and the compile-time pointer to member value, obtaining
the pointer to Bimap. This way you can avoid storing any pointer to
Bimap into left_map.

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
On Sun, Jan 27, 2013 at 9:39 AM, Ion Gaztañaga <[hidden email]> wrote:
> El 27/01/2013 9:02, Matias Capeletto escribió:
>> Any attempt to use the offset of the member to get to the bimap class
>> seems to be forbidden by the standard.
>
> Yes, i'ts undefined behaviour in theory but not in practice. Boost.Intrusive
> has a useful function implemented for many compilers:
>
> http://www.boost.org/doc/libs/1_52_0/boost/intrusive/parent_from_member.hpp

This. It is exactly what I was looking for yesterday... any chance to
make this more visible? It is a very handy tool.

> In your case the pointer to member value can be a compile-time constant (so
> &Bimap::left can be passed as a template parameter). This constant can be
> part of the type "left_map" or the compile-time pointer to member could be
> obtained by left_map by a metafunction, call parent_from_member passing
> "this" and the compile-time pointer to member value, obtaining the pointer
> to Bimap. This way you can avoid storing any pointer to Bimap into left_map.

Awesome. This is what I wanted to do, but I failed because the basic
piece in the puzzle was missing. Thanks a lot, I didn't actually
expected you to directly give it to me :)

I will do the changes then so Bimap can be used with Interprocess allocators.
Best regards
Matias

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Ion Gaztañaga
El 27/01/2013 19:00, Matias Capeletto escribió:

> This. It is exactly what I was looking for yesterday... any chance to
> make this more visible? It is a very handy tool.

What do you mean with "more visible", maybe try to put it into
boost/utility?

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] [bimap] Using raw pointers

Matias Capeletto
On Sun, Jan 27, 2013 at 10:36 PM, Ion Gaztañaga <[hidden email]> wrote:
> El 27/01/2013 19:00, Matias Capeletto escribió:
>> This. It is exactly what I was looking for yesterday... any chance to
>> make this more visible? It is a very handy tool.
>
> What do you mean with "more visible", maybe try to put it into
> boost/utility?

It can be. It would have helped me because I looked there to see if
there was something to help me.
Google didn't helped either... I looked for "parent from member C++"
and "boost parent from member C++" yesterday and there was nothing to
point me to your work.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost