[interprocess] 32-bit offset_ptr

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[interprocess] 32-bit offset_ptr

Boost - Dev mailing list
Dear Experts,

I have some old code that uses 32-bit offset_ptrs, i.e.

   typedef boost::interprocess::offset_ptr<void, int32_t, int32_t> ptr_t;

This has stopped working on 64-bit systems at some point since 1.58.  
git blame
suggests a commit on  14 Oct 2015.  There is now a static assert that
checks that
OffsetType is at least as large as a raw pointer.

In the current case, I'm using these pointers within memory-mapped
files which will
always be vastly smaller than the 32-bit limit.  Using 32-bit offsets seemed
like a worthwhile optimisation at the time, and now I am hoping to maintain
binary compatibility with those files.

In the past I've also used them to save memory on 64-bit systems with pointer-heavy
data structures (using boost.intrusive) using an allocator that
operates within a
small memory region.

Has something changed within the implementation that makes this restriction
necessary?  Was it never supposed to work as I was using it?

There is also an assert that OffsetType is unsigned, which wasn't there before.
Maybe I was doing that wrong.


Thanks for any advice.


Regards, Phil.





_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] 32-bit offset_ptr

Boost - Dev mailing list
AMDG

On 2/2/19 9:28 AM, Phil Endecott via Boost wrote:

> I have some old code that uses 32-bit offset_ptrs, i.e.
>
>   typedef boost::interprocess::offset_ptr<void, int32_t, int32_t> ptr_t;
>
> This has stopped working on 64-bit systems at some point since 1.58. 
> git blame
> suggests a commit on  14 Oct 2015.  There is now a static assert that
> checks that
> OffsetType is at least as large as a raw pointer.
>
> In the current case, I'm using these pointers within memory-mapped files
> which will
> always be vastly smaller than the 32-bit limit.  Using 32-bit offsets
> seemed
> like a worthwhile optimisation at the time, and now I am hoping to maintain
> binary compatibility with those files.
>

The offset_ptr is not restricted to existing inside
the memory-mapped file.  The library can and
does create temporaries.

> In the past I've also used them to save memory on 64-bit systems with
> pointer-heavy
> data structures (using boost.intrusive) using an allocator that operates
> within a
> small memory region.
>
> Has something changed within the implementation that makes this restriction
> necessary?  Was it never supposed to work as I was using it?
>
> There is also an assert that OffsetType is unsigned, which wasn't there
> before.
> Maybe I was doing that wrong.
>

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] 32-bit offset_ptr

Boost - Dev mailing list
On 02/02/2019 18:19, Steven Watanabe via Boost wrote:

> AMDG
>
> On 2/2/19 9:28 AM, Phil Endecott via Boost wrote:
>> I have some old code that uses 32-bit offset_ptrs, i.e.
>>
>>    typedef boost::interprocess::offset_ptr<void, int32_t, int32_t> ptr_t;
>>
>> This has stopped working on 64-bit systems at some point since 1.58.
>> git blame
>> suggests a commit on  14 Oct 2015.  There is now a static assert that
>> checks that
>> OffsetType is at least as large as a raw pointer.
>>
>> In the current case, I'm using these pointers within memory-mapped files
>> which will
>> always be vastly smaller than the 32-bit limit.  Using 32-bit offsets
>> seemed
>> like a worthwhile optimisation at the time, and now I am hoping to maintain
>> binary compatibility with those files.
>>
>
> The offset_ptr is not restricted to existing inside
> the memory-mapped file.  The library can and
> does create temporaries.

Correct. An offset_ptr placed in the stack (a temporary, a return or
argument in a function) must be able to point to an object in shared
memory. Thus in 64 bit address-spaces a 64-bit offset is needed.

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [interprocess] 32-bit offset_ptr

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Steven Watanabe <[hidden email]> wrote:
> On 2/2/19 9:28 AM, Phil Endecott via Boost wrote:
>> I have some old code that uses 32-bit offset_ptrs, i.e.
>>
>>  typedef boost::interprocess::offset_ptr<void, int32_t, int32_t> ptr_t;
>>
>> This has stopped working on 64-bit systems at some point since 1.58.

>> I'm using these pointers within memory-mapped files
>> which will always be vastly smaller than the 32-bit limit.

> The offset_ptr is not restricted to existing inside
> the memory-mapped file.  The library can and
> does create temporaries.

Thanks Steven & Ion for your replies.

I think I was getting away with this because, in general, it is OK to
copy an offset_ptr to an out-of-range temporary as long as you don't
dereference it before you copy it back.  For example:

void swap(offset_ptr& a, offset_ptr& b)
{
  offset_ptr tmp = a;
  a = b;
  b = tmp;
}

I think that works, i.e. a and b end up with the correct offsets
(provided signed arithmetic is two's complement and wraps around)
even though tmp is wrong.  On the other hand:

offset_ptr offset_ptr::operator++(int)
{
  offset_ptr r = *this;
  (*this) += 1;
  return r;
}

offset_ptr& i = somewhere_in_mapped_file;
*(i++) = 42;

That fails because the returned offset_ptr on the stack is dereferenced, and
points to the wrong place.  But we could re-write that to return a regular pointer:

pointer offset_ptr::operator++(int)
{
  pointer p = get();
  (*this) += 1;
  return p;
}

I think most of the offset_ptr methods that return offset_ptr by value can be
re-written like that - an exception is pointer_to().  (Does that change mean
it is no longer a model of a pointer, or iterator?)

Ion, can you comment on whether intrusive containers use (and dereference)
temporary pointers?


Regards, Phil.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost