Reference counting?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Reference counting?

Neal Becker
I'm looking at the glas alpha docs.  I've been studying blitz++ recently, and
I have (so far) liked the use of reference counting.

A slice of an array returns a (shared) view of the array data.  Reference
counting is used for memory management.

I read the comments in glas regarding assignment vs. copy construction.  Have
you considered using ref counting instead?
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
Hi Neal,

I have not thought of this. I am a bit puzzled. Can you explain what you
mean by reference counting? Do you mean that a container will only
disappear when all its references by other objects disappear?

Thanks,

Karl



Neal Becker wrote:

>I'm looking at the glas alpha docs.  I've been studying blitz++ recently, and
>I have (so far) liked the use of reference counting.
>
>A slice of an array returns a (shared) view of the array data.  Reference
>counting is used for memory management.
>
>I read the comments in glas regarding assignment vs. copy construction.  Have
>you considered using ref counting instead?
>_______________________________________________
>glas mailing list
>[hidden email]
>http://lists.boost.org/mailman/listinfo.cgi/glas
>  
>

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Bert Rodiers
Hello Karl,

An example:

You have a class CVec, the data is stored in seperate class CView to which CVec points. CView also has a reference count. At the moment you create a CVec a CView is recreated with reference count 1, when a new CVec is created from an existing one, the CView is being shared and the reference count is increased (with a function call). When a CVec is destroyed, you decrease the reference count (with a function call) and when the reference count becomes zero you destroy the object.
I could point you to books that discuss reference counting in more detail...

Regards,
Bert


On 12/10/2007, Karl Meerbergen <[hidden email]> wrote:
Hi Neal,

I have not thought of this. I am a bit puzzled. Can you explain what you
mean by reference counting? Do you mean that a container will only
disappear when all its references by other objects disappear?

Thanks,

Karl



Neal Becker wrote:

>I'm looking at the glas alpha docs.  I've been studying blitz++ recently, and
>I have (so far) liked the use of reference counting.
>
>A slice of an array returns a (shared) view of the array data.  Reference
>counting is used for memory management.
>
>I read the comments in glas regarding assignment vs. copy construction.  Have
>you considered using ref counting instead?
>_______________________________________________
>glas mailing list
>[hidden email]
><a href="http://lists.boost.org/mailman/listinfo.cgi/glas" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://lists.boost.org/mailman/listinfo.cgi/glas
>
>


_______________________________________________
glas mailing list
[hidden email]
<a href="http://lists.boost.org/mailman/listinfo.cgi/glas" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)"> http://lists.boost.org/mailman/listinfo.cgi/glas



_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
Hi Bert,

This is the idea of a boost::shared_ptr.
It is clear that this might help in some cases for improving reliability
of code.

If I understand Neal well, the situation could be the following:
you create a matrix.
Later, you create an object that is a proxy to a row of this matrix.
If you have reference counting, the programmer can destroy the matrix
before the row proxy. Without the reference count, the proxy should be
destroyed before the matrix.

At this stage, this mechanism is indeed not provided. In practice, it
would imply adding a boost::shared_ptr in all containers. This is
possible, but could it not create undesired overhead?

One of the great things of generic programming is that we can provide
containers and proxies that support reference counting in addition to
containers and proxies that do not use reference counting :-)

Karl



Bert Rodiers wrote:

> Hello Karl,
>
> An example:
>
> You have a class CVec, the data is stored in seperate class CView to
> which CVec points. CView also has a reference count. At the moment you
> create a CVec a CView is recreated with reference count 1, when a new
> CVec is created from an existing one, the CView is being shared and
> the reference count is increased (with a function call). When a CVec
> is destroyed, you decrease the reference count (with a function call)
> and when the reference count becomes zero you destroy the object.
> I could point you to books that discuss reference counting in more
> detail...
>
> Regards,
> Bert
>
>
> On 12/10/2007, *Karl Meerbergen* < [hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Neal,
>
>     I have not thought of this. I am a bit puzzled. Can you explain
>     what you
>     mean by reference counting? Do you mean that a container will only
>     disappear when all its references by other objects disappear?
>
>     Thanks,
>
>     Karl
>
>
>
>     Neal Becker wrote:
>
>     >I'm looking at the glas alpha docs.  I've been studying blitz++
>     recently, and
>     >I have (so far) liked the use of reference counting.
>     >
>     >A slice of an array returns a (shared) view of the array
>     data.  Reference
>     >counting is used for memory management.
>     >
>     >I read the comments in glas regarding assignment vs. copy
>     construction.  Have
>     >you considered using ref counting instead?
>     >_______________________________________________
>     >glas mailing list
>     > [hidden email] <mailto:[hidden email]>
>     > http://lists.boost.org/mailman/listinfo.cgi/glas
>     >
>     >
>
>
>     _______________________________________________
>     glas mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.boost.org/mailman/listinfo.cgi/glas
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>glas mailing list
>[hidden email]
>http://lists.boost.org/mailman/listinfo.cgi/glas
>

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Bert Rodiers
Hello Karl,

In RogueWave they do the following:
A vector has a view, which on its turn points to the actual data (a C++ data structure aggregating the data).
When you for example apply a slice on your vector you get a new vector with a different view on the same data (the sharing of the data happens with reference counting, and the view keeps track of the new start, length and stride). The data is cleaned up at the moment no views exist anymore. For matrices and higher dimension structures they also apply this so that when you extract a column from a matrix the data is shared between the original matrix and the vector which was the result of the slice.

Regards,
Bert


There you have

On 12/10/2007, Karl Meerbergen <[hidden email]> wrote:
Hi Bert,

This is the idea of a boost::shared_ptr.
It is clear that this might help in some cases for improving reliability
of code.

If I understand Neal well, the situation could be the following:
you create a matrix.
Later, you create an object that is a proxy to a row of this matrix.
If you have reference counting, the programmer can destroy the matrix
before the row proxy. Without the reference count, the proxy should be
destroyed before the matrix.

At this stage, this mechanism is indeed not provided. In practice, it
would imply adding a boost::shared_ptr in all containers. This is
possible, but could it not create undesired overhead?

One of the great things of generic programming is that we can provide
containers and proxies that support reference counting in addition to
containers and proxies that do not use reference counting :-)

Karl



Bert Rodiers wrote:

> Hello Karl,
>
> An example:
>
> You have a class CVec, the data is stored in seperate class CView to
> which CVec points. CView also has a reference count. At the moment you
> create a CVec a CView is recreated with reference count 1, when a new
> CVec is created from an existing one, the CView is being shared and
> the reference count is increased (with a function call). When a CVec
> is destroyed, you decrease the reference count (with a function call)
> and when the reference count becomes zero you destroy the object.
> I could point you to books that discuss reference counting in more
> detail...
>
> Regards,
> Bert
>
>
> On 12/10/2007, *Karl Meerbergen* < [hidden email]
> <mailto: [hidden email]>> wrote:
>
>     Hi Neal,
>
>     I have not thought of this. I am a bit puzzled. Can you explain
>     what you
>     mean by reference counting? Do you mean that a container will only
>     disappear when all its references by other objects disappear?
>
>     Thanks,
>
>     Karl
>
>
>
>     Neal Becker wrote:
>
>     >I'm looking at the glas alpha docs.  I've been studying blitz++
>     recently, and
>     >I have (so far) liked the use of reference counting.
>     >
>     >A slice of an array returns a (shared) view of the array
>     data.  Reference
>     >counting is used for memory management.
>     >
>     >I read the comments in glas regarding assignment vs. copy
>     construction.  Have
>     >you considered using ref counting instead?
>     >_______________________________________________
>     >glas mailing list
>     > [hidden email] <mailto:[hidden email]>
>     > http://lists.boost.org/mailman/listinfo.cgi/glas
>     >
>     >
>
>
>     _______________________________________________
>     glas mailing list
>     [hidden email] <mailto: [hidden email]>
>     http://lists.boost.org/mailman/listinfo.cgi/glas
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>glas mailing list
>[hidden email]
> http://lists.boost.org/mailman/listinfo.cgi/glas
>


_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas



_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Neal Becker
In reply to this post by Karl Meerbergen-2
On Friday 12 October 2007, Karl Meerbergen wrote:

> Hi Bert,
>
> This is the idea of a boost::shared_ptr.
> It is clear that this might help in some cases for improving reliability
> of code.
>
> If I understand Neal well, the situation could be the following:
> you create a matrix.
> Later, you create an object that is a proxy to a row of this matrix.
> If you have reference counting, the programmer can destroy the matrix
> before the row proxy. Without the reference count, the proxy should be
> destroyed before the matrix.
>
> At this stage, this mechanism is indeed not provided. In practice, it
> would imply adding a boost::shared_ptr in all containers. This is
> possible, but could it not create undesired overhead?
>
> One of the great things of generic programming is that we can provide
> containers and proxies that support reference counting in addition to
> containers and proxies that do not use reference counting :-)
>

Ref counting has a lot of benefits.  For example, you can return an array by
value, without a huge performance impact.  Also, a view of an array can be
treated just the same as an array.  This simplifies code.

Ref counting can be bolted-on using shared_ptr, but that has greater cost
(shared_ptr has to allocate on heap a count).  You want the ref count to be
included in the memory object to reduce the cost.  Also, without ref
counting, I believe you need to distinguish between destroying an array
(which owns the memory) and a view (which does not).  This code becomes
redundant if you have ref counting built in.

I find the separation of the memory component and the 'view of memory' (array)
that blitz++ uses to be useful.  I haven't looked at the glas design yet.
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
In reply to this post by Bert Rodiers
I see what you mean. At this moment, the glas implementation does not do
this. It is not that hard to implement this I think. As Neal points out,
it allows passing arguments by value, which indeed simplifies alot. GLAS
currently passes arguments by value, except containers. This exception
would be removed by the mechanism you describe. it would simplify the
glas implementation as well.

Currently, a view has to be destroyed before the container is destroyed.

I am not sure I understand Neal's comment about storing the count in the
data. If you allocate a vector of double, would you keep one position in
the array for the reference count?

There are consequences that should be thought of carefully:
* if v and w are two vectors (or vector views), what does the operation
w = v do ?
* the storage of vectors and matrices becomes slightly more expensive

I like the idea. Something to think of ...

Thanks,

Karl


Bert Rodiers wrote:

> Hello Karl,
>
> In RogueWave they do the following:
> A vector has a view, which on its turn points to the actual data (a
> C++ data structure aggregating the data).
> When you for example apply a slice on your vector you get a new vector
> with a different view on the same data (the sharing of the data
> happens with reference counting, and the view keeps track of the new
> start, length and stride). The data is cleaned up at the moment no
> views exist anymore. For matrices and higher dimension structures they
> also apply this so that when you extract a column from a matrix the
> data is shared between the original matrix and the vector which was
> the result of the slice.
>
> Regards,
> Bert
>
>
> There you have
>
> On 12/10/2007, *Karl Meerbergen* <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Bert,
>
>     This is the idea of a boost::shared_ptr.
>     It is clear that this might help in some cases for improving
>     reliability
>     of code.
>
>     If I understand Neal well, the situation could be the following:
>     you create a matrix.
>     Later, you create an object that is a proxy to a row of this matrix.
>     If you have reference counting, the programmer can destroy the matrix
>     before the row proxy. Without the reference count, the proxy should be
>     destroyed before the matrix.
>
>     At this stage, this mechanism is indeed not provided. In practice, it
>     would imply adding a boost::shared_ptr in all containers. This is
>     possible, but could it not create undesired overhead?
>
>     One of the great things of generic programming is that we can provide
>     containers and proxies that support reference counting in addition to
>     containers and proxies that do not use reference counting :-)
>
>     Karl
>
>
>
>     Bert Rodiers wrote:
>
>     > Hello Karl,
>     >
>     > An example:
>     >
>     > You have a class CVec, the data is stored in seperate class CView to
>     > which CVec points. CView also has a reference count. At the
>     moment you
>     > create a CVec a CView is recreated with reference count 1, when
>     a new
>     > CVec is created from an existing one, the CView is being shared and
>     > the reference count is increased (with a function call). When a
>     CVec
>     > is destroyed, you decrease the reference count (with a function
>     call)
>     > and when the reference count becomes zero you destroy the object.
>     > I could point you to books that discuss reference counting in more
>     > detail...
>     >
>     > Regards,
>     > Bert
>     >
>     >
>     > On 12/10/2007, *Karl Meerbergen* <
>     [hidden email] <mailto:[hidden email]>
>     > <mailto: [hidden email]
>     <mailto:[hidden email]>>> wrote:
>     >
>     >     Hi Neal,
>     >
>     >     I have not thought of this. I am a bit puzzled. Can you explain
>     >     what you
>     >     mean by reference counting? Do you mean that a container
>     will only
>     >     disappear when all its references by other objects disappear?
>     >
>     >     Thanks,
>     >
>     >     Karl
>     >
>     >
>     >
>     >     Neal Becker wrote:
>     >
>     >     >I'm looking at the glas alpha docs.  I've been studying blitz++
>     >     recently, and
>     >     >I have (so far) liked the use of reference counting.
>     >     >
>     >     >A slice of an array returns a (shared) view of the array
>     >     data.  Reference
>     >     >counting is used for memory management.
>     >     >
>     >     >I read the comments in glas regarding assignment vs. copy
>     >     construction.  Have
>     >     >you considered using ref counting instead?
>     >     >_______________________________________________
>     >     >glas mailing list
>     >     > [hidden email] <mailto:[hidden email]>
>     <mailto:[hidden email] <mailto:[hidden email]>>
>     >     > http://lists.boost.org/mailman/listinfo.cgi/glas
>     <http://lists.boost.org/mailman/listinfo.cgi/glas>
>     >     >
>     >     >
>     >
>     >
>     >     _______________________________________________
>     >     glas mailing list
>     >     [hidden email] <mailto:[hidden email]> <mailto:
>     [hidden email] <mailto:[hidden email]>>
>     >     http://lists.boost.org/mailman/listinfo.cgi/glas
>     >
>     >
>     >------------------------------------------------------------------------
>
>     >
>     >_______________________________________________
>     >glas mailing list
>     >[hidden email] <mailto:[hidden email]>
>     > http://lists.boost.org/mailman/listinfo.cgi/glas
>     >
>
>
>     _______________________________________________
>     glas mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.boost.org/mailman/listinfo.cgi/glas
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>glas mailing list
>[hidden email]
>http://lists.boost.org/mailman/listinfo.cgi/glas
>

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Neal Becker
In reply to this post by Bert Rodiers
On Friday 12 October 2007, Bert Rodiers wrote:

> Hello Karl,
>
> In RogueWave they do the following:
> A vector has a view, which on its turn points to the actual data (a C++
> data structure aggregating the data).
> When you for example apply a slice on your vector you get a new vector with
> a different view on the same data (the sharing of the data happens with
> reference counting, and the view keeps track of the new start, length and
> stride). The data is cleaned up at the moment no views exist anymore. For
> matrices and higher dimension structures they also apply this so that when
> you extract a column from a matrix the data is shared between the original
> matrix and the vector which was the result of the slice.
>

In blitz++ this is also applied to 'components' of an array.  For example,
complex has real and imag components.  Thus, real(array) is an lvalue!  Very
nice, IMO.
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
Neal Becker wrote:

>On Friday 12 October 2007, Bert Rodiers wrote:
>  
>
>>Hello Karl,
>>
>>In RogueWave they do the following:
>>A vector has a view, which on its turn points to the actual data (a C++
>>data structure aggregating the data).
>>When you for example apply a slice on your vector you get a new vector with
>>a different view on the same data (the sharing of the data happens with
>>reference counting, and the view keeps track of the new start, length and
>>stride). The data is cleaned up at the moment no views exist anymore. For
>>matrices and higher dimension structures they also apply this so that when
>>you extract a column from a matrix the data is shared between the original
>>matrix and the vector which was the result of the slice.
>>
>>    
>>
>
>In blitz++ this is also applied to 'components' of an array.  For example,
>complex has real and imag components.  Thus, real(array) is an lvalue!  Very
>nice, IMO.
>  
>
In GLAS, real(array) is also an lvalue ;-) At the moment, it is only
const, but this is not hard to change.

Karl


_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Russell Smiley
In reply to this post by Karl Meerbergen-2
> -----Original Message-----
> From:  Karl Meerbergen
> Subject: Re: [glas] Reference counting?
>
> * if v and w are two vectors (or vector views), what does the
> operation
> w = v do ?
>

In my implementation of a reference counted object w=v means that w
drops the reference to its current storage (with a resulting decrement
to the reference count for that object) and then picks up the reference
to the storage in v with a resulting increment to that reference count.
I think this is the most useful for general application.

If you want to copy values from the v storage into the w storage then
there are two separate methods:

w.value_copy(v) - copies values from v, but w dereferences current
storage and creates a new 'blank' storage before the copy.

w.value_copy_this_reference(v) - copies values from v into the current
storage of w.

HTH

Russell.
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Russell Smiley

> -----Original Message-----
> From: Russell Smiley
> Subject: Re: [glas] Reference counting?
>
> > -----Original Message-----
> > From:  Karl Meerbergen
> > Subject: Re: [glas] Reference counting?
> >
> > * if v and w are two vectors (or vector views), what does the
> > operation
> > w = v do ?
> >
>
> In my implementation of a reference counted object w=v means that w
> drops the reference to its current storage (with a resulting decrement
> to the reference count for that object) and then picks up the
> reference
> to the storage in v with a resulting increment to that
> reference count.
> I think this is the most useful for general application.
>
> If you want to copy values from the v storage into the w storage then
> there are two separate methods:
>
> w.value_copy(v) - copies values from v, but w dereferences current
> storage and creates a new 'blank' storage before the copy.
>
> w.value_copy_this_reference(v) - copies values from v into the current
> storage of w.
>

I forgot there is a third method:

w.value_copy() which tells w to start a reference to new storage copying
the values from the old storage.

Russell.
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
In reply to this post by Russell Smiley
Russell Smiley wrote:

>>-----Original Message-----
>>From:  Karl Meerbergen
>>Subject: Re: [glas] Reference counting?
>>
>>* if v and w are two vectors (or vector views), what does the
>>operation
>>w = v do ?
>>
>>    
>>
>
>In my implementation of a reference counted object w=v means that w
>drops the reference to its current storage (with a resulting decrement
>to the reference count for that object) and then picks up the reference
>to the storage in v with a resulting increment to that reference count.
>I think this is the most useful for general application.
>
>If you want to copy values from the v storage into the w storage then
>there are two separate methods:
>
>w.value_copy(v) - copies values from v, but w dereferences current
>storage and creates a new 'blank' storage before the copy.
>
>w.value_copy_this_reference(v) - copies values from v into the current
>storage of w.
>
>  
>
Hi Russell,

In previous discussions, it was suggested that w = v performs a deep copy.

 From a mathematics point of view, this is also what you expect.

Karl

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
In reply to this post by Russell Smiley
Dear List,

I would like to continue this discussion since it may lead to a
significant change in the development of the library.

I see several situations, where containers could be destroyed before the
view. So, a reference count is indeed useful.

One of my previous colleagues needed fixed size vectors of small size
for a finite element code. He refused to use ublas vectors because the
storage of the size created extra memory overhead.

Do you see (other) situations where the storage of a reference could
create overhead?

As for smart pointers, we can have different views. A view could take
ownership of the container (as for an auto_ptr, e.g.). The reference
count would not be needed and this would make people happy who use small
vectors.

More important, I think, is the issue about the assignment operator and
the copy constructor.

Currently, the copy constructor makes a shallow copy for all views and
expression objects, while the assignment operator is a deep copy. This
looks like a conflicting situation. UBlas does the same I think. What
about MTL?

I am not very keen on giving up the deep copy concept of w = v. Does
anyone have an interesting idea?

Best,

Karl



Russell Smiley wrote:

>>-----Original Message-----
>>From: Russell Smiley
>>Subject: Re: [glas] Reference counting?
>>
>>    
>>
>>>-----Original Message-----
>>>From:  Karl Meerbergen
>>>Subject: Re: [glas] Reference counting?
>>>
>>>* if v and w are two vectors (or vector views), what does the
>>>operation
>>>w = v do ?
>>>
>>>      
>>>
>>In my implementation of a reference counted object w=v means that w
>>drops the reference to its current storage (with a resulting decrement
>>to the reference count for that object) and then picks up the
>>reference
>>to the storage in v with a resulting increment to that
>>reference count.
>>I think this is the most useful for general application.
>>
>>If you want to copy values from the v storage into the w storage then
>>there are two separate methods:
>>
>>w.value_copy(v) - copies values from v, but w dereferences current
>>storage and creates a new 'blank' storage before the copy.
>>
>>w.value_copy_this_reference(v) - copies values from v into the current
>>storage of w.
>>
>>    
>>
>
>I forgot there is a third method:
>
>w.value_copy() which tells w to start a reference to new storage copying
>the values from the old storage.
>
>Russell.
>_______________________________________________
>glas mailing list
>[hidden email]
>http://lists.boost.org/mailman/listinfo.cgi/glas
>  
>

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Andreas P. Priesnitz
On Fri, 12 Oct 2007, Karl Meerbergen wrote:

> Do you see (other) situations where the storage of a reference could create
> overhead?

You might want to create views recursively (view into a view into a ...
container), for instance when modeling divide-and-conquer-algorithms
like quicksort that way.

In such cases, the number of views may have the same order of magnitude as
the number of container elements. Thus, the overhead for the reference
count would become significant. (and unnecessary)

(BTW: There is as well an increase in run time by creating, initializing,
incrementing, decrementing, checking, and deleting the counter.)

> As for smart pointers, we can have different views. A view could take
> ownership of the container (as for an auto_ptr, e.g.). The reference count
> would not be needed and this would make people happy who use small vectors.
>
> More important, I think, is the issue about the assignment operator and the
> copy constructor.
>
> Currently, the copy constructor makes a shallow copy for all views and
> expression objects, while the assignment operator is a deep copy. This looks
> like a conflicting situation. UBlas does the same I think. What about MTL?
>
> I am not very keen on giving up the deep copy concept of w = v. Does anyone
> have an interesting idea?

As you say, different kinds of ``smart views'' could coexist. In my
understanding, w = v should be overloaded to perform either deep or
shallow, depending on the way(s) w and v are referencing their data.

But the straightforward implementation - to define for any container/view
proper assignments from any other kind of view - is likely to cause a
considerable amount of (probably boilerplate) code.

The very interesting question would then be if and how one can express
this overload generically, as the choice of copy concept depends merely on
the kind of reference, not on the kind of data.

Regards,
Andreas

[http://www.cs.chalmers.se/~priesnit]
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Pieter Collins
I think that a general-purpose package needs to provide vector/matrix
classes taylored to various uses.  An application using many 3d vectors
will have very different requirements (fast allocation/deallocation,
efficient storage) to an application using 1000x1000 matrices (efficient
manipulation using views/slices).

As I see it, reference counting will improve safety and may allow
storage and slicing to be implemented by the same class, but does not
introduce extra mathematical functionality. The expense is probably
worth it for large objects, but not for small objects.

Regarding the copy semantics, how about introducing a new concept
"VectorView" which uses shallow copies to complement "VectorContainer"
which uses deep copies? It seems like this concept is sufficiently
important to be worth adding, though I generally think it's important to
keep the number of different concepts to a minimum.

Presumably it would be possible to separate storage (including reference
counting) from the rest of the code by using an array template argument
similar to uBlas.

Regards,

Pieter


--

Dr. Pieter Collins
Centrum voor Wiskunde en Informatica
Postbus 94079
1090 GB Amsterdam
The Netherlands

Tel: +31 20 5924196
Fax:  +31 20 5924199
Email: [hidden email]

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
Pieter Collins wrote:

>I think that a general-purpose package needs to provide vector/matrix
>classes taylored to various uses.  An application using many 3d vectors
>will have very different requirements (fast allocation/deallocation,
>efficient storage) to an application using 1000x1000 matrices (efficient
>manipulation using views/slices).
>
>As I see it, reference counting will improve safety and may allow
>storage and slicing to be implemented by the same class, but does not
>introduce extra mathematical functionality. The expense is probably
>worth it for large objects, but not for small objects.
>
>Regarding the copy semantics, how about introducing a new concept
>"VectorView" which uses shallow copies to complement "VectorContainer"
>which uses deep copies? It seems like this concept is sufficiently
>important to be worth adding, though I generally think it's important to
>keep the number of different concepts to a minimum.
>
>Presumably it would be possible to separate storage (including reference
>counting) from the rest of the code by using an array template argument
>similar to uBlas.
>
>Regards,
>
>Pieter
>
>
>  
>
Hi Pieter,

At this stage, there is no container concept in GLAS, so all options are
still open. I am in favour of having two container concepts, as you
suggest, View and Container, one with a deep copy constructor, the other
with a shallow copy constructor. I am currently using a shared_ptr for
this instead of a view.

If two types of containers exist, the user has to be aware of the
difference in behaviour between copy constructors.

I am not sure that a template argument is good enough to make clear that
the copy constructor has different behaviour ?!

Currently you can do something like

dense_vector<double> v( 10 ) ;
dense_vector<double> w( 4.0 * v ) ;

This does not make sense with a shallow copy. In the case of a shallow
copy, this could become:

dense_vector<double> v( 10 ) ;
dense_vector<double> w( size(v) ) ; w = 4.0 * v ;

Best regards,

Karl


_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Neal Becker
On Tuesday 16 October 2007, Karl Meerbergen wrote:

> Pieter Collins wrote:
> >I think that a general-purpose package needs to provide vector/matrix
> >classes taylored to various uses.  An application using many 3d vectors
> >will have very different requirements (fast allocation/deallocation,
> >efficient storage) to an application using 1000x1000 matrices (efficient
> >manipulation using views/slices).
> >
> >As I see it, reference counting will improve safety and may allow
> >storage and slicing to be implemented by the same class, but does not
> >introduce extra mathematical functionality. The expense is probably
> >worth it for large objects, but not for small objects.
> >
> >Regarding the copy semantics, how about introducing a new concept
> >"VectorView" which uses shallow copies to complement "VectorContainer"
> >which uses deep copies? It seems like this concept is sufficiently
> >important to be worth adding, though I generally think it's important to
> >keep the number of different concepts to a minimum.
> >
> >Presumably it would be possible to separate storage (including reference
> >counting) from the rest of the code by using an array template argument
> >similar to uBlas.
> >
> >Regards,
> >
> >Pieter
>
> Hi Pieter,
>
> At this stage, there is no container concept in GLAS, so all options are
> still open. I am in favour of having two container concepts, as you
> suggest, View and Container, one with a deep copy constructor, the other
> with a shallow copy constructor. I am currently using a shared_ptr for
> this instead of a view.
>
> If two types of containers exist, the user has to be aware of the
> difference in behaviour between copy constructors.
>
> I am not sure that a template argument is good enough to make clear that
> the copy constructor has different behaviour ?!
>
> Currently you can do something like
>
> dense_vector<double> v( 10 ) ;
> dense_vector<double> w( 4.0 * v ) ;
>
> This does not make sense with a shallow copy. In the case of a shallow
> copy, this could become:
>
> dense_vector<double> v( 10 ) ;
> dense_vector<double> w( size(v) ) ; w = 4.0 * v ;
>
> Best regards,
>
> Karl

That is exactly what I don't want.  Having different types for a view vs. an
array will make it much harder to interoperate with other languages.  I
believe the unified design is more elegant.

An additional benefit of separating a view and memory management is that an
externally managed blob of memory can be adopted.  This allows
inter-operation with other array libraries.  For example, suppose we want to
interoperate with python numpy objects.

If we follow blitz design, then copy constructor will reference the other
array data.  BUT, copying an expression will not.

dense_vector<double> w (v.copy()) // deep copy
dense_vector<double> w (v * 4) // no need to say (v*4).copy()
_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Reference counting?

Karl Meerbergen-2
Neal Becker wrote:

>On Tuesday 16 October 2007, Karl Meerbergen wrote:
>  
>
>>Pieter Collins wrote:
>>    
>>
>>>I think that a general-purpose package needs to provide vector/matrix
>>>classes taylored to various uses.  An application using many 3d vectors
>>>will have very different requirements (fast allocation/deallocation,
>>>efficient storage) to an application using 1000x1000 matrices (efficient
>>>manipulation using views/slices).
>>>
>>>As I see it, reference counting will improve safety and may allow
>>>storage and slicing to be implemented by the same class, but does not
>>>introduce extra mathematical functionality. The expense is probably
>>>worth it for large objects, but not for small objects.
>>>
>>>Regarding the copy semantics, how about introducing a new concept
>>>"VectorView" which uses shallow copies to complement "VectorContainer"
>>>which uses deep copies? It seems like this concept is sufficiently
>>>important to be worth adding, though I generally think it's important to
>>>keep the number of different concepts to a minimum.
>>>
>>>Presumably it would be possible to separate storage (including reference
>>>counting) from the rest of the code by using an array template argument
>>>similar to uBlas.
>>>
>>>Regards,
>>>
>>>Pieter
>>>      
>>>
>>Hi Pieter,
>>
>>At this stage, there is no container concept in GLAS, so all options are
>>still open. I am in favour of having two container concepts, as you
>>suggest, View and Container, one with a deep copy constructor, the other
>>with a shallow copy constructor. I am currently using a shared_ptr for
>>this instead of a view.
>>
>>If two types of containers exist, the user has to be aware of the
>>difference in behaviour between copy constructors.
>>
>>I am not sure that a template argument is good enough to make clear that
>>the copy constructor has different behaviour ?!
>>
>>Currently you can do something like
>>
>>dense_vector<double> v( 10 ) ;
>>dense_vector<double> w( 4.0 * v ) ;
>>
>>This does not make sense with a shallow copy. In the case of a shallow
>>copy, this could become:
>>
>>dense_vector<double> v( 10 ) ;
>>dense_vector<double> w( size(v) ) ; w = 4.0 * v ;
>>
>>Best regards,
>>
>>Karl
>>    
>>
>
>That is exactly what I don't want.  Having different types for a view vs. an
>array will make it much harder to interoperate with other languages.  I
>believe the unified design is more elegant.
>
>
>An additional benefit of separating a view and memory management is that an
>externally managed blob of memory can be adopted.  This allows
>inter-operation with other array libraries.  For example, suppose we want to
>interoperate with python numpy objects.
>

Neal,


Your concern to try and use unified types is correct.
The unified design you mention is possible to some extent, but it is not
required for interoperability with other languages. (We actually use
GLAS with BLAS, LAPACK and MUMPS.) For dynamic binding, it makes life
clearly easier. I recall this was suggested several times on the ublas
mailing list. So it could be an interesting feature of GLAS. I put it in
the todo.

For each concept, we could provide a unified class: e.g.
continuous_dense_vector<T>
strided_dense_vector<T>
Whether this contains a reference count or not is another issue.


>
>If we follow blitz design, then copy constructor will reference the other
>array data.  BUT, copying an expression will not.
>
>dense_vector<double> w (v.copy()) // deep copy
>dense_vector<double> w (v * 4) // no need to say (v*4).copy()
>  
>

Here, I see the following problem (if I understand well):

dense_vector<double> v ( w )
is a shallow copy when w is of the same type as v and a deep copy when
it is another type. I do not think this is a good idea, especially, when
you do not know the type of w, as it often happens in template programming.

I hope my analysis of your proposal is correct.


Karl

_______________________________________________
glas mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/glas

Karl.Meerbergen.vcf (417 bytes) Download Attachment
Loading...