Synapse library review starts today December 2

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Klemens Morgenstern
Am 04.12.2016 um 23:20 schrieb Emil Dotchevski:

> On Sun, Dec 4, 2016 at 4:28 AM, Klemens Morgenstern <
> [hidden email]> wrote:
>
>> Am 04.12.2016 um 11:44 schrieb Emil Dotchevski:
>>
>>> On Sat, Dec 3, 2016 at 7:06 AM, Klemens Morgenstern <
>>> [hidden email]> wrote:
>>>
>>> For your review you may wish to consider the following questions:
>>>>>        - What is your evaluation of the design?
>>>>>
>>>>> The way I see it, it tries to solve a specifiy style problem for signals
>>>> in a very simple and straight-forward way.
>>>>
>>>> But: if it uses C++11 (like with type_traits) why not go all the way and
>>>> std::shared_ptr instead of boost::shared_ptr?
>>>>
>>> In synapse/dep/smart_ptr.hpp you can change which shared_ptr/weak_ptr you
>>> want to use, though this should probably be controlled by a configuration
>>> macro. However, if this is a Boost library, it should use
>>> boost::shared_ptr.
>>>
>> Not as I understand this sentence:  "It also means using the C++ Standard
>> Library where applicable."
>> http://www.boost.org/development/requirements.html
>>
> If the rules say that Boost libraries shouldn't use boost::shared_ptr but
> std::shared_ptr if available, I'll comply (though I can't imagine why this
> would be a good idea.)
Well don't take my word for it, it's just how I understand it. Boost
libraries are intended to give you functionality the standard doesn't
and not to be an alternative STL.

>> The dependence on C++11 is only for thread_local and for correct concurrent
>>> initialization of static objects. The library is still very useful without
>>> these, it's just that the (optional) interthread communication support
>>> won't work.
>>>
>> If you have the library for the most part in C++98 and some extra features
>> in C++11 that does make sense. But I think if you want to do that, go for
>> one standard, and thus either use everything from C++11 or
>> http://www.boost.org/doc/libs/1_62_0/doc/html/thread/thread_
>> local_storage.html
>>
> I considered using boost::thread_specific_ptr but to my understanding its
> initialization must be complete before it is used by different threads.
> This makes it impossible to use in Synapse because it uses thread local
> storage per type inside function templates, which would require C++11
> concurrent initialization of static objects (please do correct me if I'm
> wrong). So, I opted for a cleaner implementation using C++11 only for these
> two features (thread_local and concurrent initialization of static objects).
Ok, that might be the case, I don#t know. I was just thinking in terms
of choosing one standard for everything.

>> Why are there no move-operations for example for connections. It doesn't
>>>> make sense to put this into a shared_ptr, when you can make it movable,
>>>> while disabling copies.
>>>>
>>>> Using shared_ptr to hold on to connections is a design choice.
>> I've gathered that much, I'm just questioning it ;). It makes sense if you
>> are pur C++98, but not if you have C++11. And I do think this is a bad
>> choice for C++11.
>>
> Strictly speaking, using shared_ptr does use move since the shared_ptr
> itself is moveable. :)
>
> I guess we'll agree to disagree, in my opinion refcounting is more
> appropriate for connections compared to move semantics.
Well that's more of a philsophical question, but here's why I wouldn't
think so: a connection is attached to two objects and thus might be
shared between those two but not any more. Thus there's no real need to
refcount it, because it won't be shared with anything else.

>
>> Also: why can't return values be combined as in boost.signals2? That
>>>> should at least be an optional feature.
>>>>
>>>> See http://zajo.github.io/boost-synapse/questions_and_answers.html.
>> I know, but that doesn't answer the question, really. Because the
>> reasoning can also be applied to boost.signals2 also, because signals in
>> general don't care how many functions are connected.
>>
> What I mean is that I can be persuaded that the benefits of this feature in
> signals2 somewhat outweight the added complexity, but in Synapse the return
> type is used to tell apart different signals with otherwise the same
> signature, which is quite practical in my experience. Perhaps I'm missing
> some important use cases but I've never missed that particular feature.
I think if you argue that better, no one would complain. E.g. saying
it's more of an event-approach, it reduces complexity and (maybe)
increases performance.

>
>> Now the slot-lifetime is determined by weak_ptr, if I understand
>>>> correctly. I don't think that's generic enough at all, there should be
>>>> more
>>>> solutions. One would be, that target can inhert some sort of slot-type,
>>>> which will handle that. That way I could use the signals on elements on
>>>> the
>>>> stack, and also do this unintrusively:
>>>>
>>>> struct signalable_vector : std::vector<int>, slot_type {};
>>>>
>>>> Some solution for unique_ptr is needed. A solution would be to implement
>>>> a
>>>> synapse::unique_slot_ptr which implements that.
>>>>
>>>> That's what the null_deleter in shared_ptr is for.
>> How does that help with unique_ptr?
>>
> Is there something like weak_ptr that works with unique_ptr? The point of
> passing weak_ptr to connect is that emit won't call the connected function
> after the emitter (or the target) have expired, even if the connection
> object is still afloat. When it is impossible to use weak_ptr for this,
> then the only option to stay safe is to control the lifetime of the
> connection object (and pass a raw pointer to connect<>) .
There is not. You could use a custom deleter for that though.

>>>>>          library?
>>>>>
>>>>> I don't think it's that useful, but I guess I would still use
>>>> boost.signals2 and boost::asio::io_service for interthread signaling. I'm
>>>> not convinced this should be a distinct library, and not a feature of
>>>> boost.signals2.
>>>>
>>>> Synapse does not claim to be a "better" Signals2; its usefulness is not
>>> as
>>> an alternative to Signals2 but as a solution to problems Signals2 can not
>>> solve. Here are three examples of this, though I have many more:
>>>
>>> http://zajo.github.io/boost-synapse/handling_events_from_an_
>>> OS_message_pump.html
>>> http://zajo.github.io/boost-synapse/adding_custom_signals_to
>>> _Qt_objects_without_MOCing.html
>>> http://zajo.github.io/boost-synapse/using_meta_signals_to_co
>>> nnect_to_a_C-style_callback_API.html
>>>
>>> How about:
>> template<typename Signature>
>> inline boost::signals2::signal<Signature> & impl()
>> {
>>     boost::signals2::signal<Signature> sig;
>>     return sig;
>> }
>>
>> template<typename Signature, typename Func>
>> boost::signals2::connection connect(Func f) {return
>> impl<Signature>().connect(f);}
>>
>> template<typename Signature, typename ...Args>
>> void emit(Args... args) {return impl<Signature>()(args);}
>>
>> Let's say I build a more elaborate version of this - how does that compare
>> to synapse?
>
> I suppose if it is elaborate enough you'd rediscover Synapse, but with a
> different low level machinery, though I suspect that both libraries contain
> incompatible intricacies which make sense in the different approaches they
> take, since non-intrusiveness (in the way Synapse is non-intrusive) was not
> a design goal of Signals2.
That may be true; there are lots of problems, which look somewhat easy
from the outside but if you try to actually implement a solution you
discover a horrible mess of details. I'm just wondering that, and I
think you're in a situation where it would be very beneficial to explain
this more, since boost.signals2 exists.

Btw. since I was the first one to write a review and I voted no, please
let me clarify: I like the idea of the library and don't think something
like that should never be in boost. I do think it can be useful if some
of the weak spots can improved etc. So it's more of a not yet, than a
never vote on my part.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Robert McInnis
In reply to this post by Emil Dotchevski-3
On Sun, Dec 4, 2016 at 6:50 PM, Emil Dotchevski <[hidden email]>
wrote:

> > I disagree with this design style, which is to say I also disagree
> > with the existing slot/signal style.
> >
> > As an example, what if we had a system with hundreds of thousands of
> > numerics... each with a particular name.  Then imagine a rule set that
> > had to maintain integrity as any particular numeric were to be
> > changed.  With slots or synapses, you're forced to search for the
> > object each time before applying the rules.  This would bog down
> > quickly as the number of updates per second increased (imagine 100k
> > objects and 2-10k entwined rules being updated 2k times per second.  
> > That's a real world scenario)
> >
>
> The search is the price one has to pay for the non-intrusive nature of
Synapse.
> Obviously, when this isn't necessary (and the overhead of the search is
significant,
> as in the real world use case you're referring to) then you can use a
different
> approach.

Why pay the price at all if there is no need?  Why use an inefficient
algorithm when
a more direct approach is available?

Additionally, you can achieve algorithmic capabilities otherwise
unobtainable using
an observer pattern as I've described.  For example, a painter's algorithm
for
updating screen components.  Using slots, the manager may be notified to
re-paint the
window, but the manager would have to maintain some understanding of the
paint order
to render it properly.  Using a map< int, Observer* > where the int is the
painting
level, observers can hook into the appropriate level and be called in order
by simply
traversing the map in-order.


> That said, in my own use cases the overhead of Synapse has never been
significant
> in comparison to the time it takes to execute the actual connected
functions. Note
> that the search is limited only to connections of the same signal type,
and that
> it can be implemented as a hash, O(1).
>

With a small test case, you would rarely see any performance issues.  Only
at scale
would the inefficiencies start to become apparent.  In my example (stock
feed), if
one stock were to change (ie: MSFT), you would be forced to traverse all
5-10k
portfolios then search each to see if they own any MSFT to be updated.
Assuming
less than 100 positions in each portfolio, O(100) is still 6 comparisons for
each
one. Assuming only 5k portfolios, where 1k actually have a MSFT position,
that's
still 30,000 searches to find all 1k portfolios.  With the subject/observer
approach,
no search would be required... just directly to the business rule on each
object.

30,000 searches from a single tick.  Now imagine 2,000 ticks/sec.  That's
the issue.

Of course, after updating the portfolio, you'd have to remember to trigger
any
associated agents the user has assigned to watch his investments.  Which in
turn would
trigger other actions.  This series of events can be easily obtained using a
proper
subject/observer pattern.

> > Objects should have events they can trigger or subjects they
contain/manage

> > which would trigger
> > given a certain situation (value changes, particular event arrives,
> > flag flips, whatever) and the observer(s), being unknown to the
> > original object, would observe the event via a loose coupling
> > mechanism.
> >
> > This allows for a system to have hundreds or thousands of objects all
> > trigging events at random but only notifying the particular observers
> > that are interested in that instance's event.
> >
> > This allows for a more flexible system while promoting dependency
> > driven updates... which would result in the best performance for an
> > event notification system.
> >
> > I've been using such a design since 1990 and posted a rendition of it
> > to this group 3 months ago:
> >
> > https://github.com/tiny/boost_observers
> >
>
> You should request a formal review, if the Boost community finds your
library
> useful it'll be accepted.

I offered it up along with any nuggets I may have gleaned long the 25+ yrs
of
experience utilizing such a design across multiple types of projects.

If the community has any questions, I'd imagine they'd ask.




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Gavin Lambert
In reply to this post by Robert McInnis
On 5/12/2016 11:51, Robert McInnis wrote:

> I disagree with this design style, which is to say I also disagree with the
> existing slot/signal style.
>
> As an example, what if we had a system with hundreds of thousands of
> numerics... each with a particular
> name.  Then imagine a rule set that had to maintain integrity as any
> particular numeric were to be
> changed.  With slots or synapses, you're forced to search for the object
> each time before applying the
> rules.  This would bog down quickly as the number of updates per second
> increased (imagine 100k objects
> and 2-10k entwined rules being updated 2k times per second.  That's a real
> world scenario)

The slot/signal model is the same as the observer model, it's just that
your observers happen to implement one signal "changed" on all objects
instead of defining different signals for different purposes.

Instance-based signals like you're talking about are what Signals2 does.
  It has the benefit of callbacks being stored locally, at the cost of
being intrusive (the object raising the signal needs to contain the
signal instance).

This library takes the other approach of being non-intrusive and storing
the signal implementations elsewhere -- the benefit is that you can have
objects emit signals on behalf of other objects that don't need to know
that the signalling framework exists, but comes with some performance
costs as a result.

Some use cases would naturally lend itself to one, while others would
lend itself to the other.  If all of the code is controlled by you then
using Signals2 probably makes more sense, since it's trivial to include
the signals in your own classes and it gives you better performance (in
theory -- YMMV with mutexes).

But Klemens is asserting that there are some cases where the alternate
design is more useful; while I can't think of any such cases I've
personally encountered, I have little doubt that they could exist and as
such a library that supports it could be useful.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Nat Goodspeed-2
On Sun, Dec 4, 2016 at 10:25 PM, Gavin Lambert <[hidden email]> wrote:

> Instance-based signals like you're talking about are what Signals2 does.  It has the benefit of callbacks being stored locally, at the cost of being intrusive (the object raising the signal needs to contain the signal instance).
>
> This library takes the other approach of being non-intrusive and storing the signal implementations elsewhere -- the benefit is that you can have objects emit signals on behalf of other objects that don't need to know that the signalling framework exists, but comes with some performance costs as a result.
>
> Some use cases would naturally lend itself to one, while others would lend itself to the other.  If all of the code is controlled by you then using Signals2 probably makes more sense, since it's trivial to include the signals in your own classes and it gives you better performance (in theory -- YMMV with mutexes).
>
> But Klemens is asserting that there are some cases where the alternate design is more useful; while I can't think of any such cases I've personally encountered, I have little doubt that they could exist and as such a library that supports it could be useful.

I think what confused me about the Synapse documentation was the use
of the term "emitter," which suggests a locally-stored signal
instance. The reason I asked earlier about a thread-local map keyed by
void* was that understanding that (possible) implementation cleared up
the conceptual model for me.

In one of our applications, we use an "event" framework in which
senders and receivers are loosely coupled. Any given sender and
receiver can communicate, regardless of the relative order in which
they are constructed, by consulting a central map with find-or-create
semantics. They need only agree on the map key. In other words, I
believe that application has use cases of the kind that Synapse
intends to address.

(But the connection need only be made once for each receiver -- the
search isn't performed all over again on every event. I expect the
same is true for proper use of Synapse.)

So for me the term "rendezvous point" makes more sense than "emitter."

To me the Synapse library looks like this:

* front-end API based on a central map of rendezvous points
* back-end implementation of rendezvous points that essentially
duplicates Boost.Signals2, with some features omitted (as in the
critiques earlier in this thread)

Our map's mapped_type is (a thin class wrapped around)
boost::signals2::signal. So while I understand the value of having a
central map of rendezvous points, I do not understand why you would
use anything other than Boost.Signals2 on the back end.

It may be that I missed a really good reason in the discussion above.
But if it's there, I did miss it. Until I understand that really good
reason, I'm not keen on introducing a whole new back-end signals
implementation into Boost.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Emil Dotchevski-3
In reply to this post by Klemens Morgenstern
On Sun, Dec 4, 2016 at 4:52 PM, Klemens Morgenstern <
[hidden email]> wrote:

> I suppose if it is elaborate enough you'd rediscover Synapse, but with a
> different low level machinery, though I suspect that both libraries contain
> incompatible intricacies which make sense in the different approaches they
> take, since non-intrusiveness (in the way Synapse is non-intrusive) was not
> a design goal of Signals2.
> That may be true; there are lots of problems, which look somewhat easy
> from the outside but if you try to actually implement a solution you
> discover a horrible mess of details. I'm just wondering that, and I think
> you're in a situation where it would be very beneficial to explain this
> more, since boost.signals2 exists.
>

I added a new documentation page discussing the similarities and
differences between Signals2 and Synapse:
http://zajo.github.io/boost-synapse/Comparison_between_Boost_Signals2_and_Synapse.html
(here I would like to apologize if I got something about Signals2 wrong,
please someone do correct me if I did.)

To answer your concerns:

First, in Signals2 signals are objects while in Synapse they are types. The
notion of emitter does not exist in Signals2, the signal object does the
emitting, but the emitter (in the traditional sense) would be the object
the signal object is a member of. This makes sense in Signals2 because it
has no use of emitters as such, but in Synapse the concept of the emitter
object is central. Both designs make sense independently, but there is a
clash of terms that would be next impossible to communicate to the average
user.

Second, while it may be possible to use Signals2 signal objects simply as a
storage for connected functions to call (in a supposed attempt to use
Signals2 as a sort of back-end for Synapse, or to "Synapsify" Signals2),
that in itself would be little more than a list of functions. Since Synapse
organizes all connections of a given type into a single data structure,
it's probably possible to turn this into a list Signals2 signal objects per
Synapse signal type, but I don't see the benefit (and there would be the
big downside of Signals2 being a dependence of Synapse.)

Third, the two libraries take a different and incompatible approach to
thread safety. The connection list maintained by Signals2 signal objects
must be thread-safe, but the Synapse connection lists need not, since they
use thread_local storage. And, there is nothing in Signals2 that resembles
the interthread communication supported by Synapse (I do agree that the
ability to change the thread affinity of connection objects can be useful
and should probably be added to Synapse.)

This is not to say that there is anything wrong with Signals2. The search
Synapse must do when emit<S> is called could be a deal breaker in some use
cases. On the other hand Synapse makes emitting signals from a third-party
object simple and elegant.

Thanks,
Emil

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Emil Dotchevski-3
In reply to this post by Nat Goodspeed-2
On Mon, Dec 5, 2016 at 9:31 AM, Nat Goodspeed <[hidden email]> wrote:

> I think what confused me about the Synapse documentation was the use
> of the term "emitter," which suggests a locally-stored signal
> instance.


I suppose that makes sense to a Signals2 user because in its design the
signal object does the emitting, but the term "emitter" would make perfect
sense to people who have experience with other signal programming
libraries, for example Qt. I added a new documentation page in an attempt
to clarify and highlight the differences in design between Synapse and
Signals2, since it seems like many readers wonder what I might think is
wrong with Signals2 (the answer is nothing, it's just different):
http://zajo.github.io/boost-synapse/Comparison_between_Boost_Signals2_and_Synapse.html
.


> In one of our applications, we use an "event" framework in which
> senders and receivers are loosely coupled. Any given sender and
> receiver can communicate, regardless of the relative order in which
> they are constructed, by consulting a central map with find-or-create
> semantics. They need only agree on the map key. In other words, I
> believe that application has use cases of the kind that Synapse
> intends to address.
>

Yes, it is possible to use Signals2 in a specific case like this, to
achieve what Synapse does in the general case non-intrusively (any object
whatsoever can be passed as an emitter to emit<>). With Synapse, that
central map would be replaced by emit<S>(e), where "e" would be the address
of either the "sender" or the "receiver", or even of some other object
known to both -- whichever makes sense to use as the Synapse emitter.


> (But the connection need only be made once for each receiver -- the
> search isn't performed all over again on every event. I expect the
> same is true for proper use of Synapse.)
>

Nope, Synapse must do the search every time emit is called, though it is
limited only to connections of the specified signal type (in Synapse,
signals are types). That is the price of it being non-intrusive, however do
note that the search can be done in O(1) using hashing.


> So for me the term "rendezvous point" makes more sense than "emitter."
>
> To me the Synapse library looks like this:
>
> * front-end API based on a central map of rendezvous points
> * back-end implementation of rendezvous points that essentially
> duplicates Boost.Signals2, with some features omitted (as in the
> critiques earlier in this thread)
>
> Our map's mapped_type is (a thin class wrapped around)
> boost::signals2::signal. So while I understand the value of having a
> central map of rendezvous points, I do not understand why you would
> use anything other than Boost.Signals2 on the back end.
>

Please see my response to Klemens Morgenstern.

Thanks,
Emil

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Emil Dotchevski-3
In reply to this post by Gavin Lambert
On Sun, Dec 4, 2016 at 7:25 PM, Gavin Lambert <[hidden email]> wrote:

> Some use cases would naturally lend itself to one, while others would lend
> itself to the other.  If all of the code is controlled by you then using
> Signals2 probably makes more sense, since it's trivial to include the
> signals in your own classes and it gives you better performance (in theory
> -- YMMV with mutexes).
>

This is exactly right, thanks. I'll just add that once you consider Synapse
as part of your arsenal, you'll probably see use cases you're not thinking
about right now. I even ended up rewriting several systems in my own code
base to use Synapse, which made them much simpler and more elegant.

Emil

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Synapse library review starts today December 2

Emil Dotchevski-3
In reply to this post by Edward Diener-3
I've made a small addition to Synapse, the ability to post arbitrary
function objects on thread_local_queues, to be executed synchronously at
the time poll is called. See http://zajo.github.io/boost-synapse/post.html.

This feature allows critical worker threads to minimize the amount of time
they consume by offloading expensive non-critical computations to another,
non-critical thread. This also removes the need for synchronization, since
the queued functions are executed synchronously in the thread that owns the
thread_local_queue object.

Emil

On Fri, Dec 2, 2016 at 6:19 AM, Edward Diener <[hidden email]>
wrote:

> The formal review of the Synapse library by Emil Dotchevski starts today,
> December 2, and is scheduled to continue through December 11. This gives
> two full weekends and the week in between for the formal review.
>
> Synapse is a signal programming library, similar to Boost Signals2 and the
> signals-slots system in Qt. The main difference is that Synapse is
> non-intrusive: the address of any object of any static type whatsoever can
> be passed to synapse::emit to emit a signal. This makes it possible to emit
> Synapse signals from objects of third-party types as well as system objects
> (e.g. standard FILE pointers, HWNDs, etc.) or any other object that can be
> converted to a pointer.
>
> The library has been formatted to fit the Boost directory and namespace
> structure. To get Synapse, clone
> 'https://github.com/zajo/boost-synapse.git' into a directory called
> 'synapse' under your boost/libs directory.
>
> See the tutorial at
> 'http://zajo.github.io/boost-synapse/Tutorial.html', or read full
> documentation at 'http://zajo.github.io/boost-synapse/index.html'. You
> can also view the documentation locally from your clone of the library at
> boost/libs/synapse/doc/index.html.
>
> Review guidelines
> =================
>
> Reviews should be submitted to the developer list ([hidden email]),
> preferably with '[synapse]' in the subject. Or if you don't wish to for
> some reason or are not subscribed to the developer list you can send them
> privately to me at 'eldiener at tropicsoft dot com'. If so, please let me
> know whether or not you'd like your review to be forwarded to the list.
>
> For your review you may wish to consider the following questions:
>
>      - What is your evaluation of the design?
>      - What is your evaluation of the implementation?
>      - What is your evaluation of the documentation?
>      - What is your evaluation of the potential usefulness of the
>        library?
>      - Did you try to use the library?  With what compiler?  Did you
>        have any problems?
>      - How much effort did you put into your evaluation? A glance? A
>        quick reading? In-depth study?
>      - Are you knowledgeable about the problem domain?
>
> And finally, every review should attempt to answer this question:
>
>      - Do you think the library should be accepted as a Boost library?
>
> Be sure to say this explicitly so that your other comments don't obscure
> your overall opinion.
>
> Even if you do not wish to give a full review any technical comment
> regarding the library is welcome as part of the review period and will help
> me as the review manager decide whether the library should be accepted as a
> Boost library. Any questions about the use of the library are also welcome.
>
> I encourage all programmers with an interest or knowledge of signal/slot
> processing to be part of the review or discussion of the Synapse library as
> your time permits.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
12