boost thread strange behaviour

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

boost thread strange behaviour

Boost - Dev mailing list
hi ,
i am trying to learn executors and executors thread pools using with
boost::async
but i found strange behabiour
when i make thread pool adaptor ea and submit tasks to it and make ea00 and
submit tasks to it
if i started ea with async , it automatically starts ea00
the attached file has the code
i thought that async will only start the passed thread pool"adaptor"
But i think there is underlying thing which gather all threads of different
pools into single entity and starts them with any call to boost::async
is this a bug???
i run this from windows 7 64 by visual studio 2017


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

executor_bug.cpp (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
On Sat, 29 Feb 2020 at 08:09, Adham Sabry via Boost <[hidden email]>
wrote:

> hi ,
> i am trying to learn executors and executors thread pools using with
> boost::async
>

I like this (HO) Concurrency.TS implementation: https://stlab.cc/
(Adobe/Sean Parent/Felix Petriconi).

The release has at least one bug, which is fixed in master (1.6), 2.0 is on
the way, I am guessing it's a milestone.

degski
--
@realdegski
https://brave.com/google-gdpr-workaround/
"We value your privacy, click here!" Sod off! - degski
"Anyone who believes that exponential growth can go on forever in a finite
world is either a madman or an economist" - Kenneth E. Boulding
"Growth for the sake of growth is the ideology of the cancer cell" - Edward
P. Abbey

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
thanks .i have already studied boost thread and i just want to make sure
that this is bug so as to avoid using boost::async with thread pool

On Sat, Feb 29, 2020 at 4:45 PM degski via Boost <[hidden email]>
wrote:

> On Sat, 29 Feb 2020 at 08:09, Adham Sabry via Boost <[hidden email]
> >
> wrote:
>
> > hi ,
> > i am trying to learn executors and executors thread pools using with
> > boost::async
> >
>
> I like this (HO) Concurrency.TS implementation: https://stlab.cc/
> (Adobe/Sean <https://stlab.cc/(Adobe/Sean> Parent/Felix Petriconi).
>
> The release has at least one bug, which is fixed in master (1.6), 2.0 is on
> the way, I am guessing it's a milestone.
>
> degski
> --
> @realdegski
> https://brave.com/google-gdpr-workaround/
> "We value your privacy, click here!" Sod off! - degski
> "Anyone who believes that exponential growth can go on forever in a finite
> world is either a madman or an economist" - Kenneth E. Boulding
> "Growth for the sake of growth is the ideology of the cancer cell" - Edward
> P. Abbey
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
On Sat, 29 Feb 2020 at 09:51, Adham Sabry via Boost <[hidden email]>
wrote:

> thanks .i have already studied boost thread and i just want to make sure
> that this is bug so as to avoid using boost::async with thread pool
>

The library is a prototype of things that expectedly will be part of C++23,
so by the time C++23 arrives (the library tracks the developing std, so by
the time its frozen, it IS the std), all you need to do, is change the
namespace and the 'include', and delete the 'old' lib (headers).

degski
--
@realdegski
https://brave.com/google-gdpr-workaround/
"We value your privacy, click here!" Sod off! - degski
"Anyone who believes that exponential growth can go on forever in a finite
world is either a madman or an economist" - Kenneth E. Boulding
"Growth for the sake of growth is the ideology of the cancer cell" - Edward
P. Abbey

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 29/02/2020 23:22, Adham Sabry wrote:

> i am trying to learn executors and executors thread pools using with
> boost::async
> but i found strange behabiour
> when i make thread pool adaptor ea and submit tasks to it and make ea00 and
> submit tasks to it
> if i started ea with async , it automatically starts ea00
> the attached file has the code
> i thought that async will only start the passed thread pool"adaptor"
> But i think there is underlying thing which gather all threads of different
> pools into single entity and starts them with any call to boost::async
> is this a bug???
> i run this from windows 7 64 by visual studio 2017

Perhaps I'm missing something, since I haven't played with Boost
executors before.  But this looks entirely normal to me.

Creating a basic_thread_pool will presumably start some background
threads that are sleeping on work.

When you submit_some(ea00) it queues some work on ea00 that will execute
at some unspecified later time (but typically relatively soon), whenever
one of the sleeping threads in ea00 wakes up.

Your use of boost::async(ea, &f1) is entirely unrelated to this --
except that it causes the test_executor_adaptor() thread to pause long
enough that you're not destroying the ea00 thread pool before it gets
around to executing the work.


You can see this by commenting out the boost::async and just sleeping
instead; the submitted work gets called in the background.


There is no "gathering from multiple pools into a single entity".  Each
thread pool is entirely separate at all times.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
thanks for your reply
where can i find the duration of sleep that basic_thread_pool does then
wakes up???it sleeps till some methods like submit are called.i thought
submit just make tasks without starting the thread and async is the one
which launch thread but i was totally wrong drived by asio paradigm
what is the reason of wake up??is it timed internally or it needs explicit
launching?we can do both using schedulers although i do not know how
also why async leads to pausing test_executor_adaptor???here async return
is assigned to variable t2 so it will not block in async destructor,but
will work asynchronously till the scope is ended and then block in
destructor of async???this i do not know answer to it????why async pause ???
what do you mean by sleeping?you meant to sleep between submit calls so
they are finished
thanks again you helped me in 3 years misunderstood behavior


On Wed, Mar 4, 2020 at 8:25 AM Gavin Lambert via Boost <
[hidden email]> wrote:

> On 29/02/2020 23:22, Adham Sabry wrote:
> > i am trying to learn executors and executors thread pools using with
> > boost::async
> > but i found strange behabiour
> > when i make thread pool adaptor ea and submit tasks to it and make ea00
> and
> > submit tasks to it
> > if i started ea with async , it automatically starts ea00
> > the attached file has the code
> > i thought that async will only start the passed thread pool"adaptor"
> > But i think there is underlying thing which gather all threads of
> different
> > pools into single entity and starts them with any call to boost::async
> > is this a bug???
> > i run this from windows 7 64 by visual studio 2017
>
> Perhaps I'm missing something, since I haven't played with Boost
> executors before.  But this looks entirely normal to me.
>
> Creating a basic_thread_pool will presumably start some background
> threads that are sleeping on work.
>
> When you submit_some(ea00) it queues some work on ea00 that will execute
> at some unspecified later time (but typically relatively soon), whenever
> one of the sleeping threads in ea00 wakes up.
>
> Your use of boost::async(ea, &f1) is entirely unrelated to this --
> except that it causes the test_executor_adaptor() thread to pause long
> enough that you're not destroying the ea00 thread pool before it gets
> around to executing the work.
>
>
> You can see this by commenting out the boost::async and just sleeping
> instead; the submitted work gets called in the background.
>
>
> There is no "gathering from multiple pools into a single entity".  Each
> thread pool is entirely separate at all times.
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
On 6/03/2020 22:47, Adham Sabry wrote:

> where can i find the duration of sleep that basic_thread_pool does then
> wakes up???it sleeps till some methods like submit are called.i thought
> submit just make tasks without starting the thread and async is the one
> which launch thread but i was totally wrong drived by asio paradigm
> what is the reason of wake up??is it timed internally or it needs explicit
> launching?we can do both using schedulers although i do not know how
> also why async leads to pausing test_executor_adaptor???here async return
> is assigned to variable t2 so it will not block in async destructor,but
> will work asynchronously till the scope is ended and then block in
> destructor of async???this i do not know answer to it????why async pause ???
> what do you mean by sleeping?you meant to sleep between submit calls so
> they are finished

There is no specific time.  The threadpool threads are asleep waiting
for work to be submitted.

As soon as you actually submit work (whether directly via submit() or
indirectly via async()), the thread is eligible to wake up (in fact it
might wake up and complete the work before submit even returns) -- that
part is entirely up to your OS's thread scheduler.

If it helps: async() is essentially internally just a call to submit()
with some additional wrapping to give you a future representing the
completion of the operation.  [This is not precisely accurate but it's a
good way to think about it for understanding purposes.]

My point about the async calls is that because you're calling async and
then future.get(), you're blocking the original thread waiting for
something else, which allows the threadpool work to complete in the
background, regardless of which threadpool the work was submitted to.

If you completely removed the future.get() calls (even if you still
called async), then you might end up destroying the threadpool before it
actually did the work, thus it would appear that submit() did nothing.
(And the asyncs would also end up doing nothing.)  But because like I
said this is up to your OS's thread scheduler, it might sometimes make
some progress on some of the work before actually being destroyed, which
can be confusing.

If you have a sleep (or any other work running long enough) in there, so
that you don't destroy the threadpool too soon, then it will complete in
the background.

Usually you need to structure your code in a way that assures that the
things you want to complete have actually completed before you destroy
them.  Waiting on futures is an easy way to do that in a toy example,
but it can get more complex in a real application.  (I personally
consider futures almost entirely useless in a real application -- but
that doesn't make learning about them useless.)

I'd recommend finding a good book on the topic to read.  (Though books
specifically on executors are probably rare, as it's relatively new
ground.)  Threading and async in C++ can be very hard to get correct
unless you have the fundamentals down strongly.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: boost thread strange behaviour

Boost - Dev mailing list
Mere moments ago, quoth I:

> My point about the async calls is that because you're calling async and
> then future.get(), you're blocking the original thread waiting for
> something else, which allows the threadpool work to complete in the
> background, regardless of which threadpool the work was submitted to.
>
> If you completely removed the future.get() calls (even if you still
> called async), then you might end up destroying the threadpool before it
> actually did the work, thus it would appear that submit() did nothing.
> (And the asyncs would also end up doing nothing.)  But because like I
> said this is up to your OS's thread scheduler, it might sometimes make
> some progress on some of the work before actually being destroyed, which
> can be confusing.
>
> If you have a sleep (or any other work running long enough) in there, so
> that you don't destroy the threadpool too soon, then it will complete in
> the background.

For another example, if you submitted some long-running tasks to ea and
some short-running tasks to ea00, and then only waited on the futures
from ea00, then they'd complete quickly and you'd exit those blocks and
destroy both threadpools -- but ea's work is very likely to be
incomplete or not even started yet.

Conversely, submitting the same tasks to the same pools but instead
waiting on futures from ea instead would increase your chances that both
pools completed their work but it's still not entirely impossible that
only ea finished and ea00 still had some unfinished work.  (You'd have
to have quite poor luck with the scheduler though.)

Finally, submitting a task to ea, then async'ing a task also to ea, then
waiting on that second task's future: this would guarantee that both
tasks finished *only if there was exactly one thread in the pool*.  If
there's more than one thread then you could still get a situation where
the second task finishes before the first.

If you want to guarantee that everything completes before you stop then
you need to wait for completion of everything in some fashion or another.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost