[Beast] Downloading multiple urls in parallel

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Beast] Downloading multiple urls in parallel

Boost - Users mailing list

Hi,

 

Are there functions within beast to allow me to start downloading multiple url’s in parallel and then handle each returned result with a different continuation function?

Any help appreciated.

Kind regards

Sean.


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Beast] Downloading multiple urls in parallel

Boost - Users mailing list
On Thu, Sep 13, 2018 at 10:49 AM Sean Farrow via Boost-users
<[hidden email]> wrote:
> Are there functions within beast to allow me to start downloading multiple url’s in parallel and then handle each returned result with a different continuation function?

Yes. This is accomplished by calling `boost::beast::http::async_read`
on two or more different instances of connected sockets.

Regards
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Beast] Downloading multiple urls in parallel

Boost - Users mailing list
Hi,

Are there any examples in the docs as to how to do this?
Does the http client do this under the hood?
Kind regards
Sean.

-----Original Message-----
From: Boost-users <[hidden email]> On Behalf Of Vinnie Falco via Boost-users
Sent: 13 September 2018 18:58
To: [hidden email]
Cc: Vinnie Falco <[hidden email]>
Subject: Re: [Boost-users] [Beast] Downloading multiple urls in parallel

On Thu, Sep 13, 2018 at 10:49 AM Sean Farrow via Boost-users <[hidden email]> wrote:
> Are there functions within beast to allow me to start downloading multiple url’s in parallel and then handle each returned result with a different continuation function?

Yes. This is accomplished by calling `boost::beast::http::async_read` on two or more different instances of connected sockets.

Regards
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Beast] Downloading multiple urls in parallel

Boost - Users mailing list
On Thu, Sep 13, 2018 at 12:13 PM Sean Farrow via Boost-users
<[hidden email]> wrote:
> Are there any examples in the docs as to how to do this?

This program crawls the list of the 10,000 most popular websites
indexed by Alexa and makes many connections simultaneously:

<https://github.com/boostorg/beast/tree/develop/example/http/client/crawl>

Regards
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users