Download links all broken?

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Download links all broken?

Boost - Dev mailing list
All the download links from www.boost.org appear to be currently broken
- which is to say no one can download Boost right now.

Ideas or fixes anyone?

Thanks, John.


--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Mon, Mar 30, 2020 at 12:37 PM John Maddock via Boost <
[hidden email]> wrote:

> All the download links from www.boost.org appear to be currently broken
> - which is to say no one can download Boost right now.
>
> Ideas or fixes anyone?
>

1. We know.
2. It will all start working again tomorrow. As we surpassed the DL limit
for the month.
3. Someone could set up a distributed web cache of some sort to fix this in
the long run?
4. People should stop using bintray for CI test jobs.


--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Mon, 30 Mar 2020 at 18:37, John Maddock via Boost <[hidden email]>
wrote:

> All the download links from www.boost.org appear to be currently broken
> - which is to say no one can download Boost right now.
>
> Ideas or fixes anyone?
>
> Thanks, John.
>
>
> --
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost


There was this recent thread about it:
http://boost.2283326.n4.nabble.com/Boost-download-links-seems-to-be-down-td4711786.html

Regards,

Pete

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list

> There was this recent thread about it:
> http://boost.2283326.n4.nabble.com/Boost-download-links-seems-to-be-down-td4711786.html

Ah, got it, and my word we're using a lot of bandwidth!

Thanks, John.



--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
CI should use tools/boostdep/depinst/depinst.py to grab only the needed
repos from github.

On Mon, Mar 30, 2020 at 11:03 AM John Maddock via Boost <
[hidden email]> wrote:

>
> > There was this recent thread about it:
> >
> http://boost.2283326.n4.nabble.com/Boost-download-links-seems-to-be-down-td4711786.html
>
> Ah, got it, and my word we're using a lot of bandwidth!
>
> Thanks, John.
>
>
>
> --
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
Top post ;-)

On Mon, Mar 30, 2020 at 1:10 PM Emil Dotchevski via Boost <
[hidden email]> wrote:

> CI should use tools/boostdep/depinst/depinst.py to grab only the needed
> repos from github.
>

Some of us test against whole Boost releases and the release archives are
the easiest to deal with. Yes, I'm partly to blame :-) But in my defense, I
did make changes to use git clones and the release tags. Even if it cost me
many minutes of CI time to do it.

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Mon, 30 Mar 2020 at 20:13, Rene Rivera via Boost
<[hidden email]> wrote:

> On Mon, Mar 30, 2020 at 1:10 PM Emil Dotchevski via Boost <[hidden email]> wrote:
>
> > CI should use tools/boostdep/depinst/depinst.py to grab only the needed
> > repos from github.
> >
>
> Some of us test against whole Boost releases and the release archives are
> the easiest to deal with. Yes, I'm partly to blame :-) But in my defense, I
> did make changes to use git clones and the release tags. Even if it cost me
> many minutes of CI time to do it.

Is it possible to mirror release archives on GitHub Releases or SF.net or both?

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 2020-03-30 21:13, Rene Rivera via Boost wrote:
>
> Some of us test against whole Boost releases and the release archives are
> the easiest to deal with. Yes, I'm partly to blame :-) But in my defense, I
> did make changes to use git clones and the release tags. Even if it cost me
> many minutes of CI time to do it.

Did you try using (a) parallel checkouts and (b) shallow git clone?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Mon, Mar 30, 2020 at 2:05 PM Andrey Semashev via Boost <
[hidden email]> wrote:

> On 2020-03-30 21:13, Rene Rivera via Boost wrote:
> >
> > Some of us test against whole Boost releases and the release archives are
> > the easiest to deal with. Yes, I'm partly to blame :-) But in my
> defense, I
> > did make changes to use git clones and the release tags. Even if it cost
> me
> > many minutes of CI time to do it.
>
> Did you try using (a) parallel checkouts and (b) shallow git clone?
>

Didn't try parallel... But did try shallow. For submodules, shallow clones
are not supported by GitHub.

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Le lundi 30 mars 2020 à 12:40 -0500, Rene Rivera via Boost a écrit :

> On Mon, Mar 30, 2020 at 12:37 PM John Maddock via Boost <
> [hidden email]> wrote:
>
> > All the download links from www.boost.org appear to be currently
> > broken
> > - which is to say no one can download Boost right now.
> >
> > Ideas or fixes anyone?
> >
>
> 1. We know.
> 2. It will all start working again tomorrow. As we surpassed the DL
> limit for the month.

Works for me as of today, although I don't know why.

> 3. Someone could set up a distributed web cache of some sort to fix
> this in the long run?

The usual solution to this is bittorrent. Providing magnet links can be
done quite easily (i can do it if there is some need for it).

Regards,
Julien


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 30.03.20 19:40, Rene Rivera via Boost wrote:
> On Mon, Mar 30, 2020 at 12:37 PM John Maddock via Boost <
> [hidden email]> wrote:
>
> 4. People should stop using bintray for CI test jobs.

you mean this for instance?
https://github.com/conan-community/conan-boost/blob/release/1.71.0/conanfile.py#L150

or this?
https://github.com/conan-io/conan-center-index/blob/master/recipes/boost/all/conandata.yml#L2-L22

TBH, if a package manager is pointing to bintray, and if there is a
limit on the download volume, something will fail (whether this is a CI
job or a human being pulling the libs).

Maybe it is time to reconsider this, for instance:
https://help.github.com/en/github/administering-a-repository/about-releases

In particular:
"Each file included in a release must be under 2 GB. There is no limit
on the total size of a release, nor bandwidth usage."

Raffi


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
John Maddock wrote:
> All the download links from www.boost.org appear to be currently broken -
> which is to say no one can download Boost right now.
>
> Ideas or fixes anyone?

Our Bintray quota is apparently 32 TB, which isn't quite enough.

I second the suggestion that people should stop downloading Boost in their
CI. It's not clear how we can effect such a change though. :-)


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Mon, Mar 30, 2020 at 3:38 PM Raffi Enficiaud via Boost <
[hidden email]> wrote:

> On 30.03.20 19:40, Rene Rivera via Boost wrote:
> > On Mon, Mar 30, 2020 at 12:37 PM John Maddock via Boost <
> > [hidden email]> wrote:
> >
> > 4. People should stop using bintray for CI test jobs.
>
> you mean this for instance?
>
> https://github.com/conan-community/conan-boost/blob/release/1.71.0/conanfile.py#L150
>
> or this?
>
> https://github.com/conan-io/conan-center-index/blob/master/recipes/boost/all/conandata.yml#L2-L22


No. Which is why I used the word "test" in my statement.

TBH, if a package manager is pointing to bintray, and if there is a
> limit on the download volume, something will fail (whether this is a CI
> job or a human being pulling the libs).
>

Yep.

Maybe it is time to reconsider this, for instance:
> https://help.github.com/en/github/administering-a-repository/about-releases
>
> In particular:
> "Each file included in a release must be under 2 GB. There is no limit
> on the total size of a release, nor bandwidth usage."
>

Someone would have to step up and write the automation scripts to deliver
the same functionality as bintray.

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On 30.03.20 22:43, Rene Rivera via Boost wrote:
> On Mon, Mar 30, 2020 at 3:38 PM Raffi Enficiaud via Boost <
> [hidden email]> wrote:

>> In particular:
>> "Each file included in a release must be under 2 GB. There is no limit
>> on the total size of a release, nor bandwidth usage."
>>
>
> Someone would have to step up and write the automation scripts to deliver
> the same functionality as bintray.
>

I am an automation freak. Where should it be plugged to?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Mon, Mar 30, 2020 at 3:48 PM Raffi Enficiaud via Boost <
[hidden email]> wrote:

> On 30.03.20 22:43, Rene Rivera via Boost wrote:
> > On Mon, Mar 30, 2020 at 3:38 PM Raffi Enficiaud via Boost <
> > [hidden email]> wrote:
>
> >> In particular:
> >> "Each file included in a release must be under 2 GB. There is no limit
> >> on the total size of a release, nor bandwidth usage."
> >>
> >
> > Someone would have to step up and write the automation scripts to deliver
> > the same functionality as bintray.
> >
>
> I am an automation freak. Where should it be plugged to?
>

 Some place here.. <
https://github.com/boostorg/release-tools/blob/develop/ci_boost_release.py#L277
>
Information on the tail end of the process is here: <
https://github.com/boostorg/wiki/wiki/Releases%3A-Preparing>

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
>> I am an automation freak. Where should it be plugged to?
>>
>   Some place here.. <
> https://github.com/boostorg/release-tools/blob/develop/ci_boost_release.py#L277
> Information on the tail end of the process is here: <
> https://github.com/boostorg/wiki/wiki/Releases%3A-Preparing>

FWIW: I recently set up automated releases on Github via Github Actions
which is triggered on tags. You can do pretty much anything and then
just hand it over to the API.
I think the same is very much possible for general boost releases, so if
this is set up in the super-repo and triggered on tags it would likely
do what we want.

See
https://github.com/boostorg/nowide/blob/develop/.github/workflows/release.yml

Note how I convert the code and create documentation before tarring and
uploading. In case of Boost it would likely be a submodule update, then tar.

Regards,
Alex




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 30.03.20 22:58, Rene Rivera via Boost wrote:

> On Mon, Mar 30, 2020 at 3:48 PM Raffi Enficiaud via Boost <
> [hidden email]> wrote:
>
>> On 30.03.20 22:43, Rene Rivera via Boost wrote:
>>> On Mon, Mar 30, 2020 at 3:38 PM Raffi Enficiaud via Boost <
>>> [hidden email]> wrote:
>>
>>>> In particular:
>>>> "Each file included in a release must be under 2 GB. There is no limit
>>>> on the total size of a release, nor bandwidth usage."
>>>>
>>>
>>> Someone would have to step up and write the automation scripts to deliver
>>> the same functionality as bintray.
>>>
>>
>> I am an automation freak. Where should it be plugged to?
>>
>
>  Some place here.. <
> https://github.com/boostorg/release-tools/blob/develop/ci_boost_release.py#L277
>>
> Information on the tail end of the process is here: <
> https://github.com/boostorg/wiki/wiki/Releases%3A-Preparing>
>

This is the first step: https://github.com/boostorg/release-tools/pull/16

Then it is a matter of preference:

* either I port the other files to python3 with some risk of breaking
things, and integrate things at the python level
* either I call this script as an external process

Best,
Raffi


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
John Maddock wrote:

> Ideas or fixes anyone?

Other ideas notwithstanding, one relatively trivial thing we could do is
stop publishing .zip and .tar.gz, and switch from .tar.bz2 to .tar.xz. This
will cut the bandwidth in half. (It's only a palliative measure, which
doesn't mean it can't help.)


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Tue, Mar 31, 2020 at 9:10 AM Peter Dimov via Boost <[hidden email]>
wrote:
> Other ideas notwithstanding, one relatively trivial thing we could do is
> stop publishing .zip and .tar.gz, and switch from .tar.bz2 to .tar.xz.
This
> will cut the bandwidth in half. (It's only a palliative measure, which
> doesn't mean it can't help.)

It will help tremendously, as it will break all the links people have in CI
scripts. :)

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Download links all broken?

Boost - Dev mailing list
On Tue, 31 Mar 2020 at 19:31, Emil Dotchevski via Boost
<[hidden email]> wrote:
> On Tue, Mar 31, 2020 at 9:10 AM Peter Dimov via Boost <[hidden email]> wrote:
> > Other ideas notwithstanding, one relatively trivial thing we could do is
> > stop publishing .zip and .tar.gz, and switch from .tar.bz2 to .tar.xz.
> This
> > will cut the bandwidth in half. (It's only a palliative measure, which
> > doesn't mean it can't help.)
>
> It will help tremendously, as it will break all the links people have in CI
> scripts. :)

They will arrive here reporting the issues.
A rare opportunity to stream direct education :)

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
12