[review queue] What to do about the library review queue?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
158 messages Options
1234 ... 8
Reply | Threaded
Open this post in threaded view
|

[review queue] What to do about the library review queue?

Boost - Dev mailing list
Dear Boost,

I see that new candidate Boost libraries entering the review queue have
exploded in recent years, with no less than *twenty-three* proposed
libraries awaiting a review.

As the ongoing strength and vitality of Boost is inextricably linked to
new growth, I think that waiting around for years for someone to
volunteer to manage a review is not healthy. If a library author has
invested the very significant effort to develop a Boost-quality library,
the least Boost can do is to try harder to provide timely reviews and
that means persuading more people to volunteer to manage reviews.

In the past people have argued that for every library you submit for
review, you should manage a review in return. Myself, Antony and a few
others have adhered to that rule, and if every library author did so
there would be no outstanding review queue. However there are problems
in that in itself in terms of moral hazard, and also because the review
manager needs to usually be fairly expert in a library being reviewed,
else it can be very hard to judge the worth and validity of reviews. A
shortage of suitably expert review managers will always be a problem for
some types of library.


I therefore ask boost-dev what to do? Some options:

1. Pay US$1000 (one thousand) dollars to each person who manages a
review. In case you're worried Boost doesn't have the money, it does in
spades, that's not a problem. For $23,000 we could clear the current
review queue assuming none of the problems mentioned yet.

2. Pay US$1000 dollars to the manager and 2x $500 dollar payments to
those writing the top two most useful reviews as judged by the review
manager. That makes the cost $2000 per library accepted or rejected, or
$46,000 to clear the current review queue.

3. In my own opinion from reviewing the review queue, a good 25% of the
libraries in the queue are not ready for review due to obvious glaring
deficiencies in the documentation or code. Spending a grand on those
libraries which will very obviously be rejected isn't worth the money.
What should we do about those? One approach could simply be to trust
review managers to not abuse the thousand dollar fee. Another could be
that before each new review, the prospective manager needs to write a
single line comment on why they did not choose the other libraries in
the queue and publish that here before starting a review. That would
quickly identify those libraries in the queue which a majority of
managers think have serious problems and could never pass any review. If
say a library in a queue accumulates three single line black marks, the
author might be encouraged to withdraw it.

4. Finally there is the problem of libraries of high quality, but not a
good fit for Boost because they are so esoteric and niche that nobody
could provide a useful review, and without useful reviews the review
manager can't really recommend acceptance. This will be an increasing
problem with time anyway as more of the low hanging C++ library fruit is
picked, but I suppose one could just kick that decision can down the
road and see if 2x $500 payments might help scare up more high quality
reviews.

5. We could try guilting more people into review managing, and redouble
banging the drum to scare up more volunteers.


I look forward to seeing what people think.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
On Tue, Mar 14, 2017 at 8:01 AM, Niall Douglas via Boost
<[hidden email]> wrote:
> I look forward to seeing what people think.

These are great ideas, and I am glad someone is bringing this up!

With respect to payments, let me offer some thoughts. Boost is used
around the world so it might be difficult to pay someone in Europe in
USDs. How about using cryptocurrency (Bitcoin for example)? This
solves the problem of receiving payment.

Where is this money coming from? Corporate sponsorships? I like it!

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
On 14/03/2017 12:20, Vinnie Falco via Boost wrote:

> On Tue, Mar 14, 2017 at 8:01 AM, Niall Douglas via Boost
> <[hidden email]> wrote:
>> I look forward to seeing what people think.
>
> These are great ideas, and I am glad someone is bringing this up!
>
> With respect to payments, let me offer some thoughts. Boost is used
> around the world so it might be difficult to pay someone in Europe in
> USDs. How about using cryptocurrency (Bitcoin for example)? This
> solves the problem of receiving payment.

Boost has a long standing payment system setup which can pay out in any
of a cheque, wire transfer, Paypal etc to any recipient in the world bar
those not permitted by US law.

It isn't handled directly by Boost, but rather by the SFF which manages
Boost's money for us.

> Where is this money coming from? Corporate sponsorships? I like it!

Boost has been accumulating more money than it spends for nearly a
decade now. I believe we are the wealthiest open source org in the SFF's
umbrella. However as has been found in the past, if there is a worthy
cause then a drum can be banged to scare up donations from corporates.
We as a community simply need to decide a consensus on what to do.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Hi,

> I think that waiting around for years for someone to
> volunteer to manage a review is not healthy.

Slitghtly out of topic: I volunteered for reviewing boost.nowide on
March 4th. Maybe I did not send my email to the right list (boost@)?
Maybe we should make it clearer how to volunteer?

Cheers,

Frédéric

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Tue, Mar 14, 2017 at 8:01 AM, Niall Douglas via Boost wrote:
> Dear Boost,
>

I personally hope none of the suggestions 1, 2, 3, or 4, are
implemented. I really worry about this desire to bring monetary value
or payment into the Boost review process.

I also have some concerns when I see the same person put forward
similar ideas that all revolve around paying or hiring individuals.

Niall, I understand you have good intentions, and I (like many others)
appreciate your administration of GSOC, but the repetition of these
ideas looks frighteningly like you want to address any employment or
financial needs that you have by getting Boost to hire you for more
things. Maybe I'm drawing too many conclusions from your blog
posts[1].

I'm curious: The libraries that too niche or esoteric for any review
manager to be interested (or reviewers to come forward) - to which
libraries are you referring?

In any case, I look forward to the mechanics of Boost being improved.
I hope they're improved by better means that do not involve solving
someone's employment needs.

Glen

[1] https://plus.google.com/+nialldouglas/posts/Ezjzxizm8Fp

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Tue, Mar 14, 2017 at 8:33 AM, Frédéric Bron via Boost wrote:
> Slitghtly out of topic: I volunteered for reviewing boost.nowide on
> March 4th. Maybe I did not send my email to the right list (boost@)?
> Maybe we should make it clearer how to volunteer?


Hi  Frédéric,

E-mail the review wizard, Ron (Ronald Garcia, (rxg at cs dot ubc dot
ca)), with your desire to manage the review.

And thanks!

Glen

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 14.03.2017 08:01, Niall Douglas via Boost wrote:
> Dear Boost,
>
> I see that new candidate Boost libraries entering the review queue have
> exploded in recent years, with no less than *twenty-three* proposed
> libraries awaiting a review.
>
> As the ongoing strength and vitality of Boost is inextricably linked to
> new growth,

Is it really ?

I maintain what I have been saying for many years: Boost (as an
organization) is crushing under its own weight. There are many other
things I would consider being important for its vitality (such as
changing its mode of organization - such as into an umbrella
organization of relatively autonomous projects). But pushing for
accelerated growth is certainly not among the things I would promote.

>  I think that waiting around for years for someone to
> volunteer to manage a review is not healthy.

I always thought that the "self-organized" nature of Boost processes
(including the review process) is a means to select the "generally
useful" submissions from the "esoteric corner case" ones. In other
words: if a library submitter can't gather enough interest in the wider
community to find reviewers and a review manager, it implies the
submitted project is finally not a good candidate for addition.

*That* is part of the strength of Boost (and of Open Source projects in
general, for that matter): It really represents the community's needs,
rather than what any particular party pushes through.


Best,
        Stefan

--

      ...ich hab' noch einen Koffer in Berlin...


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
> I personally hope none of the suggestions 1, 2, 3, or 4, are
> implemented. I really worry about this desire to bring monetary value
> or payment into the Boost review process.
>
> I also have some concerns when I see the same person put forward
> similar ideas that all revolve around paying or hiring individuals.

Every single major open source org has eventually ended up having to pay
people to do the admin which keeps that org working if they wish to keep
growing. And that's an empirical hard fact.

Boost can decide to not do that, and to date it has decided to not do
that. All empirical evidence suggests that that decision will gum up
growth and put a natural cap on Boost's size and relevance to latest C++.

A few years ago Boost had a problem of no new libraries at all in three
years. That problem has been fixed, and I'll immediately admit surprise
tinged with gratitude that it didn't require paying people to fix it.

An ongoing current problem is lack of maintenance. A system of corporate
sponsorship of maintenance has helped address that, and in some places
it's worked well, but not in others. But I'll also admit surprise tinged
with gratitude that anything non-monetary worked here at all.

A big surprise to me is how much has been done on automated testing
without paying people for it (though we did pay for some of their
hardware and in some cases for renting infrastructure). My hat is off in
thanks to those in question. Thank you.

So, one could entirely argue and with good supporting evidence that you
don't need to pay people to do this stuff, and you would be correct.

However that does not mean that if you did pay people, that it wouldn't
have gone a lot better again. And there is a much stronger argument that
the fact I raised payment for stuff and people reacted violently against
it helped get people to volunteer to solve the problem. So in a way,
it's all good and it all helps.

> Niall, I understand you have good intentions, and I (like many others)
> appreciate your administration of GSOC, but the repetition of these
> ideas looks frighteningly like you want to address any employment or
> financial needs that you have by getting Boost to hire you for more
> things. Maybe I'm drawing too many conclusions from your blog
> posts[1].

Heh. Funnily enough I'm coming towards the end of my active
participation in C++, you've got about a year left of me being annoying
before I step back majorly. Even this year you won't be seeing me at
CppCon this year as I begin to step back.

(To explain, we've been having children since 2013 for which my wife had
to give up her career and I was the sole earner. She wishes to return to
her career soon, and for which all my non-work time will be needed to
support her, so no more outside-of-work coding for me for a few years
until she's reestablished and back to earning. I'll be resigning from
all things I volunteer for including all mailing lists, and doing
nothing but basic maintenance on my open source libraries, no new code
nor new features)

You are right though that in *ideological* terms I think Boost and the
C++ Standard Foundation ought to hire people to work on the wider C++
ecosystem, just as other major programming languages do. Whether that
should be me or someone else should be the result of a competitive
public tendering process.

> I'm curious: The libraries that too niche or esoteric for any review
> manager to be interested (or reviewers to come forward) - to which
> libraries are you referring?

Safe Numerics is an excellent example. I've never used anything like
such a thing, and despite it looking to me like a great library (and I
did look over it in depth including its source code), I was aware enough
of how little I knew about that domain. I definitely could not have
review managed that library, I don't know enough about that domain at
all to have a useful opinion.

> In any case, I look forward to the mechanics of Boost being improved.
> I hope they're improved by better means that do not involve solving
> someone's employment needs.

Nobody is going to be employed permanently on bits of piece work, it's
an awful life not knowing where rent will come from. Any of the remote
working consultants I know prefer nice, long contracts where available,
ideally six months or a year or longer.

That said, when you're between those long contracts and during when
you're negotiating new contracts (which can take two months like the one
I'm negotiating right now has), if there were bits of small piece work
available, I'm sure I and many others in this profession would put a bid
in to fill the short gap at a very discounted hourly rate. That would be
excellent value for money for the C++ ecosystem, and therefore a very
rational system to establish.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 14/03/2017 12:57, Stefan Seefeld via Boost wrote:
> But pushing for
> accelerated growth is certainly not among the things I would promote.

I know what you're saying. But remember the "crushing under its own
weight" is an active choice of not reaching consensus on anything
different by the community. We, as a group, choose to be crushed. We
could choose different. We have the resources.

I try to keep asking: "what is best for the larger C++ ecosystem?"

* Is high quality peer review valuable? YES

* Is a staging ground before standardisation valuable? YES

So let's make Boost do those things as best we can. That means admitting
as many high quality C++ libraries as we can, and encouraging as many
people as possible to consider submitting their C++ library to Boost. I
am afraid that means growing as fast as we can.

> I always thought that the "self-organized" nature of Boost processes
> (including the review process) is a means to select the "generally
> useful" submissions from the "esoteric corner case" ones. In other
> words: if a library submitter can't gather enough interest in the wider
> community to find reviewers and a review manager, it implies the
> submitted project is finally not a good candidate for addition.

It's a valid point that I have significant sympathy with.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list

> I see that new candidate Boost libraries entering the review queue have
> exploded in recent years, with no less than *twenty-three* proposed
> libraries awaiting a review.
>
> As the ongoing strength and vitality of Boost is inextricably linked to
> new growth, I think that waiting around for years for someone to
> volunteer to manage a review is not healthy. If a library author has
> invested the very significant effort to develop a Boost-quality library,
> the least Boost can do is to try harder to provide timely reviews and
> that means persuading more people to volunteer to manage reviews.
>
> In the past people have argued that for every library you submit for
> review, you should manage a review in return. Myself, Antony and a few
> others have adhered to that rule, and if every library author did so
> there would be no outstanding review queue. However there are problems
> in that in itself in terms of moral hazard, and also because the review
> manager needs to usually be fairly expert in a library being reviewed,
> else it can be very hard to judge the worth and validity of reviews. A
> shortage of suitably expert review managers will always be a problem for
> some types of library.
>
>
> I therefore ask boost-dev what to do? Some options:
>
> 1. Pay US$1000 (one thousand) dollars to each person who manages a
> review. In case you're worried Boost doesn't have the money, it does in
> spades, that's not a problem. For $23,000 we could clear the current
> review queue assuming none of the problems mentioned yet.
>
> 2. Pay US$1000 dollars to the manager and 2x $500 dollar payments to
> those writing the top two most useful reviews as judged by the review
> manager. That makes the cost $2000 per library accepted or rejected, or
> $46,000 to clear the current review queue.
>
> 3. In my own opinion from reviewing the review queue, a good 25% of the
> libraries in the queue are not ready for review due to obvious glaring
> deficiencies in the documentation or code. Spending a grand on those
> libraries which will very obviously be rejected isn't worth the money.
> What should we do about those? One approach could simply be to trust
> review managers to not abuse the thousand dollar fee. Another could be
> that before each new review, the prospective manager needs to write a
> single line comment on why they did not choose the other libraries in
> the queue and publish that here before starting a review. That would
> quickly identify those libraries in the queue which a majority of
> managers think have serious problems and could never pass any review. If
> say a library in a queue accumulates three single line black marks, the
> author might be encouraged to withdraw it.
>
> 4. Finally there is the problem of libraries of high quality, but not a
> good fit for Boost because they are so esoteric and niche that nobody
> could provide a useful review, and without useful reviews the review
> manager can't really recommend acceptance. This will be an increasing
> problem with time anyway as more of the low hanging C++ library fruit is
> picked, but I suppose one could just kick that decision can down the
> road and see if 2x $500 payments might help scare up more high quality
> reviews.
>
> 5. We could try guilting more people into review managing, and redouble
> banging the drum to scare up more volunteers.

Let me be upfront. I'm strictly against correlating the review process with
money in any way.

Boost's mission is to foster the development of widely applicable, high
quality, and freely available C++ libraries. It also provides a uniform
platform allowing to distribute those libraries. Over the years it has
maintained a widely accepted quality standard which makes it desirable for
people to submit libraries and to become Boost library authors.

None of these points involve the need to increase the amount of accepted
libraries at any cost. As others have pointed out, Boost's livelihood does
not directly depend on the number of accepted libraries but just on their
quality. Adding money to the mix favors adding libraries at all cost,
non-withstanding their quality or real-world-need, emphasizing the
commercial aspect.

Having the review process being volunteer-driven guarantees a) a real-world
need for the library under review, b) fairness of the decision, c) a high
quality of the review, d) direct interest in organizing the review by the
review manager (otherwise he/she wouldn't do it).

Starting to add money to the volunteer-process of reviewing libraries will
set off the balance between quality and usability. It increases the review
managers interest in doing a review (point (d) from the list above), but
directly diminishes the importance of (a), (b), and (c).

Letting Boost pay for reviews implies that Boost as an organization is
interested in adding as many libraries as possible, regardless of real
interest in the community in a particular library. I think this is not in
the interest of the Boost organization.

Adding money to the process will immediately destroy the transparency of the
review process and it most likely will be detrimental to its quality. Both
are things we should avoid by all means possible.

This would also raise a lot of questions I wouldn't even know how to start
answering. Like:

- What's next? Letting library authors pay for their library being reviewed?
After all THEY are the most interested parties in adding their work to
Boost...
- Or perhaps accepting 'donations' from companies earmarked for paying a
review manager, further skewing the review process?
- Will every review manager receive the money? Regardless of the quality of
how the review is managed? What would be the criteria for a review manager
doing a good enough job to receive the payment? Who decides on this? How
many reviews would be a single person be allowed to perform?
- Would previous review managers receive an equally generous payment for the
libraries reviewed in the past? If yes - why, if no - why not?
- and many more...

In short, while I understand what issues this proposal is trying to solve, I
strongly believe it chooses inadequate means of doing so.

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
> None of these points involve the need to increase the amount of accepted
> libraries at any cost. As others have pointed out, Boost's livelihood does
> not directly depend on the number of accepted libraries but just on their
> quality. Adding money to the mix favors adding libraries at all cost,
> non-withstanding their quality or real-world-need, emphasizing the
> commercial aspect.

As you usual, you deliberately misquote or cherry pick quote people and
then make a big song and dance about your reinterpretation of what they
said. It's tiring and irritating.

I never said payment for adding new libraries. I did say payment for new
library REVIEWS and especially the **ADMIN** of reviewing them.

Rejection of a submitted library is just fine. Letting submitted
libraries stew for up to SEVEN years before getting reviewed is
UNACCEPTABLE if Boost is to remain even remotely relevant.

I now snip the usual Fear, Uncertainty and Doubt you like to sow Hartmut
whenever actual REAL CHANGE is proposed or discussed, leaving us with
this which had some value:

> This would also raise a lot of questions I wouldn't even know how to start
> answering. Like:
>
> - What's next? Letting library authors pay for their library being reviewed?
> After all THEY are the most interested parties in adding their work to
> Boost...

What has that to do with anything being proposed? Nothing.

> - Or perhaps accepting 'donations' from companies earmarked for paying a
> review manager, further skewing the review process?

What has that to do with anything being proposed? Nothing. Totally
separate matter.

> - Will every review manager receive the money? Regardless of the quality of
> how the review is managed? What would be the criteria for a review manager
> doing a good enough job to receive the payment? Who decides on this? How
> many reviews would be a single person be allowed to perform?

These are good questions. Plenty of possibilities. None are showstoppers
to the idea, not even remotely.

> - Would previous review managers receive an equally generous payment for the
> libraries reviewed in the past? If yes - why, if no - why not?

Another good question, and again not insolvable in either direction. Not
a showstopper.

Niall

--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Louis Dionne
In reply to this post by Boost - Dev mailing list
Boost - Dev mailing list wrote
Dear Boost,

I see that new candidate Boost libraries entering the review queue have
exploded in recent years, with no less than *twenty-three* proposed
libraries awaiting a review.
Contrary to what some people have expressed, I do think that Boost should strive
to include more libraries (but not at any cost, of course). The reason is that
part of Boost's reason to be is being a workbench for libraries that aim to get
standardized, so we can get real world experience. This does not mean that we
should trade quality or general usefulness for quantity, but I think it wouldn't
harm to clear that review queue and get more potential candidates for standardization.
One problem with C++ is the lack of a rich ecosystem of libraries (look at Java!).
Boost can help address that.

I won't make any comment regarding using money to encourage review managers;
I don't have a formed opinion about this. However, I would be comfortable with
requiring Boost authors to manage a review within [some period  of time] of
their library being accepted. Independently of this, I have decided to manage
the review for CallableTraits.

I think we should edit the page containing the review queue to better advertise
that we're looking for review managers, and also explain how to become one.
Currently, you have to scroll all the way down and read text in small font.

Louis
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list

> > None of these points involve the need to increase the amount of accepted
> > libraries at any cost. As others have pointed out, Boost's livelihood
> does
> > not directly depend on the number of accepted libraries but just on
> their
> > quality. Adding money to the mix favors adding libraries at all cost,
> > non-withstanding their quality or real-world-need, emphasizing the
> > commercial aspect.
>
> As you usual, you deliberately misquote or cherry pick quote people and
> then make a big song and dance about your reinterpretation of what they
> said. It's tiring and irritating.

I don't remember quoting anything. I have simply written down my thoughts on
the topic.

> I never said payment for adding new libraries. I did say payment for new
> library REVIEWS and especially the **ADMIN** of reviewing them.
>
> Rejection of a submitted library is just fine. Letting submitted
> libraries stew for up to SEVEN years before getting reviewed is
> UNACCEPTABLE if Boost is to remain even remotely relevant.
>
> I now snip the usual Fear, Uncertainty and Doubt you like to sow Hartmut
> whenever actual REAL CHANGE is proposed or discussed, leaving us with
> this which had some value:

I'm not sure why you have shout at me and why you have to fall back to
personally attacking me (again!). I think I have kept (and deliberately so)
my response to a list of fundamental arguments why I think paying for
reviews wouldn't be a good idea. I have not (even remotely) said or implied
anything personal with regards to you. Neither did I put on my hat as a
Boost SC member, I simply explained how I see things.  

In fact, I would have written the same response regardless of who proposed
this.

Ok, I said enough - I'm out of this discussion.

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu


PS: Niall, I'm older than you and have seen more than one younger fellow
discredit himself because they thought they knew better, so please allow me
to voice a personal advise: it is my impression that you are carrying a
personal grudge because of things that have happened in the past. This
simply clouds your ability to assess other people's opinions with an open
mind, and to accept those as what they are - opinions. I think you would be
well advised to rethink your stance related to how to professionally behave
in a community where everybody is entitled to expressing his/her thoughts,
even more if those are well explained, and even if those contradict to your
own.





_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 03/14/2017 11:01 PM, Niall Douglas via Boost wrote:
> I think that waiting around for years for someone to
> volunteer to manage a review is not healthy.

I feel that Niall identified a legitimate and long-standing problem.
Niall stepped forward with his view how it potentially might be
addressed. It might be controversial and people might feel uneasy about
his particular suggestion. I do right now. However, now I am reading the
thread and see people focusing solely on criticizing his suggestion
without suggesting any alternatives they might consider more appropriate
or better. I feel that it is one-sided, unfair and unproductive.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 3/14/17 5:01 AM, Niall Douglas via Boost wrote:
> Dear Boost,
>
> I see that new candidate Boost libraries entering the review queue have
> exploded in recent years, with no less than *twenty-three* proposed
> libraries awaiting a review.
Right
>
> As the ongoing strength and vitality of Boost is inextricably linked to
> new growth, I think that waiting around for years for someone to
> volunteer to manage a review is not healthy. If a library author has
> invested the very significant effort to develop a Boost-quality library,
> the least Boost can do is to try harder to provide timely reviews and
> that means persuading more people to volunteer to manage reviews.
Right - it also means motivating more people to write reviews

> In the past people have argued that for every library you submit for
> review, you should manage a review in return. Myself, Antony and a few
> others have adhered to that rule, and if every library author did so
> there would be no outstanding review queue. However there are problems
> in that in itself in terms of moral hazard, and also because the review
> manager needs to usually be fairly expert in a library being reviewed,
> else it can be very hard to judge the worth and validity of reviews. A
> shortage of suitably expert review managers will always be a problem for
> some types of library.

All correct.  This suggestion has been floating around since at least
2010.  I think it's time to implement it.  I propose the follwing text
to be placed in the appropriate place.

"Only those who have managed a boost review can expect their library
submissions to be to be reviewed."

This clearly states the rule and allows for some exceptions.  If howard
hinnant want so submit a library - I'm not going to demand he have acted
previously as review manager - though on second tought, maybe I should -
he'd do a great job!!!

> I therefore ask boost-dev what to do? Some options:

<snip>

I'm not crazy about suggestions involving money transfer (though I could
always use some myself!).  Let's see how the above change works out and
than talk about it.

> 4. Finally there is the problem of libraries of high quality, but not a
> good fit for Boost because they are so esoteric and niche that nobody
> could provide a useful review, and without useful reviews the review
> manager can't really recommend acceptance. This will be an increasing
> problem with time anyway as more of the low hanging C++ library fruit is
> picked, but I suppose one could just kick that decision can down the
> road and see if 2x $500 payments might help scare up more high quality
> reviews.

On one hand, we could say that if there's no one qualified to manage the
review - where are we going to find qualified people to review it?

IRIC Alfred North Whitehead and Bertrand Russell worked 20 years on
Principia Mathematica.  They submitted it to a publisher and he couldn't
find anyone to review it.  He told them - If I can't find anyone to read
it, how many copies might I sell?  The ended up publishing it at their
own expense.  So all you library writers out there who are having
trouble getting your stuff reviewed, take comfort in the fact that
you're in good company! (on the other hand they had tenure) (source
http://www.logicomix.com/en/index.html)

But let's worry about that when the case comes up.

> 5. We could try guilting more people into review managing, and redouble
> banging the drum to scare up more volunteers.
Right - we've been doing that - how's that working?

I'm actually very concerned about the number of reviews.  I've made
efforts to address this --  See my Boost 2.0 video - particularly the
last couple of minutes.  But it is a separate question. Maybe open a
second thread on this.

To summarize, I think this is valid concern, I think the first idea to
address it is a good one, I think it's worth a try and I think the
decision to try it should be non-controversial.

Robert Ramey


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
On Tue, Mar 14, 2017 at 5:53 PM, Robert Ramey via Boost
<[hidden email]> wrote:
> "Only those who have managed a boost review can expect their library
> submissions to be to be reviewed."

I've thought about volunteering to be a review manager. But I've also
considered being a reviewer. It was very intimidating to see my
progress and thoughts about a particular library versus other peoples'
experience and writings. To the point where I have doubts about my
ability to make a meaningful contribution as a reviewer or a review
manager.

In particular for libraries that lie outside my domain of knowledge, I
feel like I have nothing to offer other than perhaps attempting to
build and run the tests.

If I feel this way, is it possible there are many others who feel the
same way? It seems to me that acting as a review manager requires
specialized skills and also good knowledge and understanding of the
libraries that are in Boost. How could we expect someone whose only
knows a little Boost who then writes their own library for a formal
review, to cross this knowledge gap?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Le 14/03/2017 à 13:01, Niall Douglas via Boost a écrit :
> Dear Boost,
>
> [snip]
>

This is an interesting topic, thanks for bringing it!

> I therefore ask boost-dev what to do? Some options:
>
> 1. Pay US$1000 (one thousand) dollars to each person who manages a
> review. In case you're worried Boost doesn't have the money, it does in
> spades, that's not a problem. For $23,000 we could clear the current
> review queue assuming none of the problems mentioned yet.

I suggest another way of rewarding people:

- If the review manager is a library maintainer: by the end of the
review, he/she gets the help of the boost community (including the ppl
whose code is being reviewed) to get his/her backlog cleared. This
includes development, patches, backlog cleanup, as well as
management/coordination of those devs.

- if the review manager is the author of a library under review or
freshly reviewed for acceptance to boost, but still not part of any
release: he/she will get the help of the boost community
to make that happen as soon as possible (including open pending issues
from previous reviews, documentation, integration, migration to
boost.build, etc etc)

Of course, we can iterate further
- for each good and sound review, you get one ticket of your backlog
closed by next release

... etc etc ...

I believe this is win/win/win for the health of boost, reviewers and new
libraries.

Best,
Raffi


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
On Tue, Mar 14, 2017 at 6:12 PM, Raffi Enficiaud wrote:

> I suggest another way of rewarding people:
>
> - If the review manager is a library maintainer: by the end of the review,
> he/she gets the help of the boost community (including the ppl whose code is
> being reviewed) to get his/her backlog cleared. This includes development,
> patches, backlog cleanup, as well as management/coordination of those devs.
>
> - if the review manager is the author of a library under review or freshly
> reviewed for acceptance to boost, but still not part of any release: he/she
> will get the help of the boost community
> to make that happen as soon as possible (including open pending issues from
> previous reviews, documentation, integration, migration to boost.build, etc
> etc)
>
> Of course, we can iterate further
> - for each good and sound review, you get one ticket of your backlog closed
> by next release
>

These actually sound like excellent ideas to me.

Glen

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 3/14/2017 8:01 AM, Niall Douglas via Boost wrote:

> Dear Boost,
>
> I see that new candidate Boost libraries entering the review queue have
> exploded in recent years, with no less than *twenty-three* proposed
> libraries awaiting a review.
>
> As the ongoing strength and vitality of Boost is inextricably linked to
> new growth, I think that waiting around for years for someone to
> volunteer to manage a review is not healthy. If a library author has
> invested the very significant effort to develop a Boost-quality library,
> the least Boost can do is to try harder to provide timely reviews and
> that means persuading more people to volunteer to manage reviews.
>
> In the past people have argued that for every library you submit for
> review, you should manage a review in return. Myself, Antony and a few
> others have adhered to that rule, and if every library author did so
> there would be no outstanding review queue. However there are problems
> in that in itself in terms of moral hazard, and also because the review
> manager needs to usually be fairly expert in a library being reviewed,
> else it can be very hard to judge the worth and validity of reviews. A
> shortage of suitably expert review managers will always be a problem for
> some types of library.

I would like to concur that the number of libraries awaiting review,
because no review manager has stepped forward for those libaries, is a
real problem with Boost. I also do not believe that people should have
to wait years for a review.

>
>
> I therefore ask boost-dev what to do? Some options:
>
> 1. Pay US$1000 (one thousand) dollars to each person who manages a
> review. In case you're worried Boost doesn't have the money, it does in
> spades, that's not a problem. For $23,000 we could clear the current
> review queue assuming none of the problems mentioned yet.
>
> 2. Pay US$1000 dollars to the manager and 2x $500 dollar payments to
> those writing the top two most useful reviews as judged by the review
> manager. That makes the cost $2000 per library accepted or rejected, or
> $46,000 to clear the current review queue.

I am certainly not against paying programmers for valuable work, even if
that "work" is managing a Boost review. I am afraid, however, that
paying a review manager might mean that someone will take on the task
who is not qualified for it simply because momney is being offered.

>
> 3. In my own opinion from reviewing the review queue, a good 25% of the
> libraries in the queue are not ready for review due to obvious glaring
> deficiencies in the documentation or code. Spending a grand on those
> libraries which will very obviously be rejected isn't worth the money.
> What should we do about those? One approach could simply be to trust
> review managers to not abuse the thousand dollar fee. Another could be
> that before each new review, the prospective manager needs to write a
> single line comment on why they did not choose the other libraries in
> the queue and publish that here before starting a review. That would
> quickly identify those libraries in the queue which a majority of
> managers think have serious problems and could never pass any review. If
> say a library in a queue accumulates three single line black marks, the
> author might be encouraged to withdraw it.

I believe you have gotten too elaborate here. Many libraries on the
review queue may not be ready for review simply because the requirements
for libraries being reviewed have changed since the library was put on
the review queue.

>
> 4. Finally there is the problem of libraries of high quality, but not a
> good fit for Boost because they are so esoteric and niche that nobody
> could provide a useful review, and without useful reviews the review
> manager can't really recommend acceptance. This will be an increasing
> problem with time anyway as more of the low hanging C++ library fruit is
> picked, but I suppose one could just kick that decision can down the
> road and see if 2x $500 payments might help scare up more high quality
> reviews.
>
> 5. We could try guilting more people into review managing, and redouble
> banging the drum to scare up more volunteers.
>
>
> I look forward to seeing what people think.

Thanks for bringing this up. I do not think paying money is necessarily
the best solution, as money often corrupts judgment no matter how honest
anyone claims himself to be, but the lack of review managers for the
many libraries sitting on the review queue for a long time is a real
problem in my estimation also. I especially think it is a problem
because it stops good C++ programmers from particpating more in Boost
work and discussions because they can see that there own proposed work
is being ignored. Certainly Boost needs more qualified people, and not
less, to participate and maintain the quality of the libraries being
offered, especially as so many of those libraries need some support when
the original library author is no longer available to support it even
sporadically.

> Niall
>



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [review queue] What to do about the library review queue?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
AMDG

On 03/14/2017 04:26 PM, Glen Fernandes via Boost wrote:

> On Tue, Mar 14, 2017 at 6:12 PM, Raffi Enficiaud wrote:
>> I suggest another way of rewarding people:
>>
>> - If the review manager is a library maintainer: by the end of the review,
>> he/she gets the help of the boost community (including the ppl whose code is
>> being reviewed) to get his/her backlog cleared. This includes development,
>> patches, backlog cleanup, as well as management/coordination of those devs.
>>
>> - if the review manager is the author of a library under review or freshly
>> reviewed for acceptance to boost, but still not part of any release: he/she
>> will get the help of the boost community
>> to make that happen as soon as possible (including open pending issues from
>> previous reviews, documentation, integration, migration to boost.build, etc
>> etc)
>>
>> Of course, we can iterate further
>> - for each good and sound review, you get one ticket of your backlog closed
>> by next release
>>
>
> These actually sound like excellent ideas to me.
>

  Yes, it sounds great to me too, but...
who is volunteering to do this work?
"The boost community will do it" is much
to nebulous for a practical proposal.

In Christ,
Steven Watanabe


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
1234 ... 8