question(s) about bjam code II

[Boost - Build mailing list]

Hi Steven,

I have been following your advice and tried to merge the "make0()" function call to compute a target's fate into the state machine implemented in make1.c, to support a more dynamic build process. In general this is working great (i.e. I can now freely mix config-like targets that compute parameters that are later used in the build, with other targets). However, there is one set of use-cases that's still causing me some headache: actions that update more than one target, i.e. that require other targets to be "rebuilt".

While I understand the situation (as well as the implementation with "force_rebuilds()") well, I'm still struggling with some of the code:

Why is the loop in https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L1220-L1227 launching all the targets that are to be updated by the current command, and then letting that command wait for them to be ready ? (I see the corresponding "unlock" code is https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L939-L948, but I don't quite understand that part of the workflow.)

Many thanks for your help,

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build

[Boost - Build mailing list]

AMDG

On 10/19/2017 10:34 PM, Stefan Seefeld via Boost-build wrote:

>
> I have been following your advice and tried to merge the "make0()" function call
> to compute a target's fate into the state machine implemented in make1.c, to
> support a more dynamic build process. In general this is working great (i.e. I
> can now freely mix config-like targets that compute parameters that are later
> used in the build, with other targets). However, there is one set of use-cases
> that's still causing me some headache: actions that update more than one target,
> i.e. that require other targets to be "rebuilt".
>
> While I understand the situation (as well as the implementation with
> "force_rebuilds()") well, I'm still struggling with some of the code:
>
> Why is the loop in
> https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L1220-L1227 
> launching all the targets that are to be updated by the current command, and
> then letting that command wait for them to be ready ? (I see the corresponding
> "unlock" code is
> https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L939-L948, but
> I don't quite understand that part of the workflow.)
>

  It's basically dealing with the fact that there
isn't a one-to-one correspondence between targets
and actions.  What we need to schedule is the actions,
but the dependencies are expressed in terms of targets.
The rule is that an action can be started when all
the dependencies of all the targets it creates are
up-to-date.
  The code in question was introduced in this commit:
https://github.com/boostorg/build/commit/c1395b49614ed5c4b0958355ffaa0f794fb760a2
which replaces an older implementation which used INCLUDES.
(I don't remember the details of why INCLUDES caused
problems off the top of my head.)
  Also, keep in mind that these were fixes that were
put in much later than the original code.  The
original Perforce Jam mishandled actions
that produced multiple targets quite badly.
The mismatch between dependencies/targets/actions
is quite annoying to deal with, and the fix
to make it work is a bit of a kludge.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build

[Boost - Build mailing list]

On 20.10.2017 01:32, Steven Watanabe via Boost-build wrote:

AMDG

On 10/19/2017 10:34 PM, Stefan Seefeld via Boost-build wrote:

I have been following your advice and tried to merge the "make0()" function call 
to compute a target's fate into the state machine implemented in make1.c, to 
support a more dynamic build process. In general this is working great (i.e. I 
can now freely mix config-like targets that compute parameters that are later 
used in the build, with other targets). However, there is one set of use-cases 
that's still causing me some headache: actions that update more than one target, 
i.e. that require other targets to be "rebuilt".

While I understand the situation (as well as the implementation with 
"force_rebuilds()") well, I'm still struggling with some of the code:

Why is the loop in 
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L1220-L1227 
launching all the targets that are to be updated by the current command, and 
then letting that command wait for them to be ready ? (I see the corresponding 
"unlock" code is 
https://github.com/boostorg/build/blob/develop/src/engine/make1.c#L939-L948, but 
I don't quite understand that part of the workflow.)

  It's basically dealing with the fact that there
isn't a one-to-one correspondence between targets
and actions.  What we need to schedule is the actions,
but the dependencies are expressed in terms of targets.
The rule is that an action can be started when all
the dependencies of all the targets it creates are
up-to-date.
  The code in question was introduced in this commit:
https://github.com/boostorg/build/commit/c1395b49614ed5c4b0958355ffaa0f794fb760a2
which replaces an older implementation which used INCLUDES.
(I don't remember the details of why INCLUDES caused
problems off the top of my head.)
  Also, keep in mind that these were fixes that were
put in much later than the original code.  The
original Perforce Jam mishandled actions
that produced multiple targets quite badly.
The mismatch between dependencies/targets/actions
is quite annoying to deal with, and the fix
to make it work is a bit of a kludge.

Right, indeed. At first sight, the above logic seemed strange ("update all the targets before running the action that updates them"), but I suppose it makes sense, as "update" doesn't really mean to complete them; just get them to the state where the action can be performed.

I have to admit that I find that state engine in make1.c both ingenious and challenging, especially after I augmented it by additional steps to deal with the dynamic dependency graph manipulation. I'm constantly switching between my computer screen and a whiteboard, trying to formalize the state transitions (complicated by the asynchronous handling of commands).

But, I'm overall very happy with the results, as I believe the logic expressed in in the higher layers of the build logic (especially the fabscripts themselves) is more intuitive and straight-forward. I hope to stabilize this quickly, so I can ask for a public review.

Thanks again for your ongoing help !

_______________________________________________
Unsubscribe & other changes: https://lists.boost.org/mailman/listinfo.cgi/boost-build