[boost] [bjam] copy a file

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[boost] [bjam] copy a file

lcaminiti
Hello all,

I'd have a Jamfile that has a "doc" rule to build documentation `bjam doc`. I'd like to program another rule "config_doc" inside the Jamfile that does the following:

bjam config_doc:
1) Copy the file /etc/config.txt to ./
2) Copy the file ./src/config.txt to /etc/
3) Run the doc rule
4) Copy the file ./config.txt into /etc/
Where ./ is the current directory from where I'm running bjam. Ideally, I'd like 4) to be executed even if 3) fails.

Is there a way to do this?

Thanks a lot.
--Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: [bjam] copy a file

Steven Watanabe-4
AMDG

On 01/23/2012 10:26 AM, lcaminiti wrote:

> Hello all,
>
> I'd have a Jamfile that has a "doc" rule to build documentation `bjam doc`.
> I'd like to program another rule "config_doc" inside the Jamfile that does
> the following:
>
> bjam config_doc:
> 1) Copy the file /etc/config.txt to ./
> 2) Copy the file ./src/config.txt to /etc/
> 3) Run the doc rule
> 4) Copy the file ./config.txt into /etc/
> Where ./ is the current directory from where I'm running bjam. Ideally, I'd
> like 4) to be executed even if 3) fails.
>
> Is there a way to do this?
>

If there's any way to avoid this, you should.

a) This can't safely run in parallel with
other uses of the same rule or anything else
that uses /etc/config.txt
b) You can't guarantee that /etc/config.txt
is restored properly if the process is interrupted.

If you really, /really/ need to do this,
you should probably use a single action
that does all of this.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

[random] parallel random number generators

Thijs (M.A.) van den Berg
We are looking for a parallel random number generator for a distributed Monte Carlo project. I've heard stories that this was going to be part of boost soon, but when trying to find more info I failed to find anything.

Is there any development activity on parallel random number generators in boost?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Francisco L. Silva
2012/1/23 Thijs (M.A.) van den Berg <[hidden email]>

> We are looking for a parallel random number generator for a distributed
> Monte Carlo project. I've heard stories that this was going to be part of
> boost soon, but when trying to find more info I failed to find anything.
>
> Is there any development activity on parallel random number generators in
> boost?
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

Although it's not listed here:
http://www.boost.org/doc/libs/1_48_0/doc/html/boost_random/reference.html,

I can find a suggestive header over here:
https://svn.boost.org/svn/boost/sandbox/random/boost/random/

But I don't know any status about it, maybe forgotten on the boost sandbox.

Regards

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Heiko Bauke-4
In reply to this post by Thijs (M.A.) van den Berg
Dear Thijs,

On Mon, 23 Jan 2012 20:38:45 +0100
"Thijs (M.A.) van den Berg" <[hidden email]> wrote:

> We are looking for a parallel random number generator for a
> distributed Monte Carlo project. I've heard stories that this was
> going to be part of boost soon, but when trying to find more info I
> failed to find anything.
>
> Is there any development activity on parallel random number
> generators in boost?

I am not aware of such a development. However, you might try TRNG, see
http://numbercrunch.de/trng/ .


        Heiko

--
-- Gehirn: ein Organ, mit dem wir denken, daß wir denken.
-- (Ambrose Bierce, am. Journalist, 1842-1914)
-- Number Crunch Blog @ http://numbercrunch.de
--  Cluster Computing @ http://www.clustercomputing.de



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Thijs (M.A.) van den Berg
On Jan 23, 2012, at 8:46 PM, Francisco L. Silva wrote:
>> Is there any development activity on parallel random number
>> generators in boost?
>
> I am not aware of such a development. However, you might try TRNG, see
> http://numbercrunch.de/trng/ .
>
Thanks Heiko,
it does indeed look like we need to find an alternative lib than something available in boost




On Jan 23, 2012, at 8:46 PM, Francisco L. Silva wrote:
> Although it's not listed here:
> http://www.boost.org/doc/libs/1_48_0/doc/html/boost_random/reference.html,
>
> I can find a suggestive header over here:
> https://svn.boost.org/svn/boost/sandbox/random/boost/random/
>
> But I don't know any status about it, maybe forgotten on the boost sandbox.

Interesting! The seems to have been an attempt, but that code has ©2006, so it's very old

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [bjam] copy a file

lcaminiti
In reply to this post by Steven Watanabe-4
Steven Watanabe-4 wrote
AMDG

On 01/23/2012 10:26 AM, lcaminiti wrote:
> Hello all,
>
> I'd have a Jamfile that has a "doc" rule to build documentation `bjam doc`.
> I'd like to program another rule "config_doc" inside the Jamfile that does
> the following:
>
> bjam config_doc:
> 1) Copy the file /etc/config.txt to ./
> 2) Copy the file ./src/config.txt to /etc/
> 3) Run the doc rule
> 4) Copy the file ./config.txt into /etc/
> Where ./ is the current directory from where I'm running bjam. Ideally, I'd
> like 4) to be executed even if 3) fails.
>
> Is there a way to do this?
>

If there's any way to avoid this, you should.

a) This can't safely run in parallel with
other uses of the same rule or anything else
that uses /etc/config.txt
b) You can't guarantee that /etc/config.txt
is restored properly if the process is interrupted.

If you really, /really/ need to do this,
you should probably use a single action
that does all of this.
This is for one of my own projects (not Boost) for which I don't have to worry about parallel access to the file. However, I do want to "catch" errors and restore the file if I can't build the docs. If I can't use bjam, I might create a script to do that...

Thanks.
--Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

John Salmon-3
In reply to this post by Thijs (M.A.) van den Berg
On Mon, Jan 23, 2012 at 2:38 PM, Thijs (M.A.) van den Berg
<[hidden email]>wrote:

> We are looking for a parallel random number generator for a distributed
> Monte Carlo project. I've heard stories that this was going to be part of
> boost soon, but when trying to find more info I failed to find anything.
>
> Is there any development activity on parallel random number generators in
> boost?
>

A couple of weeks ago, I asked if there was interest in a parallel RNG in
boost.
Here's a pointer to my original post:.

http://lists.boost.org/Archives/boost/2012/01/189293.php

Our paper on parallel random number generators is here:
http://dl.acm.org/citation.cfm?doid=2063405
also available at
http://www.thesalmons.org/john/random123/papers/random123sc11.pdf

Source (non-boost) is available here:
http://deshawresearch.com/resources_random123.html

And software that might be a starting point for a boost contribution is
here:
http://thesalmons.org/john/random123/prf_boost/prf_boost.zip

Feedback is appreciated.

John Salmon

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Mathias Gaunard-2
On 01/25/2012 08:55 PM, John Salmon wrote:

> A couple of weeks ago, I asked if there was interest in a parallel RNG in
> boost.
> Here's a pointer to my original post:.
>
> http://lists.boost.org/Archives/boost/2012/01/189293.php
>
> Our paper on parallel random number generators is here:
> http://dl.acm.org/citation.cfm?doid=2063405
> also available at
> http://www.thesalmons.org/john/random123/papers/random123sc11.pdf
>
> Source (non-boost) is available here:
> http://deshawresearch.com/resources_random123.html
>
> And software that might be a starting point for a boost contribution is
> here:
> http://thesalmons.org/john/random123/prf_boost/prf_boost.zip
>
> Feedback is appreciated.

The above code seems quite more readable than the one from the
deshawresearch website, but also a bit shorter.
Does it do everything the deshawresearch does?

I see you're using inline asm for mulq on GCC, but you don't use __umulh
for MSVC.
I can't see any hit for _mm_clmulepi64_si128 (which I assume would be
useful).

Your cpuid code won't work with gcc -m32 -fPIC, and also doesn't appear
to take care of register trashing properly.
I also don't really think the code should do any cpuid check at all,
since emitting AESNI instructions requires -maes, and whenever you use
-maes the compiler is free to emit AESNI instructions whenever it wants.

 From what I see, you only use SSE instructions if AESNI is available.
Is that really desirable?

There is a library proposed for Boost that abstracts SIMD instructions
and scalar built-ins (Boost.SIMD), it might be interesting to use it /
extend it to take care of all the platform-specific bits.

In any case I think this would be a very interesting contribution to
Boost and the open-source community in general.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

John Salmon-3
On Thu, Jan 26, 2012 at 2:06 PM, Mathias Gaunard <
[hidden email]> wrote:

> On 01/25/2012 08:55 PM, John Salmon wrote:
>
>  A couple of weeks ago, I asked if there was interest in a parallel RNG in
>> boost.
>> Here's a pointer to my original post:.
>>
>> http://lists.boost.org/**Archives/boost/2012/01/189293.**php<http://lists.boost.org/Archives/boost/2012/01/189293.php>
>>
>> Our paper on parallel random number generators is here:
>> http://dl.acm.org/citation.**cfm?doid=2063405<http://dl.acm.org/citation.cfm?doid=2063405>
>> also available at
>> http://www.thesalmons.org/**john/random123/papers/**random123sc11.pdf<http://www.thesalmons.org/john/random123/papers/random123sc11.pdf>
>>
>> Source (non-boost) is available here:
>> http://deshawresearch.com/**resources_random123.html<http://deshawresearch.com/resources_random123.html>
>>
>> And software that might be a starting point for a boost contribution is
>> here:
>> http://thesalmons.org/john/**random123/prf_boost/prf_boost.**zip<http://thesalmons.org/john/random123/prf_boost/prf_boost.zip>
>>
>> Feedback is appreciated.
>>
>
> The above code seems quite more readable than the one from the
> deshawresearch website, but also a bit shorter.
> Does it do everything the deshawresearch does?
>

The functionality is the same.  The original Random123 library
(deshawresearch website) is C-compatible and also works with CUDA and
OpenCL development environments. Dropping C, CUDA and OpenCL support and
making effective use of templates and classes greatly clarifies/simplifies
the code..

>
> I see you're using inline asm for mulq on GCC, but you don't use __umulh
> for MSVC.
> I can't see any hit for _mm_clmulepi64_si128 (which I assume would be
> useful).
>

> Your cpuid code won't work with gcc -m32 -fPIC, and also doesn't appear to
> take care of register trashing properly.
> I also don't really think the code should do any cpuid check at all, since
> emitting AESNI instructions requires -maes, and whenever you use -maes the
> compiler is free to emit AESNI instructions whenever it wants.
>

> From what I see, you only use SSE instructions if AESNI is available. Is
> that really desirable?
>
> There is a library proposed for Boost that abstracts SIMD instructions and
> scalar built-ins (Boost.SIMD), it might be interesting to use it / extend
> it to take care of all the platform-specific bits.
>

These are clearly areas that need work.  In terms of portability, the
current version is at the proof-of-concept stage, not the release-ready
stage.  "Should there even be a cpuid check" is an excellent question.  If
the answer is no, it certainly makes some portability headaches go away.
How best to accomodate compiler-specific and hardware-specific intrinsics
and asm is another open question.  I'm open to considering more, or less,
use of SSE, and I'd be very happy to let another library do the heavy
lifting in terms of portability.

John Salmon


> In any case I think this would be a very interesting contribution to Boost
> and the open-source community in general.
>
>

>
> ______________________________**_________________
> Unsubscribe & other changes: http://lists.boost.org/**
> mailman/listinfo.cgi/boost<http://lists.boost.org/mailman/listinfo.cgi/boost>
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Mathias Gaunard-2
On 01/27/2012 09:29 PM, John Salmon wrote:

> The functionality is the same.  The original Random123 library
> (deshawresearch website) is C-compatible and also works with CUDA and
> OpenCL development environments. Dropping C, CUDA and OpenCL support and
> making effective use of templates and classes greatly clarifies/simplifies
> the code..

Would it possible to keep CUDA support still, since it supports C++?
There is a BOOST_GPU_ENABLED macro in Boost that will expand to __host__
__device__ if using CUDA.


> These are clearly areas that need work.  In terms of portability, the
> current version is at the proof-of-concept stage, not the release-ready
> stage.

To submit your library for review, you'll need more than
proof-of-concept, if not necessarily release-ready code.
You'll also need documentation, but that part should be pretty short.


> "Should there even be a cpuid check" is an excellent question.  If
> the answer is no, it certainly makes some portability headaches go away.

I have personally come to the conclusion that there should not be any,
since that's the only model that is safe with all compilers.
Also, this way, users don't pay for what they don't use. They're free to
build the cpuid logic on top of it themselves through compiling the code
several times with different settings.


> How best to accomodate compiler-specific and hardware-specific intrinsics
> and asm is another open question.  I'm open to considering more, or less,
> use of SSE, and I'd be very happy to let another library do the heavy
> lifting in terms of portability.

I am one of the authors of Boost.SIMD, which isn't a Boost library yet
(there are problems with compile-time performance that I'd like to fix
first, and it seriously lacks documentation).

If you wish to depend on it (which might delay getting a review, but
your code doesn't seem to be review-ready just yet in any case) I'd be
happy to help.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

John Salmon-3
On Sat, Jan 28, 2012 at 1:43 PM, Mathias Gaunard <
[hidden email]> wrote:

> On 01/27/2012 09:29 PM, John Salmon wrote:
>
>  "Should there even be a cpuid check" is an excellent question.  If
>> the answer is no, it certainly makes some portability headaches go away.
>>
>
>
I also don't really think the code should do any cpuid check at all, since
> emitting AESNI instructions requires -maes, and whenever you use -maes the
> compiler is free to emit AESNI instructions whenever it wants.
>


> I have personally come to the conclusion that there should not be any,
> since that's the only model that is safe with all compilers.
> Also, this way, users don't pay for what they don't use. They're free to
> build the cpuid logic on top of it themselves through compiling the code
> several times with different settings.
>
>
The -maes option is gcc-specific.  The Intel compiler, for example, does
not have an analogous command line option.  Unlike with gcc, you can
#include <wmmintrin.h> and you can use the _mm_aesenc_si128() without any
command line option.

I think that it's important for the library to allow the programmer the
opportunity to decide about AESNI at compile-time, and also to allow the
programmer to defer the decision till runtime.  If the decision is deferred
till runtime, I don't see how to choose a default without a cpuid check.

By the way, the code in aes_hasaesni.hpp that trashes %ebx, and is
therefore problematic with -m32 -fPIC is conditionalized on #ifdef
BOOST_HAS_AESNI.  While I agree that the #ifdef logic to set
BOOST_HAS_AESNI needs work, I think it's safe to say that BOOST_HAS_AESNI
should never be set when -m32 is in effect.

>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Jeremiah Willcock
On Sun, 29 Jan 2012, John Salmon wrote:

(snip)

> By the way, the code in aes_hasaesni.hpp that trashes %ebx, and is
> therefore problematic with -m32 -fPIC is conditionalized on #ifdef
> BOOST_HAS_AESNI.  While I agree that the #ifdef logic to set
> BOOST_HAS_AESNI needs work, I think it's safe to say that BOOST_HAS_AESNI
> should never be set when -m32 is in effect.

Is there a reason for that?  AESNI instructions use SSE registers, so they
are independent of 32-bit vs. 64-bit mode.

-- Jeremiah Willcock

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

John Salmon-3
On Sun, Jan 29, 2012 at 2:09 PM, Jeremiah Willcock <[hidden email]>wrote:

> On Sun, 29 Jan 2012, John Salmon wrote:
>
> (snip)
>
>
>  By the way, the code in aes_hasaesni.hpp that trashes %ebx, and is
>> therefore problematic with -m32 -fPIC is conditionalized on #ifdef
>> BOOST_HAS_AESNI.  While I agree that the #ifdef logic to set
>> BOOST_HAS_AESNI needs work, I think it's safe to say that BOOST_HAS_AESNI
>> should never be set when -m32 is in effect.
>>
>
> Is there a reason for that?  AESNI instructions use SSE registers, so they
> are independent of 32-bit vs. 64-bit mode.
>
> You're correct.  I mistakenly assumed that you couldn't use AESNI
instructions in 32-bit code.  I'm not sure why anyone would want to, but it
*is* possible.  I'll add the code to push and pop %ebx around cpuid.

John

-- Jeremiah Willcock
>
>
> ______________________________**_________________
> Unsubscribe & other changes: http://lists.boost.org/**
> mailman/listinfo.cgi/boost<http://lists.boost.org/mailman/listinfo.cgi/boost>
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Mathias Gaunard-2
In reply to this post by John Salmon-3
On 01/29/2012 05:47 PM, John Salmon wrote:

> The -maes option is gcc-specific.  The Intel compiler, for example, does
> not have an analogous command line option.  Unlike with gcc, you can
> #include<wmmintrin.h>  and you can use the _mm_aesenc_si128() without any
> command line option.

I'd avoid to rely on compiler-specific behaviour.


> I think that it's important for the library to allow the programmer the
> opportunity to decide about AESNI at compile-time, and also to allow the
> programmer to defer the decision till runtime.

Allowing the user to choose at compile-time is enough, since the user
can build runtime selection on top of that.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [random] parallel random number generators

Mathias Gaunard-2
In reply to this post by John Salmon-3
On 01/29/2012 10:53 PM, John Salmon wrote:

>> You're correct.  I mistakenly assumed that you couldn't use AESNI
> instructions in 32-bit code.  I'm not sure why anyone would want to, but it
> *is* possible.  I'll add the code to push and pop %ebx around cpuid.

The definition of cpuid I personally use is

#if !defined(__PIC__) || defined(__x86_64__)
     __asm__ __volatile__
     (
       "cpuid"
     : "=a" (CPUInfo[eax]), "=b" (CPUInfo[ebx])
     , "=c" (CPUInfo[ecx]), "=d" (CPUInfo[edx])
     : "a" (InfoType)
     : "cc"
     );
#else
     __asm__ __volatile__
     (
       "pushl %%ebx      \n\t" /* save %ebx */
       "cpuid            \n\t"
       "movl %%ebx, %1   \n\t" /* save what cpuid just put in %ebx */
       "popl %%ebx       \n\t" /* restore the old %ebx */
     : "=a"(CPUInfo[eax]), "=r"(CPUInfo[ebx])
     , "=c"(CPUInfo[ecx]), "=d"(CPUInfo[edx])
     : "a"(InfoType)
     : "cc"
     );
#endif




_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost