Re: [Rd] cleanup and Makevars

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Thu, 12 Jul 2007 10:05:02 +0100 (BST)

You seem extraordinarily reluctant to tell us what you are actually doing: no command appears here. I presume this is a call to R CMD INSTALL, but why do I have to keep guessing?

Since you have taken over the make action via the all: target in your Makevars, you need also to take over responsibility for cleaning. I should think

all: myclean $(SHLIB)

myclean:

         rm -f $(OBJS)

should do it. And BTW, the use of $(OBJS) to set the objects is undocumented, at least in 'Writing R Extensions'.

On Wed, 11 Jul 2007, Kasper Daniel Hansen wrote:

>
> On Jul 11, 2007, at 9:30 AM, Prof Brian Ripley wrote:
>
>> On Mon, 9 Jul 2007, Kasper Daniel Hansen wrote:
>>
>>> Hi
>>>
>>> This is a question prompted by the mac version of R, but as I see it,
>>> it should have broader interest.
>>>
>>> These days the CRAN Mac binary per default compiles every package for
>>> two architectures. First i386 and then ppc. In between the two
>>> compilation runs, any object files in pkgname/src is removed. This
>>> cleanup is necessary since otherwise Make would not recompile the
>>> object files under the next architecture.
>>>
>>> I have a package which includes a large SDK which I have put in
>>> src/fusion_sdk
>>> (and subdirectories). I take care of that by using a Makevars file
>>> with essentially (the whole Makevars file is reproduced below)
>>>
>>> PKG_SOURCES = \
>>> fusion_sdk/calvin_files/data/src/CDFData.cpp
>>>
>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>>
>>> all: $(SHLIB)
>>>
>>> However, this setup does not remove the object files in the
>>> fusion_sdk subdirectory. From a posting on R-sig-mac by Simon Urbanek
>>> I learned that I need to clean up these directories myself.
>>>
>>> I have - per R extensions - attempted to do so using a pkgname/
>>> cleanup script. While this works for cleaning up the sub directories
>>> when I do R CMD build, it seems as if this script is not being run
>>> between compilation runs (is this intentional btw.?).
>>
>> What do you mean by 'compilation runs'? The 'cleanup' script will be run
>> by R CMD INSTALL --cleanup (and without --cleanup it is intentionally not
>> run).
>>
>>> So my question is now: how do I in a simple way clean up my
>>> subdirectories? I would prefer it to be as simple as possible because
>>> so far I have not really needed anything besides a Makevars file. In
>>> fact the impression I have right now from looking at the SHLIB and
>>> INSTALL scripts is that I might need a Makefile that removes the
>>> object files _before_ starting the compilation, as I don't really see
>>> any cleanup process (eg. making a target clean) - but this may very
>>> well be due to my limited understanding of these scripts.
>>
>> I see R CMD INSTALL --clean that runs a clean up afterwards, and 2.6.0 will
>> also have R CMD INSTALL --preclean in case you forgot --clean on the last
>> run.
>>
>>> Any help/hints on how to proceed? Especially if I want it to be
>>> portable?
>>
>> What does R CMD INSTALL --clean not do that you want?
>>
>> If you are talking about R CMD SHLIB, that currently does not clean up but
>> this has been raised and it will most likely have a --clean option in 2.6.0
>> that removes $(OBJECTS).
>
> Perhaps I was not clear enough: The CRAN binary of R for Mac has two
> architectures inside $R_HOME/bin/exec, namely "ppc" and "i386". This means
> that whenever anyone installs a source package I see two compilation runs -
> the following is a shortened output:
>
> * Installing to library '/Library/Frameworks/R.framework/Resources/library'
> * Installing *source* package 'affxparser' ...
> ** libs
> ** arch - i386
> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -no-cpp-precomp
> -I/Library/Frameworks/R.framework/Resources/include
> -I/Library/Frameworks/R.framework/Resources/include/i386 -imacros
> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src
> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src
> -Ifusion_sdk/calvin_files/fusion/src
> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter
> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter
> -Ifusion_sdk/calvin_files/parameter/src -Ifusion_sdk/calvin_files/parsers/src
> -Ifusion_sdk/calvin_files/portability/src
> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src
> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file
> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -msse3 -fPIC -g -O2 -Wall
> -O0 -c fusion_sdk/calvin_files/data/src/CDFData.cpp -o
> fusion_sdk/calvin_files/data/src/CDFData.o
>
> (Now a lot of other files gets compiled)
>
> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -dynamiclib
> -Wl,-macosx_version_min -Wl,10.3 -undefined dynamic_lookup -single_module
> -multiply_defined suppress -L/usr/local/lib -o affxparser.so
> fusion_sdk/calvin_files/data/src/CDFData.o
> fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.o
> fusion_sdk/calvin_files/data/src/CDFProbeInformation.o
> fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.o
> fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.o
> fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.o
> fusion_sdk/calvin_files/data/src/CELData.o
> fusion_sdk/calvin_files/data/src/DataGroup.o
> fusion_sdk/calvin_files/data/src/DataGroupHeader.o
> fusion_sdk/calvin_files/data/src/DataSet.o
> fusion_sdk/calvin_files/data/src/DataSetHeader.o
> fusion_sdk/calvin_files/data/src/FileHeader.o
> fusion_sdk/calvin_files/data/src/GenericData.o
> fusion_sdk/calvin_files/data/src/GenericDataHeader.o
> fusion_sdk/calvin_files/exception/src/ExceptionBase.o
> fusion_sdk/calvin_files/fusion/src/CalvinAdapter/CalvinCELDataAdapter.o
> fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.o
> fusion_sdk/calvin_files/fusion/src/FusionCDFData.o
> fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.o
> fusion_sdk/calvin_files/fusion/src/FusionCELData.o
> fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.o
> fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.o
> fusion_sdk/calvin_files/parsers/src/CDFFileReader.o
> fusion_sdk/calvin_files/parsers/src/CelFileReader.o
> fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.o
> fusion_sdk/calvin_files/parsers/src/DataGroupReader.o
> fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.o
> fusion_sdk/calvin_files/parsers/src/DataSetReader.o
> fusion_sdk/calvin_files/parsers/src/FileHeaderReader.o
> fusion_sdk/calvin_files/parsers/src/FileInput.o
> fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.o
> fusion_sdk/calvin_files/parsers/src/GenericFileReader.o
> fusion_sdk/calvin_files/utils/src/AffymetrixGuid.o
> fusion_sdk/calvin_files/utils/src/DateTime.o
> fusion_sdk/calvin_files/utils/src/FileUtils.o
> fusion_sdk/calvin_files/utils/src/StringUtils.o
> fusion_sdk/calvin_files/utils/src/checksum.o fusion_sdk/file/BPMAPFileData.o
> fusion_sdk/file/BPMAPFileWriter.o fusion_sdk/file/CDFFileData.o
> fusion_sdk/file/CELFileData.o fusion_sdk/file/FileIO.o
> fusion_sdk/file/FileWriter.o R_affx_cel_parser.o R_affx_cdf_parser.o
> R_affx_cdf_extras.o R_affx_bpmap_parser.o
> -F/Library/Frameworks/R.framework/.. -framework R
> ** arch - ppc
> g++-4.0 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -no-cpp-precomp
> -I/Library/Frameworks/R.framework/Resources/include
> -I/Library/Frameworks/R.framework/Resources/include/ppc -imacros
> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src
> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src
> -Ifusion_sdk/calvin_files/fusion/src
> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter
> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter
> -Ifusion_sdk/calvin_files/parameter/src -Ifusion_sdk/calvin_files/parsers/src
> -Ifusion_sdk/calvin_files/portability/src
> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src
> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file
> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -I/usr/local/include -fPIC
> -g -O2 -Wall -O0 -c R_affx_cel_parser.cpp -o R_affx_cel_parser.o
>
> As you see from this (and sorry for all the include statements), when R
> starts compiling for arch = "ppc", it starts with a with in /src whereas the
> first compilation run (or more precise the first architecture) starts from
> /src/fusion_sdk/calvin_files/data/src. This is of course because the object
> files in the subdirectories do not get removed between the two architectures.
>
> This has - probably - something to do with R CMD SHLIB not cleaning up after
> itself. A method of removing $(OBJECTS) would be exactly what I need, and may
> I suggest that this is done per default instead of being set by an option.
> Anyone using the CRAN binary for the Mac will have the same problem as I.
>
> I am happy to see that this may be fixed for R-2.6.0. Is there anything I can
> do in the meantime for getting my package to work under R-2.5.1? (I can
> always move my source files from the /src/fusion_sdk subdirectories directly
> into /src, but I would prefer to avoid that).
>
>>>
>>> Kasper
>>>
>>> The full Makevars file:
>>>
>>> MYCXXFLAGS=-O0
>>
>> You do realize that is highly non-portable?
>
> Yes, but I do not know of any other way to downgrade the optimization level.
> The SDK I am using breaks (due to memory alignment issues if a specific
> optimization flag for GCC is include - the flag is certainly implied by -O2
> and as far as I recall also -O1). I guess the solution is to switch to an
> autoconfigure script. The main reason for not doing this is 1) time and
> actually more 2) the fact that the SDK does not come with an ac script.
>
> Thanks for the feedback, Kasper
>
>
>>> %.o: %.cpp
>>> $(CXX) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) $(MYCXXFLAGS) -c $< -o $@
>>>
>>> PKG_CPPFLAGS=\
>>> -imacros R_affx_constants.h\
>>> -Ifusion_sdk/calvin_files/array/src\
>>> -Ifusion_sdk/calvin_files/data/src\
>>> -Ifusion_sdk/calvin_files/exception/src\
>>> -Ifusion_sdk/calvin_files/fusion/src\
>>> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter\
>>> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter\
>>> -Ifusion_sdk/calvin_files/parameter/src\
>>> -Ifusion_sdk/calvin_files/parsers/src\
>>> -Ifusion_sdk/calvin_files/portability/src\
>>> -Ifusion_sdk/calvin_files/template/src\
>>> -Ifusion_sdk/calvin_files/utils/src\
>>> -Ifusion_sdk/calvin_files/writers/src\
>>> -Ifusion_sdk/file\
>>> -Ifusion_sdk/portability\
>>> -D_USE_MEM_MAPPING_
>>>
>>> PKG_SOURCES = \
>>> fusion_sdk/calvin_files/data/src/CDFData.cpp\
>>> fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.cpp\
>>> fusion_sdk/calvin_files/data/src/CDFProbeInformation.cpp\
>>> fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.cpp\
>>> fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.cpp\
>>> fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.cpp\
>>> fusion_sdk/calvin_files/data/src/CELData.cpp\
>>> fusion_sdk/calvin_files/data/src/DataGroup.cpp\
>>> fusion_sdk/calvin_files/data/src/DataGroupHeader.cpp\
>>> fusion_sdk/calvin_files/data/src/DataSet.cpp\
>>> fusion_sdk/calvin_files/data/src/DataSetHeader.cpp\
>>> fusion_sdk/calvin_files/data/src/FileHeader.cpp\
>>> fusion_sdk/calvin_files/data/src/GenericData.cpp\
>>> fusion_sdk/calvin_files/data/src/GenericDataHeader.cpp\
>>> fusion_sdk/calvin_files/exception/src/ExceptionBase.cpp\
>>> fusion_sdk/calvin_files/fusion/src/CalvinAdapter/
>>> CalvinCELDataAdapter.cpp\
>>> fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.cpp\
>>> fusion_sdk/calvin_files/fusion/src/FusionCDFData.cpp\
>>> fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.cpp\
>>> fusion_sdk/calvin_files/fusion/src/FusionCELData.cpp\
>>> fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.cpp\
>>> fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.cpp\
>>> fusion_sdk/calvin_files/parsers/src/CDFFileReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/CelFileReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/DataGroupReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/DataSetReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/FileHeaderReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/FileInput.cpp\
>>> fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.cpp\
>>> fusion_sdk/calvin_files/parsers/src/GenericFileReader.cpp\
>>> fusion_sdk/calvin_files/utils/src/AffymetrixGuid.cpp\
>>> fusion_sdk/calvin_files/utils/src/DateTime.cpp\
>>> fusion_sdk/calvin_files/utils/src/FileUtils.cpp\
>>> fusion_sdk/calvin_files/utils/src/StringUtils.cpp\
>>> fusion_sdk/calvin_files/utils/src/checksum.cpp\
>>> fusion_sdk/file/BPMAPFileData.cpp\
>>> fusion_sdk/file/BPMAPFileWriter.cpp\
>>> fusion_sdk/file/CDFFileData.cpp\
>>> fusion_sdk/file/CELFileData.cpp\
>>> fusion_sdk/file/FileIO.cpp\
>>> fusion_sdk/file/FileWriter.cpp\
>>> R_affx_cel_parser.cpp\
>>> R_affx_cdf_parser.cpp\
>>> R_affx_cdf_extras.cpp\
>>> R_affx_bpmap_parser.cpp
>>>
>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>>
>>> all: $(SHLIB)
>>
>> --
>> Brian D. Ripley, ripley_at_stats.ox.ac.uk
>> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford, Tel: +44 1865 272861 (self)
>> 1 South Parks Road, +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK Fax: +44 1865 272595

-- 
Brian D. Ripley,                  ripley_at_stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Thu 12 Jul 2007 - 09:27:55 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 12 Jul 2007 - 17:36:06 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.