Simplify declaration of making Alpaka-based library to depend on the main library of the same package #45415

makortel · 2024-07-09T16:42:20Z

Question

would it be feasible to add the this dependency from the alpaka-backend libraries to the "main library" automatically for all Alpaka-enabled packages?

Answer

The problem is that this will add a lot of extra dependencies to the alpaka libraries, including some dependencies on CUDA that we should not add to the non-CUDA backends.

It may be possible to improve this later, once

we remove the native CUDA support from the framework;

linking with --as-needed works (e.g. gcc: add --as-needed link option cmsdist#9232).

When the two items have been completed, we could try to simplify how to make the Alpaka-based libraries of a package to depend on the main library of the same package.

makortel · 2024-07-09T16:42:25Z

assign core, heterogeneous

cmsbuild · 2024-07-09T16:42:36Z

New categories assigned: core,heterogeneous

@Dr15Jones,@fwyzard,@makortel,@makortel,@smuzaffar you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2024-07-09T16:42:37Z

cms-bot internal usage

cmsbuild · 2024-07-09T16:42:38Z

A new Issue was created by @makortel.

@Dr15Jones, @antoniovilela, @makortel, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

makortel · 2024-07-10T15:53:59Z

On a further thought

The problem is that this will add a lot of extra dependencies to the alpaka libraries, including some dependencies on CUDA that we should not add to the non-CUDA backends.

isn't this happening already? I mean, for example in,

cmssw/CondFormats/EcalObjects/BuildFile.xml

Lines 1 to 19 in e232a9b

    
           <use name="FWCore/Utilities"/> 
        
           <use name="CondFormats/Serialization"/> 
        
           <use name="DataFormats/EcalDetId"/> 
        
           <use name="DataFormats/EcalDigi"/> 
        
           <use name="DataFormats/Math"/> 
        
           <use name="DataFormats/Portable"/> 
        
           <use name="DataFormats/SoATemplate"/> 
        
           <use name="HeterogeneousCore/CUDACore"/> 
        
           <use name="HeterogeneousCore/CUDAUtilities"/> 
        
           <use name="boost"/> 
        
           <use name="boost_serialization"/> 
        
           <use name="rootmath"/> 
        
           <use name="clhep"/> 
        
           <use name="cuda"/> 
        
           <use name="HeterogeneousCore/AlpakaInterface"/> 
        
           <flags ALPAKA_BACKENDS="!serial"/> 
        
           <export> 
        
             <lib name="1"/> 
        
           </export>

aren't the per-Alpaka-backend libraries linked to all the dependencies declared above anyhow?

At least in 14_1_0_pre5 I see

$ ldd $CMSSW_RELEASE_BASE/lib/el8_amd64_gcc12/libCondFormatsEcalObjectsROCmAsync.so | grep -i cuda
        libHeterogeneousCoreCUDACore.so => /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre5/lib/el8_amd64_gcc12/libHeterogeneousCoreCUDACore.so (0x00007f8857af1000)
        libHeterogeneousCoreCUDAServices.so => /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre5/lib/el8_amd64_gcc12/libHeterogeneousCoreCUDAServices.so (0x00007f8857abd000)
        libCUDADataFormatsCommon.so => /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre5/lib/el8_amd64_gcc12/libCUDADataFormatsCommon.so (0x00007f8857ab4000)
        libHeterogeneousCoreCUDAUtilities.so => /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre5/lib/el8_amd64_gcc12/libHeterogeneousCoreCUDAUtilities.so (0x00007f88579a6000)
        libcudart.so.12 => /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre5/external/el8_amd64_gcc12/lib/libcudart.so.12 (0x00007f8855f9b000)
        libnvidia-ml.so.1 => /cvmfs/cms.cern.ch/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/lib64/stubs/libnvidia-ml.so.1 (0x00007f8855b82000)
        libcuda.so.1 => /cvmfs/cms.cern.ch/el8_amd64_gcc12/external/cuda/12.4.1-fc5cb0e72dba64b6abbf00089f3a044c/lib64/stubs/libcuda.so.1 (0x00007f885561b000)

makortel · 2024-08-12T15:29:02Z

The problem is that this will add a lot of extra dependencies to the alpaka libraries, including some dependencies on CUDA that we should not add to the non-CUDA backends.

isn't this happening already?

@smuzaffar @fwyzard Any thoughts? As far as I can tell, making each per-Alpaka-backend library to depend on the main library of the same package would not make the dependencies of the per-Alpaka-backend libraries any worse they are already.

fwyzard · 2024-08-13T07:40:53Z

If adding the dependency by default does not actually add more dependencies to the alpaka-based libraries, I would be in favour of doing it.

We should start removing the CUDA-only support early in 14.2.x, which will remove some of the more problematic dependencies.

In addition -Wl,--as-needed could help removing the unused dependencies.
@smuzaffar what happened to cms-sw/cmsdist#9232 ?

fwyzard · 2024-08-13T08:56:43Z

As a test, I've checked out the full CMSSW_14_1_0_pre6, added

<use name="..." for="alpaka"/>

to all packages that build alpaka libraries, and rebuilt all packages.

Comparing the dependencies of the rebuilt alpaka libraries with the original ones, the only difference is the extra dependency on the non-alpaka library itself - all other dependencies were there already, as @makortel observed.

For example, for RecoLocalTracker/SiPixelRecHits, the change is

$ git diff RecoLocalTracker/SiPixelRecHits

diff --git a/RecoLocalTracker/SiPixelRecHits/BuildFile.xml b/RecoLocalTracker/SiPixelRecHits/BuildFile.xml
index 62787f4c989c..0654e442fbee 100644
--- a/RecoLocalTracker/SiPixelRecHits/BuildFile.xml
+++ b/RecoLocalTracker/SiPixelRecHits/BuildFile.xml
@@ -15,6 +15,7 @@
 <use name="HeterogeneousCore/CUDAUtilities"/>
 <use name="RecoLocalTracker/ClusterParameterEstimator"/>
 <flags ALPAKA_BACKENDS="1"/>
+<use name="RecoLocalTracker/SiPixelRecHits" for="alpaka"/>
 <export>
   <lib name="1"/>
 </export>

The impact on the libraries is

$ ldd $CMSSW_RELEASE_BASE/lib/$SCRAM_ARCH/libRecoLocalTrackerSiPixelRecHitsROCmAsync.so | sed -e's/ (0x\w\+)//' | sort > original.dep
$ ldd $CMSSW_BASE/lib/$SCRAM_ARCH/libRecoLocalTrackerSiPixelRecHitsROCmAsync.so | sed -e's/ (0x\w\+)//' | sort > modified.dep
$ diff -u original.dep modified.dep

--- original.dep        2024-08-13 10:54:58.524045103 +0200
+++ modified.dep        2024-08-13 10:55:03.804125494 +0200
@@ -95,6 +95,7 @@
        libPhysics.so => /data/cmssw/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre6/external/el8_amd64_gcc12/lib/libPhysics.so
        libRIO.so => /data/cmssw/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre6/external/el8_amd64_gcc12/lib/libRIO.so
        libRecoLocalTrackerClusterParameterEstimator.so => /data/user/fwyzard/test_alpaka_link/CMSSW_14_1_0_pre6/lib/el8_amd64_gcc12/libRecoLocalTrackerClusterParameterEstimator.so
+       libRecoLocalTrackerSiPixelRecHits.so => /data/user/fwyzard/test_alpaka_link/CMSSW_14_1_0_pre6/lib/el8_amd64_gcc12/libRecoLocalTrackerSiPixelRecHits.so
        libRint.so => /data/cmssw/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre6/external/el8_amd64_gcc12/lib/libRint.so
        libSmatrix.so => /data/cmssw/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre6/external/el8_amd64_gcc12/lib/libSmatrix.so
        libThread.so => /data/cmssw/el8_amd64_gcc12/cms/cmssw/CMSSW_14_1_0_pre6/external/el8_amd64_gcc12/lib/libThread.so

The only difference is the explicit dependency on libRecoLocalTrackerSiPixelRecHits.so.

smuzaffar · 2024-08-13T09:12:02Z

In addition -Wl,--as-needed could help removing the unused dependencies. @smuzaffar what happened to cms-sw/cmsdist#9232 ?

@fwyzard , this was closed in favor of CMSSW_14_1_ASNEEDED_X IBs which are build with -Wl,--as-needed. CMSSW_14_1_ASNEEDED_X IBs build without errors but we have runtime errors. Once we get ride of those then we can enable -Wl,--as-needed for all

fwyzard · 2024-08-13T09:20:50Z

Where can I find the CMSSW_14_1_ASNEEDED_X IBs ?
I didn't see any one the IB page.

smuzaffar · 2024-08-13T09:28:35Z

We built those on demand, I just have restarted one (it should be available in few hours) and also changed the configuration to automatically build 3 times a week

smuzaffar · 2024-08-13T15:44:36Z

@fwyzard , CMSSW_14_1_ASNEEDED_X_2024-08-12-2300 is available now

fwyzard · 2024-08-13T16:24:13Z

Thanks.

makortel · 2024-11-08T21:07:36Z

If adding the dependency by default does not actually add more dependencies to the alpaka-based libraries, I would be in favour of doing it.

Comparing the dependencies of the rebuilt alpaka libraries with the original ones, the only difference is the extra dependency on the non-alpaka library itself - all other dependencies were there already, as @makortel observed.

@smuzaffar Would it be feasible to make each per-Alpaka-backend library to depend on the main library? (to simplify the BuildFile.xml setup)

smuzaffar · 2024-11-13T10:21:37Z

@smuzaffar Would it be feasible to make each per-Alpaka-backend library to depend on the main library?

@makortel , yes

By the way, with #46684 ( and corresponding cmsdist PR) we should be able to enable --as-needded flag for cmssw

cmsbuild added core-pending pending-signatures heterogeneous-pending labels Jul 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplify declaration of making Alpaka-based library to depend on the main library of the same package #45415

Simplify declaration of making Alpaka-based library to depend on the main library of the same package #45415

Simplify declaration of making Alpaka-based library to depend on the main library of the same package #45415

Simplify declaration of making Alpaka-based library to depend on the main library of the same package #45415

Comments