Satu hal yang saya suka lakukan dengan kode saya adalah memastikan bahwa itu direactored menjadi potongan-potongan yang dikelola. Namun, ketika datang untuk membangun perangkat lunak, saya menemukan bahwa apa pun yang membangun perangkat lunak otomasi saya akhirnya menggunakan (akhir-akhir ini GNU Make atau SCons) akhirnya menjadi berantakan total. File input terlihat seperti skrip panjang yang tampaknya sulit untuk di-refactoring. Saya ingin dapat memperbaiki mereka dalam beberapa cara, tetapi konsep "fungsi" tidak cukup berperilaku dengan cara yang sama dalam beberapa perangkat lunak otomasi bangunan seperti halnya dalam bahasa pemrograman, jadi saya merasa sulit untuk menulis dikelola File makefiles atau SConscript untuk proyek apa pun yang cukup rumit.
Adakah yang punya saran untuk menulis file input yang dapat dikelola untuk membangun perangkat lunak otomasi? Saran agnostik perangkat lunak akan menjadi yang terbaik, tetapi bahkan saran tentang alat otomasi bangunan tertentu akan sangat membantu, terutama Make atau SCons, karena itulah yang saya gunakan untuk proyek.
Edit: Seperti yang ditunjukkan Thorbjørn, saya harus menambahkan beberapa konteks dan contoh penggunaan. Saya menyelesaikan PhD saya di bidang teknik kimia dan saya melakukan penelitian dalam ilmu komputasi. (Saya seorang mod pro di SciComp.SE, bagi Anda yang berkunjung.) Proyek saya biasanya melibatkan campuran bahasa yang dikompilasi (C, C ++, Fortran) yang melakukan beberapa bahasa pengangkatan skrip yang berat (Python , Perl) untuk membuat prototipe, dan kadang-kadang, bahasa khusus domain untuk prototipe atau tujuan teknis.
Saya telah menambahkan dua contoh di bawah ini, kira-kira dalam kisaran 250-line. Masalahnya bagi saya umumnya adalah kurangnya modularitas. Beberapa proyek ini dapat diorganisasikan ke dalam unit modular, dan akan menyenangkan untuk mengabstraksi bagian-bagian bangunan di sepanjang garis itu untuk membuatnya lebih mudah bagi saya dan pengelola masa depan untuk melacak. Memecah setiap skrip menjadi beberapa file telah menjadi salah satu solusi yang telah saya mainkan di kepala saya.
Contoh kedua sangat penting, karena saya harus segera memiliki banyak file.
Inilah yang Makefile
tampak seperti garis 265 bagi saya, diambil dari proyek yang sebenarnya, dan disusun sebaik mungkin:
#!/usr/bin/make
#Directory containing DAEPACK library folder
daepack_root = .
library = $(daepack_root)/lib
wrappers = $(daepack_root)/Wrappers/DSL48S
c_headers = parser.h problemSizes.h
f77_headers=problemSizes.f commonParam.f
f90_headers=problemSizes.f commonParam.f90
includes = -I. -Iinclude -I/usr/include/glib-2.0 \
-I/usr/lib/glib-2.0/include -I/usr/include/libxml2 \
-I/usr/include/libgdome -I/usr/include/gtest/
#Fortran 77 environment variables
f77=gfortran
fflags=-ggdb -cpp -fno-second-underscore --coverage -falign-commons \
-mcmodel=large -fbacktrace -pg
flibs=
#Fortran 90 environment variables
f90=gfortran
f90flags=-ggdb -cpp -fno-second-underscore --coverage -falign-commons \
-mcmodel=large -fbacktrace -pg
f90libs=
#C environment flags
cc=gcc
cflags=-ggdb --coverage $(includes) -mcmodel=large
clibs=
#Libraries for linking
libs=-L$(library) -ldaepack_sparse -lblas -llapack -ldl -lg2c \
-lgdome -lxml2 -lgtest -lcunit -lcholmod -lamd -lcolamd -lccolamd \
-lmetis -lspqr -lm -lblas -llapack -lstdc++ -lpcre
#Object files
objs=main.o $(dsl48sObjs) $(gdxObjs)
gdxObjs = gdxf9def.o gdxf9glu.o gamsglobals_mod.o
commonObjs=libdsl48s_model.sl cklib.o parser.o $(gdxObjs)
originalModelObjs=originalModel.o dsl48sChemkinModule.o $(commonObjs)
cspSlowModelObjs=cspSlowModel.o dsl48sChemkinModuleSlow.o cspModule.o \
$(commonObjs)
orthoProjModelObjs=orthoProjModel.o dsl48sChemkinModuleOrthoProj.o \
orthoProjModule.o basisModule.o spqrUtility.o $(commonObjs)
#Shell environment variable definitions for FUnit
FCFLAGS := $(f90flags)
LDFLAGS := libdsl48s_model.sl cklib.o gdxf9glu.o parser.o spqrUtility.o \
$(libs)
misc=*table *size.f
output=*.out
#Ftncheck flags for static analysis of Fortran 77 code
ftnchekflags= -declare -include=. -library -style=block-if,distinct-do,do-enddo,end-name,goto,labeled-stmt,structured-end
all: ckinterp.exe parserTest.exe originalModel.exe cspSlowModel.exe \
orthoProjModel.exe spqrUtilityTest.exe
#Check code style with lexical analyzer
@echo Checking program style...
ftnchek $(ftnchekflags) rhs.f
ftnchek $(ftnchekflags) resorig.f
ftnchek $(ftnchekflags) res.f
# ftnchek $(ftnchekflags) cklib.f
# ftnchek $(ftnchekflags) ckinterp.f
#Set up baseline coverage data file
@echo Set up baseline coverage data file
lcov -c -i -d . -o conpDSL48Sbase.info
#Run unit test on cspModule.f90
@echo Running unit tests on cspModule.f90...
funit cspModule
#Generate test coverage data for cspModule.f90
@echo Generating test coverage data from cspModule.f90 tests...
lcov -c -d . -o conpDSL48ScspTest.info
#Run unit test on orthoProjModule.f90
@echo Running unit tests on orthoProjModule.f90...
funit orthoProjModule
#Generate test coverage data for orthoProjModule.f90
@echo Generating test coverage data from orthoProjModule.f90 tests...
lcov -c -d . -o conpDSL48SgenProjTest.info
#Run unit tests on the parser C library
@echo Running unit tests on parser in C...
-G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind -v --tool=memcheck \
--leak-check=full --show-reachable=yes --leak-resolution=high \
--num-callers=20 --log-file=parserTest.vgdump \
./parserTest.exe > parserTest.log
#Generate test coverage data for the parser wrapper C library
@echo Generating test coverage data for the parser in C...
lcov -c -d . -o conpDSL48SparserTest.info
#Run unit tests on the SparseQR C library
@echo Running unit tests on SparseQR library in C...
./spqrUtilityTest.exe
#Generate test coverage data for the SparseQR C library
@echo Generating test coverage data for the SparseQR C library...
lcov -c -d . -o conpDSL48SsparseTest.info
#Run unit test on basisModule.f90
@echo Running unit tests on basisModule.f90...
funit basisModule
#Generate test coverage data for basisModule.f90
@echo Generating test coverage data from basisModule.f90 tests...
lcov -c -d . -o conpDSL48SbasisMod.info
#Combine test coverage data
@echo Combine baseline and test coverage data...
lcov -a conpDSL48Sbase.info \
-a conpDSL48ScspTest.info \
-a conpDSL48SgenProjTest.info \
-a conpDSL48SbasisMod.info \
-a conpDSL48SparserTest.info \
-a conpDSL48SsparseTest.info \
-o conpDSL48Stotal.info
#Post-process to remove coverage statistics from automatically
#generated source code.
@echo Removing coverage statistics for automatically generated source...
lcov -r conpDSL48Stotal.info basisModule_fun.f90 \
ckinterp.f cklib.f cspModule_fun.f90 davisSkodjeAd.f90 \
davisSkodjeJac.f90 davisSkodjeRes.f90 davisSkodjeRhs.f90 \
davisSkodjeSp.f90 gdxf9def.f90 gdxf9glu.c orthoProjModule_fun.f90 \
jac.f jacorig.f resad.f resadp.f resorigad.f resorigadp.f ressp.f \
resorigsp.f senrhs.f senrhsorig.f TestRunner.f90 \
-o conpDSL48Stotal.info
#Generate HTML report of coverage data
@echo Generate HTML report of coverage data...
genhtml conpDSL48Stotal.info
@echo Open "index.html" in browser for coverage results!
originalModel.exe: $(originalModelObjs) $(f90_headers) $(f77_headers) \
$(c_headers)
$(f90) $(f90flags) -o originalModel.exe $(originalModelObjs) $(libs)
originalModel.o: dsl48sChemkinModule.o $(commonObjs) $(f77_headers) \
$(f90_headers) $(c_headers)
$(f90) $(f90flags) -c -o originalModel.o originalModel.f90
cspSlowModel.exe: $(cspSlowModelObjs) $(f90_headers) $(f77_headers) \
$(c_headers)
$(f90) $(f90flags) -o cspSlowModel.exe $(cspSlowModelObjs) $(libs)
cspSlowModel.o: dsl48sChemkinModuleSlow.o cspModule.o $(commonObjs) \
$(c_headers) $(f77_headers)
$(f90) $(f90flags) -c -o cspSlowModel.o cspSlowModel.f90
orthoProjModel.exe: $(orthoProjModelObjs) $(f90_headers) $(f77_headers) \
$(c_headers) resOrthoFast.o
$(f90) $(f90flags) -o orthoProjModel.exe $(orthoProjModelObjs) \
resOrthoFast.o $(libs)
orthoProjModel.o: dsl48sChemkinModuleOrthoProj.o orthoProjModule.o $(commonObjs) \
$(c_headers) $(f90_headers) $(f77_headers) resOrthoFast.o basisModule.o
$(f90) $(f90flags) -c -o orthoProjModel.o orthoProjModel.f90
dsl48sChemkinModule.o: dsl48sChemkinModule.f90 cklib.o problemSizes.h \
parser.o $(c_headers) $(f90_headers)
$(f90) $(f90flags) -c -o dsl48sChemkinModule.o dsl48sChemkinModule.f90
dsl48sChemkinModuleSlow.o: dsl48sChemkinModuleSlow.f90 cspModule.o cklib.o \
problemSizes.h parser.o $(c_headers) $(f90_headers)
$(f90) $(f90flags) -c -o dsl48sChemkinModuleSlow.o \
dsl48sChemkinModuleSlow.f90
dsl48sChemkinModuleOrthoProj.o: dsl48sChemkinModuleOrthoProj.f90 \
orthoProjModule.o basisModule.o cklib.o problemSizes.h \
parser.o $(c_headers) $(f90_headers)
$(f90) $(f90flags) -c -o dsl48sChemkinModuleOrthoProj.o \
dsl48sChemkinModuleOrthoProj.f90
basisModule.o: basisModule.f90 cklib.o spqrUtility.o commonParam.f90
$(f90) $(f90flags) -c -o basisModule.o basisModule.f90
spqrUtility.o: spqrUtility.h spqrUtility.c
$(cc) $(cflags) -c -o spqrUtility.o spqrUtility.c
spqrUtilityTest.exe: spqrUtilityTest.o spqrUtility.o
$(cc) $(cflags) -o spqrUtilityTest.exe spqrUtilityTest.o \
spqrUtility.o $(libs)
spqrUtilityTest.o: spqrUtilityTest.c spqrUtility.o
$(cc) $(cflags) -c -o spqrUtilityTest.o spqrUtilityTest.c
cklib.o: cklib.f ckstrt.f
$(f77) $(fflags) -c -o cklib.o cklib.f
ckinterp.exe: ckinterp.o
$(f77) $(fflags) -o ckinterp.exe ckinterp.o
ckinterp.o: ckinterp.f
$(f77) $(fflags) -c -o ckinterp.o ckinterp.f
#Recursive makefile inherited from previous graduate students
libdsl48s_model.sl: $(f77_headers) cklibDAEPACK.f
cp $(wrappers)/makefile model.mk
make -f model.mk
resOrthoFast.o: libdsl48s_model.sl
$(f90) $(f90flags) -c -o resOrthoFast.o resOrthoFast.f90
problemSizes.f: problemSizes.fpp problemSizes.h
cpp problemSizes.fpp problemSizes.f
perl -p -i.bak -we 's/# /! /;' problemSizes.f
commonParam.f90: commonParam.f
perl -p -i.bak -we 's/^#/!/;' commonParam.f
echo "commonParam t f t fpp" | pref77tof90
echo "commonParam /" | f77tof90
perl -p -i.bak -we 's/integer a/!integer a/;' commonParam.f
perl -p -i.bak -we 's/END //;' commonParam.f90
commonParam.f: commonParam.fpp problemSizes.h
cpp commonParam.fpp commonParam.f
perl -p -i.bak -we 's/^#/!/;' commonParam.f
cspModule.o: cspModule.f90
$(f90) $(f90flags) -c -o cspModule.o cspModule.f90
orthoProjModule.o: gamsglobals_mod.o gdxf9def.o gdxf9glu.o orthoProjModule.f90 \
formatLabels.f90
$(f90) $(f90flags) -c -o orthoProjModule.o orthoProjModule.f90
gdxf9def.o: gdxf9def.f90
$(f90) $(f90flags) -c -o gdxf9def.o gdxf9def.f90
gdxf9glu.o: gdxf9glu.c gdxf9def.o
#64-bit version of wrappers (with underscores)
$(cc) $(cflags) -DCIA_LEX -DAPIWRAP_LCASE_DECOR -c -o \
gdxf9glu.o gdxf9glu.c
#64-bit version of wrappers (without underscores, for C interoperability)
# $(cc) $(cflags) -DCIA_LEX -DAPIWRAP_LCASE_NODECOR -c gdxf9glu.c
#32-bit version of wrappers
# $(cc) $(cflags) -DAPIWRAP_LCASE_DECOR -c gdxf9glu.c -Iinclude
gamsglobals_mod.o: gamsglobals_mod.f90 gdxf9def.o gdxf9glu.o
$(f90) $(f90flags) -c gamsglobals_mod.f90
parser.o: parser.c $(c_headers)
$(cc) $(cflags) -c -o parser.o parser.c
parserTest.exe: parserTest.o parser.o
$(cc) $(cflags) -o parserTest.exe parser.o \
parserTest.o $(libs)
parserTest.o: parserTest.cpp parser.o
$(cc) $(cflags) -c -o parserTest.o parserTest.cpp
clean:
-rm *.bak
-rm *.f77
-rm *.log
-rm commonParam.f90
-rm problemSizes.f
-rm commonParam.f
-make clean -f model.mk
-rm model.mk
-rm *.o
-rm *.mod
-rm $(misc)
-rm *.exe
-funit --clean
-rm *.gcno
-rm *.gcda
-rm *.info
-rm *.png
-rm *.html
-rm *.css
-rm -rf html
-rm *.pyc
-rm *.lst
Berikut adalah file 245-baris SConstruct
yang saat ini saya coba atur untuk proyek yang kira-kira sama rumitnya:
## \file SConstruct
# \brief Compiles the library and compiles tests.
#
import SCons
## \brief Build up directory names of each COIN library from package names
# and versions.
#
## Overall SCons environment
#
env = Environment();
flags = []
## Compile using debug versions?
#
debug = True
debugString = '-debug'
debugFlags = ['-ggdb']
dynamicLinkFlag = '-Wl,-rpath,'
if debug:
flags += debugFlags
## Compile Google Test from scratch.
#
GTestVersion = '1.6.0'
GTestStem = 'gtest-' + GTestVersion
GTestBuildIncDir = [GTestStem,
GTestStem + '/include',
]
GTestAllLib = env.Library('lib/libgtest.a', 'gtest-1.6.0/src/gtest-all.cc',
CPPPATH = GTestBuildIncDir,
CXXFLAGS = flags)
GTestMainLib = env.Library('lib/libgtest_main.a',
'gtest-1.6.0/src/gtest_main.cc',
CPPPATH = GTestBuildIncDir,
CXXFLAGS = flags)
GTestIncDir = GTestStem + '/include/gtest'
GTestLibDir = 'lib'
GTestLibFlags = ['gtest', 'gtest_main', 'pthread']
## Armadillo matrix library
#
ArmadilloLibFlags = ['armadillo'];
## Quick reminder of SCons flags:
# CPPPATH = path of headers (include directories)
# LIBPATH = path of libraries
# LIBS = flags of libraries
# CXXFLAGS = C++ compilation flags
#
## Locations of libraries installed on system in standard locations
#
StdIncDir = '/usr/include'
StdLibDir = '/usr/lib'
## Configuration information for COIN libraries
#
CoinUtilsVersion = '2.6.4'
ClpVersion = '1.12.0'
OsiVersion = '0.103.0'
CbcVersion = '2.5.0'
## Some standard directory locations of COIN libraries, with slashes added for
# for convenience.
#
CoinLibLocation = '/usr/local/COIN/'
StdCoinIncDir = '/include/coin'
StdCoinLibDir = '/lib'
CoinUtilsStem = 'CoinUtils-' + CoinUtilsVersion
ClpStem = 'Clp-' + ClpVersion
OsiStem = 'Osi-' + OsiVersion
CbcStem = 'Cbc-' + CbcVersion
if debug:
CoinUtilsStem += debugString
CbcStem += debugString
ClpStem += debugString
OsiStem += debugString
## Build up include directory names for COIN projects from constituent parts.
#
CoinUtilsIncDir = CoinLibLocation + CoinUtilsStem + StdCoinIncDir
ClpIncDir = CoinLibLocation + ClpStem + StdCoinIncDir
OsiIncDir = CoinLibLocation + OsiStem + StdCoinIncDir
CbcIncDir = CoinLibLocation + CbcStem + StdCoinIncDir
## Build up library names from COIN projects from constituent parts
#
CoinUtilsLibDir = CoinLibLocation + CoinUtilsStem + StdCoinLibDir
ClpLibDir = CoinLibLocation + ClpStem + StdCoinLibDir
OsiLibDir = CoinLibLocation + OsiStem + StdCoinLibDir
CbcLibDir = CoinLibLocation + CbcStem + StdCoinLibDir
## CPLEX
#
CpxStem = '/opt/ibm/ILOG/CPLEX_Studio_Academic123/cplex/'
CpxIncDir = CpxStem + 'include/ilcplex'
CpxLibDir = CpxStem + 'lib/x86-64_sles10_4.1/static_pic'
## Gurobi
#
GrbStem = '/opt/gurobi460/linux64/'
GrbIncDir = GrbStem + 'include'
GrbLibDir = GrbStem + 'lib'
OsiLibFlags = ['Osi', 'CoinUtils']
ClpLibFlags = ['Clp', 'OsiClp']
CbcLibFlags = ['Cbc', 'Cgl']
OsiCpxLibFlags = ['OsiCpx']
OsiGrbLibFlags = ['OsiGrb']
CpxLibFlags = ['cplex', 'ilocplex', 'pthread', 'm']
GrbLibFlags = ['gurobi_c++', 'gurobi46', 'pthread', 'm']
milpIncDirs = [CoinUtilsIncDir,
ClpIncDir,
OsiIncDir,
CbcIncDir,
CpxIncDir,
GrbIncDir,
GTestIncDir,
]
milpLibDirs = [CoinUtilsLibDir,
ClpLibDir,
OsiLibDir,
CbcLibDir,
CpxLibDir,
GrbLibDir,
GTestLibDir,
]
milpLibFlags = [OsiCpxLibFlags,
OsiGrbLibFlags,
CbcLibFlags,
ClpLibFlags,
OsiLibFlags,
CpxLibFlags,
GrbLibFlags,
GTestLibFlags,
]
##milpSolver = env.Object('milpSolver.cpp',
## CPPPATH = milpIncDirs,
## LIBPATH = milpLibDirs,
## CXXFLAGS = flags)
milpSolverTest = env.Program('milpSolverUnitTest',
['milpSolverTest.cpp',
'milpSolver.cpp'],
CPPPATH = milpIncDirs,
LIBPATH = milpLibDirs,
LIBS = milpLibFlags,
CXXFLAGS = flags,
LINKFLAGS = ['-Wl,-rpath,' + OsiLibDir])
env.Depends(milpSolverTest, [GTestAllLib, GTestMainLib])
## Chemkin source directories and files
#
ChemkinSourceDir = '/mnt/hgfs/DataFromOldLaptop/Data/ModelReductionResearch/Papers/AdaptiveChemistryPaper/AdaptiveChemistry/NonOpenSource/ChemkinII/';
ChemkinSourceList = ['cklib.f', 'pcmach.f','tranlib.f']
ChemkinSourceList = [ChemkinSourceDir + FileName
for FileName in ChemkinSourceList]
env.Depends('cklib.f','ckstrt.f')
## Cantera include directorie
#
CanteraStem = '/usr/local/cantera'
if debug:
CanteraStem += debugString
CanteraIncDir = CanteraStem + '/include/cantera'
CanteraLibDir = CanteraStem + '/lib'
CanteraTestingFlags = ['kinetics', 'thermo', 'tpx', 'ctbase', 'm',]
CanteraLibFlags = ['user', 'oneD', 'zeroD', 'equil', 'kinetics', 'transport',
'thermo', 'ctnumerics', 'ctmath', 'tpx', 'ctspectra',
'converters', 'ctbase', 'cvode', 'ctlapack', 'ctblas',
'ctf2c', 'ctcxx', 'ctf2c', 'm', 'm', 'stdc++']
CxxFortranFlags = ['g2c', 'gfortran'];
chemSolverIncDir = [CanteraIncDir,
StdIncDir,
'/usr/local/include',
GTestIncDir,
]
chemSolverLibDir = [StdLibDir,
CanteraLibDir,
GTestLibDir,
]
chemSolverLibFlags = [GTestLibFlags,
CxxFortranFlags,
CanteraLibFlags,
ArmadilloLibFlags,
]
chemSolverTest = env.Program('chemSolverUnitTest',
['chemSolverTest.cpp',
'chemSolver.cpp',
'ckwrapper.f90'] + ChemkinSourceList,
CPPPATH = chemSolverIncDir,
LIBPATH = chemSolverLibDir,
LIBS = chemSolverLibFlags,
CXXFLAGS = flags,
FORTRANFLAGS = flags,
F90FLAGS = flags)
env.Depends(chemSolverTest, [GTestAllLib, GTestMainLib])
#env.AddPostAction(milpSolverTest, milpSolverTest[0].abspath)
testAlias = env.Alias('test', [milpSolverTest, chemSolverTest])
AlwaysBuild(testAlias)
ckInterp = env.Program('ckinterp', ChemkinSourceDir + 'ckinterp.f')
canteraGTestLibFlags = CanteraTestingFlags + GTestLibFlags
#canteraGTestLibFlags = ['kinetics', 'thermo', 'tpx',
# 'ctbase', 'm', 'gtest', 'gtest_main', 'pthread']
canteraGTest = env.Program('canteraGTest',
'canteraGTest.cpp',
CPPPATH = chemSolverIncDir,
LIBPATH = chemSolverLibDir,
LIBS = canteraGTestLibFlags,
CXXFLAGS = flags)
env.Depends(canteraGTest, [GTestAllLib, GTestMainLib])
canteraMemTestLibFlags = CanteraTestingFlags
canteraMemTest = env.Program('canteraMemTest',
'canteraMemTest.cpp',
CPPPATH = chemSolverIncDir,
LIBPATH = chemSolverLibDir,
LIBS = canteraMemTestLibFlags,
CXXFLAGS = flags)