build: allow shell commands in `CC` and friends

When we added support for paths with spaces in the `CC`, `LD`, `AR`,
etc. variables in a previous patch, we explicitly broke support for
compiler laun

build: allow shell commands in `CC` and friends

When we added support for paths with spaces in the `CC`, `LD`, `AR`,
etc. variables in a previous patch, we explicitly broke support for
compiler launchers like `ccache`, which is usually used by prepending it
to `CC`, e.g. `CC='ccache gcc'`. This patch modifies the toolchain
detection logic to avoid sanitizing the toolchain variables for the
shell unless we are confident that they represent a path to the program
(i.e. that `which $(CC)` resolves to something).

Change-Id: I942d09cfc462d50ed07c5d22434b0275c22d1522
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: skip toolchain detection for some targets" into integration

build: skip toolchain detection for some targets

Most developers run the `clean`, `checkpatch` and other similar targets
without specifying any other additional build options. When combined
with a f

build: skip toolchain detection for some targets

Most developers run the `clean`, `checkpatch` and other similar targets
without specifying any other additional build options. When combined
with a flow where the developer passes `CROSS_COMPILE` or `CC`
explicitly, and where the default-configured tools are not on the PATH,
these targets will warn about unrecognized toolchain tools.

This change is a workaround for this whereby the toolchain makefile is
not expanded unless a target *not* named `*clean`, `check*` `doc` or
`*tool` has been specified.

Change-Id: I2f2a275964b65253df07c2207043217b14f615fe
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "makefile-cleanup" into integration

* changes:
build: improve diagnostics for unrecognized toolchain tools
build(rzg): separate BL2 and BL31 SREC generation
build(rcar

Merge changes from topic "makefile-cleanup" into integration

* changes:
build: improve diagnostics for unrecognized toolchain tools
build(rzg): separate BL2 and BL31 SREC generation
build(rcar): separate BL2 and BL31 SREC generation
build: separate preprocessing from DTB compilation
build: remove `MAKE_BUILD_STRINGS` function

show more ...

build: improve diagnostics for unrecognized toolchain tools

Up until recently the build system accepted an arbitrary value for `AS`
and, unbeknownst to anybody, was simply not making use of it. Rece

build: improve diagnostics for unrecognized toolchain tools

Up until recently the build system accepted an arbitrary value for `AS`
and, unbeknownst to anybody, was simply not making use of it. Recent
feedback has revealed that a number of contributors have `AS` explicitly
configured to use the GNU assembler, which is not a supported assembler,
and this breakage has yielded some cryptic error messages, e.g.:

aarch64-none-elf-as: unrecognized option '-x'

This change introduces human-readable diagnostics to help developers
with diagnosing unsupported toolchain tools:

The configured AArch64 assembler could not be identified and may not
be supported:

aarch64-none-elf-as

The default AArch64 assembler is:

aarch64-none-elf-gcc

The following tools are supported:

- Arm® Compiler for Embedded `armclang`
- LLVM Clang (`clang`)
- GNU GCC (`gcc`)

The build system will treat this assembler as GNU GCC (`gcc`).

Change-Id: I316036c83be2d45ee83a88846cf65c6ce7ae3c26
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "fix(build): wrap toolchain paths in double quotes" into integration

fix(build): wrap toolchain paths in double quotes

Fix issue with Windows paths containing spaces. Recent toolchain
refactoring (cc277de) caused a regression in the Windows build. Ensure
toolchain pa

fix(build): wrap toolchain paths in double quotes

Fix issue with Windows paths containing spaces. Recent toolchain
refactoring (cc277de) caused a regression in the Windows build. Ensure
toolchain path utilities wrap paths in double quoted strings.

Change-Id: I7a136e459d85cff1e9851aedf0a5272a841df09c
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
Signed-off-by: Chris Kay <chris.kay@arm.com>
Co-authored-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "feat(build): redirect stdin to nul during toolchain detection" into integration

feat(build): redirect stdin to nul during toolchain detection

It's common for Makefiles to use variables like CC, AS or LD instead of
hardcoding the name of binaries. These can be defined by the use

feat(build): redirect stdin to nul during toolchain detection

It's common for Makefiles to use variables like CC, AS or LD instead of
hardcoding the name of binaries. These can be defined by the user to
use a differnet toolchain or even as a crutch to enable cross-compilation.

In TF-A, this is not needed, as support for cross-compilation is baked
in via the CROSS_COMPILE option. TF-A still defined AS for its internal
use, but unlike most other projects, the default was setting it to the C
compiler. Overriding it wasn't possible from the environment though,
only as a make argument, so this didn't cause much issue.

With commit cc277de81692 ("build: refactor toolchain detection"), AS can
now also be set from the environment. This breaks any scripts that
supply make with a cross environment that sets AS to an assembler.

Doing so was without effect before, but now leads to a quite ugly failure
mode: As TF-A now tries to detect the toolchain, it will call AS with the
option -v, which for GNU as(1) prints the version, but doesn't exit.

Thus, as(1) will continue waiting on stdin input and the build hangs
without much indication what's wrong.

Avoid this failure mode by ensuring any tool that attempts to read stdin
during toolchain detection will immediately get EOF and exit, leading to
an error message later on instead of the build hang.

Change-Id: I79a84961f5a69250292caa7f9e879a65be4bd9f2
Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>

show more ...

Merge "build: use GCC to link by default" into integration

Merge "fix(build): don't rely on that gcc-ar is in the same directory as gcc" into integration

build: use GCC to link by default

When configuring GNU GCC as the C compiler, we usually use the GNU BFD
linker directly to link by default. However, this complicates things
because we also need to

build: use GCC to link by default

When configuring GNU GCC as the C compiler, we usually use the GNU BFD
linker directly to link by default. However, this complicates things
because we also need to support LTO, which can only be done when linking
is done via the C compiler, and we cannot change the linker later on if
some other part of the build system wants to enable LTO.

This change migrates the default choice of linker to GCC if the C
compiler is GCC, in order to enable this use-case. This should have no
impact on anything outside of the build system, as by default GCC merely
acts as a wrapper around BFD.

Change-Id: I40771be2b0571def67bbfde9e877e7629ec8cdaa
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

fix(build): don't rely on that gcc-ar is in the same directory as gcc

ccache - a fast C/C++ compiler cache.
ccache wraps gcc and g++ compilers, but not other tools like ar.
If ccache is installed, t

fix(build): don't rely on that gcc-ar is in the same directory as gcc

ccache - a fast C/C++ compiler cache.
ccache wraps gcc and g++ compilers, but not other tools like ar.
If ccache is installed, then build fails with

```
make: /usr/lib/ccache/aarch64-linux-gnu-gcc-ar: No such file or directory
```

Change-Id: I555c178aeaa0cf411cdf67b4a81e7952be762d0f
Signed-off-by: Andrey Skvortsov <andrej.skvortzov@gmail.com>
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: correct minor toolchain documentation error" into integration

build: correct minor toolchain documentation error

Default values for toolchain tools are instead provided by the main
toolchain makefile, rather than a parent makefile. This was an oversight
from a

build: correct minor toolchain documentation error

Default values for toolchain tools are instead provided by the main
toolchain makefile, rather than a parent makefile. This was an oversight
from a previous version of the original toolchain refactor patch.

Change-Id: I75752ed7874b36e1c679d94292a2664e234c484b
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: refactor toolchain detection" into integration

build: refactor toolchain detection

This change refactors how we identify the toolchain, with the ultimate
aim of eventually cleaning up the various mechanisms that we employ to
configure default to

build: refactor toolchain detection

This change refactors how we identify the toolchain, with the ultimate
aim of eventually cleaning up the various mechanisms that we employ to
configure default tools, identify the tools in use, and configure
toolchain flags.

To do this, we introduce three new concepts in this change:

- Toolchain identifiers,
- Tool class identifiers, and
- Tool identifiers.

Toolchain identifiers identify a configurable chain of tools targeting
one platform/machine/architecture. Today, these are:

- The host machine, which receives the `host` identifier,
- The AArch32 architecture, which receives the `aarch32` identifier, and
- The AArch64 architecture, which receivs the `aarch64` identifier.

The tools in a toolchain may come from different vendors, and are not
necessarily expected to come from one single toolchain distribution. In
most cases it is perfectly valid to mix tools from different toolchain
distributions, with some exceptions (notably, link-time optimization
generally requires the compiler and the linker to be aligned).

Tool class identifiers identify a class (or "role") of a tool. C
compilers, assemblers and linkers are all examples of tool classes.

Tool identifiers identify a specific tool recognized and supported by
the build system. Every tool that can make up a part of a toolchain must
receive a tool identifier.

These new identifiers can be used to retrieve information about the
toolchain in a more standardized fashion.

For example, logic in a Makefile that should only execute when the C
compiler is GNU GCC can now check the tool identifier for the C compiler
in the relevant toolchain:

ifeq ($($(ARCH)-cc-id),gnu-gcc)
...
endif

Change-Id: Icc23e43aaa32f4fd01d8187c5202f5012a634e7c
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...